Human-computer interaction method simulating light illumination acquisition for input embedding vector values

Abstract

The present application relates to the technical field of machine learning. Disclosed is a human-computer interaction method simulating light illumination acquisition to transform input embedding vector values into target embedding vectors. The method comprises: step one, configuring a simulated light source on a plane; step two, using embedding vectors extracted from the same input entity as an input embedding vector set, and input embedding vectors extracted from a plurality of input entities forming a plurality of input embedding vector sets; step three, reducing the dimensions of each input embedding vector to two dimensions and then the input embedding vector forming one node in a planar coordinate system, and connecting the nodes formed by the vectors to form a curve having several nodes; step four, on the basis of the distance between each node, which is irradiated by the simulated light source, and the light source, determining an influence weight of the input embedding vector corresponding to the node with respect to a target embedding vector; and step five, according to the method of the present application, performing calculation on the input embedding vectors obtained in step two and the weights thereof obtained in step four, so as to obtain target embedding vectors.

Description

A human-computer interaction method for simulating illumination acquisition and input embedding vector values Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a human-computer interaction method for simulating light acquisition to acquire input embedding vector values ​​and generate target embedding vectors. Background Art

[0002] In the field of machine learning, the method of creating new entities from input entities (which can be speech, images, text, videos, etc.) is to feed the input entities as machine learning materials into a pre-defined machine learning model to produce new entities. For example, multiple speech sounds can be fed into a machine learning model as learning materials to generate new speech sounds with the characteristics of the input speech sounds; multiple images can be fed into a machine learning model as learning materials to generate new images with the characteristics of the input images.

[0003] Existing human-computer interaction methods for creating new entities have the following shortcomings:

[0004] (1) The characteristics of machine learning methods themselves determine that their results are difficult to predict, and the results can only be known after completing the iterative operation;

[0005] (2) If users have preference requirements for newly created entities, this can only be achieved by selecting a large number of input entities based on preference, which is labor-intensive, costly, and time-consuming.

[0006] (3) For situations where multiple attempts are required to evaluate the new entities generated in each attempt and select the new entity that the user is most satisfied with, the cost is higher and the time is longer.

[0007] Therefore, it is necessary to design a more efficient method with controllable results to obtain new entities by inputting entities.

[0008] The general method of machine learning to extract features from any entity is to first obtain an embedding vector that expresses the characteristics of the entity through dimensionality reduction or unsupervised learning methods, and then generate a new entity based on the embedding vector (using deep neural networks or other methods) to meet the needs of different specific application scenarios.

[0009] This application simulates the human-computer interaction method of lighting to intuitively weight the embedding vector of the input entity to obtain the embedding vector value of the new entity, and then obtain the new entity corresponding to the embedding vector.

[0010] Using this method to create new entities has the following advantages:

[0011] (1) The different weights assigned to the embedding vectors of different input entities are converted into the distance between the visible light source and the node corresponding to the embedding vector (which can be visualized as the illumination intensity). This intuitive intervention in the embedding vector allows the user's tendency to be well intuitively reflected, greatly enhancing the predictability of the results for new entities;

[0012] (2) The method of obtaining new entities from input entities through machine learning multiple times for user selection is transformed into: obtaining embedding vectors for all input entities, taking multiple values ​​of different weights for the input embedding vectors through manual intervention (simulating lighting) to obtain embedding vectors of multiple new entities, and then generating multiple new entities for user selection based on the embedding vectors, which greatly improves efficiency and saves a lot of time. Summary of the Invention

[0013] This application provides a human-computer interaction method for generating a target embedding vector by simulating light acquisition input embedding vector values. This method solves the problems of high computational complexity, low efficiency, and low directional controllability when creating new entities from existing entities using machine learning methods. The method includes the following steps:

[0014] Step 1: Set a simulated light source on the interactive interface;

[0015] Step 2: The embedding vectors extracted from the same input entity are used as an input embedding vector group, and the input embedding vectors extracted from multiple input entities form multiple input embedding vector groups;

[0016] Step 3: Reduce the input embedding vector to two dimensions using principal component analysis or other methods. After reducing it to two dimensions, it becomes a node on the plane coordinate system of the interactive interface. By accessing this node, the value of the input embedding vector that formed it can be obtained. In the interactive interface, the nodes formed by the dimensionality reduction of each vector in the same input embedding vector group are connected to form a curve with multiple nodes. Multiple input embedding vector groups form multiple curves with multiple nodes.

[0017] Step 4: Receive user operations on the interactive interface (using a mouse, touch screen, or other methods) to move the simulated light source, simulate the distance between each node illuminated by the light source and the light source (which can be visualized as light intensity), and determine the influence weight of the input embedding vector corresponding to the node on the target embedding vector. When the curve in the illuminated area is defined as having shading or partial shading, the curve blocks and forms a shadow; whether the curve blocks and forms a shadow depends on the user's settings; when the curve is defined as having shading or partial shading, the algorithm for whether it blocks a node:

[0018] (1) Connect the two adjacent nodes of the curve formed by each entity with a line segment to form a line segment set A. (2) Connect the light source to each node with a line segment to form another line segment set B. (3) Determine whether there is any intersection between the two sets of line segments A and B one by one. If the line segment from the light source to the X node in the B group has two or more intersection points with the line segments in the A group with different straight-line distances from the light source, the X node is blocked (which can be visualized as being in the shadow).

[0019] Step 5: Calculate the input embedding vector obtained in step 2 and its weight obtained in step 4 as follows to obtain the target embedding vector:

[0020] The influence weight of each node's corresponding vector on the target embedding vector is: if the node is illuminated, the intensity of the illumination is determined by the node's distance from the light source. If the node in the illuminated area is obscured by a light-blocking or partially light-blocking curve (in the shadow of the light blockage), the weight can be defined as 0 or another value.

[0021] Based on the input embedding vectors and their weights, the target embedding vector is calculated by multiplying all the input embedding vectors by the weight values ​​and then adding them. There are many methods for calculating the target embedding vector based on the input embedding vectors and their weights, and they are not limited to the method used in this application.

[0022] The target embedding vector is obtained based on the above method.

[0023] It is not within the scope of this application to obtain an input embedding vector by using dimensionality reduction methods or unsupervised learning methods for input entity data. The starting point of this application is that the input embedding vector of a specific entity has been obtained.

[0024] Accordingly, the cutoff point of this application is to obtain the target embedding vector, not to obtain the target new entity. Using deep neural networks or other methods to obtain new entities based on the target embedding vector is also not within the scope of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a flow chart of a human-computer interaction method for generating a target embedding vector by simulating illumination acquisition input embedding vector values ​​provided by the present application;

[0026] FIG2 is a schematic diagram of a user interaction interface of a first embodiment of a human-computer interaction method for generating a target embedding vector by simulating light acquisition input embedding vector values ​​provided by the present application and applied to the field of speech machine learning;

[0027] FIG3 is a schematic diagram of a user interaction interface of a second embodiment of the present invention, wherein a human-computer interaction method for simulating light acquisition and inputting embedding vector values ​​to generate target embedding vectors is applied to the field of speech machine learning;

[0028] FIG4 is a schematic diagram of a user interaction interface when the simulated light source of Example 2 is moved to another position;

[0029] Figure 5 is a schematic diagram of the user interaction interface of Example 3 of the human-computer interaction method for generating target embedding vectors by simulating light acquisition input embedding vector values ​​provided in this application and applied to the field of speech machine learning. Modes for Carrying Out the Invention

[0030] The following will be combined with the accompanying drawings and specific embodiments to clearly and completely describe the technical solutions of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0032] Example 1

[0033] 2 is a schematic diagram of a user interaction interface of a first embodiment of a human-computer interaction method for generating a target embedding vector by simulating light acquisition input embedding vector values ​​provided by this application and applied to the field of speech machine learning. The specific implementation method is as follows:

[0034] The speech feature embedding vector set in the speech machine learning software for specific applications is a 32-dimensional vector, and its structure has been determined. There are methods for extracting the speech feature embedding vector value through unsupervised learning methods and methods for generating new speech based on the embedding vector using deep neural networks.

[0035] Step 1: Set a simulated light source L on the interactive plane;

[0036] Step 2: Upload a 60-second audio file (referred to as "Input Speech 1") to the interactive interface (there are various ways to upload, such as dragging and dropping, or selecting a file to upload). A pre-trained variational autoencoder (VAE) extracts the features of Input Speech 1, generating an embedding vector for every ten seconds of speech. The embedding vectors are ordered according to the speech time sequence, generating a set of six sequential input embedding vectors.

[0037] Step 3: Reduce the input embedding vectors to two dimensions using principal component analysis (PCA). Each embedding vector forms a node. The six nodes formed by the embedding vector group are connected sequentially to form a curve with nodes, which is curve 1 in Figure 2. By accessing the two-dimensional coordinates of any node, the value of the input embedding vector that formed that node can be obtained.

[0038] Step 4: The user moves the simulated light source L using the mouse in the interactive interface. The simulated light source L forms an illumination area. The distance between each node in the illumination area and the light source (visualized as light intensity) determines the weight of the embedding vector corresponding to the node. The embedding vector values ​​and corresponding weight values ​​of the six nodes are obtained respectively: weight a of node α1, weight b of node α2, weight c of node α3, weight d of node α4, weight e of node α5, and weight f of node α6.

[0039] Step 5: The embedding vector and weight of the node illuminated by the user obtained by operating the simulated light source L in the interactive interface are used to calculate the light source point embedding vector (i.e., the target embedding vector) value using the following method:

[0040]

[0041] Where X is the light source embedding vector (i.e., the target embedding vector), and α1, α2, α3, α4, α5, and α6 are the embedding vectors representing the six illuminated nodes. They are all 32-dimensional and have a fixed structure. a, b, c, d, e, and f are the weights of α1, α2, α3, α4, α5, and α6, respectively.

[0042] The embedded vector X is further processed using a pre-trained deep neural network, and finally a new speech is obtained by assigning different weights to the speech features of each segment of the 60-second input speech 1.

[0043] By moving the simulated light source L and repeating the above steps 4 and 5, multiple new voices can be obtained for the user to choose by assigning different weights to different nodes of the input voice 1.

[0044] Example 2

[0045] 3 is a schematic diagram of the user interaction interface of Example 2 of the human-computer interaction method for generating target embedding vectors by simulating light acquisition input embedding vector values ​​provided by this application and applied to the field of speech machine learning. The specific implementation method is as follows:

[0046] Step 1: Set a simulated light source L on the interactive plane;

[0047] Step 2: Upload three speech audios of 60 seconds, 30 seconds, and 40 seconds respectively (referred to as "input speech 1, input speech 2, input speech 3") to the interactive interface (there are various ways to upload, such as dragging in, selecting a file to upload, etc.), and extract the speech features through the existing unsupervised learning method. An embedding vector is formed for every five seconds of speech. The order of the embedding vectors is used as the order of the speech time sequence to generate three groups of embedding vectors. The first group consists of 12 sequential embedding vectors, namely α1, α2, α3, α4, α5, α6, α7, α8, α9, α 10 , α 11 , α 12 , the second group consists of 6 sequential embedding vectors, namely β1, β2, β3, β4, β5, β6; the third group consists of 8 sequential embedding vectors, namely γ1, γ2, γ3, γ4, γ5, γ6, γ7, γ8;

[0048] Step 3: Reduce the three sets of embedding vectors to two dimensions using principal component analysis (PCA). Each embedding vector in each set forms a node. The nodes formed by each set of embedding vectors are connected sequentially to form three curves with nodes, namely curves l1, l2, and l3 in Figure 2. By accessing the two-dimensional coordinates of any node, the value of the input embedding vector that formed that node can be obtained.

[0049] Step 4: Move the simulated light source to form an illuminated area. The distance between each node in the illuminated area and the light source (visualized as light intensity) determines the weight of the embedding vector corresponding to that node. Since all three curves are set to have light-blocking properties, the weight of the nodes in the shadow is set to 0, so the embedding vector weight corresponding to the unilluminated node is 0. Whether the node is illuminated is determined according to the algorithm in paragraph

[0018] of the invention.

[0050] There are eight nodes that are illuminated by l1, namely α3, α4, α5, α6, α7, α8, α9, α 10 , according to their distance from the simulated light source L, the weight values ​​of the 8 nodes are a1, b1, c1, d1, e1, f1, g1, h1. The remaining four nodes are blocked by the curvature of curve 1 itself, and their weights are 0, namely α1, α2, α 11 , α 12 The weight of is 0.

[0051] All six nodes l2 can be illuminated. According to the distance of the six nodes from the simulated light source L, the weight values ​​of the β1, β2, β3, β4, β5, and β6 nodes are a2, b2, c2, d2, e2, and f2 respectively.

[0052] The eight nodes of l3 are all in the shadow of the curve formed by l1. Since the curve is set to have shading properties, the weights of nodes γ1, γ2, γ3, γ4, γ5, γ6, γ7, and γ8 are all 0.

[0053] Step 5: Calculate the target embedding vector value using the same method as in Example 1:

[0054]

[0055] FIG4 is a schematic diagram of the user interaction interface when the simulated light source of Example 2 is moved to another position, as described below:

[0056] Step 4: Use the mouse to move the simulated light source L and position it at the position shown in Figure 4.

[0057] There are three nodes illuminated by l1, namely α1, α2, and α3. According to the distance from the simulated light source L, the weight values ​​of these three nodes are a1, b1, and c1 respectively. The remaining nine nodes are blocked due to the curvature of curve 1, and the weights are 0, namely α4, α5, α6, α7, α8, α9, α10, α11, α12, α13, α14, α15, α16, α17, α18, α19, α20, α21, α22, α23, α24, α25, α26, α27, α28, α29, α30, α31, α32, α33, α34, α35, α36 10 , α 11 , α 12 The weight is 0.

[0058] All six nodes l2 can be illuminated. According to the distance of the six nodes from the simulated light source L, the weight values ​​of the β1, β2, β3, β4, β5, and β6 nodes are a2, b2, c2, d2, e2, and f2 respectively.

[0059] There are three nodes illuminated by l3, namely γ1, γ2, and γ3, with weights a3, b3, and c3, that is, the weights of γ4, γ5, γ6, γ7, and γ8 are all 0.

[0060] Step 5: The light source embedding vector (i.e., target embedding vector) is calculated using the three sets of weights and embedding vectors:

[0061]

[0062] The embedded vector X is further processed using a pre-trained deep neural network, and finally a new speech is obtained by assigning different weights to the speech features of each segment of input speech 1, input speech 2, and input speech 3.

[0063] By moving the simulated light source L and repeating the above steps 4 and 5, multiple new voices can be obtained for the user to select by assigning different weights to each segment of input voice 1, input voice 2, and input voice 3.

[0064] Example 3

[0065] 5 is a schematic diagram of the user interaction interface of Example 3 of the human-computer interaction method for generating target embedding vectors by simulating light acquisition input embedding vector values ​​provided by this application and applied to the field of speech machine learning. The specific implementation method is as follows:

[0066] Step 1, step 2, and step 3 are the same as those in Example 2;

[0067] In step 4, all three curves are set to have no light-shielding property, so there is no shadow formed by the curves blocking each other.

[0068] According to the position of the simulated light source L and the illumination area it forms, the eight nodes α3, α4, α5, α6, α7, α8, α9, α 10 The remaining nodes α1, α2, α 11 , α 12 Since the set curve does not have light-shielding properties and is also illuminated, the weight values ​​of the 12 nodes are a1, b1, c1, d1, e1, f1, g1, h1, i1, j1, k1, and m1 according to the distance of each node from the simulated light source L.

[0069] All six nodes l2 can be illuminated. According to the distance of the six nodes from the simulated light source L, the weight values ​​of the β1, β2, β3, β4, β5, and β6 nodes are a2, b2, c2, d2, e2, and f2 respectively.

[0070] The eight nodes γ1, γ2, γ3, γ4, γ5, γ6, γ7, and γ8 are all illuminated. The weights of the eight nodes are a3, b3, c3, d3, e3, f3, g3, and h3, respectively, based on the distance of the eight nodes to the simulated light source L. 3。

[0071] Step 5: The light source embedding vector (i.e., target embedding vector) is calculated using the three sets of weights and embedding vectors:

[0072]

[0073] The embedded vector X is further processed using a pre-trained deep neural network, and finally a new speech is obtained by assigning different weights to the speech features of each segment of input speech 1, input speech 2, and input speech 3.

[0074] By moving the simulated light source L and repeating the above steps 4 and 5, multiple new voices can be obtained for the user to select by assigning different weights to each segment of input voice 1, input voice 2, and input voice 3.

Claims

1. A human-computer interaction method for generating a target embedding vector by simulating light acquisition input embedding vector values, characterized in that: include: Step 1: Set a simulated light source on the interactive plane; Step 2: The embedding vectors extracted from the same input entity (referred to as "input embedding vectors") are used as an input embedding vector group. The input embedding vectors extracted from multiple input entities form multiple input embedding vector groups. Step 3: Reduce the input embedding vector to two dimensions using principal component analysis (PCA) or other methods. After reducing it to two dimensions, it becomes a node (referred to as a "node") on the plane coordinate system (hidden or visible) of the interactive interface. By accessing the node, the value of the input embedding vector that formed the node can be obtained. In the interactive interface, the nodes formed by the dimensionality reduction of each vector in the same input embedding vector group are connected to form a curve with multiple nodes. Multiple input embedding vector groups form multiple curves with multiple nodes. Step 4: Receive user operations on the interactive interface (using a mouse, touch screen, or other methods) to move the simulated light source. The distance between each node illuminated by the simulated light source and the light source (which can be visualized as light intensity) determines the influence weight of the input embedding vector corresponding to the node on the target embedding vector; Step 5: Calculate the input embedding vector obtained in step 2 and its weight obtained in step 4 according to the method disclosed in this application to obtain the target embedding vector.

2. The multiple curves with nodes formed by the input embedding vector described in step 3 of claim 1 can be defined as having shading or partial shading properties, so that the curves form shadows when illuminated, and the nodes located in the shadows can be set to have an influence weight on the target embedding vector of 0 or other values. The multiple curves with nodes can also be defined as not having shading properties, so that the curves do not block each other.

3. There can be multiple methods for obtaining the target embedding vector based on the input embedding vector and its weight as described in steps 4 and 5 of claim 1, and it is not limited to the method of multiplying and adding the input embedding vector and the weight adopted in the invention content and embodiments of this application.

4. According to the method described in claims 1-3, the input embedding vectors corresponding to the nodes illuminated by the simulated light source and their influence weights are collected, and the target embedding vectors are assigned values ​​through calculation, thereby obtaining target embedding vectors that are differently affected by different input embedding vectors.

Images