Human-computer interaction method for converting computer display color into speech embedding vector

Abstract

The present application discloses a human-computer interaction method for converting a computer display color into a speech embedding vector, applicable to the field of computer machine learning software. When speech processing is performed by means of machine learning, a speech feature is pre-trained to be an embedding vector, and the embedding vector is further inputted to a subsequent deep neural network, thereby solving the specific application scenario problem. The present application provides a human-computer interaction method for converting a computer display color into a speech embedding vector in speech machine learning. Such a method can enable software users to intuitively and rapidly use colors to obtain embedding vectors in a fuzzy but targeted manner, and ensure that identical colors correspond to identical embedding vectors, thereby obtaining corresponding identical speech features; and similar colors generate spatially proximate embedding vectors, thereby obtaining corresponding similar speech features.

Description

A human-computer interaction method for converting computer display colors into speech embedding vectors Technical Field

[0001] The present application relates to the field of computer machine learning software and is a human-computer interaction method for converting computer display colors into speech embedding vectors. Background Art

[0002] The component values ​​of the embedding vector are generally obtained through machine learning of specific speech. However, when the recorded embedding vector is used to obtain the speech with the corresponding characteristics, it will face greater difficulties. The most common methods for entering the component values ​​of the general vector are:

[0003] (1) Enter the value of each component one by one through the human-computer dialogue interface designed by the software;

[0004] (2) The software provides multiple options for each component value. Through the human-computer dialogue interface designed by the software, the user selects each component value one by one;

[0005] (3) Through the slider designed by the software, each slider represents a component, and the user mainly uses the mouse to move the slider to enter the component value.

[0006] However, many speech features of embedded vectors cannot be clearly described in human language. Consequently, when entering speech features, users are unable to describe them, leaving them unaware of the resulting speech features. Only after all components of the vectors are entered (often with high dimensionality and many components that cannot be described in human language, requiring only blind entry), can they be processed by computer software and the resulting data obtained. When users need to enter embedded vectors multiple times and submit them to computer software for processing, then evaluate each result and select the one they find most satisfying, traditional data entry methods are not only inefficient and tedious, but also prevent them from experimenting with the approximate direction of the resulting speech features. Manual data entry is essentially impractical.

[0007] For speech feature generation software that starts by vaguely selecting speech with certain characteristics by monitoring the final processing results (the user himself does not know exactly what the speech characteristics that satisfy him are at this time), and continuously tries to obtain speech characteristics that satisfy the user, the entry of each component value of the embedded vector does not need to be very precise, but only needs to be rough, fuzzy, directional and in accordance with certain rules.

[0008] The present application can roughly predict the spatial distance between abstract vectors based on the color selected by the user, and thus based on the correspondence between the components of the color vector and the components of the target embedding vector and the seed variable, combined with the method adopted by the present application to generate an embedding vector that meets certain conditions through the seed variable: for similar colors, the corresponding embedding vectors are close in spatial distance; for dissimilar colors, the corresponding embedding vectors are far in spatial distance.

[0009] Many existing technologies can convert spatially close embedding vectors into sounds with similar speech features, such as using the decoder technology in a variational autoencoder (VAE) to achieve the conversion from speech feature vectors to speech. Therefore, this application does not focus on how to convert spatially close speech embedding vectors into sounds with similar speech features, nor how to convert spatially distant speech embedding vectors into sounds with significantly different speech features. Instead, it focuses on intuitively and quickly generating fuzzy and abstract embedding vectors for spatially close or distant distances by selecting similar or dissimilar colors.

[0010] In addition, the components of the embedding vector with standard normal distribution in the n-m+1 dimensional space can effectively maintain the stability of the final embedded vector.

[0011] Finally, the present application can intuitively, conveniently and quickly input the component values ​​of the fuzzy predictable embedding vector of the calculation result into the calculation software by allowing the user to select a color, so that the user can quickly obtain the embedding vector close to his or her vague idea and convert it into speech for evaluation and screening through monitoring. Summary of the Invention

[0012] Step 1: A color vector (dimension: m) is formed from one or more color model components, referred to as "vector A". Assume that the components of vector A are (X1, X2, X3.....X m ); The target embedding vector (dimension: n) to be assigned is referred to as "vector T". Assume that the components of vector T are (Y1, Y2, Y3.....Y n ). n>m. Generally speaking, n is much larger than m. All components of vector T are initially set to 0.

[0013] Step 2: Construct the component assignment correspondence between vector A and vector T according to the following rules:

[0014] At least one component of vector A is assigned to a seed variable, for example, X1 is assigned to the seed variable, and the remaining variables of vector A, X2 is assigned to Y1, X3 is assigned to Y2, X m Assign Y m-1 .

[0015] In order to make the numerical range of the components of vector A match the components of the target embedding vector to be assigned, some mathematical transformations (such as linear transformations) can be applied to transform the component values ​​of vector A before assignment.

[0016] Step 3: Input the seed variable X1 into the algorithm that can generate a specific n-m+1 dimensional embedding vector from the seed variable, and generate a vector C whose components are (Z1, Z2, Z3...Z n-m+1 ).

[0017] The algorithm of the algorithmizer is as follows:

[0018] (1) Initialization phase: First, a pseudo-random number generator is initialized using a fixed value S as the seed. The selection of the pseudo-random number generator is arbitrary, but it must ensure that the random number sequences generated by the same seed are exactly the same. Pseudo-random number generators that meet the requirements include but are not limited to LCG, MT19937, XORSHIFT, PCG, etc.

[0019] (2) Use the pseudo-random number generator initialized in (1) to generate N standard normal distribution vectors V1, V2, V3…V n .

[0020] (3) The component representing the random number seed in the color selected by the user is scaled to the range of 1 to N through linear operation and used as input x.

[0021] (4) Calculate the output coefficient r = x-floor(x) and obtain the standard deviation correction coefficient E = sqrt(r^2+(1-r)^2)).

[0022] (5) Obtain Vx = V_floor(x) and Vy = V_ceil(x). Where floor and ceil are rounded down and rounded up.

[0023] (6) Obtain the output vector (Vx*r + Vy*(1-r)) / E.

[0024] This algorithm has the following characteristics:

[0025] a. For any user input, each component of the output vector follows a standard normal distribution;

[0026] b. For the same user input, the output vector is exactly the same;

[0027] c. For user inputs with similar values, the spatial distance of the output vectors is close.

[0028] Step 4: Let Y m =Z1,Y m+1 =Z2,Y m+2=Z3......Y n =Z n-m+1 , the remaining components of vector A are assigned to those components of vector T that are not generated by the seed variable (assigned components).

[0029] The assignment results of each component of the final target speech embedding vector T are: T (X2, X3......X m ,Z1,Z2......Z n-m+1 ).

[0030] There are multiple seed variables, which are similar to this and will not be described in detail.

[0031] Compared with the prior art, the technical solution of this application has the following beneficial effects:

[0032] This application can roughly predict whether the target embedding vector will have similar effects when converted to speech based on the color selected by the user. This allows for intuitive and rapid entry of multi-component parameters of the fuzzy embedding vector based on the selected color, making it convenient for users to quickly and directionally obtain and evaluate the calculation results until they achieve satisfactory speech feature results. Specifically, it has the following advantages:

[0033] a. Improve the efficiency of high-dimensional vector parameter acquisition;

[0034] b. Use intuitive colors instead of abstract numbers to collect data;

[0035] c. By establishing a relationship between color and embedding vectors, we can roughly predict the speech feature results generated by the selected color, making it much easier for users to obtain satisfactory and stable feature speech through repeated testing;

[0036] d. Improve the fun of using software for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to clearly reveal the content of this application, we have made the process of Example 1 and Example 2 reflecting the implementation steps of this application into flow charts, see Figures 1 and 2.

[0038] To more clearly illustrate the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are some embodiments of the present application, not all embodiments. For those skilled in the art, other embodiments can be derived from these embodiments, and drawings of other embodiments can be derived from these embodiment drawings without inventive work.

[0039] FIG1 is a schematic diagram of the operation flow of a human-computer interaction method for converting computer display colors into speech embedding vectors provided by the present application;

[0040] FIG2 is a flow chart of selecting similar and dissimilar colors, converting them into corresponding embedded vectors, and calculating the corresponding vector space distances using the method provided by the present application. Modes for Carrying Out the Invention

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0042] The following two specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0043] Example 1

[0044] 1 is a schematic diagram of the operation flow of a human-computer interaction method for converting computer display colors into speech embedding vectors provided by this application, which is specifically described as follows:

[0045] Step 1: Use the RGB color model. The three components of the RGB model, R (red), G (green), and B (blue), are used as vector components to form a three-dimensional vector A(R, G, B). Select a pink color ("color 1") whose corresponding R, G, and B values ​​are 1.0 for R, 0.25 for G, and 1.0 for B. These values ​​are assigned to vector A, completing the value of vector A, namely A(1.0, 0.25, 1.0).

[0046] Step 2: The speech embedding vector suitable for a specific speech synthesis model is a 101-dimensional high-dimensional vector T1 (Y1, Y2.....Y 101 ), where the Y1 component is the gender feature component. A value of 0.0 indicates the strongest male feature and the weakest female feature, while a value of 1.0 indicates the weakest male feature and the strongest female feature. The Y2 component is the pitch feature component. A value of 0.0 indicates the highest pitch, corresponding to a MIDI pitch of 81 (frequency of 880Hz), and a value of 1.0 indicates the lowest pitch, corresponding to a MIDI pitch of 33 (frequency of 55Hz). The initial values ​​of all components of vector T1 are set to 0. Then, the R component value of vector A (1.0) is assigned to the Y1 component of vector T1, and the G component value of vector A (0.25) is assigned to the Y2 component. This selects the R\G values ​​of the RGB color model, and the gender and pitch feature values ​​of the target embedding vector are determined.

[0047] Step 3: Call the algorithm that generates the values ​​of each component of the embedding vector except gender (Y1) and pitch feature (Y2) through the seed variable. The seed variable is assigned the value of the B component of the A vector (1.0), and the vector C (Z1, Z2...Z 99 ).

[0048] The values ​​are: C(1.76, 0.40, 0.98, 2.24, 1.87, -0.98, 0.95, -0.15, -0.10, 0.41, 0.14, 1.45, 0.76, 0.12, 0.44, 0.33, 1.49, -0.21, 0.31, -0.85, -2.55, 0.65, 0.86, -0.74, 2.27, -1.45, 0.05, -0.19, 1.53, 1.47, 0.15, 0.38, -0.89, -1.98, -0.35, 0.16, 1.23, 1.20, -0.39, -0.30, , -1.05, -1.42, -1.71, 1.95, -0.51, -0.44, -1.25, 0.78, -1.61, -0.21, -0.90, 0.39, -0.51, -1.18, -0.03, 0.43, 0.07, 0.30, -0.63, -0.36, -0.67, -0.36, -0.81, -1.73, 0.18, -0.40, -1.63, 0.46, -0.91, 0.05, 0.73, 0.13, 1.14, -1.23, 0.40, -0.68, -0.87, -0.58, -0.31, 0.06, -1.17, 0.90, 0.47, -1.54, 1.49, 1.90, 1.18, -0.18, -1.07, 1.05, -0.40, 1.22, 0.21, 0.98, 0.36, 0.71, 0.01, 1.79, 0.13)

[0049] Step 4: Let Z1=Y3, Z2=Y4, ..., Z 99 =Y 101 , thus completing the assignment of all components of the target embedding vector T1. The embedded vector T1 after assignment is: T1 ([R value], [G value], Z1, Z2......Z 99), that is: T1 (1.0, 0.25, 1.76, 0.40, 0.98, 2.24, 1.87, -0.98, 0.95, -0.15, -0.10, 0.41, 0.14, 1.45, 0.76, 0.12, 0.44, 0.33, 1.49, -0.21, 0.31, -0.85, -2.55, 0.65, 0.86, -0.74, 2.27, -1.45, 0.05, -0.19, 1.53, 1.47, 0.15, 0.38, -0.89, -1.98, -0.35, 0.16, 1.23, 1.20, , -0.39, -0.30, -1.05, -1.42, -1.71, 1.95, -0.51, -0.44, -1.25, 0.78, -1.61, -0.21, -0.90, 0.39, -0.51, -1.18, -0.03, 0.43, 0.07, 0.30, -0.63, -0.36, -0.67, -0.36, -0.81, -1.73, 0.18, -0.40, -1.63, 0.46, -0.91, 0.05, 0.73, 0.13, 1.14, -1.23, 0.40, -0.68, -0.87, -0.58, -0.31, 0.06, -1.17, 0.90, 0.47, -1.54, 1.49, 1.90, 1.18, -0.18, -1.07, 1.05, -0.40, 1.22, 0.21, 0.98, 0.36, 0.71, 0.01, 1.79, 0.13).

[0050] By inputting vector T1 into its applicable speech synthesis model, a sound with specific speech features under the speech synthesis model represented by pink can be generated (this is not the content of this application, but only reflects the result of this application being used by different speech synthesis models).

[0051] Example 2

[0052] 2 , which is a flow chart showing the method provided by the present application for converting similar and dissimilar colors into corresponding embedded vectors and calculating the corresponding vector space distances.

[0053] Select color 1 from Example 1 (RGB color model R value 1.0, G value 0.25, B value 1.0); then select a slightly darker pink color that is slightly similar to color 1 ("color 2"), with an RGB color model R value of 0.8, G value 0.25, and B value of 0.8; and select a light green color that is not similar to color 1 ("color 3"), with an RGB color model R value of 0.0, G value 0.75, and B value 0.5.

[0054] According to the method of embodiment 1, the embedding vectors corresponding to color 1, color 2, and color 3 are obtained as follows:

[0055] The color 1 embedding vector T1 is the same as that in the first embodiment and is not listed again.

[0056] Color 2 embedding vector T2(0.8, 0.25, 1.81, 0.84, 0.62, 1.87, 2.05, -1.23, 1.39, -0.25, -0.28, 0.86, 0.50, 1.86, 0.96, -0.09, 0.89, 0.26, 1.64, 0.03, 0.27, -0.68, -2.25, 0.73, 0.57, -0.65, 2.52, -1.58, 0.01, -0.29, 1.94, 1.59, 0.25, 0.18, -0.73, -2.09, -0.33, -0.00, 1.36, 1.31, , -0.43, -0.20, -1.28, -1.74, -1.55, 1.93, -0.34, 0.15, -0.99, 0.53, -1.29, -0.53, -0.98, 0.36, -0.08, -1.33, -0.23, 0.39, -0.10, 0.57, -0.88, -0.63, -0.76, -0.47, -1.44, 0.19, -0.69, -1.38, 0.21, -1.25, 0.34, 0.78, 0.35, 1.18, -0.99, 0.23, -0.92, -0.68, -0.76, -0.47, -0.06, -1.13, 0.79, 0.12, -1.65, 0.90, 1.99, 0.76, -0.44, -1.03, 0.84, -0.02, 0.87, 0.27, 0.94, 0.06, 0.81, -0.03, 1.92, 0.32).

[0057] Color 3 embedding vector T3(0.0, 0.75, 1.53, 1.61, -0.26, 0.69, 2.01, -1.52, 2.05, -0.40, -0.60, 1.65, 1.15, 2.35, 1.18, -0.52, 1.66, 0.05, 1.62, 0.52, 0.11, -0.17, -1.15, 0.73, -0.17, -0.31, 2.54, -1.52, -0.07, -0.44, 2.39, 1.51, 0.40, -0.28, -0.25, -1.88, -0.22, -0.34, 1.35, 1.26, , -0.42, 0.07, -1.51, -2.06, -0.90, 1.50, 0.09, 1.38, -0.22, -0.10, -0.35, -1.08, -0.96, 0.23, 0.85, -1.36, -0.60, 0.23, -0.42, 1.01, -1.21, -1.07, -0.79, -0.61, 0.79, -0.55, 0.19, -1.15, -0.56, -0.38, -1.73, 0.88, 0.74, 0.74, 1.03, -0.27, -0.18, -1.22, -0.13, -0.98, -0.71, -0.28, -0.81, 0.39, -0.64, -1.54, -0.52, 1.78, -0.30, -0.91, -0.72, 0.22, 0.81, -0.05, 0.34, 0.66, -0.57, 0.87, -0.11, 1.81, 0.67).

[0058] Calculate the spatial distance between color 1, color 2, and color 3, using the commonly used Euclidean distance for measurement:

[0059] It can be calculated that:

[0060] The distance between T1 and T2 is 2.45

[0061] The distance between T2 and T3 is 5.32

[0062] The distance between T1 and T3 is 7.62

[0063] Results: The spatial distance between T1 and T2 was the shortest, the spatial distance between T2 and T3 was farther, and the spatial distance between T1 and T3 was the farthest.

[0064] By inputting vector T1, vector T2, and vector T3 into the applicable speech synthesis model, sounds with specific speech features under the speech synthesis models represented by color 1, color 2, and color 3 can be generated. The speech features generated by color 1 and color 2 sound closer, while the speech features generated by color 2 and color 3 sound quite different (whether the generated speech is similar also depends on the speech synthesis model and the subjective feelings of different people, but this is not the content of this application, but only reflects the results of this application being used by different speech synthesis models).

Claims

1. A human-computer interaction method for converting a computer display color into a speech embedding vector ("target embedding vector," referred to as "vector T," whose dimension is n), characterized by: Step 1: Determine one or more colors, and use the color model components of each color as a vector component to construct a color vector, referred to as "vector A", with a dimension of m; Step 2: Construct the component correspondence between vector A and vector T according to the following rules: one component of vector A is assigned to a variable (referred to as the "seed variable"), and the remaining components of vector A are assigned to specific components of vector T (referred to as the "assigned components"); Step 3: Use the algorithm disclosed in this application or a similar algorithm to generate a vector C with seed variables, whose dimension is n-m+1. The algorithm for generating vector C with seed variables satisfies the following conditions: (1) seed variables with the same value generate the same vector C; (2) vectors C generated by seed variables with similar values ​​have similar spatial distances; (3) the component values ​​of vector C are standard normally distributed in the n-m+1 dimensional space; Step 4: The assigned components (dimension: m-1) and the components of vector C (dimension: n-m+1) together form vector T (dimension: n) converted from the computer display color.

2. According to claim 1, the color model of step 1 generally has three to four components. By selecting multiple colors, each component of the color model of each color (regardless of whether they are the same type of color model) is assigned to a component of the target embedding vector to meet the requirement of a large number of assigned components.

3. According to step 2 of claim 1, one component of vector A is assigned to the seed variable, and the remaining components of vector A are assigned to specific components of vector T. In order to make the numerical range of the components of vector A match the components of the target embedding vector to be assigned, some mathematical transformations (such as linear transformations) can be applied before the assignment.

4. The algorithm for generating the n-m+1-dimensional vector C from the seed variable in step 3 of claim 1 is not the specific method (linear interpolation) disclosed in this application. Alternative methods include other methods such as LSH (Locality-Sensitive Hashing) and space-filling curves.

5. There may be more than one seed variable, and thus the component of the vector A assigned to the seed variable may also be more than one (depending on the different methods used to generate the embedding vector having the characteristics described in this application from the seed variable), but the functions of these multiple sub-variables are the same or similar to the function of the one seed variable described in this application.

6. The color model selected in step 1 according to claim 1 is not specific, such as the HSV color model, CMYK color model, RGB color model, etc.

7. The method for converting computer display colors into speech embedding vectors according to claim 1 can also be used simultaneously with a traditional method of entering data into a target embedding vector through human-computer dialogue. For example, some components of the embedding vector are obtained through color conversion, while other components can be directly achieved through traditional methods such as user entry, user parameter selection, and selection through a slider.

8. The method according to claims 1-7, wherein one or more colors are converted into speech embedding vectors corresponding to the colors.

Images