Voice instruction control method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a voice instruction control method and device, computer equipment, a computer readable storage medium and a computer program product, relates to the technical field of voice control, and can improve the voice control efficiency. The method comprises the following steps: acquiring a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by vectorizing a corresponding text instruction in the text instruction set through the server; receiving a voice instruction input by a user to the terminal, converting the voice instruction into an initial text instruction, and obtaining an initial text instruction vector corresponding to the initial text instruction; matching the initial text instruction vector with a plurality of text instruction vectors in the vector set, and determining a target text instruction vector obtained by matching; and obtaining a target text instruction corresponding to the target text instruction vector from the text instruction set, and executing a corresponding control operation based on the target text instruction.

Description

Technical Field

[0001] The present application relates to the field of voice control technology, and in particular to a voice command control method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the advancement of voice interaction technology, voice command recognition has gradually become an important function of smart terminals. Existing voice control methods mainly rely on server-side processing, that is, uploading the complete voice to the cloud for recognition and returning the command, or initially recognizing the command at the terminal and then uploading the command for the server to proofread. Although these two methods reduce the pressure on the terminal, they have obvious disadvantages: strong dependence on the network, resulting in latency problems, and huge server resource consumption, especially the need to deploy complex natural language processing models to match initial commands and target commands. These disadvantages make the efficiency of voice control very low. Summary of the invention

[0003] Based on this, it is necessary to provide a voice command control method, device, computer equipment, computer-readable storage medium and computer program product that can improve voice control efficiency in response to the above technical problems.

[0004] In a first aspect, the present application provides a voice command control method, comprising:

[0005] Acquire a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set;

[0006] Receiving a voice command input by a user to a terminal, converting the voice command into an initial text command, and determining an initial text command vector corresponding to the initial text command;

[0007] Matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector;

[0008] A target text instruction corresponding to the target text instruction vector is obtained from the text instruction set, and a corresponding control operation is performed based on the target text instruction.

[0009] In one embodiment, matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector includes:

[0010] Performing vector similarity calculations on the initial text instruction vector and the multiple text instruction vectors of the vector set respectively, to obtain the similarity between the initial text instruction and each text instruction vector in the multiple text instruction vectors;

[0011] According to the similarity, the text instruction vectors in the target vector set corresponding to the similarity are sorted in descending order, and the text instruction vector corresponding to the highest similarity is used as the target text instruction vector.

[0012] In one embodiment, each of the text instruction vectors in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set based on a multimodal model deployed on the server, and the same multimodal model is deployed on the terminal;

[0013] The determining the initial text instruction vector corresponding to the initial text instruction comprises:

[0014] Inputting the initial text instruction into the multimodal model deployed on the terminal, and obtaining text feature data based on the model output result;

[0015] The text feature data is converted into an initial text instruction vector.

[0016] In one embodiment, the step of obtaining a vector set and a text instruction set corresponding to the vector set from a server includes:

[0017] Determining version information corresponding to an initial instruction set and an initial vector set stored in the terminal;

[0018] Sending an instruction set update request carrying the version information to the server, and receiving set update information returned by the server; the set update information is determined by the server according to the version information;

[0019] The initial instruction set and the initial vector set are updated according to the update information to obtain the text instruction set and the vector set.

[0020] In one embodiment, the updating of the initial instruction set and the initial vector set according to the update information to obtain the text instruction set and the vector set includes:

[0021] According to the update information, acquiring an incremental instruction set and an incremental vector set from the server;

[0022] The acquired incremental instruction set and incremental vector set are added to the initial instruction set and initial vector set respectively to obtain the text instruction set and the vector set.

[0023] In one embodiment, the executing a corresponding control operation based on the target text instruction includes:

[0024] Determining a current control scenario according to the initial text instruction, and determining a preset similarity threshold under the current control scenario;

[0025] The similarity corresponding to the target text instruction vector is compared with the preset similarity threshold; the similarity corresponding to the target text instruction vector is the result obtained by respectively calculating the vector similarity between the initial text instruction vector and the target text instruction vector.

[0026] If the similarity is lower than the preset similarity threshold, an instruction confirmation prompt of the target text instruction is displayed to the user, and after receiving the user's confirmation operation on the instruction confirmation prompt, a corresponding control operation is performed based on the target text instruction.

[0027] In a second aspect, the present application also provides a voice command control device, comprising:

[0028] An instruction and vector management module, used for acquiring a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set;

[0029] A voice instruction conversion and vector generation module, used to receive a voice instruction input by a user to the terminal, convert the voice instruction into an initial text instruction, and determine an initial text instruction vector corresponding to the initial text instruction;

[0030] A vector matching and instruction recognition module, used for matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set, and determining a matched target text instruction vector;

[0031] The instruction determination and control module is used to obtain a target text instruction corresponding to the target text instruction vector from the text instruction set, and perform a corresponding control operation based on the target text instruction.

[0032] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0033] Acquire a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set;

[0034] Receiving a voice command input by a user to a terminal, converting the voice command into an initial text command, and determining an initial text command vector corresponding to the initial text command;

[0035] Matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector;

[0036] A target text instruction corresponding to the target text instruction vector is obtained from the text instruction set, and a corresponding control operation is performed based on the target text instruction.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0038] Acquire a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set;

[0039] Receiving a voice command input by a user to a terminal, converting the voice command into an initial text command, and determining an initial text command vector corresponding to the initial text command;

[0040] Matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector;

[0041] A target text instruction corresponding to the target text instruction vector is obtained from the text instruction set, and a corresponding control operation is performed based on the target text instruction.

[0042] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0043] Acquire a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set;

[0044] Receiving a voice command input by a user to a terminal, converting the voice command into an initial text command, and determining an initial text command vector corresponding to the initial text command;

[0045] Matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector;

[0046] A target text instruction corresponding to the target text instruction vector is obtained from the text instruction set, and a corresponding control operation is performed based on the target text instruction.

[0047] The above-mentioned voice command control method, device, computer equipment, computer-readable storage medium and computer program product are obtained by obtaining a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set; receiving a voice instruction input by a user to a terminal, converting the voice instruction into an initial text instruction, and obtaining an initial text instruction vector corresponding to the initial text instruction; matching the initial text instruction vector with multiple text instruction vectors in the vector set to determine a target text instruction vector obtained by matching; obtaining a target text instruction corresponding to the target text instruction vector from the text instruction set, and performing a corresponding control operation based on the target text instruction. In the present application, the terminal obtains a text instruction set from a server and a vector set corresponding to the instruction, so that the terminal has the ability of unilateral processing, reduces dependence on the server, improves the response speed of voice instructions, converts the voice signal into text and further vectorizes the converted text, and then performs vector matching, which can capture the semantic features of the user's instruction, rather than relying solely on the surface structure of the text, improves the accuracy of the control operation on the voice instruction, and thus improves the efficiency of voice instruction control. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] FIG1 is a diagram of an application environment of a voice command control method according to an embodiment;

[0050] FIG2 is a flow chart of a voice command control method in another embodiment;

[0051] FIG3 is a specific application framework diagram of a voice command control method in one embodiment;

[0052] FIG4 is a timing diagram of a voice command control method in one embodiment;

[0053] FIG5 is a flow chart of a voice command control method in another embodiment;

[0054] FIG6 is a structural block diagram of a voice command control device in one embodiment;

[0055] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] The voice command control method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. The terminal communicates with the server through the network. The terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0058] In an exemplary embodiment, as shown in FIG2 , a voice command control method is provided, which is described by taking the method applied to the terminal in FIG1 as an example, and includes the following steps S202 to S208 . Among them:

[0059] Step S202, obtaining a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set.

[0060] Among them, the text instruction set is the text content of all possible user input instructions stored on the server side, which provides input for the vectorization model. The vector set is a set of feature data generated by the server after extracting and vectorizing each instruction in the text instruction set through the vectorization model. Each vector represents the semantic information of the corresponding text instruction in the instruction set. In one embodiment, the server can select different pre-trained models or optimization algorithms according to the needs of different scenarios. For example, for simple voice commands, a smaller model may be used for fast vectorization; for more complex instructions, a larger and more accurate model may be used for processing.

[0061] Exemplarily, the server packages the vectorized text instruction set and vector set and synchronizes them to the terminal device through the network to ensure the integrity and consistency of the terminal instruction library. After receiving the text instruction set and vector set, the terminal stores them in a local database and prepares for instruction matching.

[0062] Optionally, the terminal obtains an adapted text instruction set and vector set from the server according to the current terminal type to ensure that it is suitable for the control requirements of the current terminal device.

[0063] Step S204, receiving a voice command input by a user to the terminal, converting the voice command into an initial text command, and obtaining an initial text command vector corresponding to the initial text command;

[0064] Among them, the initial text instruction is the text form converted into the voice instruction after voice recognition. It is a short instruction or command that expresses the user's intention and provides a standardized representation for subsequent semantic understanding and instruction matching.

[0065] The initial text instruction vector is generated by vectorization through a multimodal model or other feature extraction model after receiving the voice instruction input by the user on the terminal and converting it into an initial text instruction. The initial text instruction vector not only contains the semantic features of the user's instruction, but also expresses the position of the instruction in the multidimensional space through numerical processing, so that it can be used for accurate calculation matching. This initial text instruction vector is used as a query vector to find the target text instruction vector that best meets the user's instruction intention in the vector set in the terminal.

[0066] Exemplarily, the user issues a voice command through the terminal device, and the terminal receives the voice command through its built-in microphone. The terminal uses natural language processing technology to decompose and parse the language in the collected audio signal to generate an initial text command corresponding to the user's voice content. After the initial text command is recognized, the terminal will process the initial text command using a multimodal model to generate its corresponding initial text command vector.

[0067] Optionally, the multimodal model is a CLIP (Contrastive Language-Image Pre-Training) model, which is an effective and scalable multimodal pre-trained neural network model for learning based on natural language supervision signals. The terminal uses the CLIP model to generate a high-dimensional vector based on the position of each word and sentence in the initial text instruction in the context and its semantic meaning, which can represent the deep semantic features of the text instruction.

[0068] Step S206, matching the initial text instruction vector with a plurality of text instruction vectors in the vector set, and determining a matched target text instruction vector.

[0069] Among them, the target text instruction vector is the vector that is semantically closest to the initial text instruction vector, representing the instruction closest to the user's intention selected by the terminal from multiple text instruction vectors in the vector set, thereby guiding the terminal to perform corresponding operations.

[0070] Exemplarily, the terminal receives and loads the initial text instruction vector and the vector set, prepares to perform a matching operation, and traverses the vector set, comparing the initial text instruction vector with multiple text instruction vectors in the set one by one to identify the target vector that best meets the conditions. After the comparison is completed, the terminal determines the vector closest to the initial text instruction vector from the vector set, and sets the vector as the target text instruction vector.

[0071] Step S208, obtaining a target text instruction corresponding to the target text instruction vector from the text instruction set, and executing a corresponding control operation based on the target text instruction.

[0072] Exemplarily, the terminal retrieves the text instruction set according to the target text instruction vector matched in the previous step, and finds the target text instruction associated with the vector. This instruction directly represents the final parsing result of the user input voice instruction. After the terminal parses the target text instruction, it triggers the corresponding control operation. The control operation is pre-defined in the terminal and corresponds to the text instruction one by one.

[0073] In the above-mentioned voice command control method, a vector set and a text command set corresponding to the vector set are obtained from a server; each text command vector in the vector set is obtained by the server performing vectorization processing on the corresponding text command in the text command set; receiving a voice command input by a user to a terminal, converting the voice command into an initial text command, and obtaining an initial text command vector corresponding to the initial text command; matching the initial text command vector with multiple text command vectors in the vector set, and determining a target text command vector obtained by matching; obtaining a target text command corresponding to the target text command vector from the text command set, and performing a corresponding control operation based on the target text command. In this embodiment, the terminal obtains the text command set of the server and the vector set corresponding to the command, so that the terminal has the ability of unilateral processing, reduces the dependence on the server, improves the response speed of the voice command, converts the voice signal into text and further vectorizes the converted text, and then performs vector matching, which can capture the semantic features of the user's command, rather than relying solely on the surface structure of the text, improves the accuracy of the control operation on the voice command, and thus improves the efficiency of voice command control.

[0074] In an exemplary embodiment, step S206 matches the initial text instruction vector with multiple text instruction vectors in a vector set to determine a target text instruction vector obtained by matching, including: performing vector similarity calculations on the initial text instruction vector and multiple text instruction vectors in the vector set to obtain the similarity between the initial text instruction and each text instruction vector in the multiple text instruction vectors; sorting the text instruction vectors in the target vector set corresponding to the similarity in descending order according to the similarity, and taking the text instruction vector corresponding to the highest similarity as the target text instruction vector.

[0075] Among them, vector similarity is the degree of similarity between the initial text instruction vector and the instruction vector in the vector set. The similarity value is a numerical indicator that can be calculated by algorithms such as cosine similarity and Euclidean distance, reflecting the degree of proximity of two vectors in multidimensional space. The higher the similarity, the closer the two vectors are semantically.

[0076] Specifically, the terminal calculates the similarity of the initial text instruction vector and each text instruction vector in the vector set by using the cosine similarity algorithm to obtain the similarity value between the initial text instruction vector and each text instruction vector. The terminal sorts all the calculated similarity values ​​in descending order, and the vectors with higher similarity are ranked in the front position. By sorting in descending order, the text instruction vector closest to the initial text instruction vector can be quickly identified. After the sorting is completed, the text instruction vector with the highest similarity is selected and set as the target text instruction vector. The target text instruction vector is the confirmed best matching instruction, which indicates the closest item to the user input instruction and provides an instruction basis for subsequent control operations.

[0077] In this embodiment, through numerical similarity calculation and sorting, no additional complex logic or large-scale calculation is required, and accurate matching can be achieved with low computing resource consumption, so that the system can run efficiently in the end-side environment.

[0078] In an exemplary embodiment, each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set based on the multimodal model deployed on the server, and the same multimodal model is deployed on the terminal;

[0079] Determining the initial text instruction vector corresponding to the initial text instruction in step S204 includes: inputting the initial text instruction into a multimodal model deployed on the terminal, obtaining text feature data based on the model output result; and converting the text feature data into the initial text instruction vector.

[0080] Among them, the multimodal model is a machine learning model that can process and understand multiple data modes at the same time. The model can generate the corresponding vector expression based on the text instruction to form high-dimensional semantic features. The text feature data is the numerical data obtained after the initial text instruction is processed by the multimodal model, which contains the key semantic features of the text instruction.

[0081] Specifically, the server first inputs the text instruction set into the multimodal model for vectorization. The model extracts features from each text instruction and outputs a feature vector of the text instruction. The generated vector set is stored in the server along with the text instruction set for use when the terminal is synchronized or updated.

[0082] To ensure vector consistency, the same multimodal model as the server is deployed on the terminal to ensure that the terminal can generate the corresponding vector for the initial text instruction entered by the user locally without relying on server calculation. When the user enters the voice instruction, the terminal inputs the converted initial text instruction into the multimodal model. The model outputs the corresponding text feature data based on the text content, indicating the semantic features of the text. The terminal further converts the text feature data into the initial text instruction vector for subsequent matching with the instructions in the vector set.

[0083] In this embodiment, both the server and the terminal use the same model to generate vectors, ensuring that the vector expressions obtained for text instructions on different devices are consistent, making the matching results more reliable and improving the system's instruction recognition accuracy. At the same time, the terminal generates the initial text instruction vector locally, without the need to upload user instructions to the server, reducing the server's computing burden and network dependence, and improving the system's real-time performance and resource utilization efficiency.

[0084] In an exemplary embodiment, step S202 obtains a vector set and a text instruction set corresponding to the vector set from a server, including: determining version information corresponding to an initial instruction set and an initial vector set stored in a terminal; sending an instruction set update request carrying version information to the server, and receiving set update information returned by the server; the set update information is determined by the server based on the version information; based on the update information, the initial instruction set and the initial vector set are updated to obtain a text instruction set and a vector set.

[0085] Among them, the initial instruction set refers to the text instruction set currently stored in the terminal, and the initial instruction set can be synchronized with the update of the server to ensure that its content is consistent with the latest instructions on the server. The initial vector set is a text instruction vector set stored on the terminal, corresponding to each instruction in the initial instruction set. The vector set can also be updated when necessary to match the latest vector data of the server. In some embodiments, the initial instruction set and the initial vector set can be customized by the user or can be empty.

[0086] Specifically, the terminal first checks and determines the version information of the initial instruction set and initial vector set stored locally. The version information reflects the timestamp or version number of the data currently stored in the terminal, which is convenient for comparison with the data on the server. The terminal sends an instruction set update request containing version information to the server. The request carries the version number of the local set, and the server determines whether the instruction set and vector set of the terminal need to be updated. The server generates set update information based on the version information of the terminal and transmits the update information to the terminal. After receiving the set update information, the terminal updates the local initial instruction set and initial vector set according to the update content. After the update is completed, the terminal forms a new text instruction set and vector set for use in subsequent voice instruction matching.

[0087] In this embodiment, through the interaction of version information and set update information, the terminal can continuously maintain the latest status of the text instructions and vector sets, thereby enhancing the real-time response capability of the terminal.

[0088] In an exemplary embodiment, the initial instruction set and the initial vector set are updated according to the update information to obtain a text instruction set and a vector set, including: obtaining an incremental instruction set and an incremental vector set from a server according to the update information; and adding the obtained incremental instruction set and incremental vector set to the initial instruction set and the initial vector set, respectively, to obtain a text instruction set and a vector set.

[0089] The incremental instruction set is the instruction data increment provided by the server according to the terminal version information, and includes the text instructions newly added or updated compared with the initial instruction set. The incremental vector set is the instruction vector set corresponding to the incremental instruction set, and includes vector data corresponding to the text instructions in the incremental instruction set.

[0090] Specifically, the terminal first downloads the incremental instruction set and the incremental vector set according to the update information received from the server. The terminal adds the downloaded incremental instruction set to the local initial instruction set and the incremental vector set to the initial vector set. Through the merge operation, the terminal integrates the incremental data with the existing data to form a new text instruction set and vector set. After the merge operation is completed, the terminal generates a new text instruction set and vector set. The new set ensures that the instruction library and vector library on the terminal side are always consistent with the server, so that the latest instruction content can be recognized and matched after the user's voice instruction is input.

[0091] In this embodiment, by transmitting and merging incremental data, reloading and processing of full data is avoided, the use of system resources is optimized, the computing burden of the terminal is reduced, and the system maintains efficient operation with low resource consumption.

[0092] In an exemplary embodiment, executing a corresponding control operation based on a target text instruction includes: determining a current control scenario based on an initial text instruction, and determining a preset similarity threshold under the current control scenario; comparing the similarity corresponding to the target text instruction vector with the preset similarity threshold; the similarity corresponding to the target text instruction vector is the result of calculating the vector similarity of the initial text instruction vector and the target text instruction vector. If the similarity is lower than the preset similarity threshold, a command confirmation prompt of the target text instruction is displayed to the user, and after receiving the user's confirmation operation on the command confirmation prompt, the corresponding control operation is executed based on the target text instruction.

[0093] The preset similarity threshold is the minimum matching accuracy requirement set in a certain control scenario. The similarity threshold is used to determine whether the target text instruction is close enough to the initial text instruction to indicate whether user confirmation is required.

[0094] Specifically, after the terminal receives the initial text instruction input by the user, it determines the control scenario of the current operation according to the instruction content. According to different control scenarios, the terminal will set the corresponding preset similarity threshold to control the accuracy of the match. The terminal calculates the similarity between the target text instruction vector and the initial text instruction vector, and compares the obtained similarity value with the preset similarity threshold under the current control scenario. If the similarity between the target text instruction vector and the initial text instruction vector is lower than the preset similarity threshold. When the similarity is lower than the preset similarity threshold, the terminal determines that there is uncertainty in the matching result. Therefore, the terminal does not directly execute the instruction, but displays the content of the target text instruction to the user through the instruction confirmation prompt, asking the user whether to confirm the execution of the instruction. The terminal waits for the user's confirmation operation. If the user confirms that the instruction is correct, the corresponding operation will be executed based on the target text instruction control.

[0095] In this embodiment, by judging the similarity before the control operation, it is ensured that the instruction is directly executed in the case of high-precision matching; for low-precision matching, user confirmation is triggered, which effectively improves the accuracy of instruction execution and reduces the probability of erroneous operation.

[0096] In order to enable those skilled in the art to better understand the above steps, the embodiment of the present application is illustrated below by using an example, but it should be understood that the embodiment of the present application is not limited to this.

[0097] In an exemplary embodiment, the terminal voice command control is applied in the framework shown in FIG. 3 , the terminal includes a language recognition unit, a CLIP text encoder and an instruction vector database, and the server includes an instruction set management module.

[0098] The instruction set management module is responsible for managing the instruction set (such as adding, deleting, modifying, and checking the instruction set), managing the instruction set conversion vector and the instruction set version, and receiving the terminal's request to update the instruction set. In order to keep the vectorization result consistent with the terminal text vector, the CLIP model used by the CLIP text encoder used in the instruction set vectorization process must be consistent with the model used by the terminal.

[0099] The instruction set vector database is a database formed at the terminal when the terminal device requests the instruction set server through the network to obtain the instruction set and corresponding vector data. It is used to receive the input initial instruction vector from the CLIP encoder and obtain the target text instruction with the closest similarity through vector cosine calculation, including but not limited to the vector database, instruction set database and SQLite (Structured Query Language Lite) database. SQLite is a lightweight relational database management system that follows atomicity, consistency, isolation, and persistence, and is efficient and stable.

[0100] The speech recognition unit is responsible for the process of recognizing the user's voice command audio information in real time on the terminal and converting it into text information through the speech model.

[0101] The CLIP text encoder is used to extract text features and perform vectorization on the text instruction information received from the speech recognition unit, and use the vectorized initial instructions to retrieve target instructions in the instruction set vector database.

[0102] Specifically, according to the timing diagram shown in FIG. 4 , the language instruction control flow shown in FIG. 5 is executed.

[0103] The terminal application initiates an update request to the server, including the version information of the local instruction set, so that the server can confirm whether an update is required. If the versions are consistent, the server returns a response of "no update required", and the terminal continues to use the existing instruction set and vector data. If the versions are inconsistent (that is, the server has the latest data), the server will generate an incremental instruction set and vector data. The server returns the incremental text instruction set and the corresponding incremental vector data to the terminal, including the new instruction text and its corresponding vector representation.

[0104] After receiving the incremental data, the terminal checks whether the vector database and SQLite database have been created locally. The SQLite database is used to store the mapping relationship between the instruction text and the vector, so that the terminal can quickly find the text instruction corresponding to the target vector after completing the matching, ensuring the integrity of the data. The vector database is used to establish a vector index library to accelerate the vector matching process.

[0105] If the vector database and SQLite database do not exist yet, the terminal will create these two databases. If the vector database and SQLite database already exist, the terminal will obtain the incremental text instructions and the corresponding incremental vector instructions and put them into the terminal archive data storage. The terminal archive database is used to store the backup of all instruction sets and vector data locally, so that the terminal can still perform voice command recognition and matching when offline. The terminal transmits the vector data of the terminal archive data to the vector database, and transmits the mapping relationship between the text instructions and the vectors corresponding to the text instructions to the SQLite database.

[0106] The user inputs a voice command through the terminal device, and the terminal uses the voice recognition unit to convert the voice into text form and generate an initial text command. The generated initial text command is input into the CLIP text encoder, which uses the CLIP multimodal model to convert the text data into a semantic vector and outputs the initial text command vector.

[0107] The terminal uses the generated initial text instruction vector to search and match in the vector index library determined by the vector database to find the target text instruction closest to the user instruction. The similarity between the initial text instruction vector and each instruction vector in the vector index library is calculated by cosine similarity or other distance measurement methods, the closest target text instruction vector is screened out, and the corresponding index value is determined.

[0108] The terminal searches for the corresponding text instruction in the SQLite database according to the index value of the target text instruction vector, determines the target text instruction, and performs control operations according to the target text instruction.

[0109] In this embodiment, the voice information can be directly converted into text information by using the voice recognition unit on the terminal side without the need to rely on the server side capabilities. The terminal instruction set vector database is updated to the server side, and the instruction set vector database and instruction index are formed on the terminal side. The text information is converted into a feature vector using the terminal side CLIP text encoder, and the nearest text instruction in the instruction set is calculated using the cosine calculation algorithm, and the target text instruction is obtained through the mapping relationship. The entire process is carried out on the terminal side, which optimizes the use of system resources, enables efficient operation on terminal devices, and has high-performance matching and query capabilities.

[0110] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0111] Based on the same inventive concept, the embodiment of the present application also provides a voice command control device for implementing the voice command control method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more voice command control device embodiments provided below can refer to the limitations of the voice command control method above, and will not be repeated here.

[0112] In an exemplary embodiment, as shown in FIG6 , a voice command control device 600 is provided, comprising: a command and vector management module 610, a voice command conversion and vector generation module 620, a vector matching and command recognition module 630, and a command determination and control module 640, wherein:

[0113] The instruction and vector management module 610 is used to obtain a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set;

[0114] The voice instruction conversion and vector generation module 620 is used to receive the voice instruction input by the user to the terminal, convert the voice instruction into an initial text instruction, and determine the initial text instruction vector corresponding to the initial text instruction;

[0115] A vector matching and instruction recognition module 630 is used to match the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector;

[0116] The instruction determination and control module 640 is used to obtain a target text instruction corresponding to the target text instruction vector from the text instruction set, and perform a corresponding control operation based on the target text instruction.

[0117] In one embodiment, the vector matching and instruction recognition module 630 is also used to match the initial text instruction vector with the multiple text instruction vectors in the vector set to determine the matched target text instruction vector, including: performing vector similarity calculations on the initial text instruction vector and the multiple text instruction vectors in the vector set respectively to obtain the similarity between the initial text instruction and each text instruction vector in the multiple text instruction vectors; according to the similarity, sorting the text instruction vectors in the target vector set corresponding to the similarity in descending order, and taking the text instruction vector corresponding to the highest similarity as the target text instruction vector.

[0118] In one embodiment, the voice instruction conversion and vector generation module 620 is also used for each of the text instruction vectors in the vector set, which is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set based on the multimodal model deployed on the server, and the same multimodal model is deployed on the terminal; the initial text instruction vector corresponding to the initial text instruction includes: inputting the initial text instruction into the multimodal model deployed on the terminal, and obtaining text feature data based on the model output result; and converting the text feature data into the initial text instruction vector.

[0119] In one embodiment, the instruction and vector management module 610 is also used to obtain a vector set and a text instruction set corresponding to the vector set from the server, including: determining version information corresponding to an initial instruction set and an initial vector set stored in the terminal; sending an instruction set update request carrying the version information to the server, and receiving set update information returned by the server; the set update information is determined by the server according to the version information; and according to the update information, updating the initial instruction set and the initial vector set to obtain the text instruction set and the vector set.

[0120] In one embodiment, the instruction and vector management module 610 is also used to update the initial instruction set and the initial vector set according to the update information to obtain the text instruction set and the vector set, including: obtaining an incremental instruction set and an incremental vector set from the server according to the update information; and adding the obtained incremental instruction set and the incremental vector set to the initial instruction set and the initial vector set, respectively, to obtain the text instruction set and the vector set.

[0121] In one embodiment, the instruction determination and control module 640 is also used for executing the corresponding control operation based on the target text instruction, including: determining the current control scenario according to the initial text instruction, and determining the preset similarity threshold under the current control scenario; comparing the similarity corresponding to the target text instruction vector with the preset similarity threshold; the similarity corresponding to the target text instruction vector is the result of vector similarity calculation of the initial text instruction vector and the target text instruction vector. If the similarity is lower than the preset similarity threshold, the instruction confirmation prompt of the target text instruction is displayed to the user, and after receiving the user's confirmation operation on the instruction confirmation prompt, the corresponding control operation is executed based on the target text instruction.

[0122] Each module in the above-mentioned voice command control device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0123] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a voice command control method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0124] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0125] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0127] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0129] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0130] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0131] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A voice command control method, characterized in that: The method comprises: Acquire a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set; Receiving a voice command input by a user to a terminal, converting the voice command into an initial text command, and determining an initial text command vector corresponding to the initial text command; Matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector; A target text instruction corresponding to the target text instruction vector is obtained from the text instruction set, and a corresponding control operation is performed based on the target text instruction.

2. The method according to claim 1, characterized in that The step of matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set to determine a matched target text instruction vector includes: Performing vector similarity calculations on the initial text instruction vector and the multiple text instruction vectors of the vector set respectively, to obtain the similarity between the initial text instruction and each text instruction vector in the multiple text instruction vectors; According to the similarity, the text instruction vectors in the target vector set corresponding to the similarity are sorted in descending order, and the text instruction vector corresponding to the highest similarity is used as the target text instruction vector.

3. The method according to claim 1, characterized in that Each of the text instruction vectors in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set based on the multimodal model deployed on the server, and the same multimodal model is deployed on the terminal; The determining the initial text instruction vector corresponding to the initial text instruction comprises: Inputting the initial text instruction into the multimodal model deployed on the terminal, and obtaining text feature data based on the model output result; The text feature data is converted into an initial text instruction vector.

4. The method according to claim 1, characterized in that: The step of obtaining a vector set and a text instruction set corresponding to the vector set from a server includes: Determining version information corresponding to an initial instruction set and an initial vector set stored in the terminal; Sending an instruction set update request carrying the version information to the server, and receiving set update information returned by the server; the set update information is determined by the server according to the version information; The initial instruction set and the initial vector set are updated according to the update information to obtain the text instruction set and the vector set.

5. The method according to claim 4, characterized in that The updating of the initial instruction set and the initial vector set according to the update information to obtain the text instruction set and the vector set includes: According to the update information, obtaining an incremental instruction set and an incremental vector set from the server; The acquired incremental instruction set and incremental vector set are added to the initial instruction set and initial vector set respectively to obtain the text instruction set and the vector set.

6. The method according to claim 1, characterized in that The executing a corresponding control operation based on the target text instruction includes: Determining a current control scenario according to the initial text instruction, and determining a preset similarity threshold under the current control scenario; Compare the similarity corresponding to the target text instruction vector with the preset similarity threshold; the similarity corresponding to the target text instruction vector is the result obtained by respectively calculating the vector similarity of the initial text instruction vector and the target text instruction vector; If the similarity is lower than the preset similarity threshold, an instruction confirmation prompt of the target text instruction is displayed to the user, and after receiving the user's confirmation operation on the instruction confirmation prompt, a corresponding control operation is performed based on the target text instruction.

7. A voice command control device, characterized in that: The device comprises: An instruction and vector management module, used for acquiring a vector set and a text instruction set corresponding to the vector set from a server; each text instruction vector in the vector set is obtained by the server performing vectorization processing on the corresponding text instruction in the text instruction set; A voice instruction conversion and vector generation module, used to receive a voice instruction input by a user to the terminal, convert the voice instruction into an initial text instruction, and determine an initial text instruction vector corresponding to the initial text instruction; A vector matching and instruction recognition module, used for matching the initial text instruction vector with a plurality of the text instruction vectors in the vector set, and determining a matched target text instruction vector; The instruction determination and control module is used to obtain a target text instruction corresponding to the target text instruction vector from the text instruction set, and perform a corresponding control operation based on the target text instruction.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images