AI voice processing method and system based on intelligent device

Abstract

The invention discloses an AI voice processing method and system based on intelligent equipment, and the method comprises the steps: obtaining the voice information of a user, and extracting the intention of the user through semantic recognition and natural language understanding; user identity verification is completed in combination with the voice information and the intention, a use scene is extracted, and identity information is obtained; performing text analysis based on the intention, the identity information and the scene, generating an operation command, extracting a multi-modal characteristic parameter, and uploading the multi-modal characteristic parameter to an Internet of Things platform; the platform generates an execution instruction by utilizing AI large model reasoning; user confirmation is carried out, and an independent database is established to store multi-modal characteristic parameters; security verification information is generated and sent to the mobile device, security authentication is completed, and a cloud data access address is obtained; and accessing cloud data through the address, obtaining intelligent equipment information, establishing connection with the intelligent equipment, and executing operation. According to the method, the user identity and the use scene can be finely distinguished, so that the recognition and response accuracy of the voice system of the intelligent equipment is improved.

Description

Technical Field

[0001] The present invention relates to the field of speech processing, and in particular to an AI speech processing method based on an intelligent device. Background Art

[0002] The use of smart devices is becoming more and more popular. They are easy to use and powerful. They can automatically complete various settings according to the conditions set by the user, which is very convenient. With the development of language processing technology, more and more smart devices are equipped with language recognition functions, such as the language dial of smart watches. Speech recognition and natural language understanding technology is one of the most important technologies of AI. It enables machines to convert voice signals into corresponding text or commands through the recognition and understanding process, understand users' questions and needs, and give answers or actions. In simple terms, it allows machines to understand people's intentions and needs, and allows people to hear what the machine says, understand its meaning, and meet people's needs.

[0003] In one existing technology, language recognition and natural language understanding technology of smart devices is based on the recognition and understanding of speech. Although it can realize functions such as intelligent question and answer and human-computer dialogue, it can meet the needs of users to a certain extent. However, the use scenarios of smart devices are becoming more and more diverse, and there are also situations where multiple users use them. The existing technology only recognizes speech, which will cause the answers of smart devices to not meet the actual needs of users and have low recognition accuracy. Summary of the invention

[0004] The present invention provides an AI speech processing method and system based on intelligent devices to improve the recognition accuracy of the speech processing system.

[0005] In the first aspect, in order to solve the above technical problems, the present invention provides an AI voice processing method based on an intelligent device, comprising: The smart device obtains the voice information uploaded by the user; The smart device performs semantic recognition and natural language understanding based on the voice information to obtain the user's intention; The smart device authenticates the user and extracts the user usage scenario according to the user intention and the voice information to obtain the user identity information and the usage scenario; The smart device performs text analysis according to the user intention to obtain an operation command candidate set, removes the operation commands that do not conform to the user identity information and the usage scenario from the operation command candidate set, obtains a target operation command, and extracts multimodal feature parameters according to the target operation command; The smart device uploads the operation command and the multimodal feature parameters to the IoT platform. The IoT platform obtains the platform information and uses the AI ​​big model to infer the platform information to obtain the execution instruction: The Internet of Things platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data; The IoT platform establishes corresponding security verification information according to the execution instruction, sends it to the mobile device, performs security authentication according to the mobile device and the security verification information, and the mobile device obtains the access address of the cloud data; The mobile device accesses the cloud data according to the access address of the cloud data, obtains the device information of the corresponding smart device, establishes a connection between the mobile device and the smart device, and performs the required operation.

[0006] In an optional implementation, the smart device performs semantic recognition and natural language understanding based on the voice information to obtain the user's intention, including: The intelligent device processes the voice information into text features, analyzes and constructs a graph structure through grammatical units, and evaluates the importance of nodes using a page sorting algorithm according to the graph structure to obtain importance weights; Using natural language processing technology, the text features are mapped into vector form to obtain vector features reflecting semantic relationships; According to the importance weight and the vector feature, semantic classification is performed through a label propagation algorithm, and the user intention is obtained according to the classification result.

[0007] In an optional implementation, performing text analysis to obtain an operation command according to the user intention, the user identity information, and the usage scenario, and extracting multimodal feature parameters according to the operation command include: The intelligent device performs frame processing on the voice information, and obtains the voice features of each frame by analyzing the amplitude, frequency, and waveform of each frame of voice information; According to the voice features, combined with a preset user identity model, the user's identity is authenticated to obtain user identity information; According to the user intention, the user's usage scenario is extracted through semantic analysis.

[0008] In an optional implementation, the smart device performs text analysis to obtain an operation command according to the user intention, the user identity information, and the usage scenario, including: The smart device performs semantic analysis on the text using a natural language processing algorithm according to the user's intention to obtain an initial command; According to the initial command, the graph matching algorithm is used to match the closest operation command from the existing knowledge graph to generate a candidate set of operation commands; Eliminate the operation commands that do not conform to the user identity information and the usage scenario from the operation command candidate set to obtain a screening command candidate set; According to the screening command candidate set, an optimization process is performed using a feature learning algorithm to obtain an operation command.

[0009] In an optional implementation, the smart device uploads the operation command and the multimodal feature parameter to the IoT platform, and the IoT platform obtains platform information, and uses the AI ​​big model to infer the platform information to obtain an execution instruction, including: The smart device obtains characteristic parameters according to the operation command and the multimodal characteristic parameters based on a preset classification model, and then classifies the characteristic parameters using a classification neural network model to obtain a classification result; Extracting and matching features according to the classification results, and calculating similarity, and judging whether to upload repeatedly according to whether the similarity exceeds a preset similarity threshold; If the upload is repeated, the upload will be rejected; If there is no repeated upload, the operation command and the multimodal feature parameters are saved to the Internet of Things platform as platform information; The IoT platform uses the platform information as input, uses the AI ​​big model for reasoning, and obtains the result as the execution instruction.

[0010] In an optional implementation, the Internet of Things platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data, including: The IoT platform performs semantic understanding and analysis based on the platform information, confirms the user's identity, and establishes a historical relationship diagram of the user's interaction with the system; According to the historical relationship graph, a graph database structure is used to store relevant multimodal feature parameters to create an independent database; The independent database is queried and updated using a query language, and the sorted multimodal feature parameters are uploaded to a cloud storage to obtain cloud data.

[0011] In an optional implementation, performing security authentication according to the mobile device and the security verification information to obtain an access address of the cloud data includes: The Internet of Things platform generates a random code according to the security verification information, binds the random code to the security verification information, and obtains a verification database; Perform security verification based on the mobile device and the verification database to obtain an information verification result; The judgment is made based on the information verification result. If the verification passes, the access address of the cloud data is distributed to the mobile device. If the verification fails, the distribution is refused.

[0012] In an optional implementation, performing security verification according to the mobile device and the verification database to obtain an information verification result includes: According to the verification database, obtaining the number of verifications; If the verification times do not reach the preset verification times, the verification passes and the security verification data is obtained; If the verification times reach the preset number, the verification will be rejected and abandoned; The security verification data is compared with the random code of the verification database to obtain an information verification result.

[0013] In an optional implementation, accessing the cloud data according to the access address of the cloud data to obtain device information of the corresponding smart device includes: The mobile device obtains a calling address for connecting to an independent database according to the access address of the cloud data; By parsing the call address, determining the specific data table and field storing the smart device information in the independent database as the parsing result; The independent database is accessed according to the analysis result to obtain device information of the corresponding smart device.

[0014] In a second aspect, the present invention provides an AI voice processing method system based on an intelligent device, comprising: Voice acquisition module, used to obtain voice information uploaded by users; An intention extraction module is used to perform semantic recognition and natural language understanding based on the voice information to obtain the user's intention; An identity scenario extraction module is used for the smart device to authenticate the user and extract the user usage scenario according to the user intention and the voice information, so as to obtain the user identity information and the usage scenario; A data analysis module, configured to perform text analysis to obtain operation commands according to the user intention, the user identity information and the usage scenario, and extract multimodal feature parameters according to the operation commands; The large model inference module is used for the smart device to upload the operation command and the multimodal feature parameters to the IoT platform. The IoT platform obtains the platform information and uses the AI ​​large model to infer the platform information to obtain the execution instruction: A database establishment module, used for the Internet of Things platform to perform user confirmation according to the platform information, establish an independent database, transfer the multimodal feature parameters to the independent database, and obtain cloud data; A security authentication module, used to establish corresponding security authentication information according to the execution instruction, send it to the mobile device, perform security authentication according to the mobile device and the security authentication information, and obtain the access address of the cloud data; The operation execution module is used to access the cloud data according to the access address of the cloud data, obtain the device information of the corresponding smart device, establish a connection between the mobile device and the smart device, and execute the required operation.

[0015] In a third aspect, the present invention also provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the processor implements any one of the above-mentioned AI voice processing methods based on smart devices.

[0016] In a fourth aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned AI voice processing methods based on intelligent devices.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses an AI voice processing method based on smart devices, comprising: obtaining voice information uploaded by a user; performing semantic recognition and natural language understanding according to the voice information to obtain user intention; performing text analysis according to the user intention, and making a fine distinction between the user's identity and usage scenario to obtain an operation command, and extracting multimodal feature parameters according to the operation command; the smart device uploads the operation command and the multimodal feature parameter to an Internet of Things platform, and the Internet of Things platform obtains platform information, and uses an AI big model to infer the platform information to obtain an execution instruction: the Internet of Things platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameter to the independent database, and obtains cloud data; establishes corresponding security verification information according to the execution instruction, sends it to a mobile device, performs security authentication according to the mobile device and the security verification information, and obtains an access address of the cloud data; accesses the cloud data according to the access address of the cloud data, obtains the device information of the corresponding smart device, and establishes a connection between the mobile device and the smart device to perform the required operation.

[0018] The present invention can realize more accurate and intelligent device control and data interaction by processing the voice information uploaded by the user. In the specific implementation process, the system first extracts the user's intention through voice recognition and natural language understanding technology, and generates specific operation commands according to the intention, and then analyzes the required data. After processing, this data is uploaded to the Internet of Things platform, and reasoning and judgment are performed through the AI ​​big model to form the final execution instructions. These execution instructions will be transferred to an independent database for storage and management after the user confirms, ensuring the security and independence of the data. Then, the system generates and sends the corresponding security verification information through the mobile device, performs strict identity authentication, and ensures the access rights of the cloud data. After the user is successfully authenticated, the specific information related to the smart device can be obtained by accessing the specified cloud data, and an effective connection can be established between the mobile device and the smart device to achieve precise control and operation. The present invention can efficiently and accurately identify the user's intention, improve the convenience and intelligence level of interaction, and greatly improve the recognition accuracy of the voice processing system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of an AI voice processing method based on an intelligent device provided by the first embodiment of the present invention; Figure 2 It is a schematic diagram of the structure of an AI voice processing system based on a smart device provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Reference Figure 1 The first embodiment of the present invention provides an AI voice processing method based on an intelligent device, comprising the following steps: S11, the smart device obtains the voice information uploaded by the user; S12, the smart device performs semantic recognition and natural language understanding based on the voice information to obtain the user's intention; S13, the smart device authenticates the user and extracts the user usage scenario according to the user intention and the voice information, thereby obtaining the user identity information and the usage scenario; S14, performing text analysis to obtain an operation command according to the user intention, the user identity information and the usage scenario, and extracting multimodal feature parameters according to the operation command; S15, the smart device uploads the operation command and the multimodal feature parameters to the IoT platform, and the IoT platform obtains the platform information, and uses the AI ​​big model to infer the platform information to obtain the execution instruction: S16, the IoT platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data; S17, the IoT platform establishes corresponding security verification information according to the execution instruction, sends it to the mobile device, performs security authentication according to the mobile device and the security verification information, and obtains the access address of the cloud data; S18, the mobile device accesses the cloud data according to the access address of the cloud data, obtains the device information of the corresponding smart device, establishes a connection between the mobile device and the smart device, and performs the required operation.

[0022] In step S11, the smart device needs to obtain the voice information uploaded by the user.

[0023] It should be noted that the user's voice information is input through smart devices (smart speakers, smart phones, smart home devices, etc.). These devices have built-in voice recognition functions, which can capture the user's voice signals in real time and convert them into digital voice data. The process of acquiring voice information includes the collection of sound signals, the transmission of sound data, and the preliminary processing of voice content (such as noise reduction, echo cancellation, etc.), ensuring that the system can efficiently and accurately receive user instructions in various environments (such as noisy places). The acquisition of this voice information is crucial for subsequent operations because it contains the user's true intentions and needs. Through accurate voice data collection, the system can accurately parse the user's instructions and lay the foundation for subsequent semantic recognition and natural language understanding.

[0024] In step S12, the smart device performs semantic recognition and natural language understanding based on the voice information to obtain the user's intention.

[0025] In one implementation, performing semantic recognition and natural language understanding according to the voice information to obtain the user intention includes: The intelligent device processes the voice information into text features, analyzes and constructs a graph structure through grammatical units, and evaluates the importance of nodes using a page sorting algorithm according to the graph structure to obtain importance weights; Using natural language processing technology, the text features are mapped into vector form to obtain vector features reflecting semantic relationships; According to the importance weight and the vector feature, semantic classification is performed through a label propagation algorithm, and the user intention is obtained according to the classification result.

[0026] It should be noted that the page ranking algorithm is used to evaluate the importance of each node in the text graph structure. Its core idea is to evaluate the importance of each node based on the relationship between nodes and the strength of the link. In the semantic recognition process, the page ranking algorithm can determine which words or phrases are more critical in the user's intention by calculating the relative importance between grammatical units. For example, in the sentence "increase the temperature of the air conditioner", the algorithm will analyze the relationship between "increase" and "air conditioning" and "temperature" to evaluate the critical weight of the verb "increase", helping the system to accurately understand the user's instructions.

[0027] It is worth noting that vector features are the result of mapping text features into vector space through natural language processing technology, which is achieved through word embedding technology. Vector features can reflect the semantic relationship between words, allowing the system to better understand the semantic similarities and differences of words. When processing user voice, vector features can convert each word or phrase into a high-dimensional vector, in which words with similar meanings have similar vector representations. This vectorized representation helps capture the contextual information and semantic relationships of the sentence, allowing the system to accurately identify the user's intentions. In this way, the system can not only understand the meaning of a single word, but also understand the semantics of the word in a specific context, thereby accurately obtaining user needs.

[0028] In step S13, the smart device authenticates the user and extracts the user usage scenario according to the user intention and the voice information to obtain the user identity information and the usage scenario.

[0029] In one implementation, the smart device authenticates the user and extracts the user usage scenario according to the user intention and the voice information to obtain the user identity information and the usage scenario, including: The intelligent device performs frame processing on the voice information, and obtains the voice features of each frame by analyzing the amplitude, frequency, and waveform of each frame of voice information; According to the voice features, combined with a preset user identity model, the user's identity is authenticated to obtain user identity information; According to the user intention, the user's usage scenario is extracted through semantic analysis.

[0030] It should be noted that the reason for the framing process is that the voice signal is continuous. Framing is to divide it into multiple short time segments (frames). Each frame usually contains 20-30ms of voice data, which is about the time of a word. The framing operation is conducive to extracting the characteristics of each word spoken by the speaker, so as to obtain the user's identity more accurately. For example, if a voice signal lasts for 1 second, framing with 25ms as the frame length and 10ms as the frame shift will obtain about 75 frames of voice data.

[0031] It should be noted that the amplitude reflects the strength of the voice signal. For example, a high amplitude corresponds to a user speaking loudly, which can be used to judge the user's emotions or environmental noise. The frequency indicates the speed of the vibration of the voice signal and is related to the pitch. For example, the frequency of male voice is usually lower than that of female voice. High frequencies may correspond to crisp vowels, and low frequencies may correspond to voiced sounds. The waveform describes the shape of the voice signal and reflects the periodicity and regularity of the voice. For example, a regular waveform corresponds to a stable vowel, and an irregular waveform corresponds to a consonant or noise.

[0032] It should be noted that identity authentication is performed by combining voice features and a preset user identity model (such as a voiceprint recognition model based on deep learning). Assuming that the voiceprint model of user A has been registered, when user A speaks, his voice features are extracted and compared with the model. If the feature matching degree is higher than a threshold (such as 90%), the verification is passed and the identity information of user A is obtained.

[0033] It should be noted that semantic analysis is used to extract usage scenarios from user intent. For example, if a user says "play light music in the kitchen", the text "play light music in the kitchen" is obtained after voice recognition. Semantic analysis identifies the keywords "kitchen" and "light music", thereby extracting the usage scenario as "music playback requirements in the kitchen scenario".

[0034] In step S14, text analysis is performed to obtain operation commands based on the user intention, the user identity information and the usage scenario, and multimodal feature parameters are extracted based on the operation commands.

[0035] It should be noted that multimodal feature parameters refer to structural parameters composed of smart device identification (such as smart air conditioner ID), operation type (such as temperature adjustment), numerical parameters (such as target temperature value 25°C), operation authority information, and current environmental parameters. It can reflect the relevant characteristics of the operation instruction and facilitate the management of the operation instruction by the Internet of Things platform. Exemplarily, the operation command is a complete sentence, such as: Air conditioner 1 receives a command from a young person to adjust the indoor temperature to 20°C under the condition of room temperature of 30°C; through semantic segmentation, the required keywords are extracted from the operation command, such as: young people, room temperature of 30 degrees Celsius, adjust the indoor temperature to 20°C, and young people have full operation authority over air conditioner 1.

[0036] In one implementation, the smart device performs text analysis to obtain an operation command according to the user intention, the user identity information, and the usage scenario, including: The smart device performs semantic analysis on the text using a natural language processing algorithm according to the user's intention to obtain an initial command; According to the initial command, the graph matching algorithm is used to match the closest operation command from the existing knowledge graph to generate a candidate set of operation commands; Eliminate the operation commands that do not conform to the user identity information and the usage scenario from the operation command candidate set to obtain a screening command candidate set; According to the screening command candidate set, an optimization process is performed using a feature learning algorithm to obtain an operation command.

[0037] It should be noted that the graph matching algorithm is a technology used to find the closest operation command in the knowledge graph. The graph matching algorithm can identify the correspondence between the user's intention and the existing operation commands in the knowledge graph by calculating the similarity between the nodes and edges in the graph. In practical applications, the knowledge graph contains a large number of entities and relationships. The graph matching algorithm can quickly find the operation command that best matches the user's input intention based on these structured data. For example, when the user enters "increase the air conditioner temperature", the graph matching algorithm will match the relevant action commands in the graph based on the semantic features of "increase" and "air conditioner temperature", such as "increase temperature" or "increase air conditioner temperature".

[0038] It is worth noting that the feature learning algorithm is a data-driven method for optimizing the candidate set of operation commands. Feature learning extracts features from a large amount of historical data and trains models based on these features to evaluate the degree of match between each candidate command and user needs. In the present invention, the feature learning algorithm not only considers the semantic consistency of candidate commands, but also combines contextual information and historical behavior of users to further optimize the selection of operation commands. Feature learning can help the system understand user needs more accurately under complex language input, thereby selecting the operation command that best meets the user's intention. For example, if the user chooses to adjust the air conditioning temperature multiple times, the system will identify this preference through the feature learning model and give priority to the relevant operation commands, thereby providing more personalized and accurate services.

[0039] It is worth noting that in order to obtain the candidate set of filtered commands, the candidate set of operation commands must first be filtered according to the user identity information and usage scenarios. For example, suppose the user is an elderly person, and his usage scenario is to use a smart device to play music at home. The candidate set of operation commands may contain a variety of commands, such as "play music", "query stock information", "set alarm", etc. Since the elderly are generally not interested in stock information, and in the current music playing scenario, the "query stock information" command does not match the user identity and usage scenario, so it is eliminated. At the same time, although "set alarm" is related to the user identity, it does not meet the user's immediate needs in the current music playing scenario, so it will also be eliminated. Finally, by eliminating the operation commands that do not meet the user's identity information and usage scenarios, the filtered candidate set of commands is obtained, such as "play music", "pause playback", "adjust volume", etc. These commands are more in line with the actual needs of the elderly in the music playing scenario.

[0040] In step S15, the smart device uploads the operation command and the multimodal feature parameters to the Internet of Things platform. The Internet of Things platform obtains the platform information, uses the AI ​​big model to infer the platform information, and obtains the execution instruction.

[0041] In one implementation, the smart device uploads the operation command and the multimodal feature parameter to the IoT platform, and the IoT platform obtains platform information, and uses the AI ​​big model to infer the platform information to obtain execution instructions, including: The smart device obtains characteristic parameters according to the operation command and the multimodal characteristic parameters based on a preset classification model, and then classifies the characteristic parameters using a classification neural network model to obtain a classification result; Extracting and matching features according to the classification results, and calculating similarity, and judging whether to upload repeatedly according to whether the similarity exceeds a preset similarity threshold; If the upload is repeated, the upload will be rejected; If there is no repeated upload, the operation command and the multimodal feature parameters are saved to the Internet of Things platform as platform information; The IoT platform uses the platform information as input, uses the AI ​​big model for reasoning, and obtains the result as the execution instruction.

[0042] It should be noted that the platform information refers to a combination of the operation command and the multimodal feature parameters.

[0043] It should be noted that the preset classification model is based on historical data and previously defined features, and can extract key feature parameters from uploaded operation commands and multimodal feature parameters. These feature parameters represent the specific details of the user's intention, such as target device, operation type, data range, etc. Through the preset classification model, the system can ensure appropriate feature extraction for different types of commands and data, so as to provide effective input for subsequent reasoning steps.

[0044] It is worth mentioning that the classification neural network model is used to further classify the extracted feature parameters. The neural network model can handle complex nonlinear relationships and improve the accuracy of classification through training and optimization. In this process, the model will identify which features are the most discriminative in a given command and classify them into different operation categories. Next, the system will perform feature extraction and feature matching based on the classification results, and calculate the similarity between different operation commands and existing operations. Through this similarity calculation, the system can determine whether the current operation is repeated with the previously uploaded command. If the similarity exceeds the preset threshold, the system will refuse to upload repeatedly to avoid data redundancy or conflict; conversely, if there is no repetition, the system will execute the upload and save the operation command and multimodal feature parameters to the IoT platform as platform information for subsequent processing.

[0045] It should be noted that the AI ​​big model refers to an artificial intelligence model that has been trained with large-scale data and optimized through deep learning. It has strong reasoning capabilities and can identify potential rules and patterns from complex data. In step S15, the AI ​​big model analyzes the information on the IoT platform and performs intelligent reasoning on various environmental factors and user input to generate execution instructions. Unlike traditional models, the AI ​​big model contains billions or even more parameters, can handle more complex inputs, and has strong generalization and self-learning capabilities.

[0046] It is worth noting that the reasoning process of the AI ​​big model can combine multiple data sources and contextual information to help the system make more accurate decisions in different situations. For example, the AI ​​big model will not only analyze the operation commands and multimodal feature parameters uploaded by the user, but also infer the execution instructions that are most suitable for the current situation based on the current status of the device, historical operation records, and data from other IoT devices. Through such a reasoning process, the AI ​​big model can significantly improve the intelligence of the system, making the execution instructions more in line with user needs and able to adapt to complex and changing environmental conditions. In addition, the introduction of the AI ​​big model can also support continuous self-optimization and updating, constantly acquire knowledge from new data, and improve the overall performance and adaptability of the system.

[0047] In step S16, the IoT platform performs user confirmation based on the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data.

[0048] In one implementation, the Internet of Things platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data, including: The IoT platform performs semantic understanding and analysis based on the platform information, confirms the user's identity, and establishes a historical relationship diagram of the user's interaction with the system; According to the historical relationship graph, a graph database structure is used to store relevant multimodal feature parameters to create an independent database; The independent database is queried and updated using a query language, and the sorted multimodal feature parameters are uploaded to a cloud storage to obtain cloud data.

[0049] It should be noted that the process of confirming user identity involves in-depth mining of user identity information, including comprehensive comparison of devices, account information and historical interaction data in the platform to ensure that the current operation is initiated by an authorized user. This identity confirmation not only relies on traditional usernames and passwords, but also enhances security through device fingerprints to prevent unauthorized access.

[0050] It should be noted that the establishment of a historical relationship graph of user-system interactions provides a basis for subsequent data management and intelligent decision-making. This historical relationship graph not only records the user's operation history, but also reflects the interaction patterns, preferences, and habits between the user and the system. The system uses a graph database structure to store this relationship information. The graph database can efficiently store and query complex relationship data and support fast graph traversal and pattern matching. Through this structure, the system can accurately infer user needs in subsequent operations, improving response efficiency and operation accuracy.

[0051] It is worth noting that cloud storage can not only ensure the persistence and reliability of data, but also provide efficient data support for subsequent intelligent reasoning and analysis. Through this process, the user's multimodal feature parameters are managed efficiently and securely, and can be seamlessly accessed between different devices and systems, improving the intelligence and user experience of the entire IoT system.

[0052] In step S17, corresponding security verification information is established according to the execution instruction and sent to the mobile device, and security authentication is performed based on the mobile device and the security verification information to obtain the access address of the cloud data.

[0053] In one implementation, performing security authentication according to the mobile device and the security verification information to obtain an access address for cloud data includes: The Internet of Things platform generates a random code according to the security verification information, binds the random code to the security verification information, and obtains a verification database; Perform security verification based on the mobile device and the verification database to obtain an information verification result; The judgment is made based on the information verification result. If the verification passes, the access address of the cloud data is distributed to the mobile device. If the verification fails, the distribution is refused.

[0054] In one implementation, performing security verification according to the mobile device and the verification database to obtain an information verification result includes: According to the verification database, obtaining the number of verifications; If the verification times do not reach the preset verification times, the verification passes and the security verification data is obtained; If the verification times reach the preset number, the verification will be rejected and abandoned; The security verification data is compared with the random code of the verification database to obtain an information verification result.

[0055] It should be noted that the information verification result is obtained by comparing the security verification data with the random code in the verification database. This process is mainly based on the principle of information security verification, which confirms the user's identity and prevents malicious attacks by ensuring the uniqueness and timeliness of the verification information. The information verification principle is essentially a hash algorithm and encryption technology. During the verification process, the random code and security verification information will be hashed to generate an irreversible verification value. The verification value will be stored in the verification database, and when the user attempts to verify, the system recalculates the verification value through the same hash algorithm and compares it with the value in the database to ensure the integrity and security of the verification process. In addition, the system may also use encryption technology to ensure that the data during transmission is not tampered with, thereby ensuring the reliability of information verification.

[0056] It should be noted that the purpose of generating a random code and binding it to security verification information is to increase the security of the verification process. The generated random code changes during each authentication process, which prevents the reused security information from being intercepted and abused by attackers. The random code is combined with the security verification information and stored in the verification database to ensure that each security authentication has unique and unpredictable verification information, thereby effectively preventing potential security risks.

[0057] It is worth noting that when performing security verification based on the mobile device and the verification database, the system will first obtain the number of verifications to ensure the rationality of the verification process and prevent brute force cracking. By setting a preset number of verifications, the system can allow users to perform security verification within a certain number of times. If the verification fails for the set number of times, further verification will be automatically abandoned to prevent malicious attackers from continuously trying to obtain access rights. If the number of verifications does not reach the preset number, the system will continue to perform security verification and generate the final information verification result by comparing the random code in the verification database with the security verification data.

[0058] In step S18, the cloud data is accessed according to the access address of the cloud data, the device information of the corresponding smart device is obtained, and a connection is established between the mobile device and the smart device to perform the required operation.

[0059] In one implementation, accessing the cloud data according to the access address of the cloud data of the mobile device to obtain the device information of the corresponding smart device includes: The mobile device obtains a calling address for connecting to an independent database according to the access address of the cloud data; By parsing the call address, determining the specific data table and field storing the smart device information in the independent database as the parsing result; The independent database is accessed according to the analysis result to obtain device information of the corresponding smart device.

[0060] It should be noted that the call address connecting to the independent database is the interface between the cloud storage system and the independent database, through which the system can interact with the database storing the smart device information. The call address is an identifier pointing to a specific location in the database, which can ensure that the system accurately accesses the required database.

[0061] It is worth noting that in the process of parsing the call address, the system will identify the specific data tables and fields in the independent database that store the smart device information. The detailed information of each smart device (such as device model, status, configuration parameters, etc.) is stored in specific tables and fields in the database. The purpose of parsing the call address is to accurately locate this information. The parsing results provide a comprehensive understanding of the database structure, allowing the system to accurately retrieve and access relevant data. After obtaining the parsing results, the system will initiate a database access request based on this information and successfully extract detailed device information related to the smart device from the independent database.

[0062] The present invention discloses an AI voice processing method based on smart devices, comprising: obtaining voice information uploaded by a user; performing semantic recognition and natural language understanding according to the voice information to obtain user intention; performing text analysis according to the user intention to obtain an operation command, and extracting multimodal feature parameters according to the operation command; the smart device uploads the operation command and the multimodal feature parameters to an Internet of Things platform, the Internet of Things platform obtains platform information, and uses an AI big model to infer the platform information to obtain an execution instruction: the Internet of Things platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data; establishes corresponding security verification information according to the execution instruction, sends it to a mobile device, performs security authentication according to the mobile device and the security verification information, and obtains an access address of the cloud data; accesses the cloud data according to the access address of the cloud data, obtains device information of the corresponding smart device, and establishes a connection between the mobile device and the smart device to perform the required operation.

[0063] The present invention can realize more accurate and intelligent device control and data interaction by processing the voice information uploaded by the user. In the specific implementation process, the system first extracts the user's intention through voice recognition and natural language understanding technology, and generates specific operation commands according to the intention, and then analyzes the required data. After processing, this data is uploaded to the Internet of Things platform, and reasoning and judgment are performed through the AI ​​big model to form the final execution instructions. These execution instructions will be transferred to an independent database for storage and management after the user confirms, ensuring the security and independence of the data. Then, the system generates and sends the corresponding security verification information through the mobile device, performs strict identity authentication, and ensures the access rights of the cloud data. After the user is successfully authenticated, the specific information related to the smart device can be obtained by accessing the specified cloud data, and an effective connection can be established between the mobile device and the smart device to achieve precise control and operation. The present invention can efficiently and accurately identify the user's intention, improve the convenience and intelligence level of interaction, and greatly improve the recognition accuracy of the voice processing system.

[0064] Reference Figure 2 The second embodiment of the present invention provides an AI speech processing system based on an intelligent device, comprising: Voice acquisition module, used to obtain voice information uploaded by users; An intention extraction module is used to perform semantic recognition and natural language understanding based on the voice information to obtain the user's intention; An identity scenario extraction module is used for the smart device to authenticate the user and extract the user usage scenario according to the user intention and the voice information, so as to obtain the user identity information and the usage scenario; A data analysis module, configured to perform text analysis to obtain operation commands according to the user intention, the user identity information and the usage scenario, and extract multimodal feature parameters according to the operation commands; The large model inference module is used for the smart device to upload the operation command and the multimodal feature parameters to the IoT platform. The IoT platform obtains the platform information and uses the AI ​​large model to infer the platform information to obtain the execution instruction: A database establishment module, used for the Internet of Things platform to perform user confirmation according to the platform information, establish an independent database, transfer the multimodal feature parameters to the independent database, and obtain cloud data; A security authentication module, used to establish corresponding security authentication information according to the execution instruction, send it to the mobile device, perform security authentication according to the mobile device and the security authentication information, and obtain the access address of the cloud data; The operation execution module is used to access the cloud data according to the access address of the cloud data, obtain the device information of the corresponding smart device, establish a connection between the mobile device and the smart device, and execute the required operation.

[0065] It should be noted that the AI ​​voice processing system based on smart devices provided in an embodiment of the present invention is used to execute all the process steps of the AI ​​voice processing method based on smart devices in the above embodiment. The working principles and beneficial effects of the two correspond one to one, so they will not be repeated here.

[0066] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a voice acquisition program. When the processor executes the computer program, the steps in the above-mentioned AI voice processing method embodiment based on a smart device are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned system embodiments are realized, such as the voice acquisition module.

[0067] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0068] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0069] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0070] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0071] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0072] It should be noted that the system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the system embodiment provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0073] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. An AI voice processing method based on intelligent devices, characterized in that: include: The smart device obtains the voice information uploaded by the user; The smart device performs semantic recognition and natural language understanding based on the voice information to obtain the user's intention; The smart device authenticates the user and extracts the user usage scenario according to the user intention and the voice information to obtain the user identity information and the usage scenario; The smart device performs text analysis according to the user intention to obtain an operation command candidate set, removes the operation commands that do not conform to the user identity information and the usage scenario from the operation command candidate set, obtains a target operation command, and extracts multimodal feature parameters according to the target operation command; The smart device uploads the operation command and the multimodal feature parameters to the IoT platform. The IoT platform obtains the platform information and uses the AI ​​big model to infer the platform information to obtain the execution instruction: The Internet of Things platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data; The IoT platform establishes corresponding security verification information according to the execution instruction, sends it to the mobile device, performs security authentication according to the mobile device and the security verification information, and the mobile device obtains the access address of the cloud data; The mobile device accesses the cloud data according to the access address of the cloud data, obtains the device information of the corresponding smart device, establishes a connection between the mobile device and the smart device, and performs the required operation.

2. The AI ​​voice processing method based on intelligent device according to claim 1, characterized in that: The smart device performs semantic recognition and natural language understanding according to the voice information to obtain the user's intention, including: The intelligent device processes the voice information into text features, analyzes and constructs a graph structure through grammatical units, and evaluates the importance of nodes using a page sorting algorithm according to the graph structure to obtain importance weights; Using natural language processing technology, the text features are mapped into vector form to obtain vector features reflecting semantic relationships; According to the importance weight and the vector feature, semantic classification is performed through a label propagation algorithm, and the user intention is obtained according to the classification result.

3. The AI ​​voice processing method based on intelligent device according to claim 1, characterized in that: The smart device authenticates the user and extracts the user usage scenario according to the user intention and the voice information to obtain the user identity information and the usage scenario, including: The intelligent device performs frame processing on the voice information, and obtains the voice features of each frame by analyzing the amplitude, frequency, and waveform of each frame of voice information; According to the voice features, combined with a preset user identity model, the user's identity is authenticated to obtain user identity information; According to the user intention, the user's usage scenario is extracted through semantic analysis.

4. The AI ​​voice processing method based on intelligent device according to claim 1, characterized in that: The smart device performs text analysis according to the user intention to obtain an operation command candidate set, removes the operation commands in the operation command candidate set that do not conform to the user identity information and the usage scenario, and obtains the target operation command, including: The smart device performs semantic analysis on the text using a natural language processing algorithm according to the user's intention to obtain an initial command; According to the initial command, the graph matching algorithm is used to match the closest operation command from the existing knowledge graph to generate a candidate set of operation commands; Eliminate the operation commands that do not conform to the user identity information and the usage scenario from the operation command candidate set to obtain a screening command candidate set; According to the screening command candidate set, an optimization process is performed using a feature learning algorithm to obtain an operation command.

5. The AI ​​voice processing method based on intelligent device according to claim 1, characterized in that: The smart device uploads the operation command and the multimodal feature parameters to the IoT platform, and the IoT platform obtains platform information, and uses the AI ​​big model to infer the platform information to obtain execution instructions, including: The smart device obtains characteristic parameters according to the operation command and the multimodal characteristic parameters based on a preset classification model, and then classifies the characteristic parameters using a classification neural network model to obtain a classification result; Extracting and matching features according to the classification results, and calculating similarity, and judging whether to upload repeatedly according to whether the similarity exceeds a preset similarity threshold; If the upload is repeated, the upload will be rejected; If there is no repeated upload, the operation command and the multimodal feature parameters are saved to the Internet of Things platform as platform information; The IoT platform uses the platform information as input, uses the AI ​​big model for reasoning, and obtains the result as the execution instruction.

6. The AI ​​voice processing method based on intelligent device according to claim 1, characterized in that: The Internet of Things platform performs user confirmation according to the platform information, establishes an independent database, transfers the multimodal feature parameters to the independent database, and obtains cloud data, including: The IoT platform performs semantic understanding and analysis based on the platform information, confirms the user's identity, and establishes a historical relationship diagram of the user's interaction with the system; According to the historical relationship graph, a graph database structure is used to store relevant multimodal feature parameters to create an independent database; The independent database is queried and updated using a query language, and the sorted multimodal feature parameters are uploaded to a cloud storage to obtain cloud data.

7. The AI ​​voice processing method based on intelligent device according to claim 1, characterized in that: The performing security authentication according to the mobile device and the security verification information to obtain the access address of the cloud data includes: The Internet of Things platform generates a random code according to the security verification information, binds the random code to the security verification information, and obtains a verification database; Perform security verification based on the mobile device and the verification database to obtain an information verification result; The judgment is made based on the information verification result. If the verification passes, the access address of the cloud data is distributed to the mobile device. If the verification fails, the distribution is refused.

8. The AI ​​voice processing method based on intelligent device according to claim 7, characterized in that: The performing security verification according to the mobile device and the verification database to obtain an information verification result includes: According to the verification database, obtaining the number of verifications; If the verification times do not reach the preset verification times, the verification passes and the security verification data is obtained; If the verification times reach the preset number, the verification will be rejected and abandoned; The security verification data is compared with the random code of the verification database to obtain an information verification result.

9. The AI ​​voice processing method based on intelligent device according to claim 1, characterized in that: The mobile device accesses the cloud data according to the access address of the cloud data, and obtains device information of the corresponding smart device, including: The mobile device obtains a calling address for connecting to an independent database according to the access address of the cloud data; By parsing the call address, determining the specific data table and field storing the smart device information in the independent database as the parsing result; The independent database is accessed according to the analysis result to obtain device information of the corresponding smart device.

10. An AI voice processing system based on smart devices, characterized in that: include: Voice acquisition module, used to obtain voice information uploaded by users; An intention extraction module is used to perform semantic recognition and natural language understanding based on the voice information to obtain the user's intention; An identity scenario extraction module is used for the smart device to authenticate the user and extract the user usage scenario according to the user intention and the voice information, so as to obtain the user identity information and the usage scenario; A data analysis module, configured to perform text analysis to obtain operation commands according to the user intention, the user identity information and the usage scenario, and extract multimodal feature parameters according to the operation commands; The large model inference module is used for the smart device to upload the operation command and the multimodal feature parameters to the IoT platform. The IoT platform obtains the platform information and uses the AI ​​large model to infer the platform information to obtain the execution instruction: A database establishment module, used for the Internet of Things platform to perform user confirmation according to the platform information, establish an independent database, transfer the multimodal feature parameters to the independent database, and obtain cloud data; A security authentication module, used to establish corresponding security authentication information according to the execution instruction, send it to the mobile device, perform security authentication according to the mobile device and the security authentication information, and obtain the access address of the cloud data; The operation execution module is used to access the cloud data according to the access address of the cloud data, obtain the device information of the corresponding smart device, establish a connection between the mobile device and the smart device, and execute the required operation.

Images