Intention recognition method, device, equipment and medium for medical robots

Abstract

The present invention is applicable to the field of medical technology, and in particular relates to an intention recognition method, apparatus, device, and medium for medical robots. The present invention uses an intention recognition model to identify the initial action, initial location, and initial executor in audio features and obtain an initial instruction. The reference instruction most similar to the initial instruction is used as the target instruction, and the location information of the target action, target location, and target executor is predicted respectively. The target action, target location, target executor, and their location information are semantically reconstructed using an audio reconstruction model to obtain reconstructed audio. The intention recognition model is used to perform intention recognition on the reconstructed audio features to obtain a target intention recognition result. The reconstructed audio that eliminates the personalized information of the target patient is obtained through reconstruction and used as the object of intention recognition, thereby improving the accuracy of the intention recognition result, more accurately guiding the medical robot to make corresponding feedback, and improving the medical service quality of the medical robot.

Description

Technical Field

[0001] The present invention is applicable to the field of medical technology, and in particular relates to an intention recognition method, device, equipment and medium for a medical robot. Background Art

[0002] Intent recognition uses machine learning to analyze and identify the intent of messages sent by users to devices. By understanding various user commands, it can facilitate user operations. With the development of artificial intelligence, intent recognition technology is finding increasingly widespread applications in finance, healthcare, education, and other fields. For example, in the healthcare field, intent recognition technology can be applied to medical robots. Using the voice messages sent by users to medical robots, machine learning-related technologies such as semantic analysis can identify and understand the intent of the user's voice messages and provide corresponding feedback based on the intent recognition results. For example, feedback such as picking up medication or ringing a bell can provide corresponding assistance to the user.

[0003] Existing speech intent recognition methods typically perform semantic extraction after converting speech to text, and then perform intent recognition based on this semantic information. This increases the likelihood of errors in speech-to-text conversion and reduces accuracy. Furthermore, due to the diverse speaking habits and vocabulary choices of users, as well as the personalized nature of their audio information, existing intent recognition methods use different key points when identifying personalized speech information. This makes it difficult to identify and analyze key information related to intent, resulting in low accuracy in intent recognition results and making it difficult for medical robots to meet user needs.

[0004] Therefore, in the medical robot scenario in the field of medical technology, how to improve the accuracy of intent recognition results has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an intention recognition method, apparatus, device, and medium for a medical robot to solve the problem of low accuracy of existing intention recognition methods when performing intention recognition on personalized voice information.

[0006] In a first aspect, an embodiment of the present invention provides an intention recognition method for a medical robot, the intention recognition method comprising:

[0007] Acquiring audio information and audio features of a target patient, performing intent recognition on the audio features using an intent recognition model to obtain an initial action, an initial location, and an initial performer, and obtaining an initial instruction based on the initial action, the initial location, and the initial performer;

[0008] Calculating similarity between the initial instruction and each reference instruction in a preset reference instruction set to obtain similarity values ​​corresponding to the reference instructions, and determining the reference instruction with the largest similarity value as the target instruction, wherein the target instruction includes a target action, a target location, and a target executor;

[0009] Using a position prediction model to predict the positions of the initial action, the initial location, and the initial performer in the audio information, respectively, to obtain the corresponding position information of the initial action, the initial location, and the initial performer in the audio information, respectively, using the position information of the initial action as the position information of the target action, using the position information of the initial location as the position information of the target location, and using the position information of the initial performer as the position information of the target performer;

[0010] reconstructing the target action, the target location, the target performer, and their corresponding position information using an audio reconstruction model to obtain reconstructed audio;

[0011] Extracting reconstructed audio features of the reconstructed audio, performing intent recognition on the reconstructed audio features using the intent recognition model, and obtaining a target intent recognition result.

[0012] In a second aspect, an embodiment of the present invention provides an intention recognition device for a medical robot, the intention recognition device comprising:

[0013] An initial instruction recognition module is used to obtain audio information and audio features of a target patient, perform intent recognition on the audio features using an intent recognition model, obtain an initial action, an initial location, and an initial performer, and obtain an initial instruction based on the initial action, the initial location, and the initial performer;

[0014] a target instruction determination module, configured to calculate similarity between the initial instruction and each reference instruction in a preset reference instruction set, obtain similarity values ​​corresponding to the reference instructions, and determine the reference instruction with the largest similarity value as the target instruction, wherein the target instruction includes a target action, a target location, and a target executor;

[0015] a position information prediction module, configured to use a position prediction model to predict the positions of the initial action, the initial location, and the initial performer in the audio information, respectively, to obtain the corresponding position information of the initial action, the initial location, and the initial performer in the audio information, respectively, and use the position information of the initial action as the position information of the target action, the position information of the initial location as the position information of the target location, and the position information of the initial performer as the position information of the target performer;

[0016] An audio reconstruction module, configured to reconstruct the target action, the target location, the target performer, and their corresponding position information using an audio reconstruction model to obtain reconstructed audio;

[0017] The target intention recognition module is used to extract the reconstructed audio features of the reconstructed audio, use the intention recognition model to perform intention recognition on the reconstructed audio features, and obtain a target intention recognition result.

[0018] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the intention recognition method as described in the first aspect when executing the computer program.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the intention recognition method as described in the first aspect is implemented.

[0020] The beneficial effects of the embodiments of the present invention compared with the prior art are: by obtaining the audio information and audio features of the target patient, using the intention recognition model to perform intent recognition on the audio features, obtaining the initial action, initial location and initial executor, and obtaining the initial instruction based on the initial action, initial location and initial executor, respectively, performing similarity calculations on the initial instruction and each reference instruction in the preset reference instruction set, obtaining the similarity value of the corresponding reference instruction, determining the reference instruction with the largest similarity value as the target instruction, the target instruction including the target action, target location and target executor, using the position prediction model to predict the positions of the initial action, initial location and initial executor in the audio information, respectively, obtaining the corresponding position information of the initial action, initial location and initial executor in the audio information, and using the position information of the initial action as the position of the target action Information, the location information of the initial location is used as the location information of the target location, the location information of the initial executor is used as the location information of the target executor, and the target action, target location and target executor and their corresponding location information are reconstructed using an audio reconstruction model to obtain reconstructed audio, and the reconstructed audio features of the reconstructed audio are extracted. The intention recognition model is used to perform intent recognition on the reconstructed audio features to obtain a target intention recognition result. By determining the reference instruction that is most similar to the initial instruction as the target instruction, and the location information of the target action, target location and target executor in the target instruction in the audio information, the reconstructed audio that eliminates the personalized information of the target patient is obtained as the object of intent recognition, thereby improving the accuracy of the intention recognition result, thereby more accurately guiding the medical robot to make corresponding feedback, and improving the medical service quality of the medical robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a schematic diagram of an application environment of an intention recognition method for a medical robot provided in the first embodiment of the present invention;

[0023] Figure 2 This is a flow chart of a method for identifying intentions of a medical robot provided in the first embodiment of the present invention;

[0024] Figure 3 This is a schematic structural diagram of an intention recognition device for a medical robot provided in a second embodiment of the present invention;

[0025] Figure 4 This is a structural diagram of a computer device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0026] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0027] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0028] It will also be understood that the term "and / or" used in the present description and appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0029] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0030] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] Embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0033] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0034] It should be understood that the order of execution of the steps in the following embodiments does not necessarily mean the order in which they are executed. The order in which each process is executed should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0035] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0036] The first embodiment of the present invention provides an intention recognition method for a medical robot, which can be applied in the following situations: Figure 1 In an application environment, a client communicates with a server. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0037] See also Figure 2 , is a flow chart of a method for intention recognition of a medical robot provided by the first embodiment of the present invention. The above-mentioned intention recognition method can be applied to Figure 1 In the client, the intent recognition method may include the following steps:

[0038] Step S201: obtain the audio information and audio features of the target patient, use the intention recognition model to perform intention recognition on the audio features, obtain the initial action, initial location and initial executor, and obtain the initial instruction based on the initial action, initial location and initial executor.

[0039] The intent recognition model is used to analyze and identify the intent of input content, obtain corresponding intent recognition results, and provide corresponding assistance to users. It has a wide range of applications in finance, healthcare, education, and other fields. For example, in the medical field, the intent recognition model can be applied to medical robots. Through the voice information sent by the user to the medical robot, the robot uses machine learning technologies such as semantic analysis to identify and understand the intention of the target patient's audio information. Based on the intent recognition results, the medical robot can provide corresponding feedback, such as providing feedback such as taking medicine or ringing a bell to call, to provide corresponding assistance to the target patient.

[0040] The audio information of the target patient is the speech of the target patient when expressing his / her intention. The audio features are features extracted from the audio information. In this embodiment, the intention recognition model is used to analyze and recognize the audio features to obtain the initial action, initial location, and initial executor. The initial action can represent the intended action of the target patient, such as taking medicine; the initial location can represent the intended location of the target patient, such as a pharmacy; the initial executor can represent the intended object of the target patient, such as Xiao A; the initial instruction is formed based on the initial action, initial location, and initial executor. The intended action, intended location, and intended object of the target patient can be represented based on the initial instruction to guide the medical robot to make corresponding feedback, that is, to guide Xiao A to take medicine from the pharmacy to meet the intention of the target patient.

[0041] Due to patients' diverse speaking habits and vocabulary, their audio information is personalized. Different patients may express the same intent in different ways. For example, different patients may refer to the same robot as "Xiao A," "Xiao a," "Robot A," and so on. They may also refer to the pharmacy as "pharmacy," "Room 103," "10th Floor," and so on. The intent "Xiao A picks up medicine from the pharmacy" can also be expressed as "Xiao a picks up medicine from the pharmacy," "Xiao a picks up medicine from Room 103," "Robot A picks up medicine from the 10th Floor," and so on. As a result, existing intent recognition methods use different key points when recognizing personalized speech, making it difficult to identify and analyze key information related to the intent, resulting in lower accuracy.

[0042] Therefore, this embodiment takes the initial action, initial location and initial executor as the initial intention recognition result, and uses the initial intention recognition result as the basis for correcting the audio information and intention recognition model, which can improve the accuracy of the intention recognition result to meet the intention needs of the target patient.

[0043] Optionally, audio information and audio features of the target patient are obtained, including:

[0044] Acquire audio information of a target patient, segment the audio information using a preset window, and obtain at least one segmented frame;

[0045] The phoneme features of each segmentation frame are extracted respectively, and the phoneme features corresponding to all segmentation frames are spliced ​​according to the arrangement order of each segmentation frame in the audio information to obtain the audio features.

[0046] Among them, when extracting features from audio information, the audio information is first segmented using a preset window to obtain at least one segmented frame, and the phoneme features of each segmented frame are extracted separately to represent the semantic feature information of the corresponding segmented frame. Then, according to the arrangement order of each segmented frame in the audio information, the phoneme features corresponding to all segmented frames are spliced ​​to obtain audio features to represent the semantic feature information of the audio information.

[0047] Optionally, extracting the phoneme features of each segmented frame includes:

[0048] Extract the Mel cepstral coefficients of each segmented frame respectively;

[0049] The speech recognition model is used to extract the Mel-frequency cepstral coefficients to obtain the phoneme features of the corresponding segmentation frames.

[0050] The frequency band division of the Mel-frequency cepstral coefficients is equidistant on the Mel-frequency scale. This better approximates the human auditory system than the linearly spaced frequency bands used in normal logarithmic cepstral spectra and has wide applications in fields such as speech recognition. Therefore, this embodiment extracts the Mel-frequency cepstral coefficients for each segmented frame and uses a speech recognition model to perform feature extraction on the Mel-frequency cepstral coefficients to obtain the phoneme features of the corresponding segmented frame to represent the semantic feature information of the corresponding segmented frame.

[0051] Optionally, an intent recognition model is used to perform intent recognition on the audio features to obtain the initial action, initial location, and initial performer, including:

[0052] Performing intent recognition on the audio features using an intent recognition model to obtain a first recognition probability corresponding to at least one preset action, a second recognition probability corresponding to at least one preset location, and a third recognition probability corresponding to at least one preset executor;

[0053] The preset action corresponding to the maximum value of the first recognition probability is determined as the initial action, the preset location corresponding to the maximum value of the second recognition probability is determined as the initial location, and the preset executor corresponding to the maximum value of the third recognition probability is determined as the initial executor.

[0054] Among them, the preset action, preset location and preset executor are pre-set according to actual conditions. The intention recognition model performs intent recognition on audio features to obtain a first recognition probability corresponding to the preset action, a second recognition probability corresponding to the preset location and a third recognition probability corresponding to the preset executor. Among them, the first recognition probability can represent the probability that the corresponding preset action is the initial action, the second recognition probability can represent the probability that the corresponding preset location is the initial location, and the third recognition probability can represent the probability that the corresponding preset executor is the initial executor.

[0055] Therefore, the preset action corresponding to the maximum value of the first recognition probability is determined as the initial action, the preset location corresponding to the maximum value of the second recognition probability is determined as the initial location, and the preset executor corresponding to the maximum value of the third recognition probability is determined as the initial executor. The initial action, initial location and initial executor can be obtained as the initial intention recognition result, and serve as the basis for correcting the audio information and intention recognition model.

[0056] The above-mentioned method obtains the audio information and audio features of the target patient, uses the intention recognition model to perform intention recognition on the audio features, obtains the initial action, initial location and initial executor, and obtains the steps of the initial instruction based on the initial action, initial location and initial executor. The initial action, initial location and initial executor are obtained as the initial intention recognition result, and serve as the basis for correcting the audio information and intention recognition model, which can improve the accuracy of the intention recognition result to meet the intention needs of the target patient.

[0057] In step S202, similarity calculation is performed on the initial instruction and each reference instruction in the preset reference instruction set to obtain similarity values ​​of the corresponding reference instructions, and the reference instruction with the largest similarity value is determined as the target instruction, which includes a target action, a target location, and a target executor.

[0058] Among them, the reference instruction set includes several pre-set reference instructions, each reference instruction includes a corresponding reference action, reference location and reference executor, each reference instruction corresponds to an intention, and the reference action, reference location and reference executor in each reference instruction can be regarded as the official instruction corresponding to the intention, which is the basis for guiding the medical robot to provide intention feedback.

[0059] For the personalized initial instruction, the similarity between the initial instruction and each reference instruction in the preset reference instruction set can be calculated respectively, and the reference instruction most similar to the initial instruction can be determined as the target instruction. Correspondingly, the target instruction includes the target action, target location and target executor, which serves as the basis for guiding the medical robot to provide intention feedback, so as to reduce the impact of personalized voice information on the intention recognition results, thereby improving the accuracy of the intention recognition results.

[0060] Optionally, the reference instruction includes a reference action, a reference location, and a reference executor;

[0061] The initial instruction is respectively similar to each reference instruction in the preset reference instruction set to obtain a similarity value of the corresponding reference instruction, including:

[0062] For any reference instruction, calculating the similarity between the reference action and the initial action of the reference instruction to obtain a first similarity, calculating the similarity between the reference location and the initial location of the reference instruction to obtain a second similarity, and calculating the similarity between the reference executor and the initial executor of the reference instruction to obtain a third similarity;

[0063] Multiplying the first similarity by the first recognition probability corresponding to the initial action to obtain a first multiplication result, multiplying the second similarity by the second recognition probability corresponding to the initial location to obtain a second multiplication result, and multiplying the third similarity by the third recognition probability corresponding to the initial performer to obtain a third multiplication result;

[0064] The first multiplication result, the second multiplication result, and the third multiplication result are added to obtain an addition result, and the addition result is determined to be a similarity value corresponding to the reference instruction.

[0065] Among them, the similarity between the reference action and the initial action is calculated to obtain the first similarity, the similarity between the reference location and the initial location is calculated to obtain the second similarity, and the similarity between the reference executor and the initial executor is calculated to obtain the third similarity, and the first recognition probability, second recognition probability and third recognition probability corresponding to the initial action, initial location and initial executor are used as weights, and the corresponding first similarity, second similarity and third similarity are weightedly summed respectively, and the addition result of the weighted summation results is determined as the similarity value of the corresponding reference instruction, which serves as the basis for matching the initial instruction and the reference instruction.

[0066] The above-mentioned similarity calculation is performed on the initial instruction and each reference instruction in the preset reference instruction set to obtain the similarity value of the corresponding reference instruction, and the reference instruction with the largest similarity value is determined as the target instruction. The target instruction includes the steps of target action, target location and target executor. By determining the reference instruction most similar to the initial instruction as the target instruction, and using the target instruction as the basis for guiding the medical robot to provide intention feedback, the influence of personalized voice information on the intention recognition result can be reduced, and the accuracy of the intention recognition result can be improved.

[0067] Step S203: Use the position prediction model to predict the positions of the initial action, initial location, and initial performer in the audio information, respectively, and obtain the corresponding position information of the initial action, initial location, and initial performer in the audio information, respectively. The position information of the initial action is used as the position information of the target action, the position information of the initial location is used as the position information of the target location, and the position information of the initial performer is used as the position information of the target performer.

[0068] Among them, since the initial action, initial location and initial executor are obtained by performing intent recognition on audio features through the intention recognition model, and are not the original content in the audio information, in order to determine the position information of the target action, target location and target executor in the audio information, this embodiment first uses the position prediction model to predict the corresponding position information of the initial action, initial location and initial executor in the audio information. Correspondingly, the position information of the initial action is used as the position information of the target action, the position information of the initial location is used as the position information of the target location, and the position information of the initial executor is used as the position information of the target executor. The position information corresponding to the target action, target location and target executor can be determined respectively as the basis for reconstructing the audio information to reduce the impact of personalized information in the audio information on the intention recognition results, thereby improving the accuracy of the intention recognition results.

[0069] The above-mentioned step of using the position prediction model to predict the positions of the initial action, initial location and initial executor in the audio information respectively, and obtaining the corresponding position information of the initial action, initial location and initial executor in the audio information respectively, using the position information of the initial action as the position information of the target action, the position information of the initial location as the position information of the target location, and the position information of the initial executor as the position information of the target executor, using the predicted position information of the initial action, initial location and initial executor in the audio information as the corresponding position information of the target action, target location and target executor in the audio information as the basis for reconstructing the audio information, can reduce the influence of personalized information in the audio information on the intention recognition results, and improve the accuracy of the intention recognition results.

[0070] Step S204 : reconstructing the target action, target location, target performer and their corresponding position information using the audio reconstruction model to obtain reconstructed audio.

[0071] Among them, compared with the initial action, initial location and initial executor, the target action, target location and target executor can more accurately represent the intended action, intended location and intended object of the target patient. However, since the initial action, initial location and initial executor are identified by the intention recognition model based on the audio information of the target patient, the target action, target location and target executor are obtained based on the initial action, initial location and initial executor. In order to fundamentally eliminate the influence of personalized information in the audio information on the intention recognition results, this embodiment uses an audio reconstruction model to reconstruct the target action, target location and target executor and their corresponding position information to obtain reconstructed audio. Compared with the audio information, the reconstructed voice eliminates the personalized information of the target patient as the object of intention recognition, thereby improving the accuracy of the intention recognition results.

[0072] The above-mentioned audio reconstruction model is used to reconstruct the target action, target location, target executor and their corresponding position information to obtain reconstructed audio. The reconstructed audio that eliminates the personalized information of the target patient is used instead of the audio information as the object of intention recognition, thereby improving the accuracy of the intention recognition results.

[0073] Step S205: extract reconstructed audio features of the reconstructed audio, use the intention recognition model to perform intent recognition on the reconstructed audio features, and obtain a target intent recognition result.

[0074] Among them, compared with the audio information, the reconstructed voice eliminates the personalized information of the target patient, and the reconstructed audio features of the reconstructed audio are extracted, and the intention recognition model is used to perform intent recognition on the reconstructed audio features to obtain the target intention recognition results to characterize the target patient's target intention action, target intention location and target intention object, so as to guide the medical robot to make corresponding feedback and provide corresponding help to the target patient.

[0075] Optionally, extracting the reconstructed audio features of the reconstructed audio includes:

[0076] Using a preset window to segment the reconstructed audio to obtain at least one target segmentation frame;

[0077] Extract the target Mel-frequency cepstral coefficients of each target segmented frame respectively;

[0078] Use the speech recognition model to extract the target Mel-frequency cepstral coefficients to obtain the target phoneme features corresponding to the target segmentation frame;

[0079] According to the arrangement order of each target segmentation frame in the reconstructed audio, the target phoneme features corresponding to all target segmentation frames are spliced ​​to obtain the reconstructed audio features.

[0080] Among them, when extracting features from the reconstructed audio, the reconstructed audio is first segmented using a preset window to obtain at least one target segmentation frame, and the target Mel-cepstral coefficients of each target segmentation frame are extracted respectively. The target Mel-cepstral coefficients are then feature extracted from the target Mel-cepstral coefficients using a speech recognition model to obtain target phoneme features of the corresponding target segmentation frame to characterize the semantic feature information of the corresponding target segmentation frame. Then, according to the arrangement order of each target segmentation frame in the reconstructed audio, the target phoneme features corresponding to all target segmentation frames are spliced ​​to obtain reconstructed audio features to characterize the semantic feature information of the reconstructed audio.

[0081] The above steps of extracting the reconstructed audio features of the reconstructed audio, using the intention recognition model to perform intent recognition on the reconstructed audio features, and obtaining the target intention recognition results, use the intention recognition model to perform intent recognition on the reconstructed audio features of the reconstructed audio that has eliminated the influence of the target patient's personalized information, thereby improving the accuracy of the intention recognition results, thereby more accurately guiding the medical robot to make corresponding feedback, and improving the medical service quality of the medical robot.

[0082] An embodiment of the present invention obtains audio information and audio features of a target patient, uses an intention recognition model to perform intention recognition on the audio features, obtains an initial action, an initial location, and an initial executor, obtains an initial instruction based on the initial action, the initial location, and the initial executor, calculates similarity between the initial instruction and each reference instruction in a preset reference instruction set, obtains similarity values ​​corresponding to the reference instructions, determines that the reference instruction with the largest similarity value is the target instruction, the target instruction includes a target action, a target location, and a target executor, uses a position prediction model to predict the positions of the initial action, the initial location, and the initial executor in the audio information, obtains the corresponding position information of the initial action, the initial location, and the initial executor in the audio information, uses the position information of the initial action as the position information of the target action, and uses the position information of the initial location as the position information of the target action. The position information is used as the position information of the target location, and the position information of the initial executor is used as the position information of the target executor. The target action, target location and target executor and their corresponding position information are reconstructed using an audio reconstruction model to obtain reconstructed audio, and the reconstructed audio features of the reconstructed audio are extracted. The intention recognition model is used to perform intent recognition on the reconstructed audio features to obtain a target intention recognition result. By determining the reference instruction that is most similar to the initial instruction as the target instruction, and the position information of the target action, target location and target executor in the target instruction in the audio information, the reconstructed audio that eliminates the personalized information of the target patient is obtained as the object of intent recognition, thereby improving the accuracy of the intention recognition result, thereby more accurately guiding the medical robot to make corresponding feedback, and improving the medical service quality of the medical robot.

[0083] Corresponding to the intention recognition method of the above embodiment, Figure 3 A structural block diagram of an intention recognition device for a medical robot provided in a second embodiment of the present invention is given. For ease of explanation, only the parts related to the embodiment of the present invention are shown.

[0084] See also Figure 3 , the intention recognition device includes:

[0085] An initial instruction recognition module 31 is used to obtain audio information and audio features of a target patient, perform intent recognition on the audio features using an intent recognition model, obtain an initial action, an initial location, and an initial performer, and obtain an initial instruction based on the initial action, initial location, and initial performer;

[0086] The target instruction determination module 32 is configured to calculate the similarity between the initial instruction and each reference instruction in the preset reference instruction set, obtain similarity values ​​corresponding to the reference instructions, and determine the reference instruction with the largest similarity value as the target instruction, wherein the target instruction includes a target action, a target location, and a target executor;

[0087] a position information prediction module 33 for predicting the positions of the initial action, initial location, and initial performer in the audio information using a position prediction model, obtaining the corresponding position information of the initial action, initial location, and initial performer in the audio information, and using the position information of the initial action as the position information of the target action, the position information of the initial location as the position information of the target location, and the position information of the initial performer as the position information of the target performer;

[0088] An audio reconstruction module 34 is configured to reconstruct the target action, target location, target performer, and their corresponding position information using an audio reconstruction model to obtain reconstructed audio;

[0089] The target intention recognition module 35 is used to extract the reconstructed audio features of the reconstructed audio, use the intention recognition model to perform intention recognition on the reconstructed audio features, and obtain the target intention recognition result.

[0090] Optionally, the initial instruction recognition module 31 includes:

[0091] a recognition probability determination submodule, configured to perform intent recognition on the audio features using an intent recognition model, and obtain a first recognition probability corresponding to at least one preset action, a second recognition probability corresponding to at least one preset location, and a third recognition probability corresponding to at least one preset executor;

[0092] The initial instruction determination submodule is used to determine the preset action corresponding to the maximum value of the first recognition probability as the initial action, determine the preset location corresponding to the maximum value of the second recognition probability as the initial location, and determine the preset executor corresponding to the maximum value of the third recognition probability as the initial executor.

[0093] Optionally, the initial instruction recognition module 31 includes:

[0094] A first audio segmentation submodule is configured to obtain audio information of a target patient and segment the audio information using a preset window to obtain at least one segmentation frame;

[0095] The first audio feature splicing submodule is used to extract the phoneme features of each segmentation frame respectively, and splice the phoneme features corresponding to all segmentation frames according to the arrangement order of each segmentation frame in the audio information to obtain audio features.

[0096] Optionally, the first audio feature splicing submodule includes:

[0097] A first Mel-cepstral coefficient extraction unit is used to extract the Mel-cepstral coefficient of each segmented frame respectively;

[0098] The first phoneme feature extraction unit is used to perform feature extraction on the Mel-frequency cepstral coefficients using a speech recognition model to obtain phoneme features corresponding to the segmented frames.

[0099] Optionally, the reference instruction includes a reference action, a reference location, and a reference executor. The target instruction determination module 32 includes:

[0100] a similarity calculation submodule, configured to calculate, for any reference instruction, the similarity between a reference action and an initial action of the reference instruction to obtain a first similarity, calculate the similarity between a reference location and an initial location of the reference instruction to obtain a second similarity, and calculate the similarity between a reference executor and an initial executor of the reference instruction to obtain a third similarity;

[0101] a data multiplication submodule, configured to multiply the first similarity by the first recognition probability corresponding to the initial action to obtain a first multiplication result, multiply the second similarity by the second recognition probability corresponding to the initial location to obtain a second multiplication result, and multiply the third similarity by the third recognition probability corresponding to the initial performer to obtain a third multiplication result;

[0102] The similarity value calculation submodule is used to add the first multiplication result, the second multiplication result and the third multiplication result to obtain an addition result, and determine the addition result as the similarity value of the corresponding reference instruction.

[0103] Optionally, the target intention recognition module 35 includes:

[0104] A second audio segmentation submodule is configured to segment the reconstructed audio using a preset window to obtain at least one target segmentation frame;

[0105] The second Mel-cepstral coefficient extraction submodule is used to extract the target Mel-cepstral coefficient of each target segmented frame respectively;

[0106] The second phoneme feature extraction submodule is used to extract features of the target Mel-frequency cepstral coefficients using a speech recognition model to obtain target phoneme features corresponding to the target segmentation frame;

[0107] The second audio feature splicing submodule is used to splice the target phoneme features corresponding to all target segmentation frames according to the arrangement order of each target segmentation frame in the reconstructed audio to obtain the reconstructed audio features.

[0108] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0109] Figure 4 This is a schematic diagram of the structure of a computer device provided in the third embodiment of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps of any of the above-mentioned embodiments of the method for identifying intentions are implemented.

[0110] The computer device may include, but is not limited to, a processor and a memory. It will be understood by those skilled in the art that Figure 4 The above is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0111] The processor may be a CPU, or other general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor, or any conventional processor.

[0112] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be the hard disk of the computer device, and in other embodiments, it can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Furthermore, the memory can also include both the internal storage unit of the computer device and the external storage device. The memory is used to store the operating system, application programs, boot loaders (BootLoader), data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or is about to be output.

[0113] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned method embodiment. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0114] The present invention may implement all or part of the processes in the above-mentioned method embodiments, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0115] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0116] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0117] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for identifying intentions of a medical robot, characterized in that: The intention recognition method includes: Acquiring audio information and audio features of a target patient, performing intent recognition on the audio features using an intent recognition model to obtain an initial action, an initial location, and an initial performer, and obtaining an initial instruction based on the initial action, the initial location, and the initial performer; Calculating similarity between the initial instruction and each reference instruction in a preset reference instruction set to obtain similarity values ​​corresponding to the reference instructions, and determining the reference instruction with the largest similarity value as the target instruction, wherein the target instruction includes a target action, a target location, and a target executor; Using a position prediction model to predict the positions of the initial action, the initial location, and the initial performer in the audio information, respectively, to obtain the corresponding position information of the initial action, the initial location, and the initial performer in the audio information, respectively, using the position information of the initial action as the position information of the target action, using the position information of the initial location as the position information of the target location, and using the position information of the initial performer as the position information of the target performer; reconstructing the target action, the target location, the target performer, and their corresponding position information using an audio reconstruction model to obtain reconstructed audio; Extracting reconstructed audio features of the reconstructed audio, and performing intent recognition on the reconstructed audio features using the intent recognition model to obtain a target intent recognition result; The using the intention recognition model to perform intention recognition on the audio feature to obtain the initial action, initial location, and initial performer includes: Performing intent recognition on the audio features using the intent recognition model to obtain a first recognition probability corresponding to at least one preset action, a second recognition probability corresponding to at least one preset location, and a third recognition probability corresponding to at least one preset executor; Determine the preset action corresponding to the maximum value of the first recognition probability as the initial action, determine the preset location corresponding to the maximum value of the second recognition probability as the initial location, and determine the preset executor corresponding to the maximum value of the third recognition probability as the initial executor; The reference instruction includes a reference action, a reference location and a reference executor; The calculating similarity between the initial instruction and each reference instruction in a preset reference instruction set to obtain a similarity value of the corresponding reference instruction includes: For any reference instruction, calculating the similarity between a reference action of the reference instruction and the initial action to obtain a first similarity, calculating the similarity between a reference location of the reference instruction and the initial location to obtain a second similarity, and calculating the similarity between a reference executor of the reference instruction and the initial executor to obtain a third similarity; multiplying the first similarity by a first recognition probability corresponding to the initial action to obtain a first multiplication result, multiplying the second similarity by a second recognition probability corresponding to the initial location to obtain a second multiplication result, and multiplying the third similarity by a third recognition probability corresponding to the initial performer to obtain a third multiplication result; The first multiplication result, the second multiplication result, and the third multiplication result are added to obtain an addition result, and the addition result is determined as a similarity value corresponding to the reference instruction.

2. The intention recognition method according to claim 1, characterized in that The step of obtaining the audio information and audio features of the target patient includes: Acquiring audio information of a target patient, and segmenting the audio information using a preset window to obtain at least one segmented frame; The phoneme features of each segmentation frame are extracted respectively, and the phoneme features corresponding to all segmentation frames are spliced ​​according to the arrangement order of each segmentation frame in the audio information to obtain the audio features.

3. The intention recognition method according to claim 2, characterized in that The extracting of the phoneme features of each segmentation frame includes: Extract the Mel cepstral coefficients of each segmented frame respectively; A speech recognition model is used to perform feature extraction on the Mel-frequency cepstral coefficients to obtain phoneme features of corresponding segmentation frames.

4. The intention recognition method according to claim 3, characterized in that The extracting the reconstructed audio features of the reconstructed audio comprises: Using the preset window to segment the reconstructed audio to obtain at least one target segmentation frame; Extract the target Mel-frequency cepstral coefficients of each target segmented frame respectively; Using the speech recognition model to perform feature extraction on the target Mel-frequency cepstral coefficients to obtain target phoneme features corresponding to the target segmentation frame; According to the arrangement order of each target segmentation frame in the reconstructed audio, the target phoneme features corresponding to all target segmentation frames are spliced ​​to obtain the reconstructed audio features.

5. An intention recognition device for a medical robot, characterized in that: The intention recognition device includes: An initial instruction recognition module is used to obtain audio information and audio features of a target patient, perform intent recognition on the audio features using an intent recognition model, obtain an initial action, an initial location, and an initial performer, and obtain an initial instruction based on the initial action, the initial location, and the initial performer; a target instruction determination module, configured to calculate similarity between the initial instruction and each reference instruction in a preset reference instruction set, obtain similarity values ​​corresponding to the reference instructions, and determine the reference instruction with the largest similarity value as the target instruction, wherein the target instruction includes a target action, a target location, and a target executor; a position information prediction module, configured to use a position prediction model to predict the positions of the initial action, the initial location, and the initial performer in the audio information, respectively, to obtain the corresponding position information of the initial action, the initial location, and the initial performer in the audio information, respectively, and use the position information of the initial action as the position information of the target action, the position information of the initial location as the position information of the target location, and the position information of the initial performer as the position information of the target performer; An audio reconstruction module, configured to reconstruct the target action, the target location, the target performer, and their corresponding position information using an audio reconstruction model to obtain reconstructed audio; a target intent recognition module, configured to extract reconstructed audio features of the reconstructed audio, perform intent recognition on the reconstructed audio features using the intent recognition model, and obtain a target intent recognition result; The using the intention recognition model to perform intention recognition on the audio feature to obtain the initial action, initial location, and initial performer includes: Performing intent recognition on the audio features using the intent recognition model to obtain a first recognition probability corresponding to at least one preset action, a second recognition probability corresponding to at least one preset location, and a third recognition probability corresponding to at least one preset executor; Determine the preset action corresponding to the maximum value of the first recognition probability as the initial action, determine the preset location corresponding to the maximum value of the second recognition probability as the initial location, and determine the preset executor corresponding to the maximum value of the third recognition probability as the initial executor; The reference instruction includes a reference action, a reference location and a reference executor; The calculating similarity between the initial instruction and each reference instruction in a preset reference instruction set to obtain a similarity value of the corresponding reference instruction includes: For any reference instruction, calculating the similarity between a reference action of the reference instruction and the initial action to obtain a first similarity, calculating the similarity between a reference location of the reference instruction and the initial location to obtain a second similarity, and calculating the similarity between a reference executor of the reference instruction and the initial executor to obtain a third similarity; multiplying the first similarity by a first recognition probability corresponding to the initial action to obtain a first multiplication result, multiplying the second similarity by a second recognition probability corresponding to the initial location to obtain a second multiplication result, and multiplying the third similarity by a third recognition probability corresponding to the initial performer to obtain a third multiplication result; The first multiplication result, the second multiplication result, and the third multiplication result are added to obtain an addition result, and the addition result is determined as a similarity value corresponding to the reference instruction.

6. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the intention recognition method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the intention recognition method according to any one of claims 1 to 4 is implemented.

Images