A method and system for natural human-computer interaction based on embodied intelligence

Abstract

The application discloses a kind of man-machine natural interaction method and system based on embodied intelligence, and the application relates to information interaction communication technical field, solve the technical problem that the segment processing of user continuous action or complex instruction lacks dynamic adaptability, the present application is segmented based on motion trajectory turning point and voice pause dynamically, support personalized adjustment, improve instruction segmentation accuracy, solve continuous action semantic segmentation ambiguity, introduce dynamic time regulation+cosine similarity multilayer matching algorithm, preferentially screen the same position pre-stored instruction, and support combination instruction reorganization, improve the efficiency of abnormal instruction repair, reduce the secondary input burden of user, improve the fluency of interaction, simultaneously build "recognition-verification-reorganization-execution" closed loop, by similarity mean evaluation preferred combination instruction, and support incremental learning, improve system adaptability.

Description

Technical Field

[0001] This invention relates to the field of information interaction and communication technology, specifically to a human-computer natural interaction method and system based on embodied intelligence. Background Technology

[0002] With the development of artificial intelligence and sensor technology, human-computer interaction systems based on embodied intelligence are gradually moving from the laboratory to practical applications.

[0003] According to patent application CN112462940A, a smart home multimodal human-computer natural interaction system and method are disclosed. The system includes a gesture recognition model pre-training module, which trains a network model using a gesture dataset that matches the scenario and saves the trained gesture recognition model; a speech recognition model pre-training module, which trains an acoustic model and a language model sequentially using a Chinese speech dataset and saves the trained speech recognition model; a gesture recognition module, which uses the saved gesture recognition model to predict the collected gestures; a speech recognition module, which calls the saved speech recognition model to recognize the collected audio; and a multimodal fusion module, which fuses the results from the gesture recognition module and the speech recognition module to derive the final command.

[0004] Traditional human-computer interaction methods (such as keyboard, mouse, and touch) rely on users learning specific instructions, which has problems such as insufficient naturalness of interaction and poor cross-scene adaptability. At the same time, they lack dynamic adaptability to segmented processing of continuous user actions or complex instructions, making it difficult to accurately capture pause points and semantic boundaries. Furthermore, the error correction mechanism for abnormal instructions relies on fixed template matching and cannot intelligently reorganize based on context and user habits, resulting in low interaction efficiency. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a human-computer natural interaction method and system based on embodied intelligence, which solves the problem of lack of dynamic adaptability in segmented processing of continuous user actions or complex commands.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a human-computer natural interaction method based on embodied intelligence, which specifically includes the following steps:

[0007] Collect user interaction information, determine whether it can be recognized, and generate interaction recognition signals or secondary interaction input signals;

[0008] The interaction recognition signal is analyzed to obtain the user's error feedback and generate an interaction correction analysis signal. Based on this, the interaction command is segmented according to the turning point and matched with the pre-stored command to obtain the abnormal segmented command.

[0009] The instruction with the highest similarity between the filtered abnormal segmentation instruction and the pre-stored instruction is recorded as the pre-selected instruction, and interactive results or secondary interactive input signals are generated in combination with user feedback;

[0010] The secondary interactive input signal is analyzed, the secondary input interactive information is segmented to obtain the secondary interactive segmented instruction, and it is matched and filtered with the pre-stored combined instruction information to obtain the combined instruction to be analyzed.

[0011] The system retrieves the segmented instructions from the combined instructions to be analyzed that do not match the segmented instructions of the secondary interaction, calculates their similarity to the corresponding pre-stored instructions, and sums the mean similarity. Then, it uses the combined instructions to be analyzed with the maximum mean similarity as the standard for interaction and generates the interaction result.

[0012] As a further aspect of the present invention, the specific method for analyzing the interactive recognition signal is as follows:

[0013] The parsed interactive instructions are presented to the user in a visual form, and user feedback is collected. If the user feedback is correct, the interactive instructions are used as the standard for interaction analysis, and the corresponding interaction results are generated. Otherwise, an interaction correction analysis signal is generated.

[0014] As a further aspect of the present invention, the specific method for obtaining the abnormal segmentation instruction is as follows:

[0015] The interaction commands are obtained, and the segmented commands are divided into segments based on the turning points of the user's action trajectory. The segmented command numbers are denoted as i, where i = 1, 2, ..., j, and j represents the number of segmented commands. Then, the obtained segmented commands i are matched with the pre-stored commands, which are the segmented commands that have been trained. The segmented commands that cannot be recognized are recorded as abnormal segmented commands.

[0016] As a further aspect of the present invention, the specific method for generating interactive results or secondary interactive input signals by combining user feedback is as follows:

[0017] The abnormal segmentation instruction is labeled as n, and n = 1, 2, ..., m, where m represents the number of abnormal segmentation instructions. The similarity between the instruction and all pre-stored instructions is calculated, and the pre-stored instruction with the largest similarity value is selected as the pre-selected instruction.

[0018] The pre-selected instructions and the segmented instructions in the interaction instructions are recombined to obtain the recombined interaction instructions. Then, user feedback is obtained, and it is determined whether the recombined interaction instructions are correct. If they are correct, they are used as the standard for interaction to generate an interaction result. Otherwise, if they are incorrect, a secondary interaction input signal is generated.

[0019] As a further aspect of the present invention, the specific method for obtaining the combined instructions to be analyzed is as follows:

[0020] The secondary input interaction information is obtained and segmented to obtain secondary interaction segment instructions, which are labeled as a, where a = 1, 2, ..., b, and b represents the number of secondary interaction segment instructions. Then, the similarity between the secondary interaction segment instruction a and the pre-stored instructions is calculated and sorted from largest to smallest according to the similarity of the corresponding pre-stored instructions. At the same time, all combination instruction information corresponding to the pre-stored instructions is obtained, and the combination instructions to be analyzed that have intersection with the secondary interaction segment instructions are filtered.

[0021] As a further aspect of the present invention, the specific method for generating the interactive result is as follows:

[0022] Obtain the combined instruction to be analyzed and label it as k, where k = 1, 2, ..., p, and p is the number of combined instructions to be analyzed. Match the pre-stored instructions in the combined instruction k with the secondary interactive segmented instruction a, and filter out the segmented instructions in the secondary interactive segmented instruction a that do not match. At the same time, calculate the similarity between the segmented instruction and the corresponding pre-stored instructions in the combined instruction k, and calculate the mean similarity.

[0023] The system selects the combined instruction with the highest average similarity as the standard, generates a standard combined instruction, and then uses it as the standard to generate the corresponding interactive results.

[0024] A human-computer natural interaction system based on embodied intelligence, the system comprising:

[0025] The interactive information recognition module is used to collect user interactive information, determine whether it can be recognized, generate interactive recognition signals or secondary interactive input signals, and transmit both respectively.

[0026] The interaction recognition and analysis module is used to acquire and analyze interaction recognition signals, obtain error feedback corresponding to user feedback, generate interaction correction analysis signals, then segment the interaction instructions according to the turning points to obtain segmented instructions, and match and filter them with pre-stored instructions to obtain abnormal segmented instructions.

[0027] The instruction with the highest similarity between the filtered abnormal segmentation instruction and the pre-stored instruction is recorded as the pre-selected instruction, and interactive results or secondary interactive input signals are generated in combination with user feedback;

[0028] The secondary interaction analysis module is used to analyze the acquired secondary interaction input signals, obtain secondary input interaction information and segment it to obtain secondary interaction segment instructions, calculate its similarity with pre-stored instructions, obtain combined instruction information, and filter it with secondary interaction segment instructions to obtain combined instructions to be analyzed.

[0029] The mismatched instructions in the secondary interactive segmented instructions are obtained as segmented instructions based on the combined instructions to be analyzed. At the same time, the similarity between the segmented instructions and the pre-stored instructions in the combined instructions to be analyzed is calculated and the average value is calculated. Then, the combined instructions to be analyzed with the largest average value are used as the standard for interaction, the interaction result is generated and transmitted to the interaction information output module.

[0030] The interactive information output module is used to display the generated interactive results to the corresponding interactive users.

[0031] This invention provides a human-computer natural interaction method and system based on embodied intelligence. Compared with existing technologies, it has the following advantages:

[0032] This invention dynamically segments instructions based on motion trajectory turning points and speech pauses, supporting personalized adjustments, improving the accuracy of instruction segmentation, resolving semantic ambiguity in continuous action segmentation, and introducing a multi-layer matching algorithm based on dynamic time warping and cosine similarity. It prioritizes filtering pre-stored instructions at the same position and supports the recombination of combined instructions, improving the efficiency of abnormal instruction repair, reducing the burden of secondary input for users, and enhancing the smoothness of interaction. At the same time, it constructs a closed loop of "recognition-verification-recombination-execution", optimizes the combination of instructions through the mean similarity evaluation, and supports incremental learning to improve the system's adaptability. Attached Figure Description

[0033] Figure 1 This is a diagram illustrating the steps and methods of the present invention;

[0034] Figure 2 This is a system block diagram of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1: Please refer to Figure 1 This application provides a human-computer natural interaction method based on embodied intelligence, which specifically includes the following steps:

[0037] Step S1: Collect multi-source sensor data from the user. Here, the obtained multi-source sensor data is cleaned and fused accordingly. Specifically, Kalman filtering, wavelet transform and other methods are used to eliminate sensor drift (such as camera shake, electromagnetic interference). Then, multi-modal data is synchronized by timestamp (error ≤ 5ms). A unified coordinate mapping is established in space (such as projecting gesture video and depth data to the same three-dimensional space). Interaction information is obtained by recognizing the interaction information. At the same time, interaction recognition judgment is performed on the interaction information. If it can be recognized, an interaction recognition signal is generated. Otherwise, if it cannot be recognized, a secondary interaction input signal is generated.

[0038] Step S2: Analyze the obtained interaction recognition signal and obtain the corresponding interaction command. At the same time, feed the interaction command back to the corresponding user and make a judgment based on the user's feedback. Here, the user feedback specifically answers the question of whether the interaction command is correct. If the interaction command is correct after feedback, it means that the user's feedback is correct. Then, the interaction analysis is performed based on the interaction command as the standard, and the corresponding interaction result is generated and displayed to the corresponding user. Conversely, if the interaction command is incorrect after feedback, it means that the user's feedback is wrong, and an interaction correction analysis signal is generated.

[0039] The obtained interactive correction analysis signal is processed to obtain the interactive command, which is then segmented. This segmentation is based on pause points within the interactive command. A specific pause point represents a turning point in the user's movement trajectory. For example, during head shaking, the head will sway left and right, creating a turning point, which is recorded as a pause point. For instance, in a user's continuous actions "drawing a circle with the left hand → pointing to the window → waving," the system divides the command into three segments based on the turning points.

[0040] Instruction 1: Draw a circle with your left hand (this may indicate "cycle" or "adjust").

[0041] Command 2: Point to window (target is window);

[0042] Instruction 3: Wave your hand (may indicate "close" or "stop").

[0043] The segmented instruction is labeled i, where i = 1, 2, ..., j, and j represents the number of segmented instructions. The segmented instruction i is then matched with pre-stored instructions, which are also segmented instructions stored by the operator. Abnormal segmented instructions are then identified; these are those that cannot be recognized by the pre-stored instructions. Dynamic Time Warping (DTW) or cosine similarity is used to calculate the matching degree between the segmented instruction and the pre-stored template. For example, if the user instruction "draw a circle with your left hand → point to the window" matches the second segment of the pre-stored instruction "raise your hand → point to the window" with a 90% match, but the first segment does not match, it is marked as an abnormal segment. Segmented instructions with a matching degree below a threshold (e.g., 70%) are also considered abnormal.

[0044] For example, in a smart home environment, a user says "turn off" and quickly points to a light fixture. The system initially interprets this as a "turn off the light fixture" command and asks the user for clarification. The user shakes their head to indicate that this is incorrect.

[0045] The system generates a correction signal, segmenting the user's action "rapidly pointing → shaking head" into segments based on the turning point;

[0046] The first segment "quick pointing" matches the pre-stored instruction "pointing to the light fixture", but the second segment "shaking head" does not match and is marked as abnormal;

[0047] Based on historical data, the system anticipates potential user error and prompts, "Do you want to turn on the lights?" The user nods in confirmation and executes the correct command.

[0048] Step S3: Obtain all abnormal segmentation instructions and label them as n, where n = 1, 2, ..., m, and m represents the number of abnormal segmentation instructions. Then, calculate the similarity between the abnormal segmentation instructions and all pre-stored instructions, and select the pre-stored instruction with the highest similarity value as the pre-selected instruction. Here, the pre-selected instruction means that it belongs to the same movement position as the abnormal segmentation instruction. At the same time, recombine the obtained pre-selected instruction with the segmentation instructions to obtain the recombined interaction instruction. Then, obtain user feedback and determine whether the recombined interaction instruction is correct. If it is correct, use it as the standard to perform interaction and generate an interaction result. Otherwise, if it is incorrect, generate a secondary interaction input signal.

[0049] The similarity of motion trajectories (such as hand movement path similarity) is calculated by using the coordinates of skeletal key points. Simultaneously, the Dynamic Time Warping (DTW) algorithm is used to match the temporal rhythm of the motion, prioritizing pre-stored instructions that start at the same position as the abnormal instruction. For example, if the user's abnormal action is "right hand swinging from left to right," only motion templates starting with "right hand from left" are selected from the pre-stored instruction library. A similarity threshold is set (e.g., ≥0.7). If the similarity is below this value, no pre-selected instruction is generated, and the user directly enters the secondary interaction.

[0050] Replace the original faulty segment with the pre-selected instruction, while retaining other correct segments. For example:

[0051] Original segmented instruction sequence: [wave → point to TV → clench fist] (where "clench fist" is an abnormal segment);

[0052] Pre-selected instruction: [Click action] (highest similarity, and starting position is close to "clenched fist");

[0053] Recombined instructions: [Wave → Point at TV → Click action].

[0054] Step S4: Analyze the generated secondary interactive input signal to obtain secondary input interactive information, and segment it. The segmentation method here is the same as in step S2. Obtain the secondary interactive segment instructions and label them as a, where a = 1, 2, ..., b, where b represents the number of secondary interactive segment instructions. Then, calculate the similarity between the secondary interactive segment instruction a and the pre-stored instructions, and sort them from largest to smallest according to the similarity of the corresponding pre-stored instructions. Here, the first step is to filter the pre-stored instructions with a similarity greater than 70% based on the similarity. Then, sort the filtered pre-stored instructions from largest to smallest. Specifically, sort the pre-stored instructions corresponding to all secondary interactive segment instructions. For example, if the secondary interactive segment instruction a = 1, first calculate its similarity with the pre-stored instructions, and filter the pre-stored instructions with a similarity greater than 70%. Then, sort the filtered pre-stored instructions from largest to smallest according to the similarity. At the same time, obtain all the combined instruction information corresponding to the pre-stored instructions. Here, the combined instruction information represents the complete interactive instructions that the interactive system can recognize.

[0055] Next, the secondary interactive segmented instruction a is filtered based on the combined instruction information to obtain the combined instructions to be analyzed. The filtering method here is based on the secondary interactive segmented instruction a. If a secondary interactive segmented instruction exists in the combined instruction information, it is recorded as a combined instruction to be analyzed. Otherwise, if it does not exist, it is not selected and labeled as k, where k = 1, 2, ..., p, where p represents the number of combined instructions to be analyzed. Then, the pre-stored instructions in the combined instructions to be analyzed k are matched with the secondary interactive segmented instruction a, and the segmented instructions that do not match in the secondary interactive segmented instruction a are filtered. Here, the segmented instructions refer to the secondary interactive segmented instructions that differ from the pre-stored instructions. At the same time, the similarity between the segmented instructions and the corresponding pre-stored instructions in the combined instructions to be analyzed k is calculated, and the mean similarity is calculated. Specifically, all the segmented instructions in the combined instructions to be analyzed are obtained, and the similarity between the segmented instructions and their corresponding pre-stored instructions is calculated. Then, the mean of all similarities is calculated.

[0056] For example, constructing a secondary interactive segmented instruction matrix M a×n (where a is the number of segments and n is the number of pre-stored instructions), calculate the matching degree matrix P for each combination instruction k. kelement p u，o Let represent the similarity between the u-th segment and the o-th pre-stored instruction, where u and o are in corresponding positions. Calculate the mean similarity of the combined instruction k.

[0057] Similarly, the average similarity of segmented instructions for all combined instructions k to be analyzed is calculated, and the combined instruction with the highest average similarity is selected as the standard to generate a standard combined instruction. Then, the corresponding interactive results are generated using this standard.

[0058] For example, if a user makes an unusual interaction in a smart home environment, the system triggers a secondary interaction process, as follows:

[0059] User input: Voice "Turn down...this" + gesture (left hand swipes down → points to air conditioner);

[0060] Segmentation results: a=1 (voice "turn down"), a=2 (voice "this" + gesture).

[0061] Segment a=1 matches the pre-stored instructions: "lower temperature" (similarity 0.85), "lower volume" (similarity 0.72);

[0062] Segment a=2 matches the pre-stored instructions: "point to air conditioner" (similarity 0.91), "point to light" (similarity 0.68, filtered);

[0063] The combined commands to be analyzed are: k=1: [lower temperature → point to air conditioner], with a mean similarity of 0.88; k=2: [lower volume → point to air conditioner], with a mean similarity of 0.77.

[0064] Choose k=1 as the standard combination instruction.

[0065] Example 2: Please refer to Figure 2 This application provides a human-computer natural interaction system based on embodied intelligence, which includes an interaction information recognition module, an interaction recognition and analysis module, a secondary interaction analysis module, and an interaction information output module.

[0066] The interactive information recognition module is used to collect user interactive information, determine whether it can be recognized, generate interactive recognition signal or secondary interactive input signal, and transmit both at the same time. The specific processing method is the same as the processing process of step S1 in embodiment one.

[0067] The interaction recognition and analysis module is used to acquire and analyze interaction recognition signals, acquire error feedback corresponding to user feedback, generate interaction correction analysis signals, then segment the interaction instructions according to the turning point to obtain segmented instructions, and match and filter them with pre-stored instructions to obtain abnormal segmented instructions. The specific processing method is the same as the processing process in step S2 of embodiment one.

[0068] The instruction with the highest similarity between the filtered abnormal segmentation instruction and the pre-stored instruction is recorded as the pre-selected instruction, and the interactive result or secondary interactive input signal is generated in combination with user feedback. The specific processing method is the same as the processing process in step S3 of embodiment one.

[0069] The secondary interaction analysis module is used to analyze the acquired secondary interaction input signals, obtain secondary input interaction information and segment it to obtain secondary interaction segment instructions, calculate its similarity with pre-stored instructions, obtain combined instruction information, and filter it with secondary interaction segment instructions to obtain combined instructions to be analyzed.

[0070] Using the combined instruction to be analyzed as the standard, the mismatched instruction in the secondary interactive segmented instruction is recorded as the segmented instruction. At the same time, the similarity between the segmented instruction and the pre-stored instruction in the combined instruction to be analyzed is calculated and the average value is calculated. Then, the combined instruction to be analyzed with the largest average value is used as the standard for interaction, generating the interaction result and transmitting it to the interaction information output module. The specific processing method is the same as the processing process in step S4 of embodiment one.

[0071] The interactive information output module is used to display the generated interactive results to the corresponding interactive users.

[0072] The data in the above formulas are all calculated using numerical values, without substituting the units of the parameters. In addition, the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0073] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for natural human-computer interaction based on embodied intelligence, characterized in that, The method specifically comprises the following steps: Collect the interaction information of the user, and determine whether it can be recognized to generate an interaction recognition signal or a secondary interaction input signal; Analyze the interaction recognition signal to obtain an error feedback of the user to generate an interaction correction analysis signal, segment the interaction instruction according to a turning point, and match the segmented instruction with a pre-stored instruction to obtain an abnormal segmented instruction; The specific way of obtaining the abnormal segmented instruction is as follows: Obtain the interaction instruction, segment the instruction according to the trajectory turning point in the user action trajectory to obtain a segmented instruction label marked as i, and i = 1, 2, …, j, wherein j represents the number of segmented instructions, then match the obtained segmented instruction i with a pre-stored instruction, the pre-stored instruction is a segmented instruction after training, and the segmented instruction that cannot be recognized is screened as an abnormal segmented instruction; Screen the maximum similarity instruction of the abnormal segmented instruction and the pre-stored instruction as a pre-selected instruction, and generate an interaction result or a secondary interaction input signal in combination with the user feedback; The specific way of generating the interaction result in combination with the user feedback is as follows: Mark the abnormal segmented instruction label as n, and n = 1, 2, …, m, wherein m represents the number of abnormal segmented instructions, calculate the similarity of the abnormal segmented instruction with all pre-stored instructions, and select the pre-stored instruction with the maximum similarity value as the pre-selected instruction; Recombine the pre-selected instruction with the segmented instruction in the interaction instruction to obtain a recombined interaction instruction, then obtain the user feedback, and determine whether the recombined interaction instruction is correct, if yes, use it as a standard to interact to generate an interaction result, otherwise, generate a secondary interaction input signal; Segment the secondary interaction input signal to obtain secondary interaction segmented instructions, and match and screen the secondary interaction segmented instructions with pre-stored combination instruction information to obtain combination instructions to be analyzed; Obtain the segmented instruction in the combination instruction to be analyzed that does not match the secondary interaction segmented instruction, calculate the similarity of the segmented instruction with the corresponding pre-stored instruction, sum the similarity values, then use the combination instruction to be analyzed with the maximum similarity value as a standard to interact to generate an interaction result.

2. The method according to claim 1, wherein, The specific way of analyzing the interaction recognition signal is as follows: Display the analyzed interaction instruction to the user in a visual form, collect the user feedback, if the user feedback is correct, use the interaction instruction as a standard to interact and analyze, and generate a corresponding interaction result, otherwise, generate an interaction correction analysis signal.

3. The method of claim 1, wherein the method further comprises: The specific way of obtaining the combination instruction to be analyzed is as follows: Obtain the secondary interaction input information, segment the information to obtain secondary interaction segmented instructions and label them as a, and a = 1, 2, …, b, wherein b represents the number of secondary interaction segmented instructions, then calculate the similarity of the secondary interaction segmented instruction a with the pre-stored instruction, sort the pre-stored instructions according to the similarity from large to small, obtain all combination instruction information corresponding to the pre-stored instructions, and screen the combination instruction to be analyzed that has an intersection with the secondary interaction segmented instruction.

4. The method of claim 1, wherein the method further comprises: The specific way of generating the interaction result is as follows: Obtain the combination instruction to be analyzed and mark it as k, and k=1, 2, …, p, and p is the number of combination instructions to be analyzed, match the pre-stored instruction in the combination instruction to be analyzed k with the secondary interaction segment instruction a, and screen the segment instructions that do not match in the secondary interaction segment instruction a, calculate the similarity of the segment instructions and the pre-stored instruction in the corresponding combination instruction k, and calculate the average value of the similarity; And select the combination instruction to be analyzed with the maximum average value of the similarity as the standard to generate a standard combination instruction, and then interact with it as the standard to generate the corresponding interaction result.

5. A natural human-computer interaction system based on embodied intelligence, used for executing the natural human-computer interaction method of any one of claims 1-4, characterized in that, The system comprises: An interaction information identification module is configured to collect interaction information of a user, determine whether the interaction information can be identified, generate an interaction identification signal or a secondary interaction input signal, and transmit the two signals respectively; An interaction identification analysis module is configured to obtain the interaction identification signal and analyze it, obtain error feedback corresponding to user feedback, analyze and generate an interaction correction analysis signal, segment the interaction instruction according to a turning point to obtain segment instructions, and match and screen the segment instructions with pre-stored instructions to obtain abnormal segment instructions; The specific way of obtaining the abnormal segment instructions is as follows: Obtain the interaction instruction, segment the segment instructions based on the trajectory turning points in the user action trajectory, and mark the segment instructions as i, and i=1, 2, …, j, where j represents the number of segment instructions, then match the obtained segment instructions i with pre-stored instructions respectively, and the pre-stored instructions are segment instructions after training, and screen the segment instructions that cannot be identified as abnormal segment instructions; Screen the maximum similarity instruction of the abnormal segment instruction and the pre-stored instruction as a pre-selected instruction, and generate an interaction result or a secondary interaction input signal in combination with user feedback; The specific way of generating an interaction result or a secondary interaction input signal in combination with user feedback is as follows: Mark the abnormal segment instruction as n, and n=1, 2, …, m, where m represents the number of abnormal segment instructions, calculate the similarity of the abnormal segment instruction with all pre-stored instructions, and select the pre-stored instruction with the maximum similarity value as a pre-selected instruction; Recombine the pre-selected instruction with the segment instructions in the interaction instruction to obtain a recombined interaction instruction, then obtain user feedback, and determine whether the recombined interaction instruction is correct, if yes, interact with it as the standard to generate an interaction result, otherwise, generate a secondary interaction input signal; A secondary interaction analysis module is configured to analyze the obtained secondary interaction input signal, obtain secondary input interaction information and segment it to obtain secondary interaction segment instructions, calculate the similarity of the secondary interaction segment instructions with pre-stored instructions, obtain combination instruction information, screen the secondary interaction segment instructions with the combination instruction information to obtain combination instructions to be analyzed, and mark the non-matching instructions in the secondary interaction segment instructions as segment instructions, calculate the similarity of the segment instructions with the pre-stored instructions in the combination instructions to be analyzed, sum and calculate the average value, then interact with the combination instruction to be analyzed with the maximum average value as the standard to generate an interaction result, and transmit the interaction result to an interaction information output module; An interaction information output module is configured to display the generated interaction result to a corresponding interaction user. ​

Images