Intelligent robot instruction receiving and intention analysis method and system

Abstract

The invention discloses an intelligent robot instruction receiving and intention analysis method and system. The method specifically comprises the technologies of voice wake-up recognition, semantic retrieval fusion, structured instruction disassembly and execution optimization and the like. The method comprises the following steps: realizing high-precision and low-delay voice wake-up and transcription through a locally deployed lightweight DF-Conformer model; utilizing a Bge-large embedding model and a retrieval enhancement generation technology based on FAISS to associate the instruction text with historical memory and business knowledge to form an enhanced business token vector; based on a pre-trained Qwen3 large language model in an inspection scene, an intention analysis agent is constructed through LangChain, and a ReAct multi-step reasoning mechanism is adopted to gradually disassemble a complex instruction into structured task steps which can be executed by a machine. The technical problems that in the prior art, voice processing precision is low, intention analysis lacks context association, complex instruction disassembling is incomplete and the like are effectively solved, and the autonomous task execution capacity of the intelligent robot in a complex scene is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent robot instruction processing and intention recognition, and in particular to an intelligent robot instruction receiving and intention analysis method and system. BACKGROUND

[0002] With the development of artificial intelligence technology, intelligent robots are increasingly widely used in various complex environments. Intelligent robots receive user instructions through natural language, accurately understand the user's intention, and execute corresponding tasks.

[0003] The existing intelligent robot instruction processing scheme has the following significant technical defects: traditional speech processing models mostly use a single Transformer architecture, which needs to be simplified due to the power limit of local device deployment, resulting in high keyword wake-up false trigger rate in noisy environments, low speech-to-text accuracy, and inability to stably receive natural language instructions; most current schemes are based on current instruction text analysis only, without associating historical instruction memory and robot current state, resulting in incomplete complex instruction disassembly and easy omission of key business information; if complex text disassembly is involved, existing technologies mostly rely on fixed rules or a single language model for instruction disassembly, which cannot effectively handle multi-subtask nested inspection instructions, and the disassembled steps lack executability, and in severe cases, even produce logical confusion.

[0004] Therefore, there is an urgent need for an intelligent robot intention analysis system that integrates multi-task learning, state-aware retrieval, and structured reinforcement learning to significantly improve speech recognition robustness, semantic retrieval accuracy, and instruction execution reliability. SUMMARY

[0005] The present application relates to the technical field of intelligent robot instruction processing and intention recognition, and in particular to an intelligent robot instruction receiving and intention analysis method and system.

[0006] To realize the conversion of natural language instructions to structured task steps and improve the accuracy of intelligent robot instruction intention analysis, the present application proposes an intelligent robot instruction receiving and intention analysis method suitable for complex inspection instruction processing of intelligent robots in various scenarios.

[0007] TECHNICAL SCHEME

[0008] The present application proposes an intelligent robot instruction receiving and intention analysis method, comprising:

[0009] S1: deploying a DF-Conformer model on the local device of the intelligent robot to monitor the environment speech in real time, setting the wake-up keyword, and triggering the local device to enter the instruction receiving state when the model detects the wake-up keyword;

[0010] S2: combine the preset prompt word template to construct a standardized input, call the pre-trained bge-large model to convert the instruction text in the standardized input into a token vector; adopt a two-stage retrieval enhancement mechanism to retrieve the token vector to obtain an enhanced business token vector;

[0011] The two-stage retrieval enhancement mechanism comprises: performing preliminary retrieval by using a FAISS vector library, calculating an accurate similarity score by a cross-encoder, and performing vector fusion to obtain the enhanced business token vector.

[0012] S3: constructing an intent analysis intelligent agent by using a large language model, the intent analysis intelligent agent receives the enhanced business token vector, starts a ReAct cycle to disassemble instructions, and outputs executable steps.

[0013] Further, the DF-Conformer model is trained in a multi-task joint manner, and the loss function used includes wake-up recognition loss, speech recognition loss and contrast learning loss; an adaptive noise enhancement strategy is introduced in the model training stage.

[0014] Further, the dimension of the token vector is consistent with the dimension of the pre-trained bge-large model.

[0015] Further, the preliminary retrieval by using the FAISS vector library comprises: constructing a context vector library by FAISS, including historical instructions and business knowledge; retrieving the token vector by using the context vector library, associating recent historical memory with current business supplement; and retrieving Top-K candidates by cosine similarity.

[0016] Further, the vector fusion comprises:

[0017] State-aware gating function Calculate the fusion weight , the formula is:

[0018]

[0019] In the formula, is the accurate similarity score, which represents the semantic matching score of the cross-encoder between the query vector and the first candidate business token vector ; represents the output of the state-aware gating function, state is the current state information of the system, and The similarity score is controlled by learnable hyperparameters or trainable scalar coefficients And gate signal The relative importance in weight calculation

[0020] Generate enhanced business token vector The formula is:

[0021]

[0022] In the formula, the weight is obtained after softmax normalization, indicating the contribution proportion of each candidate token in the final fusion, And b respectively represent the learnable linear transformation matrix and the bias term, Indicates the layer normalization function.

[0023] Further, the large language model is pre-trained in the intelligent robot inspection professional scene, and the training data includes multiple inspection instruction samples; the training stage of the large language model adopts a hybrid strategy of imitation learning and reinforcement learning.

[0024] Further, the intent analysis agent is constructed based on the LangChain framework, and the enhanced business token vector and the current state of the intelligent robot Start the ReAct cycle.

[0025] Further, the ReAct cycle includes:

[0026] (1) Call the large language model interface, analyze the current instruction complexity and disassemble it, and output the disassembly result;

[0027] (2) Search the context vector library, generate a structured intermediate representation through the large language model, check whether the structured intermediate representation meets the resource, power and path constraint conditions through the constraint validator, if not, return the exception information to the large language model to rewrite, until it passes the verification;

[0028] (3) Generate subtask steps according to the result of (2), and repeat (1) to (2) with the subtask steps as input, until the command disassembly is executable steps, and output the executable steps.

[0029] Further, the executable steps include step number, execution action and judgment standard, which are directly called by the intelligent robot.

[0030] The application further provides an intelligent robot instruction receiving and intent analysis system, which includes:

[0031] The voice recognition module is configured to deploy a DF-Conformer model locally on the intelligent robot to monitor the environment in real time, set a wake-up keyword, and trigger the local device to enter an instruction receiving state when the model detects the wake-up keyword;

[0032] The semantic retrieval module is configured to construct a standardized input in combination with a preset prompt word template, call a pre-trained bge-large model to convert instruction text in the standardized input into a token vector, and perform retrieval on the token vector by using a two-stage retrieval enhancement mechanism to obtain an enhanced business token vector.

[0033] The two-stage retrieval enhancement mechanism includes preliminary retrieval by using a FAISS vector library, calculation of an accurate similarity score by using a cross-encoder, and vector fusion to obtain the enhanced business token vector.

[0034] The instruction analysis module is configured to construct an intent analysis agent by using a large language model, receive the enhanced business token vector by using the intent analysis agent, start a ReAct cycle to disassemble the instruction, and output executable steps.

[0035] Advantages:

[0036] Compared with the prior art, the present application has the following significant advantages:

[0037] In the voice wake-up and transcription stage, a DF-Conformer model with multi-task joint training is introduced, which realizes high transcription accuracy and low false wake-up rate while reducing the model size, and is suitable for edge computing devices; and by combining a contrast learning and an adaptive noise enhancement strategy, the robustness of voice recognition is significantly enhanced.

[0038] In the text semantic retrieval stage, a two-stage retrieval enhancement mechanism including FAISS coarse retrieval and cross-encoder fine retrieval is adopted, so that the intelligent robot can recall historical tasks and master business knowledge, and the historical information association accuracy is improved; by combining a state-aware gating attention fusion model, the context association accuracy is improved.

[0039] In the complex instruction analysis stage, the present application converts fuzzy and complex instructions into a logical and executable action sequence by using ReAct multi-step reasoning, improves the complex instruction disassembly completeness, and further verifies the logical consistency and executability of the constructed structured intermediate representation by using a constraint verifier, and combines an imitation learning and a reinforcement learning strategy to realize high reliability of task disassembly and execution. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The flowchart of the method of the present application;

[0041] Figure 2 This is a flowchart of step S1 of the present invention;

[0042] Figure 3 This is a flowchart of step S2 of the present invention. Detailed Implementation

[0043] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0044] like Figure 1 The diagram shows a flowchart of a method for receiving and parsing instructions for an intelligent robot according to the present invention. Specifically, it includes the following steps:

[0045] S1: Deploy the lightweight deep learning model DF-Conformer on the robot's local device to jointly optimize wake-up classification, speech recognition, and speech feature comparison tasks. During the model's training phase, adaptive noise enhancement using samples with varying signal-to-noise ratios is introduced to improve the model's robustness in complex environments. The flowchart for step S1 is as follows: Figure 2 As shown.

[0046] The total loss function during the model training phase Defined as:

[0047]

[0048] in, To awaken classification loss, For speech recognition loss, To compare the learning losses, the specific expressions for each loss are as follows:

[0049]

[0050]

[0051]

[0052] In the formula, The tag used to trigger the keyword takes a value of 0 or 1; The wake-up probability output by the model is determined by the wake-up classification head of the DF-Conformer model; For speech feature sequences, Let be the encoder output vectors for two positive samples, respectively. The encoder output vector representing the negative sample, i.e., the vector with... Embedding of different other speech segments, Temperature is a parameter used to adjust the distribution range of similarity scores and control the discrimination between positive and negative samples in contrastive learning.

[0053] The preset wake-up keyword, the DF-Conformer monitors the environmental voice in real time, when the wake-up keyword is detected, the local device is triggered to enter the instruction receiving state, the wake-up response delay is ≤300ms, and the false wake-up rate is ≤1%. After the device enters the instruction receiving state, the DF-Conformer continuously collects natural language instructions of the user, converts the voice signal into text data, and in an environment with a signal-to-noise ratio ≥15dB, the transcription accuracy is ≥95%.

[0054] The DF-Conformer model is based on a convolution-enhanced Transformer algorithm, which extracts local features of the voice through a local convolution module, and combines the global modeling capability of the Transformer to ensure the accuracy of voice processing while reducing the model parameter size by 40%.

[0055] S2: Implement a two-stage retrieval enhancement mechanism, including: using the FAISS vector library for preliminary retrieval, calculating the accurate similarity score through the cross-encoder, and performing vector fusion based on the state-aware gating function to obtain the enhanced business token vector. The flowchart of step S2 is shown in Figure 3 .

[0056] The pre-trained embedding model bge-large is called, and the current state of the intelligent robot and the preset prompt word template are combined to construct a standardized input. The bge-large model converts the instruction text in the standardized input into a high-dimensional token vector, and the vector dimension is consistent with the model pre-training dimension, ensuring accurate representation of the text semantics.

[0057] The two-stage retrieval enhancement mechanism specifically includes:

[0058] (1) First stage: The token vector is retrieved by the context vector library constructed by FIASS (Fast Approximate Nearest Neighbor Search Algorithm), and two types of information are associated, and the Top-K candidate is retrieved by the cosine similarity.

[0059] The context vector library includes historical instructions and business knowledge, and the two types of information are specifically: (1) Historical memory: such as the inspection instruction record and abnormal processing record of "commercial complex 1st floor exhibition hall" of this robot in the past 7 days; (2) Current business supplement: such as the basic business terminology corresponding to "fire facility inspection" (inspection items of fire extinguishers and fire hydrants).

[0060] (2) Second stage: The cross-encoder is used to calculate the accurate similarity score , and the state-aware gating function is used to calculate the fusion weight , and the formula is:

[0061]

[0062] wherein, is the cross-encoder similarity score between the query vector and the th candidate business token vector ; represents the output of the state-aware gating function, state is the current state information of the system, and are learnable hyperparameters or trainable scalar coefficients, respectively controlling the relative importance of the similarity scores and the gating signal in the weight calculation;

[0063] The enhanced business token vector is generated, and the formula is:

[0064]

[0065] wherein, the weight is obtained after softmax normalization, indicating the contribution proportion of each candidate token in the final fusion, and b represent a learnable linear transformation matrix and a bias term, respectively, and norm represents a layer normalization function. After the above process, the enhanced business token vector is obtained, which integrates the current instruction semantics, historical memory and business knowledge.

[0066] S3: An intent analysis agent constructed by a large language model, which uses a ReAct cycle mechanism to perform multi-step reasoning on the enhanced business token vector, generates a structured intermediate representation using a large language model, and outputs executable steps after verification by a constraint verifier.

[0067] This embodiment uses a large language model Qwen3, which is pre-trained in the intelligent robot inspection professional scene. The training data set contains more than 100,000 inspection instruction samples. The training phase of the large language model uses a hybrid strategy of imitation learning (IL) and reinforcement learning (RL):

[0068]

[0069]

[0070] wherein, is the time step, represents the model parameter at the time step t, the probability of generating the gold label under the given historical information, is the policy Down-sampled trajectories The obtained expected return, is a KL divergence regularization coefficient, is a divergence loss. If it is necessary to adapt to a specific scene (such as an underground parking lot), further fine-tuning is performed, and the above optimization can significantly improve the executability and safety of task decomposition.

[0071] A large language model is completed by using the pre-training, and an intent analysis intelligent agent Agent is constructed based on a LangChain framework scene . The analysis intelligent agent Agent scene receives the enhanced business token vector output by the receiving step S2, combines the current state of the intelligent robot , and starts a ReAct cycle. The ReAct cycle specifically includes the following links:

[0072] (1) Thinking: calling a Qwen3 model interface, analyzing the current instruction complexity (such as judging whether “exhibition hall fire-fighting facility inspection” includes a subtask), and outputting a thinking result (such as “it needs to be decomposed into three subtasks of ‘path navigation to exhibition hall -> identifying fire-fighting facilities -> checking facility status’ ”).

[0073] (2) Observation: searching a context vector library to verify the rationality of the thinking result (such as confirming that “exhibition hall fire-fighting facilities” include two types of fire extinguishers and fire hydrants, which need to be checked respectively), and supplementing subtask details. The process of verifying the rationality of the thinking result is specifically: using a large language model Qwen3 to generate a structured intermediate representation TG, the structured intermediate representation including task nodes, dependency relationships and constraint conditions, and being represented as:

[0074]

[0075] TG is checked by a constraint verifier to see whether it meets resource, power and path constraint conditions, and if a violation is found, abnormal information is returned to the model for rewriting until the verification is passed.

[0076] (3) Execution: generating a preliminary subtask step according to the observation result, and repeating the processes of (1) to (2) with the step as input until the subtask decomposition is executable and structured specific steps are output. The specific steps include step numbers, execution actions and judgment criteria, which can be directly called by the robot to execute subsequent modules.

[0077] In this embodiment, after one subtask is decomposed into executable specific steps, the output is as follows:

[0078] 1. Navigate from the current location to the exhibition hall entrance on the first floor;

[0079] 2. Start the visual recognition module to identify the fire extinguishers in the exhibition hall.

[0080] 3. Check if the pressure gauge of the fire extinguisher is in the normal range (0.8-1.2MPa);

[0081] 4. Record the status of the fire extinguisher;

[0082] 5. Check the fire hydrant in the exhibition hall in the same way;

[0083] 6. Summarize the results of all facility inspections.

[0084] In order to verify the effectiveness of the present application, comparative experiments were conducted on different instruction receiving and intention analysis methods including the present application in the same test environment, a number of representative indexes were selected to evaluate different methods, and the results are shown in Tables 1 and 2:

[0085] Table 1 Comparison of overall performance of instruction processing

[0086] Performance indicators Fixed rule-based scheme Single LLM parsing scheme The present invention Instruction reception success rate 72.5% 88.0% 98.5% Intention parsing accuracy 65.0% 82.5% 96.0% Complex instruction disassembly completeness 58.5% 75.0% 94.5% Historical information association accuracy 30.0% 55.0% 92.5%

[0087] Table 2 Comparison of voice module performance (signal-to-noise ratio 15dB)

[0088] Performance indicators Traditional Transformer model The present invention (DF-Conformer) Wake-up false trigger rate 5.2% 0.8% Speech transcription accuracy 84.5% 95.8% Wake-up response delay 450ms 280ms

[0089] The above experimental data show that the present application is significantly better than the traditional scheme in all core performance indicators, especially in handling complex instructions and associated historical information, which fully proves its advancement and practicality.

Claims

1. A method for receiving and parsing instructions for an intelligent robot, characterized in that, include: S1: Deploy the DF-Conformer model on the local device of the intelligent robot to monitor the ambient voice in real time, set wake-up keywords, and trigger the local device to enter the instruction receiving state when the model detects the wake-up keywords. S2: Construct standardized input by combining preset prompt word templates, and call the pre-trained bge-large model to convert the instruction text in the standardized input into a token vector; A two-stage retrieval enhancement mechanism is used to retrieve the token vector to obtain the enhanced business token vector; The two-stage retrieval enhancement mechanism includes: performing preliminary retrieval using the FAISS vector library, calculating accurate similarity scores through cross-encoder and performing vector fusion; S3: Construct an intent parsing agent using a large language model. The intent parsing agent receives the enhanced business token vector, starts a ReAct loop to decompose instructions, and outputs executable steps.

2. The instruction receiving and intent parsing method according to claim 1, characterized in that, The DF-Conformer model is trained using a multi-task joint approach, and the loss functions used include wake-up recognition loss, speech recognition loss, and contrastive learning loss; an adaptive noise enhancement strategy is introduced during the model training phase.

3. The instruction receiving and intent parsing method according to claim 1, characterized in that, The dimension of the token vector is the same as that of the pre-trained bge-large model.

4. The instruction receiving and intent parsing method according to claim 1, characterized in that, The preliminary retrieval using the FAISS vector library includes: constructing a context vector library using FAISS, including historical instructions and business knowledge; retrieving token vectors using the context vector library, associating recent historical memories with current business supplements; and retrieving Top-K candidates using cosine similarity.

5. The instruction receiving and intent parsing method according to claim 1, characterized in that, The vector fusion process includes: State-aware gating function Calculate fusion weights The formula is: In the formula, For accurate similarity scores, representing the cross-encoder's performance on the query vector. With the Candidate business token vectors Semantic matching score between them; This represents the output of the state-aware gating function, where `state` represents the current state information of the system. and The similarity score is controlled by either learnable hyperparameters or trainable scalar coefficients. and gating signals Relative importance in weight calculation; Generate enhanced business token vector The formula is: In the formula, the weights This is obtained after softmax normalization, representing the contribution ratio of each candidate token in the final fusion. and b represent the learnable linear transformation matrix and bias term, respectively. The representation layer normalization function.

6. The instruction receiving and intent parsing method according to claim 1, characterized in that, The large language model is pre-trained in a professional scenario of intelligent robot inspection, and the training data includes multiple inspection instruction samples; the training phase of the large language model adopts a hybrid strategy of imitation learning and reinforcement learning.

7. The instruction receiving and intent parsing method according to claim 1, characterized in that, The intent parsing agent is built on the LangChain framework, combining the enhanced business token vector with the current state of the intelligent robot. Start the ReAct loop.

8. The instruction receiving and intent parsing method according to claim 1, characterized in that, The ReAct loop includes: (1) Call the large language model interface to analyze the complexity of the current instruction and decompose it, and output the decomposition results; (2) Retrieve the context vector library, generate a structured intermediate representation through the large language model, and check whether the structured intermediate representation meets the resource, power and path constraints through the constraint validator. If it does not meet the constraints, return the abnormal information to the large language model for rewriting until it passes the verification. (3) Generate subtask steps based on the results of (2), and then use them as input to repeat (1) to (2) until the command is broken down into executable steps and the executable steps are output.

9. The instruction receiving and intent parsing method according to claim 1 or 8, characterized in that, The executable steps include a step number, an action to be performed, and a judgment criterion, which can be directly invoked by the intelligent robot.

10. A system for receiving and interpreting instructions for an intelligent robot, characterized in that, include: The speech recognition module is used to deploy the DF-Conformer model on the local device of the intelligent robot to monitor the ambient speech in real time, set wake-up keywords, and trigger the local device to enter the instruction receiving state when the model detects the wake-up keywords; The semantic retrieval module is used to construct standardized input by combining preset prompt word templates, and call the pre-trained bge-large model to convert the instruction text in the standardized input into a token vector; A two-stage retrieval enhancement mechanism is used to retrieve the token vector to obtain the enhanced business token vector; The two-stage retrieval enhancement mechanism includes: performing preliminary retrieval using the FAISS vector library, calculating accurate similarity scores through cross-encoder and performing vector fusion; The instruction parsing module is used to construct an intent parsing agent using a large language model. The intent parsing agent receives the enhanced business token vector, starts a ReAct loop to decompose the instruction, and outputs executable steps.

Images