Multilingual command processing method, device and equipment for vehicle-mounted external robot

Abstract

The application provides a multi-language command processing method, device and equipment for a vehicle-mounted external robot, and the method comprises the following steps: converting a user voice command into a user text with a language label by an ASR module; preprocessing the user text according to a preset text normalization rule to obtain a normalized text; calling an action rule table corresponding to the language label, matching the normalized text according to the action rule table to obtain a composite action sequence; removing non-compliant action items in the composite action sequence according to a preset safety policy to obtain an executable action sequence; adding the executable action sequence to a serial command queue; mapping each action in the serial command queue into a corresponding MCP control instruction in sequence and issuing the MCP control instruction to the vehicle-mounted external robot for serial execution. The application realizes accurate matching of multi-language commands and improves the execution safety of the vehicle-mounted external robot in a multi-language environment.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted robot technology, and in particular to a multilingual command processing method, apparatus, and device for vehicle-mounted external robots. Background Technology

[0002] With the continuous development of in-vehicle intelligent cockpit technology, external desktop robots, as a new type of interactive terminal, effectively enhance the vividness and fun of human-vehicle interaction through various actions such as nodding, waving, raising hands, clapping, dancing, and displaying facial expressions. Users typically give action commands to the robot via voice, such as "raise your left hand," "look to the right," or "wave your hand when angry." Due to the diversity of in-vehicle users, voice commands often involve multiple languages ​​such as Chinese, English, and Japanese. Furthermore, the voice recognition results suffer from a high false recognition rate due to factors such as in-vehicle noise and accents. The recognized text often contains colloquial modifiers, complex clauses, or omitted contextual elements, increasing the complexity of command parsing.

[0003] Existing voice control solutions typically pass speech recognition results directly to a language model for semantic understanding, and then drive the robot to execute commands based on the tool name, action parameters, or communication protocol fields output by the model. However, this approach has significant security flaws. The language model may generate action names that the device does not support, or misinterpret interrogative sentences, negative sentences, or descriptive statements as executable commands; for complex instructions involving multiple steps, existing solutions often only parse the last action, leading to the loss of the user's true intent; and during robot execution, if multiple actions are executed concurrently, it can easily cause overlapping servo motor commands or timing conflicts.

[0004] Therefore, existing voice control solutions for in-vehicle robots suffer from low execution security and poor command parsing accuracy. Summary of the Invention

[0005] This invention provides a multilingual command processing method, apparatus, and device for vehicle-mounted external robots, aiming to solve the problems of low execution security and poor command parsing accuracy in existing vehicle-mounted robot voice control schemes.

[0006] In a first aspect, embodiments of the present invention provide a multilingual command processing method for vehicle-mounted external robots, the method comprising: The ASR module transcribes user voice commands into user text with language tags. The user text is preprocessed according to preset text normalization rules to obtain normalized text; The action rule table corresponding to the language tag is invoked, and the normalized text is matched according to the action rule table to obtain a composite action sequence; According to the preset security policy, non-compliant action items in the composite action sequence are removed to obtain an executable action sequence. Add the executable action sequence to the serial command queue; The actions in the serial command queue are mapped sequentially to the corresponding MCP control instructions and then sent to the vehicle-mounted external robot for serial execution.

[0007] Secondly, embodiments of the present invention also provide a multilingual command processing device for vehicle-mounted external robots, the device comprising: The transcription unit is used to transcribe user voice commands into user text with language tags via the ASR module. The preprocessing unit is used to preprocess the user text according to preset text normalization rules to obtain normalized text; The matching unit is used to call the action rule table corresponding to the language tag, and match the normalized text according to the action rule table to obtain a composite action sequence; A security verification unit is used to remove non-compliant action items from the composite action sequence according to a preset security policy, so as to obtain an executable action sequence. An adding unit is used to add the executable action sequence to a serial command queue; The execution unit is used to map each action in the serial command queue into a corresponding MCP control instruction in sequence, and send it to the vehicle-mounted external robot for serial execution.

[0008] Thirdly, embodiments of the present invention also provide an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect above.

[0009] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, can implement the method described in the first aspect.

[0010] This invention provides a multilingual command processing method, apparatus, and device for vehicle-mounted external robots. The method includes: transcribing user voice commands into user text with language tags using an ASR module; preprocessing the user text according to preset text normalization rules to obtain normalized text; calling an action rule table corresponding to the language tags and matching the normalized text according to the action rule table to obtain a composite action sequence; removing non-compliant action items from the composite action sequence according to a preset safety policy to obtain an executable action sequence; adding the executable action sequence to a serial command queue; and sequentially mapping each action in the serial command queue to a corresponding MCP control instruction and issuing it to the vehicle-mounted external robot for serial execution. This invention achieves accurate matching of multilingual commands by calling the action rule table corresponding to the language tags and improves the execution safety of the vehicle-mounted external robot in a multilingual environment by removing non-compliant action items from the composite action sequence according to a preset safety policy. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a multilingual command processing method for vehicle-mounted external robots provided in an embodiment of the present invention. Figure 2 This is a schematic block diagram of a multilingual command processing device for vehicle-mounted external robots provided in an embodiment of the present invention. Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0015] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0016] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. Embodiments of this invention provide a multilingual command processing method, apparatus, and device for vehicle-mounted external robots. The method is applied to the server side of a vehicle-mounted robot control system, which also includes a vehicle-mounted external robot communicatively connected to the server. Specifically, the server opens a network port, and the vehicle-mounted external robot, acting as a WebSocket client, actively initiates a connection to the network port. After connection, the server drives the vehicle-mounted external robot to perform actions through the network port. The invention will now be described in detail through specific embodiments.

[0017] Figure 1 This is a flowchart illustrating a multilingual command processing method for vehicle-mounted external robots provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110-S160.

[0018] S110. The user's voice commands are transcribed into user text with language tags via the ASR module.

[0019] In this embodiment, the user's voice commands are transcribed into user text with language tags by the ASR (Automatic Speech Recognition) module. The language tags can be Chinese, English, or Japanese.

[0020] S120. Preprocess the user text according to preset text normalization rules to obtain normalized text.

[0021] In this embodiment, irrelevant characters such as wake words, filler words, punctuation, and spaces in the user text are first removed, and the text format is standardized (such as standardizing the capitalization of English letters). Then, interrogative sentences, negative sentences, and other expressions in the user text that should not trigger actions are filtered out to avoid mistaking ability inquiries, prohibition commands, ordinary descriptions, and small talk as action commands.

[0022] S130. Call the action rule table corresponding to the language tag, and match the normalized text according to the action rule table to obtain a composite action sequence.

[0023] In this embodiment, the action rule table includes matching keywords, optional parameters, and exclusion words corresponding to each preset action. Among them, the optional parameters are additional constraint fields for the execution details of the corresponding action. For example, if the action is "raise hand", the optional parameters can be specified as "raise left hand" or "raise both hands", etc. More specifically, if the optional parameters corresponding to a certain action are empty, the action is mapped to a specific action according to the default value. For example, the action "put down hand" is mapped to "put both hands down" according to the corresponding default value.

[0024] In one embodiment, step S130 includes: loading a corresponding action rule table according to the language tag; splitting the normalized text into multiple word and sentence fragments according to the compound delimiters in the normalized text; performing keyword matching on each word and sentence fragment according to the action rule table to obtain the robot action corresponding to each word and sentence fragment; wherein each robot action corresponds to an execution parameter; identifying emotion modifiers in each word and sentence fragment and converting them into corresponding facial expressions; wherein each facial expression corresponds to an expression parameter; and combining each robot action and each facial expression according to the order of the normalized text to obtain a composite action sequence.

[0025] In this embodiment, the compound delimiter can be "again", "next", "after", etc. Based on the compound delimiter in the normalized text, the normalized text is split into multiple word and sentence fragments. More specifically, if a compound delimiter is identified, but the normalized text contains only one word or sentence fragment, the normalized text is marked as low confidence and will not enter the subsequent execution path.

[0026] Based on the action rule table, keyword matching is performed on each of the word and phrase fragments to obtain the corresponding robot actions for each word and phrase fragment. Each robot action corresponds to an execution parameter, which defines the specific execution method of the robot action, including angle, direction, duration, etc. Emotion modifiers in each word and phrase fragment are identified and converted into corresponding facial expressions. Each facial expression corresponds to an expression parameter, which defines the specific execution method of the facial expression, such as duration, etc. The robot actions and facial expressions are combined according to the order of the normalized text to obtain a composite action sequence.

[0027] More specifically, the robot actions and facial expressions are combined according to the order of the standardized text to obtain a composite action sequence to be verified; the composite action sequence to be verified is subjected to legality verification, including the number of actions, action whitelist, prohibited actions, etc.; if the number of actions in the composite action sequence to be verified exceeds the preset maximum number of actions, or contains actions not registered in the whitelist, or contains action types prohibited from entering the sequence, then subsequent execution is refused and terminated; if all actions in the composite action sequence to be verified pass the legality verification, then the composite action sequence to be verified is recognized as a composite action sequence.

[0028] In one embodiment, after splitting the normalized text into multiple word and sentence fragments according to the compound delimiter in the normalized text, the method further includes: performing keyword matching on each word and sentence fragment according to the action rule table; if any word or sentence fragment cannot be matched with the corresponding robot action, the word and sentence fragment that failed to match the keyword is parsed according to the preset ASR misidentification fault tolerance mechanism to obtain the corresponding robot action.

[0029] In this embodiment, the ASR misidentification tolerance mechanism is triggered only when both "hand object" and "hand action" appear simultaneously. Specifically, the ASR misidentification tolerance mechanism is triggered when the word segment contains both hand action and hand object, or a hand misidentification substitute word, or a hand action ending word. Here, the hand misidentification substitute word refers to the misidentification result of hand object due to speech recognition error. For example, in a noisy environment, it is easy to misidentify "hands" as "mulberry tree". "Mulberry tree" is the hand misidentification substitute word, which is mapped to "hands" to achieve a fallback for speech misidentification.

[0030] S140. According to the preset security policy, non-compliant action items in the composite action sequence are removed to obtain an executable action sequence.

[0031] In this embodiment, non-compliant action items in the composite action sequence are removed according to a preset security policy. Non-compliant action items include, but are not limited to: unknown actions, actions that require servo drive when the current battery level is lower than a preset battery level threshold, actions that exceed the hardware capability limit, and actions that occur after the time interval between two adjacent similar actions is less than a preset interval threshold. After removing non-compliant action items from the composite action sequence, an executable action sequence is obtained.

[0032] In one embodiment, after step S140, the method further includes: determining whether the executable action sequence meets a preset restriction rule and obtaining a determination result; if the determination result is yes, adding the executable action sequence to a serial command queue and attaching a sequence number to each action in the executable action sequence; if the determination result is no, terminating the subsequent execution of the executable action sequence.

[0033] In this embodiment, the limiting rules include, but are not limited to: the number of actions does not exceed the preset maximum number of actions, the sequence is not empty, all actions are registered in the whitelist, there are no action types that are prohibited from entering the sequence, there is no sequence currently being executed, and the execution parameters of each action do not exceed the hardware capability limit; If the executable action sequence meets the preset restriction rules, the executable action sequence is added to the serial command queue, and a sequence number is added to each step of the executable action sequence; if the executable action sequence does not meet the preset restriction rules, the subsequent execution of the executable action sequence is terminated.

[0034] S150. Add the executable action sequence to the serial command queue.

[0035] In this embodiment, the serial command queue adopts a first-in-first-out scheduling mechanism, and is consumed serially by an independent dedicated execution thread. Only one control command is scheduled to be issued and executed at the same time, thus preventing overlapping servo actions caused by multiple parallel commands from the scheduling level.

[0036] S160. Map each action in the serial command queue to the corresponding MCP control instruction in sequence, and send them to the vehicle-mounted external robot for serial execution.

[0037] In this embodiment, after the current MCP (Model Context Protocol) control instruction is completed, the next MCP control instruction is retrieved from the serial command queue, and an independent execution thread is started for this instruction, which is then sent to the hardware driver layer of the vehicle-mounted external robot for execution. Only one execution thread is allowed to be running at any given time. The scheduling process for the next MCP control instruction is only started after the current MCP control instruction is completed and the corresponding execution thread exits, thus avoiding hardware driver conflicts and abnormal action superposition caused by multiple parallel instructions.

[0038] Before step S160, the procedure further includes: performing a lightweight safety check on each action in the serial command queue before it is sent to the hardware for execution; the safety check includes at least a current emergency stop status check and a battery power status check; by adding this check step, abnormal changes that occur during the action queuing period are captured; if the current state is an emergency stop or the battery power is lower than a preset execution threshold, the check is determined to have failed and the current action is terminated; if the check passes, the current action is mapped to the corresponding MCP control command and sent to the vehicle-mounted external robot for execution.

[0039] In one embodiment, after step S160, the method further includes: an action execution process, wherein if a new executable action sequence is detected, the new executable action sequence is rejected from being added to the serial command queue, and the serial execution of the current executable action sequence is maintained.

[0040] In this embodiment, during the action execution process, if a new executable action sequence is detected, the new executable action sequence is rejected from being added to the serial command queue, and the serial execution of the current executable action sequence is maintained. This prevents multiple sequences from queuing in parallel and being executed in an interleaved manner, ensuring the determinism of the action execution timing and the security of hardware operation.

[0041] In one embodiment, after step S160, the method further includes: detecting in real time whether the execution duration of the current action is greater than the preset execution duration; if the execution duration of the current action is detected to be greater than the preset execution duration, exiting the current action; after the current action exits, starting the next action.

[0042] In this embodiment, after the current MCP control instruction is completed, the next MCP control instruction is retrieved from the serial command queue, and an independent execution thread is started for this instruction, which is then sent to the hardware driver layer of the vehicle-mounted external robot for execution. The execution time of the current execution thread is monitored in real time to see if it exceeds the preset execution time. If the execution time of the current execution thread is detected to be greater than the preset execution time, the current execution thread is exited. After the current execution thread exits, the scheduling process of the next MCP control instruction is started to avoid abnormal action superposition.

[0043] In one embodiment, after step S160, the method further includes: when a stop command is detected, triggering a highest priority preemption mechanism to clear the serial command queue and terminate the currently executing action.

[0044] In this embodiment, when a disconnection or connection switch occurs, a stop command is automatically generated, triggering the highest priority preemption mechanism, clearing the serial command queue and terminating the currently executing action (such as robot actions or facial expressions) to avoid action residue.

[0045] In one embodiment, after step S160, the method further includes: when a current sequence cancellation instruction is received, obtaining a target sequence number according to the current sequence cancellation instruction, and comparing the target sequence number with the sequence number of the current executable action sequence; when the target sequence number is consistent with the sequence number of the current executable action sequence, and the current executable action sequence is being executed, canceling the current executable action sequence; specifically, the current executable action sequence can be canceled by clearing the serial command queue and terminating the currently executing action.

[0046] In summary, this invention achieves precise matching of multilingual commands by calling the action rule table corresponding to the language tag; and removes non-compliant action items from the composite action sequence according to a preset security policy, thereby improving the execution security of the vehicle-mounted external robot in a multilingual environment.

[0047] Figure 2 This is a schematic block diagram of a multilingual command processing device for vehicle-mounted external robots provided in an embodiment of the present invention. Figure 2 As shown, corresponding to the above-described multilingual command processing method for vehicle-mounted external robots, this invention also provides a multilingual command processing device for vehicle-mounted external robots. The device is configured on the server side of the vehicle-mounted robot control system. The system also includes a vehicle-mounted external robot that is communicatively connected to the server. Specifically, the server opens a network port, and the vehicle-mounted external robot, acting as a WebSocket client, actively initiates a connection to this network port. After connection, the server drives the vehicle-mounted external robot to perform actions through the network port. For details, please refer to [link to relevant documentation]. Figure 2 The multilingual command processing device 700 for vehicle-mounted external robots includes: The transcription unit 701 is used to transcribe user voice commands into user text with language tags via the ASR module. The preprocessing unit 702 is used to preprocess the user text according to preset text normalization rules to obtain normalized text; The matching unit 703 is used to call the action rule table corresponding to the language tag, and match the normalized text according to the action rule table to obtain a composite action sequence; The security verification unit 704 is used to remove non-compliant action items from the composite action sequence according to a preset security policy to obtain an executable action sequence. Adding unit 705 is used to add the executable action sequence to the serial command queue; The execution unit 706 is used to map each action in the serial command queue into a corresponding MCP control instruction in sequence, and send it to the vehicle-mounted external robot for serial execution.

[0048] In some embodiments, when the matching unit 703 executes the step of calling the action rule table corresponding to the language tag, matching the normalized text according to the action rule table, and obtaining a composite action sequence, it is specifically used for: Load the corresponding action rule table according to the language tag; split the normalized text into multiple word and sentence fragments according to the compound delimiters in the normalized text; perform keyword matching on each word and sentence fragment according to the action rule table to obtain the robot action corresponding to each word and sentence fragment; wherein, each robot action corresponds to an execution parameter; identify the emotion modifiers in each word and sentence fragment and convert them into corresponding facial expressions; wherein, each facial expression corresponds to an expression parameter; combine each robot action and each facial expression according to the order of the normalized text to obtain a composite action sequence.

[0049] In some embodiments, after performing the step of splitting the normalized text into multiple word / sentence fragments based on the compound delimiters in the normalized text, the matching unit 703 is further configured to: The keyword matching of each word and phrase fragment is performed according to the action rule table; if any word or phrase fragment cannot be matched with the corresponding robot action, the word or phrase fragment that failed to match the keyword is parsed according to the preset ASR misidentification fault tolerance mechanism to obtain the corresponding robot action.

[0050] In some embodiments, after performing the step of removing non-compliant action items from the composite action sequence according to a preset security policy to obtain an executable action sequence, the security verification unit 704 is further configured to: Determine whether the executable action sequence meets the preset restriction rules and obtain the determination result; if the determination result is yes, add the executable action sequence to the serial command queue and add a sequence number to each step of the executable action sequence; if the determination result is no, terminate the subsequent execution of the executable action sequence.

[0051] In some embodiments, after executing the sequential mapping of each action in the serial command queue to the corresponding MCP control instructions and issuing them to the vehicle-mounted external robot for serial execution, the execution unit 706 is further configured to: During the action execution process, if a new executable action sequence is detected, the new executable action sequence is rejected from being added to the serial command queue, and the serial execution of the current executable action sequence is maintained.

[0052] In some embodiments, after executing the sequential mapping of each action in the serial command queue to the corresponding MCP control instructions and issuing them to the vehicle-mounted external robot for serial execution, the execution unit 706 is further configured to: The system continuously monitors whether the execution time of the current action exceeds the preset execution time. If the execution time of the current action exceeds the preset execution time, the current action is exited. After the current action is exited, the next action is started.

[0053] In some embodiments, after executing the sequential mapping of each action in the serial command queue to the corresponding MCP control instructions and issuing them to the vehicle-mounted external robot for serial execution, the execution unit 706 is further configured to: When a stop command is detected, the highest priority preemption mechanism is triggered, the serial command queue is cleared, and the currently executing action is terminated.

[0054] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned multilingual command processing device for vehicle-mounted external robots and its various units can be referred to the corresponding descriptions in the foregoing method embodiments. For the sake of convenience and brevity, these details will not be repeated here.

[0055] The aforementioned multilingual command processing device for vehicle-mounted external robots can be implemented as a computer program, which can be used in, for example... Figure 3 It runs on the electronic device shown.

[0056] Please see Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. The electronic device 800 can be a terminal or a server. The terminal can be an electronic device with communication functions. The server can be a standalone server or a server cluster composed of multiple servers.

[0057] See Figure 3 The electronic device 800 includes a processor 802, a memory, and a network interface 805 connected via a system bus 801. The memory may include a non-volatile storage medium 803 and internal memory 804.

[0058] The non-volatile storage medium 803 may store an operating system 8031 ​​and a computer program 8032. The computer program 8032 includes program instructions that, when executed, cause the processor 802 to perform a multilingual command processing method for an onboard external robot.

[0059] The processor 802 provides computing and control capabilities to support the operation of the entire electronic device 800.

[0060] The internal memory 804 provides an environment for the operation of the computer program 8032 in the non-volatile storage medium 803. When the computer program 8032 is executed by the processor 802, the processor 802 can execute a multilingual command processing method for vehicle-mounted external robots.

[0061] This network interface 805 is used for network communication with other devices. Those skilled in the art will understand that... Figure 3The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the electronic device 800 to which the present invention is applied. The specific electronic device 800 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0062] The processor 802 is used to run a computer program 8032 stored in the memory to perform the following steps: The ASR module transcribes user voice commands into user text with language tags; the user text is preprocessed according to preset text normalization rules to obtain normalized text; the action rule table corresponding to the language tag is called, and the normalized text is matched according to the action rule table to obtain a composite action sequence; non-compliant action items in the composite action sequence are removed according to a preset security policy to obtain an executable action sequence; the executable action sequence is added to a serial command queue; each action in the serial command queue is mapped to the corresponding MCP control instruction in sequence and sent to the vehicle-mounted external robot for serial execution.

[0063] In some embodiments, when the processor 802 implements the step of calling the action rule table corresponding to the language tag, matching the normalized text according to the action rule table, and obtaining a composite action sequence, the specific implementation is as follows: Load the corresponding action rule table according to the language tag; split the normalized text into multiple word and sentence fragments according to the compound delimiters in the normalized text; perform keyword matching on each word and sentence fragment according to the action rule table to obtain the robot action corresponding to each word and sentence fragment; wherein, each robot action corresponds to an execution parameter; identify the emotion modifiers in each word and sentence fragment and convert them into corresponding facial expressions; wherein, each facial expression corresponds to an expression parameter; combine each robot action and each facial expression according to the order of the normalized text to obtain a composite action sequence.

[0064] In some embodiments, after implementing the step of splitting the normalized text into multiple word / sentence fragments based on the compound delimiters in the normalized text, the processor 802 further implements the following steps: The keyword matching of each word and phrase fragment is performed according to the action rule table; if any word or phrase fragment cannot be matched with the corresponding robot action, the word or phrase fragment that failed to match the keyword is parsed according to the preset ASR misidentification fault tolerance mechanism to obtain the corresponding robot action.

[0065] In some embodiments, after the processor 802 implements the step of removing non-compliant action items from the composite action sequence according to a preset security policy to obtain an executable action sequence, it further implements the following steps: Determine whether the executable action sequence meets the preset restriction rules and obtain the determination result; if the determination result is yes, add the executable action sequence to the serial command queue and add a sequence number to each step of the executable action sequence; if the determination result is no, terminate the subsequent execution of the executable action sequence.

[0066] In some embodiments, after the processor 802 sequentially maps each action in the serial command queue to the corresponding MCP control instruction and issues it to the onboard external robot for serial execution, it also performs the following steps: During the action execution process, if a new executable action sequence is detected, the new executable action sequence is rejected from being added to the serial command queue, and the serial execution of the current executable action sequence is maintained.

[0067] In some embodiments, after the processor 802 sequentially maps each action in the serial command queue to the corresponding MCP control instruction and issues it to the onboard external robot for serial execution, it also performs the following steps: The system continuously monitors whether the execution time of the current action exceeds the preset execution time. If the execution time of the current action exceeds the preset execution time, the current action is exited. After the current action is exited, the next action is started.

[0068] In some embodiments, after the processor 802 sequentially maps each action in the serial command queue to the corresponding MCP control instruction and issues it to the onboard external robot for serial execution, it also performs the following steps: When a stop command is detected, the highest priority preemption mechanism is triggered, the serial command queue is cleared, and the currently executing action is terminated.

[0069] It should be understood that, in this embodiment of the invention, the processor 802 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0070] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0071] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the following steps: The ASR module transcribes user voice commands into user text with language tags; the user text is preprocessed according to preset text normalization rules to obtain normalized text; the action rule table corresponding to the language tag is called, and the normalized text is matched according to the action rule table to obtain a composite action sequence; non-compliant action items in the composite action sequence are removed according to a preset security policy to obtain an executable action sequence; the executable action sequence is added to a serial command queue; each action in the serial command queue is mapped to the corresponding MCP control instruction in sequence and sent to the vehicle-mounted external robot for serial execution.

[0072] In one embodiment, when the processor executes the program instructions to call the action rule table corresponding to the language tag, and matches the normalized text according to the action rule table to obtain a composite action sequence, the specific steps are as follows: Load the corresponding action rule table according to the language tag; split the normalized text into multiple word and sentence fragments according to the compound delimiters in the normalized text; perform keyword matching on each word and sentence fragment according to the action rule table to obtain the robot action corresponding to each word and sentence fragment; wherein, each robot action corresponds to an execution parameter; identify the emotion modifiers in each word and sentence fragment and convert them into corresponding facial expressions; wherein, each facial expression corresponds to an expression parameter; combine each robot action and each facial expression according to the order of the normalized text to obtain a composite action sequence.

[0073] In one embodiment, after executing the program instructions to split the normalized text into multiple word / sentence fragments based on compound delimiters in the normalized text, the processor further performs the following steps: The keyword matching of each word and phrase fragment is performed according to the action rule table; if any word or phrase fragment cannot be matched with the corresponding robot action, the word or phrase fragment that failed to match the keyword is parsed according to the preset ASR misidentification fault tolerance mechanism to obtain the corresponding robot action.

[0074] In one embodiment, after the processor executes the program instructions to remove non-compliant action items from the composite action sequence according to a preset security policy to obtain an executable action sequence, it further performs the following steps: Determine whether the executable action sequence meets the preset restriction rules and obtain the determination result; if the determination result is yes, add the executable action sequence to the serial command queue and add a sequence number to each step of the executable action sequence; if the determination result is no, terminate the subsequent execution of the executable action sequence.

[0075] In one embodiment, after the processor executes the program instructions to sequentially map each action in the serial command queue to the corresponding MCP control instructions and issues them to the onboard robot for serial execution, it further performs the following steps: During the action execution process, if a new executable action sequence is detected, the new executable action sequence is rejected from being added to the serial command queue, and the serial execution of the current executable action sequence is maintained.

[0076] In one embodiment, after the processor executes the program instructions to sequentially map each action in the serial command queue to the corresponding MCP control instructions and issues them to the onboard robot for serial execution, it further performs the following steps: The system continuously monitors whether the execution time of the current action exceeds the preset execution time. If the execution time of the current action exceeds the preset execution time, the current action is exited. After the current action is exited, the next action is started.

[0077] In one embodiment, after the processor executes the program instructions to sequentially map each action in the serial command queue to the corresponding MCP control instructions and issues them to the onboard robot for serial execution, it further performs the following steps: When a stop command is detected, the highest priority preemption mechanism is triggered, the serial command queue is cleared, and the currently executing action is terminated.

[0078] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0079] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0080] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0081] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0082] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0083] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multilingual command processing method for vehicle-mounted external robots, characterized in that, The method includes: The ASR module transcribes user voice commands into user text with language tags. The user text is preprocessed according to preset text normalization rules to obtain normalized text; The action rule table corresponding to the language tag is invoked, and the normalized text is matched according to the action rule table to obtain a composite action sequence; According to the preset security policy, non-compliant action items in the composite action sequence are removed to obtain an executable action sequence. Add the executable action sequence to the serial command queue; The actions in the serial command queue are mapped sequentially to the corresponding MCP control instructions and then sent to the vehicle-mounted external robot for serial execution.

2. The multilingual command processing method for vehicle-mounted external robots according to claim 1, characterized in that, The step involves calling the action rule table corresponding to the language tag, matching the normalized text according to the action rule table, and obtaining a composite action sequence, including: Load the corresponding action rule table based on the language tag; Based on the compound delimiters in the normalized text, the normalized text is split into multiple word and sentence fragments; Based on the action rule table, keyword matching is performed on each of the word and phrase fragments to obtain the robot action corresponding to each word and phrase fragment; wherein, each robot action corresponds to an execution parameter; Identify the emotion modifiers in each of the aforementioned word and sentence fragments and convert them into corresponding facial expressions; wherein each facial expression corresponds to an expression parameter; The robot actions and facial expressions are combined according to the order of the standardized text to obtain a composite action sequence.

3. The multilingual command processing method for vehicle-mounted external robots according to claim 2, characterized in that, After splitting the normalized text into multiple word / sentence fragments based on the compound delimiters in the normalized text, the process further includes: Keyword matching is performed on each of the aforementioned word and phrase fragments according to the action rule table; If any word or phrase fragment cannot be matched with the corresponding robot action, the word or phrase fragment that failed to match the keyword is parsed according to the preset ASR misidentification fault tolerance mechanism to obtain the corresponding robot action.

4. The multilingual command processing method for vehicle-mounted external robots according to claim 1, characterized in that, After removing non-compliant action items from the composite action sequence according to a preset security policy to obtain an executable action sequence, the method further includes: Determine whether the executable action sequence meets preset restriction rules, and obtain the determination result; If the determination result is yes, the executable action sequence is added to the serial command queue, and a sequence number is added to each step of the executable action sequence; If the determination result is negative, the subsequent execution of the executable action sequence is terminated.

5. The multilingual command processing method for vehicle-mounted external robots according to claim 1, characterized in that, After mapping each action in the serial command queue to its corresponding MCP control instruction and issuing it to the onboard robot for serial execution, the process further includes: During the action execution process, if a new executable action sequence is detected, the new executable action sequence is rejected from being added to the serial command queue, and the serial execution of the current executable action sequence is maintained.

6. The multilingual command processing method for vehicle-mounted external robots according to claim 1, characterized in that, After mapping each action in the serial command queue to its corresponding MCP control instruction and issuing it to the onboard robot for serial execution, the process further includes: Real-time detection of whether the execution time of the current action exceeds the preset execution time; If the execution time of the current action is detected to be longer than the preset execution time, exit the current action; After the current action is completed, the next action is started.

7. The multilingual command processing method for vehicle-mounted external robots according to claim 1, characterized in that, After mapping each action in the serial command queue to its corresponding MCP control instruction and issuing it to the onboard robot for serial execution, the process further includes: When a stop command is detected, the highest priority preemption mechanism is triggered, the serial command queue is cleared, and the currently executing action is terminated.

8. A multilingual command processing device for vehicle-mounted external robots, characterized in that, The device includes: The transcription unit is used to transcribe user voice commands into user text with language tags via the ASR module. The preprocessing unit is used to preprocess the user text according to preset text normalization rules to obtain normalized text; The matching unit is used to call the action rule table corresponding to the language tag, and match the normalized text according to the action rule table to obtain a composite action sequence; A security verification unit is used to remove non-compliant action items from the composite action sequence according to a preset security policy, so as to obtain an executable action sequence. An adding unit is used to add the executable action sequence to a serial command queue; The execution unit is used to map each action in the serial command queue into a corresponding MCP control instruction in sequence, and send it to the vehicle-mounted external robot for serial execution.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multilingual command processing method for vehicle-mounted external robots as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which includes program instructions that, when executed by a processor, cause the processor to perform the multilingual command processing method for vehicle-mounted external robots as described in any one of claims 1-7.

Images