Leg-arm robot autonomous motion sensing control method and system based on large model

Abstract

The invention provides a leg-arm robot autonomous motion sensing control method and system based on a large model, and relates to the technical field of robot motion control, and the method comprises the steps: obtaining environment information and a user voice instruction; identifying and analyzing the acquired environment information and the user voice instruction to obtain an analyzed text and a control signal; a control command, a control parameter and a feedback text are generated based on the analysis text and the control signal, the control parameter is transmitted to a four-foot single-arm robot motion controller according to the generated control command, and motion control over the four-foot single-arm robot is achieved; performing reverse speech synthesis according to the feedback text to realize speech feedback of the quadruped single-arm robot; according to the method, the natural language understanding and execution precision is improved, the autonomous learning and emotion interaction ability is enhanced, the adaptability and autonomy of the robot are improved, the compatibility and transportability are optimized, and cross-platform application is achieved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of robot motion control, and in particular to a method and system for autonomous motion perception control of a leg-arm robot based on a large model. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] With the rapid development of artificial intelligence technology, AI big models have become a key force in promoting the advancement of intelligent systems. Compared with traditional rule-based control systems, control systems based on AI big models can automatically extract features and optimize decision-making processes by learning large amounts of data, thereby making more accurate and flexible responses in uncertain environments.

[0004] Leg-arm robots have unique advantages in terms of passability and robust operation capabilities in complex terrain. Leg-arm robots can walk stably on rugged ground, stairs or other obstacles, and perform delicate operations through their arms, which is particularly important in scenarios such as search and rescue, field exploration, and construction. However, the control strategies of leg-arm robots such as quadrupeds with arms and humanoids are usually designed based on specific algorithms. These algorithms are often optimized for specific tasks and environments, but at the same time they also limit the robot's adaptability when facing unknown or changing environments. In addition, the interactive capabilities of traditional robots are usually limited to simple commands or preset interactive modes. They lack the ability to learn autonomously and transfer knowledge, and have difficulty understanding complex user intentions or natural language instructions. When encountering new or unforeseen situations, it is often difficult to respond flexibly. This limitation causes traditional leg-arm robots to perform poorly in application scenarios that require high autonomy, adaptability, and intelligent decision-making.

[0005] In summary, the existing control methods of leg-arm robots still have the following limitations:

[0006] 1) At present, the control strategies of quadruped-armed and humanoid leg-arm robots are usually designed based on specific algorithms. However, these specific algorithms are often optimized for specific tasks and environments, which also limits the robot's ability to adapt to unknown or changing environments, and its autonomous control and adaptability in unknown areas are insufficient;

[0007] 2) The interactive capabilities of traditional robots are usually limited to simple commands or preset interactive modes. They lack the ability to learn autonomously and transfer knowledge. They have difficulty parsing complex user intentions or natural language instructions, and thus have difficulty making flexible responses to unknown situations.

[0008] 3) Traditional leg-arm robots still have shortcomings in application scenarios that require a high degree of autonomy, adaptability and intelligent decision-making. Summary of the invention

[0009] In order to solve the above problems, the present disclosure proposes a leg-arm robot autonomous motion perception control method and system based on a large model. By integrating the AI ​​large model, voice commands are parsed and recognized to achieve natural language communication and voice command control between the leg-arm robot and the user, thereby improving the interaction between the user and the robot. The reasoning and decision-making ability of the AI ​​large model is used to achieve autonomous navigation, behavior control and autonomous operation of the quadruped single-arm robot. According to some embodiments, the present disclosure adopts the following technical solutions:

[0010] The autonomous motion perception and control method of the leg-arm robot based on a large model includes:

[0011] Obtain environmental information and user voice commands;

[0012] Recognize and analyze the acquired environmental information and user voice commands to obtain analyzed text and control signals;

[0013] Generate control commands, control parameters and feedback text based on the parsed text and the control signal, and transmit the control parameters to the motion controller of the quadruped single-arm robot according to the generated control commands to realize the motion control of the quadruped single-arm robot;

[0014] Reverse speech synthesis is then performed based on the feedback text to achieve speech feedback of the quadruped single-arm robot.

[0015] According to some embodiments, the present disclosure adopts the following technical solutions:

[0016] The autonomous motion perception control system of the leg-arm robot based on a large model includes:

[0017] The acquisition module is used to obtain environmental information and user voice commands;

[0018] The parsing module is used to recognize and parse the acquired environmental information and user voice commands to obtain parsed text and control signals;

[0019] A processing module, used for generating a control command, a control parameter and a feedback text based on the parsed text and the control signal;

[0020] A control module transmits control parameters to a motion controller of a quadruped single-arm robot according to a generated control command, thereby realizing motion control of the quadruped single-arm robot;

[0021] The output module is used to perform reverse speech synthesis according to the feedback text to realize the speech feedback of the quadruped single-arm robot.

[0022] According to some embodiments, the present disclosure adopts the following technical solutions:

[0023] A non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the autonomous motion perception control method of a leg-arm robot based on a large model is implemented.

[0024] According to some embodiments, the present disclosure adopts the following technical solutions:

[0025] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the large model-based leg-arm robot autonomous motion perception control method.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The disclosed large-model-based leg-arm robot autonomous motion perception and control method, by integrating AI large-model technology, improves the robot's semantic parsing capability for natural language commands and image information, enabling the robot to more accurately capture the user's intentions and perform corresponding actions or tasks, thereby reducing erroneous actions caused by misunderstandings and improving the success rate of task execution. Compared with traditional robots, it improves the natural language understanding and execution accuracy.

[0028] The disclosed large-scale model-based autonomous motion perception control method for leg-arm robots enables the robot to have autonomous learning and emotional interaction capabilities, providing a more personalized and emotional interactive experience. Over time, the robot can also optimize itself based on the user's behavior and preferences, further improving the level of personalized services and enhancing autonomous learning and emotional interaction capabilities.

[0029] The disclosed large-model-based autonomous motion perception control method for a leg-arm robot provides an intelligent leg-arm robot. Due to the intelligent behavior scheduling and multimodal perception capabilities based on the AI ​​large language model, the robot's autonomy and adaptability are improved, and it can better adapt to a variety of task scenarios, reduce dependence on manual intervention, and improve the independence and robustness of operations. The leg-arm robot achieves precise motion control and intelligent gait adjustment under AI scheduling, so that it has excellent autonomous adaptability and stability in complex terrains such as uneven ground and stairs. Combined with the flexibility and precision of its arms, it is particularly suitable for performing tasks in complex scenarios, such as rescue in ruins or exploration and construction in mining areas with rugged terrain and complex environments.

[0030] The disclosed large-model-based autonomous motion perception control method for leg-arm robots optimizes compatibility and portability and realizes cross-platform applications. By adopting modular design and standardized interfaces, the robot can be easily integrated with other robot platforms or remote control platforms to achieve rapid expansion of functions and diversification of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0032] Figure 1 It is a flow chart of the intelligent behavior method and system of a quadruped single-arm robot integrating an AI large model according to an embodiment of the present disclosure;

[0033] Figure 2 A schematic diagram of the mechanical structure of a quadruped single-arm robot according to an embodiment of the present disclosure;

[0034] Figure 3 It is a schematic diagram of the structure of the electrical and communication connections of the quadruped single-arm robot according to an embodiment of the present disclosure;

[0035] Figure 4 This is a flow chart of controlling a quadruped single-arm robot integrated with an AI large model according to an embodiment of the present disclosure;

[0036] Figure 5 This is an AI big model instruction parsing diagram of an embodiment of the present disclosure;

[0037] Figure 6 This is a structural block diagram of the leg-arm robot system that integrates the AI ​​large model according to an embodiment of the present disclosure.

[0038] Among them, 1. Torso (which contains the controller and voice transceiver system, the roll hip joint of each leg is also in the torso and connected to the pitch hip joint), 2. Joints and connecting rods of the robotic arm, 3. Robotic arm gripper, 4. LiDAR, 5. Depth camera, 6. Pitch hip joint, 7. Pitch knee joint, 8. Thigh connecting rod, 9. Calf connecting rod. DETAILED DESCRIPTION

[0039] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0040] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0041] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0042] Example 1

[0043] In one embodiment of the present disclosure, a large-model-based leg-arm robot autonomous motion perception control method is provided, comprising:

[0044] Obtain environmental information and user voice commands;

[0045] Recognize and analyze the acquired environmental information and user voice commands to obtain analyzed text and control signals;

[0046] Generate control commands, control parameters and feedback text based on the parsed text and the control signal, and transmit the control parameters to the motion controller of the quadruped single-arm robot according to the generated control commands to realize the motion control of the quadruped single-arm robot;

[0047] Reverse speech synthesis is then performed based on the feedback text to achieve speech feedback of the quadruped single-arm robot.

[0048] As an embodiment, the present disclosure proposes a leg-arm robot autonomous motion perception control method based on a large model, wherein: Figure 2 and Figure 3 As shown, the quadruped part of the quadruped single-arm robot is driven by electricity, has 12 degrees of freedom, includes a trunk 1 and four legs connected to the trunk, each leg includes a thigh link 8, a calf link 9 and three DC brushless motors, respectively, a roll hip joint (set inside the trunk 1 and connected to the pitch hip joint 6), a pitch hip joint 6 and a pitch knee joint 7. The quadruped single-arm robot includes a lithium battery, an actuator, a motion controller, a power management board, and an SPI signal adapter board. The robotic arm part is driven by 9 electric servos, has 7 degrees of freedom, includes a robotic arm gripper 3 and the joints and links 2 of the robotic arm, and uses a USB to RS485 to communicate with the controller.

[0049] The motion controller uses the UP Board to run the autonomous motion perception control method of the quadruped robot. The Ubuntu operating system is deployed on it, and the PREEMPT-RT patch is installed to achieve soft real-time operation. The SPI signal adapter board communicates with the controller via SPI, and then sends the controller instructions to the actuator via the CAN interface, and reads the status data fed back by the actuator and sends it to the controller.

[0050] In this embodiment, the large model control is mainly implemented for the quadruped single-arm robot. When a specific command is issued, the data acquisition module collects voice information and environmental information, and recognizes the voice command through the recognition function. If the command is of natural language interaction type, the natural language interaction between the quadruped single-arm robot and the user is realized, such as reciting poetry, giving life advice, etc.; if the command is of motion control type, the posture change of the quadruped single-arm robot from lying down to standing up, running fast to jogging, etc., the movement of the front and back, left and right, and the movement of the robot arm to draw a circle are realized; if the command is of autonomous operation type, the target and environment are analyzed, the navigation path and behavior planning are generated, and the operation task is executed. At the same time, after the operation is completed, the information is fed back through the broadcast module for the user to adjust.

[0051] Furthermore, the microphone, depth camera 5, and laser radar 4 are used to collect voice command information and environmental information, wherein the microphone is responsible for collecting sound, the depth camera is responsible for obtaining the depth image and RGB image of the environment, and the laser radar is responsible for obtaining the environmental point cloud data. The nuc mini computer is used as a processor to be responsible for the operation of the vision and large language model algorithms, and directly uses the existing large model, which can be Zero One Everything, Baidu Wenxin large model, etc., directly by calling the large model API, by inputting the prompt word, the large model parses and feedbacks the data, which is explained in detail in the next paragraph. It is also responsible for the control of the robotic arm. The Bluetooth speaker is used as the output of the voice broadcast, and the status of the voice recognition can be detected at any time. Different voice broadcasts are used for different recognition states, which is convenient for personnel debugging. The wireless module is connected to the wireless network of the main controller of the quadruped single-arm robot to achieve remote control, and a battery is set to provide energy input for the entire voice controller.

[0052] As an example, Figure 5 As shown, the specific process of the language large model control disclosed in the present invention is: when the commander gives a control voice command, the microphone user command is collected, and then the collected voice is recognized through the online semantic recognition function package, and the ros nodes of the camera and radar are turned on at the same time, and the camera and radar are used to collect environmental information for subsequent perception and navigation. The generated command text is sent to the large model by calling the large model API for online processing, and the control command, control parameters and feedback text are generated. According to the control command, the control parameters are passed to the quadruped single-arm robot motion controller through lcm to realize the movement of the quadruped single-arm robot. If the navigation and perception functions are needed, the image information and point cloud information are sent to the visual module for processing, and the image information is sent to the large model by calling the large model API for parsing to generate control commands and parsing text, and the control command is sent to the motion controller to realize the perception and autonomous navigation of the robot. At the same time, the feedback text is generated by calling the speech synthesis function package online to generate feedback voice, which is broadcast through the Bluetooth speaker.

[0053] As an embodiment, the single-arm robot disclosed in the present invention can realize natural language human-computer interaction. For example, when asked "Are you ready?", the robot will broadcast the answer "Ready, ready to go" after being processed by the large model. Since the large model introduces a certain degree of randomness, the robot's answers to the same question may be different. If you want the robot to respond to the user in a specific way, you can add a prompt template to the large language control algorithm. For example, after defining "You are a cyber bark", ask again "Are you ready?" The robot answers "Bark is ready, please indicate." Also note that when writing the prompt template, it is necessary to limit the robot's output json to not carry any punctuation marks to ensure that the feedback text can be broadcast smoothly.

[0054] As an embodiment, the quadruped single-arm robot disclosed in the present invention can realize motion control based on a large language model. It can be roughly divided into two categories: motion planning and autonomous operation. Motion planning includes basic movements of the robot, such as: moving forward, backward, turning left, turning right, standing up, lying down, etc.; gait switching, such as: walking, running, jogging, jumping, etc.; specific movements, such as: front flip, back flip, waving, bowing, etc. Autonomous operation includes navigation and obstacle avoidance (such as planning paths), body operation (such as load transportation), arm operation (such as clamping objects), leg-arm collaborative operation (such as material handling), etc.

[0055] Specifically, when the robot receives a user command, it will set the corresponding parameters, including mode, state, and response, according to the analysis of the AI ​​big model. The mode selects the control mode, that is, analyzes the command type, including simple dialogue, motion control, autonomous operation, etc.; the state selects the robot posture and motion parameters, and passes a label and required parameters according to the recognized command. The label and parameters are passed to the motion controller of the quadruped single-arm robot to realize the control of different motion states of the quadruped single-arm robot. The big language's feedback text for the command is stored in the response, which can be generated autonomously or manually by the big language, or the prompt template can be used to limit the big language's reply format and reply tone to meet the needs of certain specific scenarios.

[0056] When the large model controller recognizes a command with perception or navigation, it will call the camera and radar to collect environmental information and process it in the visual controller to achieve environmental perception and autonomous navigation. It should be noted that the user's commands do not have to be the same each time. The large language model will analyze the user's commands autonomously. For example, if the command "stand", "stand up", "stand at attention" is sent, the robot will recognize it as a command to change the standing posture, and the sent mode and state are the same.

[0057] like Figure 5As shown, specifically, when the robot recognizes that the voice command is a simple conversation, the AI ​​large model analyzes the user's voice and automatically sets the conversation flag, that is, mode is 0, state is none, and generates feedback text and stores it in the response. After voice synthesis, the feedback text is broadcast using a Bluetooth speaker to achieve human-computer interaction.

[0058] Among them, 1) When the command is parsed as a robot posture or gait change, such as giving the command "Stand up", after being parsed by the AI ​​large model, the standing flag is automatically set, that is, mode is 1, state is 1, and feedback text is generated and stored in the response. The parameters are passed to the motion controller, the robot state is changed to standing, and the content in the response is broadcast by voice. When the command is "Run", after being parsed by the AI ​​large model, the diagonal trot flag is automatically set, that is, mode is 1, state is 3, feedback text is generated and stored in the response, the parameters are passed to the motion controller, the robot state is changed to diagonal trot, and the content in the response is broadcast by voice. In addition to standing up and lying down, switching between different gaits can be switched directly without buffering. Users can also issue commands such as "speed up" and "lift your feet". After being parsed by the AI ​​large model, the robot does not change the current motion gait, but only changes parameters such as step frequency and step height to adapt to complex terrain.

[0059] 2) When the command is interpreted as an arm action display, such as when the command "wave" is given, the AI ​​large model will automatically set the mode to 1 and the state to 6 after analysis, generate feedback text and store it in the response, pass the parameters to the robotic arm controller, use MoveIt to plan the robotic arm's motion path, and use the underlying servo to realize the robotic arm's action, and voice broadcast the content in the response.

[0060] 3) When the command is interpreted as speed movement, such as "advance at a speed of one meter per second", after being parsed by the AI ​​large model, the mode is automatically set to 2, the state is {'x':1, 'z':0}, and the feedback text is generated and stored in the response. The parameters in the state: x-axis speed 1m / s, angular velocity 0 are passed to the motion controller, the robot state is changed to advance at a speed of one meter per second, and the content in the response is broadcasted by voice. Correspondingly, when the command is recognized as retreat, the x-axis speed is set to -1m / s, and the rest remains unchanged.

[0061] 4) When the voice command is other motion planning types, after being parsed by the AI ​​large model, mode and state are automatically set to the corresponding action label or motion parameter. For safety reasons, when sending speed parameters, the state label is not sent to the controller, that is, only the speed parameters are sent without changing the current state of the robot, ensuring that the robot only moves in the appropriate state machine.

[0062] 5) When the command is interpreted as object recognition, such as "see what's around", after being parsed by the AI ​​big model, the mode is automatically set to 4 and the state is none. The depth camera is called to collect RGB images and depth images and pass them to the visual controller. With the calibration algorithm, the semantic information and location information of the environment image are obtained, and then this information is sent to the AI ​​big model for parsing. The feedback text is stored in the response and broadcasted through voice. For example, in the office scene test, the robot replied, "There is a white table and two black chairs in front of the picture. There are two blue cups on the table. The environment is clean and tidy, and the light is bright. It is estimated that it is in the office area or other indoor environment."

[0063] 6) When the command is parsed as a navigation type, such as "navigate to target point 2, 1", after being parsed by the AI ​​large model, the mode is automatically set to 3, the state is {'x': 2, 'y': 1}, and the feedback text is generated and stored in the response. The parameters in the state: the x-axis coordinate 2 and the y-axis coordinate 1 of the robot starting position are passed to the visual controller, and the depth camera and radar are called to generate a planned path through the command navigation algorithm. If the speed is not specified, the robot moves toward the target point at the default speed of 1m / s, and the content in the response is broadcasted by voice.

[0064] 7) When the robot recognizes that there is complex terrain ahead during navigation, it will pass the terrain information collected by the depth camera and radar to the AI ​​large model for analysis, intelligently configure the robot's leg and foot movement mode, and can also combine the preset terrain adaptation algorithm to accurately plan and execute movement to the target point. For example, the large model recognizes that there are stairs ahead based on the terrain information passed back, and will call the visual controller to identify the height of the stairs and the travel distance. Combined with these terrain parameters, the AI ​​large model generates a motion strategy and intelligently adjusts the parameters in the motion controller to increase the step height, slow down the step frequency, and slow down the speed to ensure that the robot can go up and down the stairs smoothly. Similarly, when the robot recognizes that there is rugged terrain such as grass or gravel ahead, it can automatically adjust the motion parameters according to the motion strategy generated by the AI ​​large model to ensure the stability of the motion.

[0065] 8) When the instruction is parsed as a job type, such as "Go find the man in black and get the water cup", after being parsed by the AI ​​large model, the mode is automatically set to 4 and the state is set to 1. The parameters are passed to the visual controller, and the depth camera and radar are called. First, the location of the target is analyzed by parsing the environmental image, and feedback text is generated and stored in the response. Then, the planning path is generated through the command-based navigation algorithm. If there is no speed specified, the robot moves toward the target point at the default speed of 1m / s. When the robot reaches the target point, it turns to a standing posture. The AI ​​large model calls the visual controller to determine the target position, plans the gripping path according to the end position control, extends the robotic arm to complete the operation, and voice broadcasts the content in the response. In this experiment, the target is set behind a section of stairs and gravel, and obstacles such as cardboard boxes are placed on the road. The robot can still use the powerful recognition and parsing capabilities of the AI ​​large model and the flexible terrain adaptability of the legs and feet to show good working results in complex environments.

[0066] Example 2

[0067] In one embodiment of the present disclosure, a leg-arm robot autonomous motion perception control system based on a large model is provided, comprising:

[0068] The acquisition module is used to obtain environmental information and user voice commands;

[0069] The parsing module is used to recognize and parse the acquired environmental information and user voice commands to obtain parsed text and control signals;

[0070] A processing module, used for generating a control command, a control parameter and a feedback text based on the parsed text and the control signal;

[0071] A control module transmits control parameters to a motion controller of a quadruped single-arm robot according to a generated control command, thereby realizing motion control of the quadruped single-arm robot;

[0072] The output module is used to perform reverse speech synthesis according to the feedback text to realize the speech feedback of the quadruped single-arm robot.

[0073] Example 3

[0074] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the large-model-based leg-arm robot autonomous motion perception control method is implemented.

[0075] Example 4

[0076] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the large model-based leg-arm robot autonomous motion perception control method.

[0077] This disclosure successfully realizes the multimodal perception, natural language interaction and autonomous behavior control of the quadruped single-arm robot, significantly improving its autonomy and adaptability in complex environments. By integrating AI large model technology, the robot can understand and execute the user's natural language instructions, realize the natural language interaction between the quadruped single-arm robot and the user, and realize the functions of autonomous analysis of the environment and autonomous planning of the path from simple dialogue communication to complex action control. The solution also has good compatibility and can be easily transplanted to other robot platforms or remote control systems.

[0078] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0080] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A large-model-based autonomous motion perception and control method for a leg-arm robot, characterized in that: include: Obtain environmental information and user voice commands; Recognize and analyze the acquired environmental information and user voice commands to obtain analyzed text and control signals; Generate control commands, control parameters and feedback text based on the parsed text and the control signal, and transmit the control parameters to the motion controller of the quadruped single-arm robot according to the generated control commands to realize the motion control of the quadruped single-arm robot; Reverse speech synthesis is then performed based on the feedback text to achieve speech feedback of the quadruped single-arm robot.

2. The large-model-based autonomous motion perception control method for a leg-arm robot as claimed in claim 1, characterized in that: The four legs of the quadruped single-arm robot are driven by electricity and have multiple degrees of freedom, including a trunk and four legs connected to the trunk. Each leg includes three DC brushless motors, namely a roll hip joint, a pitch hip joint and a pitch knee joint.

3. The large-model-based autonomous motion perception control method for a leg-arm robot as claimed in claim 1, characterized in that: The robotic arm of the quadruped single-arm robot is driven by an electric servo, also includes multiple degrees of freedom and a gripper is set on the head.

4. The large-model-based autonomous motion perception control method for a leg-arm robot as claimed in claim 1, characterized in that: A microphone, a depth camera and a lidar are used to acquire environmental information and user voice commands. The microphone is used to collect user voice commands, the depth camera is used to acquire depth images and RGB images of the environment, and the lidar is used to acquire point cloud data in the environment.

5. The large-model-based autonomous motion perception control method for a leg-arm robot according to claim 1, characterized in that: A large speech model is used to identify and parse the acquired environmental information and user voice commands, and control commands are generated based on the parsed text and control signals. The control commands include motion planning and autonomous operation. Motion planning includes the robot's basic movements, gait switching and specific movements. The basic movements include moving forward, moving backward, turning left, turning right, standing up and lying down; gait switching includes walking, sprinting, jogging and jumping; the specific movements include front flips, back flips, waving and bowing; autonomous operations include environment recognition, object interaction, automatic obstacle avoidance and autonomous navigation.

6. The large-model-based autonomous motion perception control method for a leg-arm robot as claimed in claim 1, characterized in that: The large language model generates control commands and sets corresponding control parameters, including mode, state and response. Mode selects the control mode, that is, analyzes the command type, including simple dialogue, motion control and autonomous operation. State selects the robot posture and motion parameters. The label and required control parameters are passed according to the identified control command, and the label and control parameters are passed to the motion controller of the quadruped single-arm robot to realize the control of different motion states of the quadruped single-arm robot. The response stores the feedback text of the large language model for the control command.

7. The large-model-based autonomous motion perception control method for a leg-arm robot as claimed in claim 6, characterized in that: The feedback text is generated autonomously or manually by the big language model, and a prompt template can also be used to limit the reply format and reply tone of the big language to meet the needs of certain specific scenarios.

8. The autonomous motion perception control system of the leg-arm robot based on a large model is characterized by: include: The acquisition module is used to obtain environmental information and user voice commands; The parsing module is used to recognize and parse the acquired environmental information and user voice commands to obtain parsed text and control signals; A processing module, used for generating a control command, a control parameter and a feedback text based on the parsed text and the control signal; A control module transmits control parameters to a motion controller of a quadruped single-arm robot according to a generated control command, thereby realizing motion control of the quadruped single-arm robot; The output module is used to perform reverse speech synthesis according to the feedback text to realize the speech feedback of the quadruped single-arm robot.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the large-model-based leg-arm robot autonomous motion perception control method is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the large model-based leg-arm robot autonomous motion perception control method as described in any one of claims 1-7.

Images