Robot control method and device and robot
By identifying and converting user task instructions, optimizing algorithm parameters, and combining image and sensor data generation control strategies, the problem of low execution efficiency of robots in complex scenarios in the existing technology is solved, the execution efficiency and accuracy are improved, and the user is improved Experience.
Patent Information
- Application Number
- CN202410286909.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-13
- Publication Date
- 2024-11-12
AI Technical Summary
Existing robot control methods are difficult to apply to complex and flexible application scenarios, resulting in low execution efficiency and flexibility, which affects the user experience.
By identifying and converting user task instructions, target control instructions are generated, and algorithm parameters are optimized based on feedback information, and image information and sensor data are generated to improve the execution efficiency and accuracy of the robot in complex scenarios.
It improves the execution efficiency and flexibility of the robot in complex scenarios, enhances the accuracy and security of tasks, and improves the user experience.
Smart Images

Figure CN118927231A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of robots, and in particular, relates to a robot control method, device and robot. Background Art
[0002] Robots are widely used in home automation, industrial manufacturing, and service industries. Most of the related technologies rely on strictly predefined programs and instruction sets to control robots to perform corresponding actions, so that robots can only execute relatively simple operation instructions and cannot be applied to complex and flexible application scenarios. The execution efficiency and flexibility of robots are low, which affects the user experience. Summary of the invention
[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a robot control method, device and robot, which can be applied to complex and flexible application scenarios, improve the execution efficiency and flexibility of the robot, and improve the accuracy of the robot in executing tasks in subsequent applications, which helps to improve the user experience.
[0004] In a first aspect, the present application provides a robot control method, the method comprising:
[0005] Identify and convert the acquired user task instructions to obtain target control instructions;
[0006] Controlling the robot based on the target control instruction and obtaining feedback information;
[0007] Based on the feedback information, target algorithm parameters are optimized; the target algorithm parameters include at least one of a model parameter for identifying the user task instruction and a control strategy for controlling the robot based on the target control instruction.
[0008] According to the robot control method of the present application, by identifying and converting the acquired user task instructions, target control instructions that can be recognized by the robot are obtained, and the target algorithm parameters are optimized based on the feedback information corresponding to the execution results. This method can be applied to complex and flexible application scenarios, improve the execution efficiency and flexibility of the robot, and improve the accuracy of the robot's task execution in subsequent applications, which helps to improve the user experience.
[0009] According to an embodiment of the present application, the step of identifying and converting the acquired user task instruction to obtain the target control instruction includes:
[0010] Inputting the user task instruction into the large language model, and obtaining the intermediate instruction output by the large language model;
[0011] The intermediate instruction is converted to obtain the target control instruction.
[0012] According to an embodiment of the present application, the converting the intermediate instruction to obtain the target control instruction includes:
[0013] Standardizing the basic information included in the intermediate instruction to generate standardized data;
[0014] Based on the task information included in the intermediate instruction, matching a target function corresponding to the task information from a plurality of candidate functions;
[0015] The target control instruction is generated based on the standardized data and the target function.
[0016] According to one embodiment of the present application, when the target control instruction includes a plurality of sub-control instructions, controlling the robot based on the target control instruction includes:
[0017] Determining a target execution order of the plurality of sub-control instructions based on a priority level of each of the sub-control instructions;
[0018] storing the plurality of sub-control instructions into a task stack based on the target execution order;
[0019] When the target sub-control instruction among the plurality of sub-control instructions is executed based on the task stack, the target sub-control instruction is deleted from the task stack.
[0020] According to one embodiment of the present application, controlling the robot based on the target control instruction includes:
[0021] Generate a target control strategy based on the image information collected by the robot and the target control instruction;
[0022] Based on the target control strategy, the robot is controlled.
[0023] According to an embodiment of the present application, optimizing target algorithm parameters based on the feedback information includes:
[0024] The model parameters are optimized based on at least a portion of the feedback information within the first duration and a user task instruction corresponding to the feedback information.
[0025] According to an embodiment of the present application, optimizing target algorithm parameters based on the feedback information includes:
[0026] The control strategy is optimized based on at least part of the feedback information within a second duration and a target control instruction corresponding to the feedback information.
[0027] According to one embodiment of the present application, controlling the robot based on the target control instruction includes:
[0028] Predicting risk information based on sensor data collected by the robot and position information of the robot;
[0029] Based on the risk information, a target operation is performed.
[0030] In a second aspect, the present application provides a robot control device, the device comprising:
[0031] The first processing module is used to identify and convert the acquired user task instructions to obtain target control instructions;
[0032] A second processing module, used to control the robot based on the target control instruction and obtain feedback information;
[0033] The third processing module is used to optimize the target algorithm parameters based on the feedback information; the target algorithm parameters include at least one of the model parameters for identifying the user task instructions and the control strategy for controlling the robot based on the target control instructions.
[0034] According to the robot control device of the present application, the acquired user task instructions are identified and converted to obtain target control instructions that can be recognized by the robot, and the target algorithm parameters are optimized based on the feedback information corresponding to the execution results. It can be applied to complex and flexible application scenarios, improve the execution efficiency and flexibility of the robot, and improve the accuracy of the robot's task execution in subsequent applications, which helps to improve the user experience.
[0035] In a third aspect, the present application provides a robot, comprising:
[0036] Robotic arm;
[0037] As described in the second aspect, the robot control device is electrically connected to the robot arm.
[0038] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the robot control method as described in the first aspect above.
[0039] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the robot control method as described in the first aspect above.
[0040] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0041] By identifying and converting the acquired user task instructions, the target control instructions that the robot can recognize are obtained, and the target algorithm parameters are optimized based on the feedback information corresponding to the execution results. This can be applied to complex and flexible application scenarios, improve the execution efficiency and flexibility of the robot, and improve the accuracy of the robot's task execution in subsequent applications, which helps to improve the user experience.
[0042] Furthermore, by combining the visual perception results to determine the target control strategy to control the robot to perform the corresponding task, the efficiency and accuracy of the robot in performing the task can be improved.
[0043] Furthermore, by predicting risk information through sensor data, potential safety risks can be detected so that when safety risks are determined to exist, appropriate actions can be taken to avoid the risks, thereby improving the safety of robot operation and the service life of the robot.
[0044] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0046] Figure 1 This is one of the flowcharts of the robot control method provided in the embodiment of the present application;
[0047] Figure 2 This is the second flow chart of the robot control method provided in the embodiment of the present application;
[0048] Figure 3 is a schematic diagram of the structure of a robot control device provided in an embodiment of the present application;
[0049] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0051] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0052] The robot control method, robot control device, robot and readable storage medium provided in the embodiments of the present application are described in detail below through specific embodiments and their application scenarios in combination with the accompanying drawings.
[0053] Among them, the robot control method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.
[0054] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).
[0055] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.
[0056] The robot control method provided in the embodiment of the present application may be executed by a robot, or a functional module or functional entity in the robot that can implement the robot control method, or a device connected to the robot for communication. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The robot control method provided in the embodiment of the present application is described below using a robot as an example of the execution subject.
[0057] like Figure 1 As shown, the robot control method includes: step 110, step 120 and step 130.
[0058] Step 110: Identify and convert the acquired user task instructions to obtain target control instructions;
[0059] In this step, the user task instruction is an instruction used to control the robot to perform corresponding actions.
[0060] The user task instruction may be a voice instruction.
[0061] The user task instruction may be a simple word instruction such as "sweep the floor" or "mop the floor", or a more complex language instruction such as "clean out the trash can in the living room at 3 pm today".
[0062] The target control instruction is an instruction recognizable by the robot and is used to control the robot to perform actions corresponding to the user's task instruction.
[0063] In the actual execution process, conventional semantic recognition technology can be used to semantically recognize user task instructions and convert them into a language recognizable by the robot to obtain target control instructions.
[0064] Of course, in other embodiments, the user task instructions may also be other instructions besides voice instructions, such as instructions input through a human-computer interaction panel or action instructions collected by an image sensor, etc., which is not limited in this application.
[0065] Step 120: Control the robot based on the target control instruction and obtain feedback information;
[0066] In this step, the robot may be any type of robot, including but not limited to: a household robot, an industrial robot, or a robot used in the service field, etc.
[0067] The feedback information may include: the execution results returned by the robot when the robot is controlled to perform related tasks based on the target control instructions, and the evaluation results fed back by the user based on the results of the robot performing related tasks.
[0068] In some embodiments, the execution result may include execution completion or error reporting, etc.
[0069] Step 130: Optimize target algorithm parameters based on the feedback information.
[0070] In this step, the target algorithm parameters include at least one of a model parameter for identifying a user task instruction and a control strategy for controlling the robot based on the target control instruction.
[0071] The model corresponding to the model parameters can be any network model, such as a large language model.
[0072] The optimized target algorithm parameters can be used in the next robot control process.
[0073] For example, when the target algorithm parameter is a model parameter for identifying a user task instruction, the next time a user task instruction is received, semantic recognition of the user task instruction can be performed based on the optimized model.
[0074] For example, when the target algorithm parameters are used to control the control strategy of the robot based on the target control instructions, after the user task instructions are converted into target control instructions next time, the robot can be controlled to perform corresponding actions based on the target control instructions and the optimized target algorithm parameters.
[0075] In the present application, by identifying and converting the acquired user task instructions, the target control instructions that the robot can recognize are obtained, so that the robot can recognize complex user instructions, and thus can be applied to complex and flexible application scenarios, thereby improving the execution efficiency and flexibility of the robot and improving the user experience;
[0076] In addition, the target algorithm parameters are optimized through feedback information to achieve autonomous optimization and learning of the target algorithm parameters, so as to improve the accuracy of the robot's task execution in subsequent applications and further improve the user experience.
[0077] According to the robot control method provided in the embodiment of the present application, the acquired user task instructions are identified and converted to obtain target control instructions that can be recognized by the robot, and the target algorithm parameters are optimized based on the feedback information corresponding to the execution results. This method can be applied to complex and flexible application scenarios, improve the execution efficiency and flexibility of the robot, and improve the accuracy of the robot's task execution in subsequent applications, which helps to improve the user experience.
[0078] like Figure 2 As shown, in some embodiments, step 110 may include:
[0079] Input the user task instruction into the large language model and obtain the intermediate instruction output by the large language model;
[0080] The intermediate instructions are converted and processed to obtain the target control instructions.
[0081] In this embodiment, the large language model (LLM) is a deep learning model that uses advanced natural language processing technology and is trained based on massive text data. It can generate natural language text, deeply understand the meaning of the text, and handle various natural language tasks.
[0082] In some embodiments, the large language model may be a deep learning model, including but not limited to GPT or BERT.
[0083] Intermediate instructions are obtained by semantically recognizing user task instructions.
[0084] It is understandable that the intermediate instructions are a language in an intermediate state and cannot be directly recognized by the machine.
[0085] For example, for the user task instruction "Clean out the trash can in the living room at 3 o'clock this afternoon", the large language model can extract the keywords to obtain the intermediate instructions: "3 o'clock in the afternoon", "living room", "trash can" and "clean up" and other keywords used to represent time, place and tasks.
[0086] In some embodiments, the large language model can also identify and correct common language errors, such as colloquial expressions, grammatical errors, etc., to ensure correct understanding of the instructions.
[0087] The target control instruction is a language instruction that can be directly recognized by the machine after the intermediate instruction is standardized.
[0088] In some embodiments, converting the intermediate instruction to obtain the target control instruction may include:
[0089] Standardize the basic information included in the intermediate instructions to generate standardized data;
[0090] Based on the task information included in the intermediate instruction, a target function corresponding to the task information is matched from a plurality of candidate functions;
[0091] Based on the standardized data and the target function, target control instructions are generated.
[0092] In this embodiment, the basic information is attribute information used to characterize the task, including but not limited to: time and location.
[0093] Standardized data is data that can be recognized by the robot.
[0094] The task information is information related to the task to be performed, such as sweeping, dragging, grabbing, and opening a door.
[0095] The candidate functions are pre-set control instructions for executing different actions.
[0096] In the actual execution process, an association table between the task information and the candidate functions may be pre-established, so as to match the candidate functions corresponding to the task information based on the association table.
[0097] Continuing with the user task instruction "Clean the trash can in the living room at 3 o'clock this afternoon" as an example, the large language model can extract the keywords to obtain intermediate instructions: "3 o'clock in the afternoon", "living room", "trash can" and "clean up". After that, the time information can be further converted into "2023010115:00", and then search whether there is a living room in the map corresponding to the room and whether there is a trash can in the living room. When the trash can is found, match the ID corresponding to the trash can in the map to obtain the location information of the trash can, and then match the target function "sweep" corresponding to the task information "clean up" from the pre-set multiple candidate functions based on the task information "clean up", and map "clean up" to the "sweep" function.
[0098] In some embodiments, when the target control instruction includes multiple sub-control instructions, controlling the robot based on the target control instruction may include:
[0099] Determining a target execution order of the plurality of sub-control instructions based on the priority level of each sub-control instruction;
[0100] storing multiple sub-control instructions into a task stack based on the target execution order;
[0101] When the target sub-control instruction among the plurality of sub-control instructions is executed based on the task stack, the target sub-control instruction is deleted from the task stack.
[0102] In this embodiment, the priority level may be determined based on the importance of the task and the execution sequence.
[0103] The priority level may be user-defined or automatically generated by the system, and this application does not limit this.
[0104] The target execution order is the execution order of each sub-control instruction.
[0105] The task stack is a container for placing activity instances in the form of a stack, following the first-in-last-out principle, where activity instances correspond one-to-one to sub-control instructions.
[0106] The target sub-control instruction may be any sub-control instruction among the multiple sub-control instructions, and the target sub-control instruction is a sub-task instruction currently being executed in the task stack.
[0107] After the target sub-control instruction is executed, the target sub-control instruction may be deleted from the task stack.
[0108] During the actual execution process, the robot can count all the sub-control instructions to be executed the next day on the previous night, and then determine the execution order corresponding to each sub-control instruction to obtain the target execution order, and then store multiple sub-control instructions in the task stack based on the target execution order; on the second day, the robot executes each sub-control instruction in sequence based on the storage order of the task stack, and deletes a sub-control instruction from the task stack after executing it, until all sub-control instructions in the task stack are cleared.
[0109] In some embodiments, when a new sub-control instruction is added, the new sub-control instruction can be placed in a task stack to wait for the robot to execute until the robot completes executing all the sub-control instructions.
[0110] According to the robot control method provided in the embodiment of the present application, by storing sub-control instructions in the form of a task stack, it is possible to realize automatic task hosting, so that the robot can perform various tasks sequentially, improve the stability of the robot's operation and the orderliness of task execution, and further improve the level of intelligence.
[0111] Continue to refer Figure 2 In some embodiments, controlling the robot based on the target control instruction may include:
[0112] Generate target control strategy based on the image information and target control instructions collected by the robot;
[0113] Control the robot based on the target control strategy.
[0114] In this embodiment, the target control strategy is a specific task execution strategy corresponding to the target control instruction that the robot can execute, which is used to plan the robot's running path and the corresponding posture and other information at each point in the path.
[0115] Image information is used to represent environmental information, obstacles, mission objectives and other information during the operation of the robot.
[0116] In some embodiments, the environmental information may include lighting, complex background, etc.
[0117] The image information can be collected based on an image sensor arranged on the robot.
[0118] In the actual execution process, image information can be used for object recognition, environmental analysis, spatial positioning and semantic segmentation through image recognition, and based on the recognition results, combined with the target control instructions, the path and posture of the robot from the current position to the location of the task target are generated, and then the robot is controlled to run based on the determined path and achieve the corresponding posture at the corresponding position.
[0119] In some embodiments, generating a target control strategy based on the image information and target control instructions collected by the robot may also include:
[0120] Generate target control strategies based on image information, target control instructions and other sensor data.
[0121] In this embodiment, other sensor data may include but are not limited to tactile, sound, and infrared sensor data.
[0122] By combining visual data with other sensor data from other sensors to obtain the target control strategy, the precision and accuracy of the target control strategy can be further improved so that the robot can make more accurate decisions.
[0123] According to the robot control method provided in the embodiment of the present application, by combining the visual perception results to determine the target control strategy to control the robot to perform the corresponding task, the efficiency and accuracy of the robot in performing the task can be improved.
[0124] In some embodiments, the method may further include:
[0125] Perform feature recognition on the collected image information to obtain target features and position information corresponding to the target features;
[0126] Based on the target feature and the position information corresponding to the target feature, a target control strategy is obtained by matching from multiple control strategies.
[0127] In this embodiment, the target feature may be an object to be processed that is identified by the robot.
[0128] The control strategy may be a pre-set strategy for the robot to perform different actions or functions.
[0129] Taking a sweeping robot as an example, the control strategies may include sweeping, vacuuming, mopping, and navigating.
[0130] It is understandable that the strategies to be executed by the robot may vary depending on the target characteristics.
[0131] For example, when the target feature is melon seed shells, the implemented strategy may be sweeping the floor; when the target feature is water stains, the implemented strategy may be mopping the floor.
[0132] The position information is used to characterize the actual position of the target feature in the current scene.
[0133] During the actual execution process, if the robot recognizes that there are a bunch of melon seed shells in the collected image information, it can determine the melon seed shells as the target features and identify the location information of the melon seed shells. Then, the melon seed shells and the corresponding location information of the melon seed shells are sent to the large language model. The large language model matches the corresponding target control strategy based on the received information, such as matching the corresponding control strategy for sweeping the floor.
[0134] Among them, the large language model can be pre-trained to determine whether the received target features need to be processed, and if it is determined that processing is required, further match the corresponding control strategy.
[0135] According to the robot control method provided in the embodiment of the present application, by combining the visual perception results to collect the on-site environment image, the corresponding control strategy is automatically matched based on the image analysis results to control the robot to execute the relevant strategy without the need for the user to issue instructions, and has a higher degree of automation and intelligence.
[0136] Continue to refer Figure 2 In some embodiments, step 130 may include:
[0137] The model parameters are optimized based on at least part of the feedback information within the first duration and the user task instruction corresponding to the feedback information.
[0138] In this embodiment, the first duration can be based on user customization, such as being set to one month, half a year or one year, etc., which is not limited in this application.
[0139] Feedback information may include user evaluation information.
[0140] The user evaluation information is the user's evaluation information on the execution result of the robot executing the user task instruction, such as the cleaning effect or the satisfaction of coffee making.
[0141] For example, the robot collects user evaluation information given by users based on the execution results each time a user task instruction is executed. Half a year later, the robot combines the user evaluation information collected within half a year and the user task instructions corresponding to the user evaluation information to optimize the large language model in turn and improve the accuracy and precision of the model parameters, so that the intermediate instructions recognized by the large language model are closer to the user's true intentions.
[0142] According to the robot control method provided in the embodiment of the present application, the accuracy and precision of identifying complex user instructions can be improved by optimizing model parameters through feedback information, so that more complex or varied tasks can be performed, thereby broadening the application scenarios and functionality.
[0143] Continue to refer Figure 2 In some embodiments, step 130 may include:
[0144] The control strategy is optimized based on at least part of the feedback information within the second duration and the target control instruction corresponding to the feedback information.
[0145] In this embodiment, the second duration can be based on user customization, such as being set to one day, two days, or one week, etc., which is not limited in this application.
[0146] In some embodiments, the second duration may be shorter than the first duration.
[0147] Feedback information may include user evaluation information and error reporting information.
[0148] Error reporting information is information reported when the robot fails to execute the target control instruction.
[0149] The error reporting information may include: fault status, fault cause, and time information of the fault occurrence, etc.
[0150] For example, the error reporting information reported by the robot each time it fails to perform a task is collected. At the end of the day, data recognition and learning are performed on one or more collected error reporting information to optimize the control strategy to reduce the probability of task failure the next time the same task is performed.
[0151] For example, based on the user evaluation information of each task execution result, the user's preferences, such as the frequency of cleaning or the concentration of coffee, are learned to optimize the control strategy so that the final task execution result is closer to the user's preference.
[0152] According to the robot control method provided in the embodiment of the present application, the control strategy is optimized by feedback information during the robot's task execution, and autonomous optimization and learning of the control strategy are achieved. This can not only improve the accuracy and precision of the control strategy and reduce the error rate of subsequent task execution; it can also learn the user's preferences to make the control strategy closer to the user's preferences, achieve user personalization and stylized services, help improve the user experience, and have a high degree of intelligence.
[0153] Continue to refer Figure 2 In some embodiments, controlling the robot based on the target control instruction may further include:
[0154] Predict risk information based on the sensor data collected by the robot and the robot's posture information;
[0155] Based on the risk information, targeted actions are performed.
[0156] In this embodiment, the posture information includes the position information and posture information of the robot.
[0157] The sensor data may be data collected by any one or more sensors.
[0158] Sensor data may include, but is not limited to, image data, radar data, infrared data, audio information, and tactile data.
[0159] The risk information may be potential safety risk information, including but not limited to: collision risk, operation error risk and other risks.
[0160] The target operation may include but is not limited to: outputting warning information to the user, controlling the robot to suspend the execution of tasks, and other operations.
[0161] When the robot is executing the target control instruction, it can also collect sensor data in the environment in real time to identify and analyze the sensor data, such as collision analysis and error operation analysis, to determine whether there will be risks such as collision when the robot continues to execute the target control instruction based on the current target control strategy;
[0162] When it is predicted that there is risk information, corresponding measures can be taken based on the risk information to avoid the risk, such as controlling the robot to stop running when an imminent collision is detected.
[0163] When it is predicted that there is no risk information, the robot can be controlled to maintain the current target control strategy and continue to execute the target control instructions.
[0164] According to the robot control method provided in the embodiment of the present application, risk information is predicted by sensor data, so that potential safety risks can be detected so that when it is determined that there is a safety risk, corresponding operations can be taken to avoid the risk, thereby improving the safety of the robot's operation and the service life of the robot.
[0165] Continue to refer Figure 2 In the actual execution process, you can set Figure 2 Each module shown is used to execute corresponding steps to realize the control of the robot.
[0166] In the robot control method provided in the embodiment of the present application, the execution subject may be a robot control device. In the embodiment of the present application, a robot control device executing the robot control method is taken as an example to illustrate that the robot control device provided in the embodiment of the present application is provided.
[0167] An embodiment of the present application also provides a robot control device.
[0168] like Figure 3 As shown, the robot control device includes: a first processing module 310 , a second processing module 320 and a third processing module 330 .
[0169] The first processing module 310 is used to identify and convert the acquired user task instructions to obtain target control instructions;
[0170] The second processing module 320 is used to control the robot based on the target control instruction and obtain feedback information;
[0171] The third processing module 330 is used to optimize the target algorithm parameters based on the feedback information; the target algorithm parameters include at least one of the model parameters for identifying the user task instructions and the control strategy for controlling the robot based on the target control instructions.
[0172] According to the robot control device provided in the embodiment of the present application, the target control instructions that can be recognized by the robot are obtained by identifying and converting the acquired user task instructions, and the target algorithm parameters are optimized based on the feedback information corresponding to the execution results. It can be applied to complex and flexible application scenarios, improve the execution efficiency and flexibility of the robot, and improve the accuracy of the robot's task execution in subsequent applications, which helps to improve the user experience.
[0173] In some embodiments, the first processing module 310 may also be used to:
[0174] Input the user task instruction into the large language model and obtain the intermediate instruction output by the large language model;
[0175] The intermediate instructions are converted and processed to obtain the target control instructions.
[0176] In some embodiments, the first processing module 310 may also be used to:
[0177] Standardize the basic information included in the intermediate instructions to generate standardized data;
[0178] Based on the task information included in the intermediate instruction, a target function corresponding to the task information is matched from a plurality of candidate functions;
[0179] Based on the standardized data and the target function, target control instructions are generated.
[0180] In some embodiments, the second processing module 320 may also be used to:
[0181] Determining a target execution order of the plurality of sub-control instructions based on the priority level of each sub-control instruction;
[0182] storing multiple sub-control instructions into a task stack based on the target execution order;
[0183] When the target sub-control instruction among the plurality of sub-control instructions is executed based on the task stack, the target sub-control instruction is deleted from the task stack.
[0184] In some embodiments, the second processing module 320 may also be used to:
[0185] Generate target control strategy based on the image information and target control instructions collected by the robot;
[0186] Control the robot based on the target control strategy.
[0187] In some embodiments, the third processing module 330 may also be used to:
[0188] The model parameters are optimized based on at least part of the feedback information within the first duration and the user task instruction corresponding to the feedback information.
[0189] In some embodiments, the third processing module 330 may also be used to:
[0190] The control strategy is optimized based on at least part of the feedback information within the second duration and the target control instruction corresponding to the feedback information.
[0191] In some embodiments, the second processing module 320 may also be used to:
[0192] Predict risk information based on the sensor data collected by the robot and the robot's posture information;
[0193] Based on the risk information, targeted actions are performed.
[0194] The robot control device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.
[0195] The robot control device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0196] The robot control device provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0197] In some embodiments, the embodiments of the present application also provide a robot, comprising: a robotic arm and a robot control device as described in any of the above embodiments.
[0198] In this embodiment, the robot control device is electrically connected to the robot arm.
[0199] The robot control device is used to execute the robot control method described in any of the above embodiments.
[0200] According to the robot provided in the embodiment of the present application, the acquired user task instructions are identified and converted to obtain target control instructions that the robot can recognize, and the target algorithm parameters are optimized based on the feedback information corresponding to the execution results. This can be applied to complex and flexible application scenarios, improve the execution efficiency and flexibility of the robot, and improve the accuracy of the robot's task execution in subsequent applications, which helps to improve the user experience.
[0201] like Figure 4 As shown, an embodiment of the present application also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, the various processes of the above-mentioned robot control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0202] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0203] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned robot control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0204] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0205] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned robot control method when executed by a processor.
[0206] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0207] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned robot control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0208] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0209] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0210] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0211] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
[0212] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0213] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. A robot control method, It is characterized in that include: Identify and convert the acquired user task instructions to obtain target control instructions; Controlling the robot based on the target control instruction and obtaining feedback information; Based on the feedback information, target algorithm parameters are optimized; the target algorithm parameters include at least one of a model parameter for identifying the user task instruction and a control strategy for controlling the robot based on the target control instruction.
2. The robot control method according to claim 1, It is characterized in that The step of identifying and converting the acquired user task instructions to obtain target control instructions includes: Inputting the user task instruction into the large language model, and obtaining the intermediate instruction output by the large language model; The intermediate instruction is converted to obtain the target control instruction.
3. The robot control method according to claim 2, It is characterized in that The converting process of the intermediate instruction to obtain the target control instruction includes: Standardizing the basic information included in the intermediate instruction to generate standardized data; Based on the task information included in the intermediate instruction, matching a target function corresponding to the task information from a plurality of candidate functions; The target control instruction is generated based on the standardized data and the target function.
4. The robot control method according to any one of claims 1 to 3, It is characterized in that In the case where the target control instruction includes a plurality of sub-control instructions, controlling the robot based on the target control instruction includes: Determining a target execution order of the plurality of sub-control instructions based on a priority level of each of the sub-control instructions; storing the plurality of sub-control instructions into a task stack based on the target execution order; When the target sub-control instruction among the plurality of sub-control instructions is executed based on the task stack, the target sub-control instruction is deleted from the task stack.
5. The robot control method according to any one of claims 1 to 3, It is characterized in that The controlling the robot based on the target control instruction comprises: Generate a target control strategy based on the image information collected by the robot and the target control instruction; Based on the target control strategy, the robot is controlled.
6. The robot control method according to any one of claims 1 to 3, It is characterized in that The optimizing the target algorithm parameters based on the feedback information includes: The model parameters are optimized based on at least a portion of the feedback information within the first duration and a user task instruction corresponding to the feedback information.
7. The robot control method according to any one of claims 1 to 3, It is characterized in that The optimizing the target algorithm parameters based on the feedback information includes: The control strategy is optimized based on at least part of the feedback information within a second duration and a target control instruction corresponding to the feedback information.
8. The robot control method according to any one of claims 1 to 3, It is characterized in that The controlling the robot based on the target control instruction comprises: Predicting risk information based on sensor data collected by the robot and position information of the robot; Based on the risk information, a target operation is performed.
9. A robot control device, It is characterized in that include: The first processing module is used to identify and convert the acquired user task instructions to obtain target control instructions; A second processing module, used to control the robot based on the target control instruction and obtain feedback information; The third processing module is used to optimize the target algorithm parameters based on the feedback information; the target algorithm parameters include at least one of the model parameters for identifying the user task instructions and the control strategy for controlling the robot based on the target control instructions.
10. A robot, It is characterized in that include: Robotic arm; The robot control device according to claim 9, wherein the robot control device is electrically connected to the robot arm.
11. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the robot control method according to any one of claims 1 to 8 is implemented.
12. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the robot control method according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Task management method and system in robot
CN105786605A
Robot motion behavior control method and device and computer readable medium
CN113246134A
Robot self-collision detection method and system based on model prediction and robot
CN116985184A
Robot control method and device, electronic equipment and readable storage medium
CN117021073A
Model adjustment method and device, electronic equipment and readable storage medium
CN117532607A
Cited By
Urinary interrogation method and device based on LLM context embedding assistance
CN119811647A
Robot control method
WO2025190181A1
