Mechanical arm pose control method and related device

Abstract

The application provides a mechanical arm pose control method and related equipment. The mechanical arm pose control method comprises: performing format processing on initial data input by a user to obtain prompt data; inputting the prompt data into a generative pre-training model to obtain a prediction script; when the number of times of currently generating the prediction script reaches a preset number of times, verifying the syntax of the prediction script to obtain a syntax verification result of the prediction script; and when the syntax verification result is verified, determining that the prediction script is a target script, wherein the target script is used to control the pose of the mechanical arm. The application relates to the technical field of script generation, and can improve the accuracy of generating a script for controlling the pose of a mechanical arm.

Description

Technical Field

[0001] This application relates to the field of robotic arm control technology, specifically to the field of robotic arm pose control technology, and particularly to a robotic arm pose control method and related equipment, wherein the related equipment includes a robotic arm pose control device, electronic equipment and storage medium. Background Technology

[0002] With the development of the manufacturing industry, more and more manufacturers are inclined to use mechanized assembly line operations to increase production capacity. In mechanized assembly line operations, robotic arms are typically used to complete a series of operations such as material handling or product assembly. Before the robotic arm can operate, electronic devices are usually used to run robotic arm control scripts to control its posture and thus achieve different functions. Currently, developers typically write the scripts required for robotic arm operation using scripting languages, or use graphical interfaces to drag and drop configurations of the robotic arm's operation process. This approach is limited by the developers' coding abilities and suffers from low accuracy in the generated robotic arm control scripts. Summary of the Invention

[0003] In view of the above, it is necessary to propose a robotic arm pose control method and related equipment to solve the technical problem of low accuracy in writing robotic arm control scripts. Robotic Arm Pose Control This application provides a robotic arm pose control method applied to an electronic device, wherein the electronic device is communicatively connected to the robotic arm. The method includes: formatting initial data input by a user to obtain prompt data; inputting the prompt data into a generative pre-trained model to obtain a prediction script; verifying the syntax of the prediction script when the number of times the prediction script has been generated reaches a preset number, and obtaining a syntax verification result of the prediction script; when the syntax verification result is successful, determining the prediction script as a target script, wherein the target script is used to control the pose of the robotic arm.

[0004] In some embodiments, the method further includes: when the number of times the prediction script has been generated has not reached a preset number, determining updated prompt data based on pre-stored question information and the prediction script; continuing to input the updated prompt data into the generative pre-trained model to obtain a prediction script, until the number of times the prediction script has been generated reaches the preset number.

[0005] In some embodiments, when the syntax verification result is a verification failure, the method further includes: constructing updated prompt data based on the verification result, the prediction script, and pre-stored prompt information; and inputting the updated prompt data into the generative pre-trained model.

[0006] In some embodiments, the method for training the generative pre-trained model includes: formatting pre-stored training data to obtain training hint data; inputting the training hint data into a pre-stored initial generative model to obtain a training prediction script corresponding to the training hint data; verifying the syntax of the training prediction script to obtain a syntax verification result of the training prediction script; determining a reward value for the training prediction script based on the syntax verification result; determining updated training hint data based on the reward value, the syntax verification result, and the training prediction script; and continuing to input the updated training hint data into the initial generative model until a preset termination condition is met to obtain the generative pre-trained model trained to a convergent state.

[0007] In some embodiments, determining the reward value of the training prediction script based on the verification result includes: when the syntax verification result is a successful verification, determining the reward value of the training prediction script as a first score; when the verification result is a failed verification, determining the reward value of the training prediction script as a second score; wherein the first score is greater than the second score.

[0008] In some embodiments, determining the updated prompt data based on the reward value, the syntax verification result, and the training prediction script includes: filling the reward value, the syntax verification result, and the training prediction script into a pre-stored prompt template to obtain the updated prompt data.

[0009] In some embodiments, the preset termination condition includes: the number of times the updated training hint data is obtained reaches a preset number of iterations; or the difference between the training prediction script and the sample script recorded in the training data is less than a preset termination threshold.

[0010] This application embodiment also provides a robotic arm pose control device, the device comprising: a formatting module for formatting initial data input by a user to obtain prompt data; a generation module for inputting the prompt data into a generative pre-trained model to obtain a prediction script; a verification module for verifying the syntax of the prediction script when the number of times the prediction script has been generated reaches a preset number, and obtaining a syntax verification result of the prediction script; the verification module is further configured to determine the prediction script as a target script when the syntax verification result is a pass, wherein the target script is used to control the pose of the robotic arm.

[0011] This application also provides an electronic device, which includes: a memory for storing at least one instruction; and a processor for executing the instructions stored in the memory to implement the robotic arm pose control method.

[0012] This application also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the robotic arm pose control method.

[0013] As can be seen from the above technical solutions, the embodiments of this application format the initial data in various formats input by the user to obtain prompt data in a unified format, and ensure that the prompt data contains as much effective information as possible, thereby improving the accuracy of the prediction script generated by the subsequent generative pre-training model. The prompt data is input into the generative pre-training model to obtain the prediction script. When the number of times the prediction script is generated reaches a preset number, the syntax of the prediction script is verified. Thus, the prompt data is continuously updated based on the generated prediction script, and the prediction script is regenerated based on the updated prompt data. This allows for the adjustment of parameters in the generative pre-training model, thereby improving the accuracy of the prediction script generated by the generative pre-training model. Finally, when the syntax verification result is passed, the prediction script is determined to be the target script for controlling the pose of the robotic arm, ensuring that there are no syntax errors in the target script, which further improves the accuracy of the generated target script. Attached Figure Description

[0014] Figure 1 This is an application scenario diagram of the robotic arm pose control method provided in one embodiment of this application.

[0015] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0016] Figure 3 This is a flowchart of a robotic arm pose control method provided in an embodiment of this application.

[0017] Figure 4 This is a flowchart of a method for training a generative pre-trained model provided in an embodiment of this application.

[0018] Figure 5 This is a functional block diagram of a robotic arm posture control device provided in an embodiment of this application. Detailed Implementation

[0019] To better understand the purpose, features, and advantages of this application, a detailed description of the application is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Numerous specific details are set forth in the following description to provide a thorough understanding of this application; the described embodiments are only a part of the embodiments of this application, and not all of them.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] This application provides a robotic arm pose control method that can be applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0023] Electronic devices can be any electronic product that allows human-computer interaction with a customer, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0024] Electronic devices may also include network devices and / or client devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0025] The networks in which electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0026] like Figure 1As shown, the robotic arm script generation method provided in this application can be applied to an electronic device 100, which is communicatively connected to a robotic arm 200. The electronic device 100 receives initial data input by the user, generates a script for controlling the pose of the robotic arm 200 based on the initial data, and runs the script, causing the robotic arm 200 to move to a pose corresponding to the point information in the initial data, so that the robotic arm can perform a preset operation at that pose (e.g., grasping a fixture located in a preset area).

[0027] In one embodiment of this application, the electronic device 100 may be an external electronic device that is communicatively connected to the robotic arm 200, or it may be the central control device of the robotic arm itself. This application does not limit it in this respect.

[0028] Please see Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 includes a memory 120 and a processor 130. The memory 120 is used to store computer-readable instructions, and the processor 130 is used to execute the computer-readable instructions stored in the memory 120 to implement the robotic arm pose control method described in any of the above embodiments.

[0029] In one embodiment of this application, the electronic device 100 further includes a bus and a computer program stored in the memory 120 and executable on the processor 130, such as a robotic arm pose control program.

[0030] Figure 2 Only an electronic device 100 with a memory 120 and a processor 130 is shown. It will be understood by those skilled in the art that... Figure 2 The structure shown does not constitute a limitation on the electronic device 100, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0031] The memory 120 includes at least one type of readable storage medium, which can be non-volatile or volatile. The readable storage medium includes flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 120 can be an internal storage unit of the electronic device 100, such as the portable hard drive of the electronic device 100. In other embodiments, the memory 120 can also be an external storage device of the electronic device 100, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 100. The memory 120 can be used not only to store application software and various types of data installed on the electronic device 100, such as the code of a robotic arm posture control program, but also to temporarily store data that has been output or will be output.

[0032] In one embodiment of this application, a collector 121, a generative pre-trained model 122, a repeater 123, and a compiler 124 run in the memory 120. The collector 121, generative pre-trained model 122, repeater 123, and compiler 124 are all pre-written programs or collections of programs. Specifically, the collector 121 formats the initial data input by the user and outputs prompt data. The initial data includes a point table, a variable table, and a prompt template. The generative pre-trained model 122 outputs a corresponding prediction script based on the prompt data. To improve the accuracy of the generated prediction script, the generative pre-trained model 122 can be used to generate the prediction script multiple times. The repeater 123 is used to determine the number of times the generative pre-trained model 122 generates the prediction script. When this number is less than a preset number (e.g., 3 times), it indicates that the repeater 123 has not been used to generate the prediction script multiple times, and the accuracy of the generated prediction script may be low. Therefore, the repeater 123 can be used to determine the updated prompt data based on the pre-stored question information and the prediction script, and continue to input the updated prompt data into the generative pre-trained model 122 to obtain the prediction script. When the repeater 123 determines that the number of times the prediction script has been generated has reached the preset number, it indicates that the generative pre-trained model 122 has generated the prediction script multiple times, and therefore the parameters in the generative pre-trained model 122 have been adjusted multiple times, so the accuracy of the currently generated prediction script is high. The compiler 124 can be used to determine whether there are syntax errors in the prediction script. When a syntax error is found in the prediction script, the prompt data can be updated according to the syntax error, and the generative pre-trained model 122 can be used to regenerate the prediction script based on the updated prompt data. When no syntax error is found in the prediction script, the prediction script can be determined as the target script. The electronic device 100 can run the target script, thereby controlling the pose of the robotic arm 200 according to the target script.

[0033] In some embodiments, the processor 130 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 130 is the control unit of the electronic device 100, connecting to various components of the electronic device 100 via various interfaces and lines. It executes programs or modules stored in the memory 120 (e.g., executing a robotic arm pose control program) and calls data stored in the memory 120 to perform various functions and process data of the electronic device 100.

[0034] The processor 130 executes the operating system of the electronic device 100 and various installed applications. The processor 130 executes the applications to implement the steps in the above-described embodiments of the robotic arm pose control methods.

[0035] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 120 and executed by the processor 130 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device 100. For example, the computer program may be divided into a collector 121, a generative pre-trained model 122, a repeater 123, and a compiler 124.

[0036] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the robotic arm pose control methods described in the various embodiments of this application.

[0037] If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0038] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, and other memory.

[0039] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0040] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, in... Figure 2 The symbol is represented by only one arrow, but this does not indicate that there is only one bus or one type of bus. The bus is configured to enable communication between the memory 120 and at least one processor 130, etc.

[0041] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 100 and does not constitute a limitation on the electronic device 100. The electronic device 100 may be a bus-type structure or a star-type structure. The electronic device 100 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, the electronic device 100 may also include input / output devices, network access devices, etc.

[0042] It should be noted that electronic device 100 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0043] like Figure 3 The diagram shown is a flowchart of a robotic arm pose control method according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The robotic arm pose control method provided in this embodiment includes the following steps.

[0044] S30: Obtain initial data and format the initial data to obtain prompt data.

[0045] In one embodiment of this application, in order to generate an accurate script to accurately control the pose of the robotic arm, the user can first input initial data into an electronic device, causing the electronic device to generate a script corresponding to the initial data, and then run the script to control the pose of the robotic arm. The initial data includes a point table, a variable table, and a prompt template. The point table contains information including data for controlling the pose of the robotic arm, representing the need to control the pose of the robotic arm; the variable table contains information including data for controlling various components of the robotic arm (e.g., the gripper of the robotic arm), representing the need to control each component of the robotic arm; the prompt template contains formatted text data (e.g., JSON formatted text data), used to convert the information in the point table and variable table into formatted prompt data. This prompt data can represent the need to adjust the pose of the robotic arm and can be input data that a generative pre-trained model can understand.

[0046] In one embodiment of this application, the point table can be a table that records point names and point coordinates, with each point name corresponding to a point coordinate. Specifically, the point coordinates can be in the form of a six-dimensional vector; for example, the point coordinates can be... , where x, y, and z are used to represent the coordinates of the pose of the robotic arm at a certain point in three-dimensional space; An angle used to characterize the angle between the line connecting the end effector of the robotic arm and the center of the robotic arm base and the x-axis. An angle used to characterize the angle between the line connecting the end effector of the robotic arm and the center of the robotic arm base and the y-axis. The angle used to characterize the angle between the line connecting the end of the robotic arm and the center of the robotic arm base and the z-axis.

[0047] In one embodiment of this application, the variable table can also be a table, which includes variable names and variable values, with each variable name corresponding to a variable value. Specifically, the contents recorded in the variable table are used to characterize the control requirements of the robotic arm. For example, when the variable name is "gripper switch" and the variable value corresponding to that variable name is 1, it indicates a need to open the gripper of the robotic arm.

[0048] In one embodiment of this application, the prompt template can be JSON-formatted text data, and the prompt template includes one or more of the following prompt information: background prompts, instruction prompts, and constraints. The background prompts characterize the context of the data operation requirement; for example, the background prompt information could be "You are an intelligent assistant controlling the pose of a robotic arm, possessing intent understanding capabilities." The instruction prompt information characterizes the operation to be completed and the output result; for example, the instruction prompt information could be "Generate a script to control the pose of the robotic arm based on the point table and variable table." The constraint information characterizes the constraints on the output data; for example, the constraint information could be "maximum number of bytes output," "output language," "output format," etc.

[0049] For example, when the information in the point table includes "{[starting point, coordinate 1], [transfer point, coordinate 2], [grabbing point, coordinate 3], [ending point, coordinate 4]}", and the information in the variable table includes "{[variable 1, variable value 1], [variable 2, variable value 2], [variable 3, variable value 3], [variable 4, variable value 4]}", then the prompt data could be: Background information: You are an intelligent assistant that controls the pose of a robotic arm and has the ability to understand intent.

[0050] Instruction prompt: Generate a script to control the pose of the robotic arm based on the position table and variable table.

[0051] The point table includes: {[starting point, coordinate 1], [transfer point, coordinate 2], [grabbing point, coordinate 3], [ending point, coordinate 4]}; The variable table includes: {[variable1, variable value1], [variable2, variable value2], [variable3, variable value3], [variable4, variable value4]}; Constraint: Output the robotic arm control script.

[0052] In one embodiment of this application, since the prompt template is in JSON format, the prompt data obtained after filling the point table and variable table with data from the prompt template can also be in JSON format. This unifies the format of all initial data, obtaining a format that the generative pre-trained model can understand, facilitating the generation of prediction scripts based on the prompt data using the generative pre-trained model.

[0053] S31, Input the prompt data into the generative pre-trained model to obtain the prediction script.

[0054] In one embodiment of this application, the generative pre-trained model can be any model with script generation capabilities. For example, the generative pre-trained model can be a GPT model (General Pre-Trained Transform model), which is used to output corresponding responses based on input data.

[0055] In one embodiment of this application, the generative pre-trained model can be pre-trained to a convergent state to ensure that the generative pre-trained model can accurately understand the semantics of the information recorded in the prompt data and improve the accuracy of the output prediction script. Specifically, for the method of training the generative pre-trained model, please refer to [link to relevant documentation]. Figure 4 The corresponding detailed description.

[0056] S32, determine the number of times the prediction script is generated.

[0057] In one embodiment of this application, a generative pre-trained model can be used to generate prediction scripts corresponding to the prompt data multiple times, thereby ensuring that the parameters in the generative pre-trained model are adjusted multiple times according to the prompt data, and thus improving the accuracy of the prediction scripts output by the generative pre-trained model.

[0058] In one embodiment of this application, to ensure that the generative pre-trained model generates prediction scripts corresponding to the prompt data a sufficient number of times, the number of times the generative pre-trained model generates prediction scripts based on the prompt data can be pre-recorded. Subsequently, the accuracy of the prediction scripts input to the generative pre-trained model can be evaluated based on the number of prediction script generation counts. It should be noted that the initial value of the number of prediction script generation counts can be 0, and the count increases by 1 each time the generative pre-trained model generates a prediction script.

[0059] S33, determine whether the current number of times the prediction script has been generated has reached a preset number; if the current number of times the prediction script has been generated has reached the preset number, execute step S34; if the current number of times the prediction script has not reached the preset number, execute step S35.

[0060] In one embodiment of this application, when the number of times the prediction script is generated reaches a preset number (e.g., 3 times, 5 times, or 7 times), it indicates that the generative pre-trained model has repeatedly generated the prediction script. Therefore, the parameters in the generative pre-trained model have been adjusted multiple times, and the accuracy of the currently generated prediction script is high. Therefore, step S34 can be executed to determine whether the syntax of the prediction script is qualified.

[0061] In one embodiment of this application, when the number of times the prediction script is generated is less than a preset number, it indicates that the generative pre-trained model generates the prediction script less frequently, which means that the accuracy of the currently generated prediction script is low. Therefore, step S35 can be executed to generate the prediction script multiple times, thereby improving the accuracy of the prediction script generated by the generative pre-trained model.

[0062] S34, verify the syntax of the prediction script, obtain the syntax verification result of the prediction script, and determine whether the syntax verification result is a pass. If the syntax verification result is a pass, proceed to step S36; if the syntax verification result is a fail, proceed to step S37.

[0063] In one embodiment of this application, a pre-stored validator can be used to verify the syntax of the prediction script. The pre-stored validator can be a pre-written program or set of programs with syntax verification functionality. When the syntax verification result of the prediction script passes, it indicates that the prediction script has no syntax errors, and the electronic device can run the prediction script to control the pose of the robotic arm. When the syntax verification result fails, it indicates that the prediction script has syntax errors, and the electronic device needs to update the prompt data based on the syntax verification failure information, obtain the updated prompt data, and regenerate the prediction script based on the updated prompt data.

[0064] In one embodiment of this application, verifying the syntax of the prediction script and obtaining the syntax verification result of the prediction script specifically includes: segmenting the prediction script to obtain multiple code words in the prediction script; determining the part-of-speech of each code word using a pre-stored text classification model; for any two adjacent code words, determining the word that appears earlier in the prediction script as the first word and the word that appears later in the prediction script as the second word; determining the first node corresponding to the part-of-speech of the first word from a pre-stored syntax tree; when the second node corresponding to the part-of-speech of the second word is a child node of the first node, determining that the first word and the second word do not have syntax errors; when it is determined that all two adjacent code words in the prediction script do not have syntax errors, determining that the syntax verification result of the prediction script is verified as passed; when it is determined that any two adjacent code words in the prediction script have syntax errors, determining that the syntax verification result of the prediction script is verified as failed.

[0065] For example, when the prediction script is "Get YouBot Handle", the part-of-speech tag for "Get" is determined to be part-of-speech tag 1, for "YouBot" it is determined to be part-of-speech tag 2, and for "Handle" it is determined to be part-of-speech tag 3. In the pre-stored syntax tree, if part-of-speech tag 1 corresponds to the first node, and the second node corresponding to part-of-speech tag 2 is a child node of the first node, then "Get" and "YouBot" have no syntax errors. Similarly, if part-of-speech tag 2 corresponds to the first node, and the second node corresponding to part-of-speech tag 3 is a child node of that first node, then "YouBot" and "Handle" also have no syntax errors. Therefore, the syntax verification result of the prediction script can be determined as successful.

[0066] S35, update the prompt data to obtain the updated prompt data and execute step S31.

[0067] In one embodiment of this application, when the number of times the prediction script is generated has not reached a preset number, it indicates that the generative pre-trained model generates prediction scripts less frequently, the parameters in the generative pre-trained model are adjusted less frequently, and therefore the accuracy of the currently generated prediction script is low.

[0068] In one embodiment of this application, in order to improve the accuracy of the generated prediction script, the updated prompt data can be determined based on the pre-stored question information and the prediction script; the updated prompt data is then input into the generative pre-trained model to obtain the prediction script, until the number of times the prediction script has been generated reaches the preset number.

[0069] The pre-stored question information is used to represent the need for the generative pre-trained model to self-check the prediction script and regenerate it. The pre-stored question information can be text data in JSON format. For example, the pre-stored question information could be "Please confirm whether the prediction script is correct, please regenerate the prediction script".

[0070] S36, determine that the prediction script is the target script, and control the pose of the robotic arm by executing the target script.

[0071] In one embodiment of this application, when the syntax verification result of the prediction script passes, it indicates that the prediction script has no syntax errors. Therefore, it is determined that the target script can be successfully executed by the electronic device and used to adjust the pose of the robotic arm. Thus, the electronic device can run the target script to control the pose of the robotic arm, adjusting the pose of the robotic arm to the pose recorded in the position table.

[0072] S37. When the syntax verification result is that the verification fails, update the prompt data to obtain the updated prompt data, and re-execute step S31.

[0073] In one embodiment of this application, when the syntax verification result is "verification failed," it indicates that the prediction script contains syntax errors. The electronic device then needs to update the prompt data based on the "verification failed" information, obtain the updated prompt data, and regenerate the prediction script based on the updated prompt data. Specifically, the updated prompt data can be constructed based on the verification result, the prediction script, and pre-stored prompt information; the updated prompt data is then input into the generative pre-trained model. The pre-stored prompt information can be JSON-formatted text data, used to convert the verification result and prediction script into a format that the generative pre-trained model can understand.

[0074] For example, when the validation result is "Validation failed, syntax error exists", and the predicted script is {script A}, the updated prompt data could be: "The validation result for the currently generated prediction script: {script A} is that the validation failed due to a syntax error. Please regenerate the prediction script."

[0075] Thus, by indicating a syntax error in the prediction script generated by the generative pre-trained model based on the updated prompt data, the generative pre-trained model can promptly correct errors in the prediction script generation process, thereby adjusting the parameters in the generative pre-trained model and further improving the accuracy of the generated prediction script.

[0076] As can be seen from the above technical solutions, the embodiments of this application format the initial data in various formats input by the user to obtain prompt data in a unified format, and ensure that the prompt data contains as much effective information as possible, thereby improving the accuracy of the prediction script generated by the subsequent generative pre-training model. The prompt data is input into the generative pre-training model to obtain the prediction script. When the number of times the prediction script is generated reaches a preset number, the syntax of the prediction script is verified. Thus, the prompt data is continuously updated based on the generated prediction script, and the prediction script is regenerated based on the updated prompt data. This allows for the adjustment of parameters in the generative pre-training model, thereby improving the accuracy of the prediction script generated by the generative pre-training model. Finally, when the syntax verification result is passed, the prediction script is determined to be the target script for controlling the pose of the robotic arm, ensuring that there are no syntax errors in the target script, which further improves the accuracy of the generated target script.

[0077] Please see Figure 4 , Figure 4 This is a flowchart illustrating a method for training a generative pre-trained model according to an embodiment of this application. The order of steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements. The method for training a generative pre-trained model according to an embodiment of this application includes the following steps.

[0078] S40 formats the pre-stored training data to obtain training prompt data.

[0079] In one embodiment of this application, the pre-stored training data includes a position table, a variable table, a cue template, and a sample script. The information recorded in the position table represents the need to control the pose of the robotic arm; the information recorded in the variable table represents the need to control the robotic arm's device; the cue template converts the information in the position table and variable table into formatted cue data, which can be input data that the generative pre-trained model can understand, and represents the need to adjust the pose of the robotic arm; the sample script represents the script that needs to be run when the electronic device controls the robotic arm to adjust to the pose corresponding to the position table and variable table.

[0080] In one embodiment of this application, the point location table can be tabular data, including point names and point coordinates, with each point name corresponding to a point coordinate. Specifically, the point coordinates can be in the form of a six-dimensional vector; for example, the point coordinates can be... , where x, y, and z are used to represent the coordinates of the pose of the robotic arm at a certain point in three-dimensional space; An angle used to characterize the angle between the line connecting the end effector of the robotic arm and the center of the robotic arm base and the x-axis. An angle used to characterize the angle between the line connecting the end effector of the robotic arm and the center of the robotic arm base and the y-axis. The angle used to characterize the angle between the line connecting the end of the robotic arm and the center of the robotic arm base and the z-axis.

[0081] In one embodiment of this application, the variable table can also be tabular data, including variable names and variable values, with each variable name corresponding to a variable value. Specifically, the contents recorded in the variable table are used to characterize the control requirements of the robotic arm. For example, when the variable name is "gripper switch" and the variable value corresponding to that variable name is 1, it indicates a need to open the robotic arm gripper.

[0082] In one embodiment of this application, the prompt template can be JSON-formatted text data, and the prompt template includes one or more of the following prompt information: background prompts, instruction prompts, and constraints. The background prompts characterize the context of the data operation requirement; for example, the background prompt information could be "You are an intelligent assistant controlling the pose of a robotic arm, possessing intent understanding capabilities." The instruction prompt information characterizes the operation to be completed and the output result; for example, the instruction prompt information could be "Generate a script to control the pose of the robotic arm based on the point table and variable table." The constraint information characterizes the constraints on the output data; for example, the constraint information could be "maximum number of bytes output," "output language," "output format," etc.

[0083] In one embodiment of this application, the sample script is used to characterize the script that needs to be run when the electronic device controls the robotic arm to adjust to the pose corresponding to the position table and variable table. For example, the sample script may take the form of "function sysCall_init()". -- Get YouBot Handle you_bot = sim.getObjectHandle('youBot') -- Prepare initial values ​​for four wheels wheel_joints = {-1,-1,-1,-1} -- front left, rear left, rear right, front right”.

[0084] For example, when the information in the point table includes "{[starting point, coordinate 1], [transfer point, coordinate 2], [grabbing point, coordinate 3], [ending point, coordinate 4]}", and the information in the variable table includes "{[variable 1, variable value 1], [variable 2, variable value 2], [variable 3, variable value 3], [variable 4, variable value 4]}", then the training prompt data could be: Background information: You are an intelligent assistant that controls the pose of a robotic arm and has the ability to understand intent.

[0085] Instruction prompt: Generate a script to control the pose of the robotic arm, as shown in the sample script below, based on the point table and variable table; The point table includes: {[starting point, coordinate 1], [transfer point, coordinate 2], [grabbing point, coordinate 3], [ending point, coordinate 4]}; The variable table includes: {[variable1, variable value1], [variable2, variable value2], [variable3, variable value3], [variable4, variable value4]}; Sample script: function sysCall_init() -- Get YouBot Handle you_bot = sim.getObjectHandle('youBot') -- Prepare initial values ​​for four wheels wheel_joints = {-1,-1,-1,-1} -- front left, rear left, rear right, front right; Constraint: Output the robotic arm control script.

[0086] In one embodiment of this application, since the prompt template is in JSON format, the prompt data obtained after filling the point table and variable table with data from the prompt template can also be in JSON format. This unifies the format of all initial data, obtaining a format that the generative pre-trained model can understand, facilitating the generation of prediction scripts based on the prompt data using the generative pre-trained model.

[0087] S41, input the training prompt data into the pre-stored initial generation model to obtain the training prediction script corresponding to the training prompt data.

[0088] In one embodiment of this application, the pre-stored initial generative model can be a generative model of any structure, for example, the initial generative model can be the ChatGPT model.

[0089] S42, verify the syntax of the training prediction script and obtain the syntax verification result of the training prediction script.

[0090] In one embodiment of this application, the method for verifying the syntax of the training prediction script is the same as the detailed description corresponding to step S34, and will not be repeated here. The syntax verification result of the training prediction script includes: verification passed and verification failed. When the syntax verification result of the training prediction script passes, it indicates that the training prediction script has no syntax errors; when the syntax verification result is verification failed, it indicates that the training prediction script has syntax errors.

[0091] S43, determine the reward value of the training prediction script based on the syntax verification result.

[0092] In one embodiment of this application, in order to evaluate the quality of the training prediction script generated by the initial generative model and thus adjust the parameters of the initial generative model, the reward value of the training prediction script can be determined based on the syntax validation result. Specifically, if the syntax validation result of the training prediction script is "validated," the reward value of the training prediction script can be determined to be a high value, indicating that the quality of the currently generated training prediction script is high; if the syntax validation result is "validated but not passed," the reward value of the training prediction script can be determined to be a low value, indicating that the quality of the currently generated prediction script is low.

[0093] In one embodiment of this application, when the syntax verification result is that the verification is passed, the reward value of the training prediction script is determined to be a first score; when the verification result is that the verification is failed, the reward value of the training prediction script is determined to be a second score; wherein, the first score is greater than the second score.

[0094] For example, the first score can be 10 and the second score can be 0. In this way, the quality of the training prediction script is represented by the reward value, thereby ensuring that the initial generative model can accurately and efficiently determine whether the generated training prediction script can be successfully used to control the pose of the robotic arm, thus improving the accuracy of the initial generative model in generating training prediction scripts.

[0095] S44, determine the updated training hint data based on the reward value, the syntax verification result, and the training prediction script.

[0096] In one embodiment of this application, in order to adjust the parameters in the initial generative model to improve its performance, the reward value, the syntax validation result, and the training prediction script can be filled into a pre-stored prompt template to obtain updated training prompt data. The updated training prompt data is used to characterize the quality of the most recently generated training prediction script and to determine the need to regenerate the training prediction script.

[0097] For example, if the most recently generated training prediction script contains a syntax error and the reward value corresponding to that script is 0, the updated training hint data could be: Background information: The previously generated training prediction script contained a syntax error and had a reward value of 0.

[0098] Instruction prompt: Please regenerate the script for controlling the pose of the robotic arm as shown in the sample script below, based on the point table and variable table; The point table includes: {[starting point, coordinate 1], [transfer point, coordinate 2], [grabbing point, coordinate 3], [ending point, coordinate 4]}; The variable table includes: {[variable1, variable value1], [variable2, variable value2], [variable3, variable value3], [variable4, variable value4]}; Sample script: function sysCall_init() -- Get YouBot Handle you_bot = sim.getObjectHandle('youBot') -- Prepare initial values ​​for four wheels wheel_joints = {-1,-1,-1,-1} -- front left, rear left, rear right, front right; Constraint: Output the robotic arm control script.

[0099] In this way, updating the training hint data based on the reward value and the grammar validation results allows the initial generative model to obtain information on the quality of the most recently generated training prediction script based on the updated training hint data, thereby updating the parameters of the initial generative model and improving its performance.

[0100] S45, determine whether the preset termination condition is met. If the preset termination condition is not met, repeat step S41; if the preset termination condition is met, execute step S46.

[0101] In one embodiment of this application, the preset termination conditions include: the number of times the updated training hint data is obtained reaches a preset number of iterations; or the difference between the training prediction script and the sample script recorded in the training data is less than a preset termination threshold.

[0102] In one embodiment of this application, when the updated training hint data reaches a preset number of iterations, it indicates that the training hint data has been updated multiple times based on the reward value and syntax verification results of the training prediction script, and the training prediction script is generated from the updated training hint data. Therefore, the parameters of the initial generative model have been adjusted multiple times, and the accuracy of the training prediction script generated by the initial generative model is high.

[0103] In one embodiment of this application, the smaller the difference between the training prediction script and the sample script recorded in the training data, the more accurately the training prediction script generated by the initial generative model can represent the pose of the robotic arm moving to the position table and variable table, i.e., the accuracy of the training prediction script is high. Specifically, since both the training prediction script and the sample script are text-formatted scripts, methods for determining the difference between the training prediction script and the sample script include: Euclidean distance algorithm, cosine similarity algorithm, Hamming distance algorithm, etc., which are not limited in this application.

[0104] In one embodiment of this application, if any of the above-mentioned preset termination conditions are not met, it indicates that the difference between the training prediction script and the sample script is large. In this case, step S41 can be executed to make the initial generation model regenerate the training prediction script based on the updated training prompt data.

[0105] S46, Determine the current initial generative model as a generative pre-trained model trained to convergence.

[0106] In one embodiment of this application, when any one of the aforementioned preset termination conditions is met, it indicates that the difference between the current training prediction script and the sample script is small. Therefore, training can be terminated, and the current initial generative model can be determined as a generative pre-trained model trained to a convergent state. The generative pre-trained model trained to a convergent state is used to generate a target script for controlling the pose of a robotic arm based on formatted prompt data.

[0107] Please see Figure 5 , Figure 5 This is a functional block diagram of a robotic arm pose control device provided in one embodiment of this application. The robotic arm pose control device 51 includes a formatting module 510, a generation module 511, a verification module 512, and a control module 513. The module / unit referred to in this application refers to a series of computer-readable instruction segments that can be executed by the processor 130 and perform a fixed function, and which are stored in the memory 120. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0108] The formatting module 510 is used to format the initial data input by the user to obtain prompt data.

[0109] The generation module 511 is used to input the prompt data into the generative pre-trained model to obtain the prediction script.

[0110] The verification module 512 is used to verify the syntax of the prediction script when the number of times the prediction script is generated reaches a preset number, and obtain the syntax verification result of the prediction script.

[0111] The verification module 512 is further configured to determine the predicted script as the target script when the syntax verification result is a successful verification.

[0112] The control module 513 is used to control the pose of the robotic arm by executing the target script.

[0113] In some embodiments, the verification module 512 is further configured to: when the number of times the prediction script has been generated has not reached a preset number, determine the updated prompt data based on the pre-stored question information and the prediction script; continue to input the updated prompt data into the generative pre-trained model to obtain the prediction script, until the number of times the prediction script has been generated reaches the preset number.

[0114] In some embodiments, when the syntax verification result is a verification failure, the verification module 512 is further configured to: construct updated prompt data based on the verification result, the prediction script, and the pre-stored prompt information; and input the updated prompt data into the generative pre-trained model.

[0115] In some embodiments, the method by which the generation module 511 trains the generative pre-trained model includes: formatting pre-stored training data to obtain training hint data; inputting the training hint data into a pre-stored initial generative model to obtain a training prediction script corresponding to the training hint data; verifying the syntax of the training prediction script to obtain a syntax verification result of the training prediction script; determining a reward value for the training prediction script based on the syntax verification result; determining updated training hint data based on the reward value, the syntax verification result, and the training prediction script; and continuing to input the updated training hint data into the initial generative model until a preset termination condition is met to obtain the generative pre-trained model trained to a convergent state.

[0116] In some embodiments, the generation module 511 determines the reward value of the training prediction script based on the verification result by: when the syntax verification result is a successful verification, determining the reward value of the training prediction script as a first score; when the verification result is a failed verification, determining the reward value of the training prediction script as a second score; wherein the first score is greater than the second score.

[0117] In some embodiments, the generation module 511 determines the updated prompt data based on the reward value, the syntax verification result, and the training prediction script by: filling the reward value, the syntax verification result, and the training prediction script into a pre-stored prompt template to obtain the updated prompt data.

[0118] This application also provides a computer-readable storage medium (not shown), which stores computer-readable instructions. These computer-readable instructions are executed by a processor in an electronic device to implement the robotic arm pose control method described in any of the above embodiments.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0120] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0122] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in the specification may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A robotic arm pose control method, applied to an electronic device, wherein the electronic device is communicatively connected to the robotic arm, characterized in that, The method includes: The initial data input by the user is formatted to obtain prompt data. The initial data includes a point table and a prompt template. The point table is used to record the point data that controls the posture of the robotic arm, and the prompt template is used to format the point data. The prompt data is input into a generative pre-trained model to obtain a prediction script; wherein, the generative pre-trained model is used to generate a script for controlling the pose of the robotic arm based on the prompt data; When the number of times the prediction script is generated reaches a preset number, the syntax of the prediction script is verified, and the syntax verification result of the prediction script is obtained. When the syntax verification result is "verification passed", the predicted script is determined to be the target script; The pose of the robotic arm is controlled by executing the target script; The generative pre-trained model is trained by: formatting pre-stored training data to obtain training hint data; inputting the training hint data into a pre-stored initial generative model to obtain a training prediction script corresponding to the training hint data; verifying the syntax of the training prediction script to obtain a syntax verification result; determining a reward value for the training prediction script based on the syntax verification result; determining updated training hint data based on the reward value, the syntax verification result, and the training prediction script; and continuing to input the updated training hint data into the initial generative model until a preset termination condition is met, thereby obtaining the generative pre-trained model trained to a convergent state.

2. The robotic arm pose control method as described in claim 1, characterized in that, The method further includes: If the number of times the prediction script is generated has not reached the preset number, the updated prompt data is determined based on the pre-stored question information and the prediction script. Continue inputting the updated prompt data into the generative pre-trained model to obtain a prediction script, until the number of times the prediction script has been generated reaches the preset number.

3. The robotic arm pose control method as described in claim 1, characterized in that, When the syntax validation result is a validation failure, the method further includes: The updated prompt data is constructed based on the syntax verification results, the prediction script, and the pre-stored prompt information; Input the updated prompt data into the generative pre-trained model.

4. The robotic arm pose control method as described in claim 1, characterized in that, The reward value of the training prediction script is determined based on the syntax verification results, including: When the syntax verification result is successful, the reward value of the training prediction script is determined to be the first score; When the syntax verification result is that the verification fails, the reward value of the training prediction script is determined to be the second score; The first score is greater than the second score.

5. The robotic arm pose control method as described in claim 1, characterized in that, The updated prompt data, determined based on the reward value, the syntax validation result, and the training prediction script, includes: The reward value, the syntax verification result, and the training prediction script are filled into the pre-stored prompt template to obtain the updated prompt data.

6. The robotic arm pose control method as described in claim 1, characterized in that, The preset termination conditions include: The number of times the updated training hint data is obtained reaches a preset number of iterations; or The difference between the training prediction script and the sample script recorded in the training data is less than a preset termination threshold.

7. A robotic arm posture control device, characterized in that, The device includes a module for implementing the robotic arm pose control method as described in any one of claims 1 to 6, and the device includes: A formatting module is used to format the initial data input by the user to obtain prompt data. The initial data includes a point table and a prompt template. The point table is used to record the point data that controls the posture of the robotic arm, and the prompt template is used to format the point data. The generation module is used to input the prompt data into the generative pre-trained model to obtain the prediction script; The verification module is used to verify the syntax of the prediction script when the number of times the prediction script has been generated reaches a preset number, and to obtain the syntax verification result of the prediction script. The verification module is further configured to determine the predicted script as the target script when the syntax verification result is that the verification is passed; The control module is used to control the pose of the robotic arm by executing the target script.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being used to implement the robotic arm pose control method as described in any one of claims 1 to 6 when executing a computer program stored in the memory.

9. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the robotic arm pose control method as described in any one of claims 1 to 6.

Images