Robot position planning method and device based on reinforcement learning

Abstract

The present invention proposes a robot position planning method and device based on reinforcement learning. The method includes: using a preset positioning pen module to digitally describe the position, shape and posture information of the object to be manipulated by the robot, as the input of the deep neural network of the robot's corresponding simulation platform, and the manipulation position of the robot-controlled object as the output, to train the robot's initial position planning model; based on the application scenario of the object, the corresponding observation space and task feedback logic are selected to optimize the initial position planning model and obtain the robot position planning model; the robot position planning model predicts the digital object data of the object to be manipulated, and obtains the target position of the robot manipulating the object to be manipulated. Therefore, based on the trained robot position planning model, the robot planning accuracy meets the implementation requirements of various industrial scenarios, and greatly reduces the workload of manual operation.

Description

Technical Field

[0001] The present invention relates to the technical field of robot motion planning, and in particular to a robot position planning method, device, electronic device and storage medium based on reinforcement learning. Background Art

[0002] When a robot performs a task, it is often necessary for the robot as a whole or a part of it to move to a specified position, and use sensors to detect information such as the target position and posture in the task. Among them, guiding the robot to detect at an appropriate position and posture, and feeding back perception data that can meet the needs of the target detection task is one of the key links in task execution. In related technologies, the use of pre-recorded robot positions for fixed objects has great limitations in applicability, or through manual control, and guiding the robotic arm to the corresponding position based on a teach pendant, there is a high degree of manual participation, high labor costs, and low efficiency. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] To this end, the first purpose of the present invention is to propose a robot position planning method based on reinforcement learning. Based on the trained robot position planning model, the robot planning accuracy meets the implementation requirements of various industrial scenarios and greatly reduces the workload of manual operation.

[0005] The second object of the present invention is to provide a robot position planning device based on reinforcement learning.

[0006] A third objective of the present invention is to provide an electronic device.

[0007] A fourth object of the present invention is to provide a non-transitory computer-readable storage medium storing computer instructions.

[0008] To achieve the above object, a first embodiment of the present invention proposes a robot position planning method based on reinforcement learning, the method comprising:

[0009] Through the preset positioning pen module, the position, shape and posture information of the object to be manipulated by the robot are digitally described to obtain digital object data;

[0010] The digital object data is used as input of a deep neural network of a simulation platform corresponding to the robot, and the manipulation position of the object controlled by the robot is used as output to train an initial position planning model of the robot;

[0011] Based on the application scenario of the object that the robot needs to manipulate, select the observation space and task reward logic corresponding to the initial position planning model;

[0012] Based on the observation space and task reward logic, the initial position planning model is optimized to obtain an optimized robot position planning model, and deployed on the robot;

[0013] The digitized object data of the object to be controlled is predicted based on the robot position planning model to obtain a target position of the object to be controlled by the robot.

[0014] To achieve the above object, a second embodiment of the present invention provides a robot position planning device based on reinforcement learning, the device comprising:

[0015] A digital description module is used to digitally describe the position, shape and posture information of the object to be manipulated by the robot through a preset positioning pen module to obtain digital object data;

[0016] A training module, used to use the digital object data as input to a deep neural network of a simulation platform corresponding to the robot, and the manipulation position of the object controlled by the robot as output, so as to train an initial position planning model of the robot;

[0017] The selection module is used to select the observation space and task return logic corresponding to the initial position planning model based on the application scenario of the object that the robot needs to manipulate;

[0018] An optimization module, used to perform scenario optimization on the initial position planning model based on the observation space and task reward logic to obtain an optimized robot position planning model, and deploy it on the robot;

[0019] The prediction module is used to predict the digital object data of the object to be controlled based on the robot position planning model to obtain the target position of the robot controlling the object to be controlled.

[0020] To achieve the above-mentioned purpose, the third aspect of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0021] In order to achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0022] The robot position planning method, device, electronic device and storage medium based on reinforcement learning provided by the embodiments of the present invention use the digital object data that digitally describes the position, shape and posture information of the object to be manipulated by the robot through a preset positioning pen module as the input of the deep neural network of the robot's corresponding simulation platform, and the manipulation position of the robot-controlled object as the output to train the robot's initial position planning model; based on the application scenario of the object, the corresponding observation space and task feedback logic are selected to optimize the initial position planning model and obtain the robot position planning model; the robot position planning model predicts the digital object data of the object to be manipulated and obtains the target position of the robot manipulating the object to be manipulated. Therefore, based on the trained robot position planning model, the robot planning accuracy meets the implementation requirements of various industrial scenarios and greatly reduces the workload of manual operation.

[0023] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A schematic diagram of a process flow of a robot position planning method based on reinforcement learning provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of a deep neural network structure provided by an embodiment of the present invention;

[0027] Figure 3 An application flow chart of another robot position planning method based on reinforcement learning provided by an embodiment of the present invention;

[0028] Figure 4 A schematic structural diagram of a robot position planning device based on reinforcement learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0030] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of relevant laws and regulations.

[0031] The following describes a robot position planning method, device, electronic device, and storage medium based on reinforcement learning according to an embodiment of the present invention with reference to the accompanying drawings.

[0032] Figure 1 A schematic flow chart of a robot position planning method based on reinforcement learning provided in an embodiment of the present invention.

[0033] like Figure 1 As shown, the method comprises the following steps:

[0034] Step 101 , using a preset positioning pen module, digitally describe the position, shape and posture information of an object to be manipulated by the robot to obtain digital object data.

[0035] In some embodiments, the preset positioning pen module is a type of prompt module, which is a language template used to guide the large language model (LLM) to perform specific tasks or generate expected content. Prompt helps users interact with the large model by providing clear instructions or questions, reducing the uncertainty of the model, thereby improving the accuracy and structure of the output.

[0036] Specifically, the positioning pen module can be operated very conveniently to digitally describe the position, shape and posture information of the object, and transmit the information to the robot as a prompt for the robot to perform task planning.

[0037] The posture information of the object includes but is not limited to horizontal, vertical, and tilted, which is not specifically limited in this embodiment.

[0038] Step 102, using the digitized object data as input to the deep neural network of the robot's corresponding simulation platform, and the manipulation position of the robot-controlled object as output, so as to train an initial position planning model of the robot.

[0039] In some embodiments, when the application scenario of the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the IsaacSim simulation platform is used for deep neural network training, and the deep neural network structure adopts a full connection plus relu activation method, contains 7 hidden layer structures, and the number of neurons is [64, 64, 32, 32, 16, 8, 8].

[0040] Specifically, Figure 2 As shown, the number of neurons in the input layer and output layer of the deep neural network can be 6 ( , ), the neurons in the hidden layer ([64, 64, 32, 32, 16, 8, 8]) are , , , , , , .

[0041] Furthermore, in the case where the application scenario of the object to be manipulated by the robot is weld detection, and the robot is a mechanical arm, the position, shape and posture information of the weld corresponding to the digital data of the weld is used as the input of the deep neural network of the robot corresponding to the simulation platform, and the six-axis position of the mechanical arm is used as the output to train the initial position planning model of the robot. Therefore, based on the IsaacSim simulation platform, the richness of the scene and the concurrency of the learning and training process can be greatly and efficiently improved, and the cost is extremely low compared to the intensive learning process in the actual environment.

[0042] Among them, when using the IsaacSim simulation platform for deep neural network training, the number of concurrent connections (training data) can be designed to be 4096.

[0043] Step 103, based on the application scenario of the object that the robot needs to manipulate, select the observation space and task reward logic corresponding to the initial position planning model.

[0044] In some embodiments, when the application scenario based on the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the observation space corresponding to the object includes the weld information collected by the positioning pen, the current position of the robotic arm, and the joint information of the robotic arm. The task feedback logic includes the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welded workpiece, whether the camera image covers all welds, robotic arm collision penalty, and abnormal action penalty. Therefore, based on rich application scenarios and large-scale simulation parallel training, the robot planning accuracy meets the needs of industrial implementation, and the planning efficiency is far ahead of traditional solutions.

[0045] Step 104, based on the observation space and task reward logic, the initial position planning model is optimized to obtain an optimized robot position planning model, and deployed on the robot.

[0046] In some embodiments, when the application scenario based on the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the initial position planning model is optimized according to the weld information, the current position of the robotic arm, the joint information of the robotic arm, the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welding workpiece, whether the camera image covers all the welds, the robotic arm collision penalty, and the abnormal action penalty, so as to obtain the optimized robot position planning model and deploy it on the robot. Therefore, in the stage of searching for a specified weld on the welding machine wall, it is not only suitable for weld detection in different positions and postures in various non-standard scenarios, but also greatly reduces the workload of manual operation.

[0047] Step 105 , predicting the digital object data of the object to be manipulated based on the robot position planning model to obtain a target position of the object to be manipulated by the robot.

[0048] In some embodiments, taking weld detection as an example, the position, shape and posture information of the weld to be welded can be obtained through a positioning pen, and the corresponding real-time weld digital data can be input into the robot position planning model, thereby predicting the target position of the robot arm when welding the weld, and moving to the target position to complete the weld quickly and accurately without human control.

[0049] The robot position planning method based on reinforcement learning in the embodiment of the present invention uses a preset positioning pen module to digitally describe the position, shape and posture information of the object to be manipulated by the robot, and uses the digital object data as the input of the deep neural network of the robot's corresponding simulation platform, and the manipulation position of the robot-controlled object as the output, to train the robot's initial position planning model; based on the application scenario of the object, the corresponding observation space and task feedback logic are selected to optimize the initial position planning model in the scenario to obtain the robot position planning model; the robot position planning model predicts the digital object data of the object to be manipulated, and obtains the target position of the robot manipulating the object to be manipulated. Therefore, based on the trained robot position planning model, the robot planning accuracy meets the implementation requirements of various industrial scenarios, and greatly reduces the workload of manual operation.

[0050] In order to clearly explain the above embodiment, Figure 3The present invention provides an application flow chart of a robot position planning method based on reinforcement learning provided by an embodiment of the present invention. Specifically, the position, shape and posture information of the object to be manipulated by the robot can be digitally described through a preset prompt module (positioning pen module) to obtain digital object data; then deep reinforcement learning task modeling is performed, and the deep reinforcement learning task modeling includes: the definition of the object corresponding to the observation space, the deep neural network structure, the deep neural network output, and the task reward logic of the deep neural network; then simulation and deep reinforcement learning model training are performed, and the simulation and deep reinforcement learning model training include simulation environment modeling (building the robot's initial position planning model based on the deep neural network structure and the deep neural network output) and simulation training (simulating and training the robot's position neural network model based on the observation space and task reward logic), thereby deploying the neural network model as a robot position planning model on the robot (mechanical arm) body.

[0051] In order to implement the above embodiment, the present invention also proposes a robot position planning device based on reinforcement learning.

[0052] Figure 4 A schematic structural diagram of a robot position planning device based on reinforcement learning provided in an embodiment of the present invention.

[0053] like Figure 4 As shown, the robot position planning device 40 based on reinforcement learning includes: a digital description module 41, a training module 42, a selection module 43, an optimization module 44 and a prediction module 45.

[0054] The digital description module 41 is used to digitally describe the position, shape and posture information of the object to be manipulated by the robot through a preset positioning pen module to obtain digital object data;

[0055] A training module 42, used to use the digital object data as input to a deep neural network of a simulation platform corresponding to the robot, and the manipulation position of the object controlled by the robot as output, so as to train an initial position planning model of the robot;

[0056] A selection module 43 is used to select the observation space and task return logic corresponding to the initial position planning model based on the application scenario of the object to be manipulated by the robot;

[0057] An optimization module 44 is used to perform scenario optimization on the initial position planning model based on the observation space and task return logic to obtain an optimized robot position planning model and deploy it on the robot;

[0058] The prediction module 45 is used to predict the digital object data of the object to be controlled based on the robot position planning model to obtain the target position of the robot controlling the object to be controlled.

[0059] Furthermore, in a possible implementation of an embodiment of the present invention, when the application scenario of the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the observation space corresponding to the object includes the weld information collected by the positioning pen, the current position of the robotic arm, and the joint information of the robotic arm. The task report logic includes the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welding workpiece, whether the camera image covers all the welds, the robotic arm collision penalty, and the abnormal action penalty.

[0060] Furthermore, in a possible implementation of the embodiment of the present invention, the training module 42 is specifically used to:

[0061] In the case where the application scenario based on the object that the robot needs to manipulate is weld detection, and the robot is a robotic arm, the position, shape and posture information of the weld corresponding to the weld digital data is used as the input of the deep neural network of the robot's corresponding simulation platform, and the six-axis position of the robotic arm is used as the output to train the robot's initial position planning model.

[0062] Furthermore, in a possible implementation of the embodiment of the present invention, the optimization module 44 is specifically configured to:

[0063] When the application scenario based on the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the initial position planning model is optimized according to the weld information, the current position of the robotic arm, the joint information of the robotic arm, the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welding workpiece, whether the camera image covers all the welds, the robotic arm collision penalty, and the abnormal action penalty to obtain the optimized robot position planning model and deploy it on the robot.

[0064] Furthermore, in a possible implementation method of an embodiment of the present invention, when the application scenario of the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the IsaacSim simulation platform is used to perform deep neural network training, and the deep neural network structure adopts a fully connected plus relu activation method, contains 7 hidden layer structures, and the number of neurons is [64, 64, 32, 32, 16, 8, 8].

[0065] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and will not be repeated here.

[0066] The robot position planning device based on reinforcement learning in the embodiment of the present invention uses a preset positioning pen module to digitally describe the position, shape and posture information of the object to be manipulated by the robot, and uses the digital object data as the input of the deep neural network of the robot's corresponding simulation platform, and the manipulation position of the robot-controlled object as the output, to train the robot's initial position planning model; based on the application scenario of the object, the corresponding observation space and task feedback logic are selected to optimize the initial position planning model in the scenario to obtain the robot position planning model; the robot position planning model predicts the digital object data of the object to be manipulated, and obtains the target position of the robot manipulating the object to be manipulated. Therefore, based on the trained robot position planning model, the robot planning accuracy meets the implementation requirements of various industrial scenarios, and greatly reduces the workload of manual operation.

[0067] In order to implement the above embodiment, the present invention further provides an electronic device, including:

[0068] at least one processor; and

[0069] a memory communicatively connected to the at least one processor; wherein,

[0070] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the aforementioned method.

[0071] In order to implement the above embodiment, the present invention further proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the above method.

[0072] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0073] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0074] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0075] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0076] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0077] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0078] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules. If the integrated module is implemented in the form of software functional modules and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0079] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A robot position planning method based on reinforcement learning, characterized in that: The method comprises: Through the preset positioning pen module, the position, shape and posture information of the object to be manipulated by the robot are digitally described to obtain digital object data; The digital object data is used as input of a deep neural network of a simulation platform corresponding to the robot, and the manipulation position of the object controlled by the robot is used as output to train an initial position planning model of the robot; Based on the application scenario of the object that the robot needs to manipulate, select the observation space and task reward logic corresponding to the initial position planning model; Based on the observation space and task reward logic, the initial position planning model is optimized to obtain an optimized robot position planning model, and deployed on the robot; Predicting the digital object data of the object to be controlled based on the robot position planning model to obtain a target position of the object to be controlled by the robot; When the application scenario based on the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the observation space corresponding to the object includes the weld information collected by the positioning pen, the current position of the robotic arm, and the joint information of the robotic arm. The task report logic includes the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welding workpiece, whether the camera image covers all the welds, the robotic arm collision penalty, and the abnormal action penalty.

2. The method according to claim 1, characterized in that The method uses the digitized object data as input of a deep neural network corresponding to a simulation platform of the robot, and the robot controls the manipulation position of the object as output, so as to train an initial position planning model of the robot, including: In the case where the application scenario based on the object that the robot needs to manipulate is weld detection, and the robot is a robotic arm, the position, shape and posture information of the weld corresponding to the weld digital data is used as the input of the deep neural network of the robot's corresponding simulation platform, and the six-axis position of the robotic arm is used as the output to train the robot's initial position planning model.

3. The method according to claim 1, characterized in that The initial position planning model is optimized based on the observation space and task return logic to obtain an optimized robot position planning model, and deployed on the robot, including: When the application scenario based on the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the initial position planning model is optimized according to the weld information, the current position of the robotic arm, the joint information of the robotic arm, the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welding workpiece, whether the camera image covers all the welds, the robotic arm collision penalty, and the abnormal action penalty to obtain the optimized robot position planning model and deploy it on the robot.

4. The method according to claim 1, characterized in that: In the case where the application scenario of the object to be manipulated by the robot is weld inspection and the robot is a robotic arm, the IsaacSim simulation platform is used for deep neural network training. The deep neural network structure adopts a fully connected plus relu activation method, contains 7 hidden layer structures, and the number of neurons is [64, 64, 32, 32, 16, 8, 8].

5. A robot position planning device based on reinforcement learning, characterized in that: The device comprises: A digital description module is used to digitally describe the position, shape and posture information of the object to be manipulated by the robot through a preset positioning pen module to obtain digital object data; A training module, used to use the digital object data as input to a deep neural network of a simulation platform corresponding to the robot, and the manipulation position of the object controlled by the robot as output, so as to train an initial position planning model of the robot; The selection module is used to select the observation space and task return logic corresponding to the initial position planning model based on the application scenario of the object that the robot needs to manipulate; An optimization module, used to perform scenario optimization on the initial position planning model based on the observation space and task reward logic to obtain an optimized robot position planning model, and deploy it on the robot; A prediction module, used for predicting the digital object data of the object to be controlled based on the robot position planning model, so as to obtain a target position of the robot controlling the object to be controlled; When the application scenario based on the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the observation space corresponding to the object includes the weld information collected by the positioning pen, the current position of the robotic arm, and the joint information of the robotic arm. The task report logic includes the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welding workpiece, whether the camera image covers all the welds, the robotic arm collision penalty, and the abnormal action penalty.

6. The device according to claim 5, characterized in that The training module is specifically used for: In the case where the application scenario based on the object that the robot needs to manipulate is weld detection, and the robot is a robotic arm, the position, shape and posture information of the weld corresponding to the weld digital data is used as the input of the deep neural network of the robot's corresponding simulation platform, and the six-axis position of the robotic arm is used as the output to train the robot's initial position planning model.

7. The device according to claim 5, characterized in that The optimization module is specifically used for: When the application scenario based on the object to be manipulated by the robot is weld detection, and the robot is a robotic arm, the initial position planning model is optimized according to the weld information, the current position of the robotic arm, the joint information of the robotic arm, the distance from the camera on the robotic arm to the weld, the angle between the camera and the bottom plate of the welding workpiece, whether the camera image covers all the welds, the robotic arm collision penalty, and the abnormal action penalty to obtain the optimized robot position planning model and deploy it on the robot.

8. The device according to claim 5, characterized in that In the case where the application scenario of the object to be manipulated by the robot is weld inspection and the robot is a robotic arm, the IsaacSim simulation platform is used for deep neural network training. The deep neural network structure adopts a fully connected plus relu activation method, contains 7 hidden layer structures, and the number of neurons is [64, 64, 32, 32, 16, 8, 8].

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

Images