Composite robot control method, control equipment, composite robot and storage medium

Abstract

The invention discloses a composite robot control method and device, a composite robot and a storage medium. The method comprises the following steps: acquiring a control request, wherein the control request comprises material data corresponding to a target material and instruction data; in response to the control request, controlling the 3D camera to work, and obtaining actually measured 3D data collected by the 3D camera; processing the actually measured 3D data, the material data and the instruction data by adopting a target VLA model, and determining the initial position of the collaborative robot; controlling the collaborative robot to move to an initial position, controlling a 2D camera arranged on the collaborative robot to work, and obtaining 2D data collected by the 2D camera; and based on the 2D data, determining a target position of the collaborative robot, controlling the collaborative robot to move to the target position, controlling an end effector arranged on the collaborative robot, and executing target operation corresponding to the instruction data on the target material. According to the scheme, the positioning efficiency of visual positioning is guaranteed through 3D data, and the positioning precision of visual positioning is guaranteed through 2D data.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a composite robot control method, a control device, a composite robot, and a storage medium. Background Art

[0002] With the rapid development of science and technology, collaborative robots have been widely used in industrial production, logistics, metallurgy and medical care, and 3C industry. They are used to replace human hands in repetitive and mechanical tasks such as grasping and carrying, which can significantly improve production efficiency. In addition, collaborative robots can also replace human hands in some complex and dangerous tasks, ensuring the personal safety of operators to a large extent.

[0003] Currently, collaborative robots require visual positioning of target materials before performing tasks such as loading and unloading, gripping, or welding. However, existing collaborative robots struggle to balance positioning efficiency and accuracy when performing visual positioning. Summary of the Invention

[0004] Embodiments of the present invention provide a composite robot control method, a control device, a composite robot, and a storage medium to solve the problem that in the visual positioning process of existing collaborative robots, both positioning efficiency and positioning accuracy cannot be taken into account.

[0005] A composite robot control method, comprising: Obtaining a control request, wherein the control request includes material data and instruction data corresponding to a target material; In response to the control request, controlling the 3D camera to operate and obtaining measured 3D data collected by the 3D camera; Processing the measured 3D data, the material data, and the instruction data using a target VLA model to determine an initial position of the collaborative robot; Controlling the collaborative robot to move to the initial position, controlling a 2D camera provided on the collaborative robot to operate, and acquiring 2D data collected by the 2D camera; determining a target position of the collaborative robot based on the 2D data, and controlling the collaborative robot to move to the target position; The end effector provided on the collaborative robot is controlled to perform the target operation corresponding to the instruction data on the target material.

[0006] Preferably, controlling the 3D camera to operate and obtaining the measured 3D data collected by the 3D camera includes: Controlling the 3D camera to operate and obtain initial 3D data collected by the 3D camera; performing target tracking based on the initial 3D data and material data corresponding to the target material, and determining a current shooting angle of the 3D camera shooting the target material; If the current shooting angle is within a preset angle range, determining the initial 3D data as the measured 3D data; If the current shooting angle is not within the preset angle range, the position and / or posture of the 3D camera is adjusted, and the 3D camera control operation is repeatedly performed to obtain initial 3D data collected by the 3D camera.

[0007] Preferably, the composite robot comprises a middle cabinet, a collaborative robot and an induction pan-tilt platform arranged on the middle cabinet; The collaborative robot includes a plurality of robotic arms that are rotatably connected in sequence, wherein two adjacent robotic arms are rotatably connected, the first robotic arm is mounted on the middle cabinet via a fixed base, and the last robotic arm is connected to the end effector via an adapter, wherein the adapter is provided with a first 3D camera and the 2D camera; The induction pan-tilt platform includes a lifting mechanism, a rotating mechanism and a swing mechanism. The lifting mechanism is installed on the middle cabinet, the rotating mechanism is installed on the lifting mechanism, the swing mechanism is installed on the rotating mechanism, and a second 3D camera is provided on the swing mechanism. If the current shooting angle is not within the preset angle range, adjusting the position and / or posture of the 3D camera includes: If the current shooting angle of the first 3D camera to the target material is not within a first preset angle range, controlling at least one of the plurality of robotic arms to rotate to adjust the position and / or posture of the first 3D camera; If the current shooting angle of the second 3D camera for the target material is not within a second preset angle range, at least one of the lifting mechanism, the rotating mechanism and the swing mechanism is controlled to operate to adjust the position and / or posture of the second 3D camera.

[0008] Preferably, a movable chassis is provided under the middle cabinet; Before controlling the 3D camera to operate and obtaining the measured 3D data collected by the 3D camera, the composite robot control method further includes: Control the mobile chassis to move the middle cabinet to the working point where the target material is located; The working point where the target material is located is determined based on working scene data, and the working scene data is data obtained by photographing the working scene with a working scene camera.

[0009] Preferably, a first sensing device is provided on the mobile chassis; The control of the mobile chassis to move the middle cabinet to the working point where the target material is located includes: Controlling the mobile chassis to move based on the target path, and acquiring first sensing data collected by the first sensing device in real time; performing obstacle detection based on the first sensing data, and determining an obstacle detection result; If the obstacle detection result indicates that an obstacle exists, a target obstacle avoidance strategy is executed to control the operation of the mobile chassis; If the obstacle detection result is that there is no obstacle, the mobile chassis is controlled to continue moving along the target path until the middle cabinet moves to the working point where the target material is located.

[0010] Preferably, the process of processing the measured 3D data, the material data and the instruction data using the target VLA model to determine the initial position of the collaborative robot includes: Converting the measured 3D data to determine target point cloud data; A target VLA model is used to extract features of the target point cloud data, the material data and the instruction data, and point cloud features, material features and instruction features are determined respectively. The point cloud features, material features and instruction features are processed to determine the initial position of the collaborative robot.

[0011] Preferably, the measured 3D data includes first 3D data collected by a first 3D camera and second 3D data collected by a second 3D camera, wherein the first 3D camera is arranged on the collaborative robot, and the second 3D camera is arranged outside the collaborative robot; The converting the measured 3D data to determine target point cloud data includes: Converting the first 3D data and the second 3D data to determine first point cloud data corresponding to the first 3D data and second point cloud data corresponding to the second 3D data; determining, based on position and posture data corresponding to the first 3D camera and position and posture data corresponding to the second 3D camera, a first fusion coefficient corresponding to the first 3D camera and a second fusion coefficient corresponding to the second 3D camera; Target point cloud data is determined based on the first point cloud data, the second point cloud data, the first fusion coefficient, and the second fusion coefficient.

[0012] Preferably, controlling the collaborative robot to move to the initial position includes: Controlling the movement of the collaborative robot based on the dynamic position output by the target VLA model, and determining measured movement data of the collaborative robot moving to the dynamic position; If the measured movement data does not meet the preset safety conditions, controlling the collaborative robot to stop working; The dynamic position is that the target VLA model processes the measured 3D data to determine the position to be moved at the next moment.

[0013] Preferably, the collaborative robot comprises a plurality of mechanical arms rotatably connected in sequence, and each of the mechanical arms is connected to a drive motor; The composite robot control method further includes: During the movement of the collaborative robot, obtaining measured motor data corresponding to a plurality of drive motors; When the measured motor data corresponding to at least one of the drive motors meets the motor risk condition, controlling the collaborative robot to stop working; Wherein, the measured motor data includes a measured current loop and a measured torque loop; The motor risk condition is that the measured current loop is greater than the current loop threshold, or the measured torque loop is greater than the torque loop threshold.

[0014] Preferably, the end effector is provided with a second sensing device; After controlling the end effector provided on the collaborative robot to perform the target operation corresponding to the instruction data on the target material, the composite robot control method further includes: Obtaining a measured sensing distance corresponding to a second sensing device, where the measured sensing distance is used to represent a distance between the end effector and the material placement frame; When the measured sensing distance is less than the preset safety distance, the end effector is controlled to stop working.

[0015] A control device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned composite robot control method is implemented.

[0016] A composite robot comprises the above-mentioned control device.

[0017] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the composite robot control method is implemented.

[0018] The aforementioned hybrid robot control method, control device, hybrid robot, and storage medium utilize a target VLA model to process real-time measured 3D data. This model dynamically determines the collaborative robot's initial position based on the 3D data from different viewpoints. The initial position has a low accuracy of approximately 10 mm. After the collaborative robot is controlled to move to the initial position, a 2D camera mounted on the collaborative robot is controlled to acquire 2D data from the collaborative robot's viewpoint. Based on this 2D data, the corresponding target position is determined with a high accuracy of approximately 0.5 mm, enabling precise positioning and guidance of the collaborative robot. After the collaborative robot reaches the target position, the end effector on the collaborative robot is controlled to perform the target operation corresponding to the instruction data on the target material. In this example, 3D data is used to ensure the efficiency of visual positioning, while 2D data is used to ensure its accuracy. This approach helps reduce on-site setup work for large and complex industrial scenarios. In various industrial scenarios, the target VLA model can be used to drive the collaborative robot's mechanical motion without pre-programming or teaching, requiring only the calibration of the 2D camera's coordinates. This improves the flexibility, dynamism, and high-precision requirements of the hybrid robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 is a schematic diagram of a robot scheduling system according to an embodiment of the present invention; Figure 2 is another schematic diagram of a robot scheduling system according to an embodiment of the present invention; Figure 3 is a structural diagram of a composite robot according to an embodiment of the present invention; Figure 4 is a flow chart of a composite robot control method according to an embodiment of the present invention.

[0021] In the figure: 10. Composite robot; 11. Mobile chassis; 12. Middle cabinet; 13. Collaborative robot; 14. Induction pan / tilt; 15. Hopper mechanism; 16. Clamping mechanism; 17. First 3D camera; 18. 2D camera; 19. Second 3D camera; 20. Main control unit. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] The embodiment of the present invention provides a control method for a composite robot, wherein the composite robot 10 is a robot that integrates multiple robot functions and technologies, and the composite robot 10 may be a composite robot 10 in a robot scheduling system. Figure 1 and Figure 2 As shown, the robot scheduling system includes a main control computer 20 and at least one composite robot 10 communicatively connected to the main control computer 20. The main control computer 20 and the at least one composite robot 10 communicate via the TCP / IP communication protocol, and other communication protocols are also possible. The main control computer 20 is integrated with a warehouse management system for managing warehouse materials. The main control computer 20 is equipped with a visual BI dashboard, allowing users to perform relevant operations based on the content displayed on the visual BI dashboard. Each composite robot 10 can communicate with a terminal device via the TCP / IP communication protocol or other communication protocols. The terminal devices herein include but are not limited to personal computers, laptops, smartphones, tablet computers, and portable wearable devices.

[0024] like Figure 3 As shown, the mechanical structure of the composite robot 10 includes a mobile chassis 11, a mid-level cabinet 12 mounted on the mobile chassis 11, a collaborative robot 13 mounted on the mid-level cabinet 12, a sensing platform 14, and a silo mechanism 15. The collaborative robot 13 is detachably connected to an end effector via a quick-change mechanism. The end effector may be, but is not limited to, a clamping mechanism 16. The collaborative robot 13 is equipped with a first 3D camera 17 and a 2D camera 18; the sensing platform 14 is equipped with a second 3D camera 19.

[0025] like Figure 1 and Figure 2As shown, the control system of the compound robot 10 includes a central control node, a 3D camera, a 2D camera, a collaborative robot controller, a mobile chassis controller and a PLC controller connected to the central control node through a switch or other communication methods; the central control node can control the operation of the 3D camera and the 2D camera, process the 3D data collected by the 3D camera and the 2D data collected by the 2D camera, and output control instructions to the collaborative robot controller, the mobile chassis controller and the PLC controller to respectively control the operation of the collaborative robot, the mobile chassis, the clamping mechanism and the silo mechanism. In this example, the 3D camera includes a first 3D camera 17 set on the collaborative robot 13 and a second 3D camera 19 set on the sensing gimbal 14. The first 3D camera 17 is used to collect first 3D data, which is 3D data collected from the perspective of the collaborative robot; the second 3D data is 3D data collected from a third-party perspective other than the perspective of the collaborative robot; the second 3D data is used to collect second 3D data. The 2D camera node is set on the collaborative robot 13 for collecting 2D data, which is 2D data collected from the perspective of the collaborative robot.

[0026] In this example, the composite robot 10 also includes a carrier node, a business processing node, and a data processing node. The carrier node is a node used to control the operation of the silo mechanism 15. The business processing node is a node used to perform business processing. The data processing node is a node used to perform data processing.

[0027] In this example, at least one of the central control nodes of the master control machine 20 and the composite robot 10 deploys a target VLA model. The target VLA model is a VLA model pre-determined based on training industrial data. The VLA (Vision Language Action) model, also known as the visual language action model, is a multimodal artificial intelligence technology that combines visual perception, language understanding, and action execution, and is mainly used in the fields of autonomous driving and robotics. The training industrial data here is the industrial data used for model training. This industrial data is formed based on material data of different materials, instruction data corresponding to different control operations, and different 3D data. The embodiment of the present invention provides a composite robot control method, which is described by taking the method applied to the central control node as an example. Figure 4 As shown, the composite robot control method includes: S1: Obtain a control request, wherein the control request includes material data and instruction data corresponding to the target material; S2: In response to the control request, controlling the 3D camera to operate and obtain measured 3D data collected by the 3D camera; S3: Processing the measured 3D data, the material data, and the instruction data using a target VLA model to determine an initial position of the collaborative robot; S4: controlling the collaborative robot to move to the initial position, controlling the 2D camera provided on the collaborative robot to operate, and acquiring 2D data collected by the 2D camera; S5: determining a target position of the collaborative robot based on the 2D data, and controlling the collaborative robot to move to the target position; S6: Control the end effector provided on the collaborative robot to execute the target operation corresponding to the instruction data.

[0028] The control request is a request triggered by the master computer to control the operation of the composite robot.

[0029] As an example, in step S1, the central control node may receive a control request output by the main control computer via WiFi or other wireless communication technologies, parse the control request, and determine the material data and instruction data corresponding to the target material. The target material here refers to the material that requires operation. Material data represents data about the material, which may be image data of the target material, or voice data and / or text data used to represent the location and characteristics of the target material. Instruction data refers to the voice data and / or text data corresponding to the instruction required to control the target material to perform the target operation, for example, the target material needs to be moved from the first work point A to the second work point B.

[0030] As an example, in step S2, after receiving a control request from the host computer 20, the central control node can control the 3D camera in communication with it to operate, causing the 3D camera to collect measured 3D data and transmit the measured 3D data to the central control node, allowing the central control node to obtain the measured 3D data collected by the 3D camera. The 3D camera can be installed on or off the collaborative robot. The measured 3D data is the data collected in real time by the 3D camera and used to determine the initial position.

[0031] In this example, the 3D camera includes a first 3D camera provided on the collaborative robot and a second 3D camera provided outside the collaborative robot, and the measured 3D data includes first 3D data and second 3D data. The first 3D data is 3D data collected by the first 3D camera provided on the collaborative robot, and is 3D data collected from the perspective of the collaborative robot. The second 3D data is 3D data collected by the collaborative robot that is not provided on the collaborative robot, and is 3D data collected from a third-party perspective other than the perspective of the collaborative robot. That is, after receiving the control request from the main control machine, the central control node can control the first 3D camera to collect the first 3D data in real time, and control the second 3D camera to collect the second 3D data in real time, so that the central control node can obtain the first 3D data from the perspective of the collaborative robot and the second 3D data from the perspective of the third party, so as to collect 3D data from different perspectives.

[0032] As an example, in step S3, after obtaining the measured 3D data, the central control node can input the material data and instruction data corresponding to the target material parsed in the control request, as well as the measured 3D data collected in real time by the 3D camera, into the target VLA model. The target VLA model processes the multimodal data and determines the collaborative robot's initial position, which is a highly accurate position, based on the target VLA model's output. The initial position here refers to the position at which the collaborative robot needs to switch to collect 2D data during movement. In this example, because the target VLA model can perceive multimodal data such as images, text, and voice, it can dynamically analyze and determine the target material's first position at the first work point A based on the material data corresponding to the target material and the measured 3D data. It can also dynamically analyze and determine the target material's need to move to a second position at the second work point B based on the instruction data and the measured 3D data. A planned path between the first and second positions is determined based on the planned path. A position at which the collaborative robot's 2D camera has a better viewing angle is determined based on the planned path, and this position is determined as the collaborative robot's initial position.

[0033] In this example, the target VLA model is pre-trained based on training industrial data to fine-tune the model parameters. The training industrial data is proprietary data within the enterprise, ensuring data security. Using this proprietary training industrial data, the open-source, certified, and secure VLA model is trained, fine-tuned, and deployed to form the enterprise's own target VLA model for its specific vertical domain. This target VLA model, a dedicated enterprise-specific model, accurately identifies real-time, measured 3D data to determine the collaborative robot's initial position. Intelligent grasping based on the target VLA model leverages the VLA model's capabilities, eliminating the need for extensive material labeling and pre-programming. By collecting high-quality training industrial data, fine-tuning, and deploying it, intelligent grasping control of various materials can be achieved. Furthermore, the target VLA model is an end-to-end, large-scale model, eliminating the need for pre-programming of the collaborative robot's operational processes. This significantly reduces the time required for field deployment and eliminates the need for teaching the collaborative robot to each workstation.

[0034] As an example, in step S4, after determining the collaborative robot's initial position based on the target VLA model, the central control node first controls the collaborative robot to move to the initial position and then controls the 2D camera mounted on the collaborative robot to collect 2D data. The 2D camera follows the collaborative robot's movement, collecting 2D data from the collaborative robot's perspective in real time. This allows the precise determination of the target material's position based on this 2D data, helping to ensure the control accuracy of subsequent operations. The 2D camera here is an industrial 2D camera with a 20-megapixel pixel count and a field of view of 500x400mm, which can fully cover a variety of material sizes. Generally speaking, 2D cameras can capture an object's surface contour and color information, and have relatively high image quality. 3D cameras can capture richer information about an object's three-dimensional shape, depth, stereoscopic position, and texture, but their image quality is lower than that of 2D cameras.

[0035] As an example, in step S5, after acquiring the 2D data captured by the 2D camera, the central control node can convert the 2D data based on the transformation relationship between the camera coordinate system corresponding to the 2D camera and the robot coordinate system of the collaborative robot, thereby determining the target position of the collaborative robot. The central control node then controls the collaborative robot to move to the target position, where the target position refers to the collaborative robot stopping. After the collaborative robot reaches the target position, the central control node can control the end effector on the collaborative robot to perform the target operation corresponding to the instruction data on the target material.

[0036] For example, when the end effector is a clamping mechanism, when the target material needs to be moved from the first working point A to the second working point B, the clamping mechanism can be controlled to perform the unloading operation first, and the target material on the first working point A can be moved to the silo mechanism of the composite robot; when the mobile chassis is controlled to move near the second working point B, the clamping mechanism is controlled to execute the loading mechanism, and the target material on the silo mechanism of the composite robot can be moved to the second working point B, so as to guide the collaborative robot to perform more precise material grasping or material placement operations. For example, when the target material is a tray, the gap between the material placement frame and the tray in the working point can be set to about 1.5mm to achieve refined control. In this example, after controlling the end effector to operate on the target material, the target LVA model can be switched back to dynamically drive the collaborative robot to work, so that the collaborative robot retracts to its original position.

[0037] In this example, a target VLA model is used to process the measured 3D data, the material data, and the instruction data to dynamically determine the initial position of the collaborative robot based on the measured 3D data. The initial position has a low accuracy of approximately 10 mm. After the collaborative robot is controlled to move to the initial position, the 2D camera installed on the collaborative robot is controlled to collect 2D data from the collaborative robot's perspective. Based on the 2D data, the corresponding target position is determined. The target position has a high accuracy of approximately 0.5 mm, so as to achieve precise positioning and guidance of the collaborative robot. After the collaborative robot moves to the target position, the end effector on the collaborative robot is controlled to perform the target operation corresponding to the instruction data on the target material. For large and complex industrial scenarios, this helps save on-site layout work. In different industrial scenarios, the mechanical movement of the collaborative robot driven by the target VLA model does not require pre-programming and teaching. Only the coordinates of the 2D camera need to be calibrated, which helps improve the flexibility, activity, and high-precision requirements of the composite robot.

[0038] In one embodiment, step S2, i.e., controlling the 3D camera to obtain measured 3D data collected by the 3D camera, includes: S21: controlling the 3D camera to operate and obtaining initial 3D data collected by the 3D camera; S22: performing target tracking based on the initial 3D data and the material data corresponding to the target material, and determining a current shooting angle of the 3D camera shooting the target material; S23: If the current shooting angle is within a preset angle range, determining the initial 3D data as the measured 3D data; S24: If the current shooting angle is not within the preset angle range, the position and / or posture of the 3D camera is adjusted, and the 3D camera control operation is repeatedly performed to obtain initial 3D data collected by the 3D camera.

[0039] As an example, after receiving a control request from the host computer 20, the central control node can control the operation of a 3D camera to obtain initial 3D data captured by the 3D camera. This initial 3D data is 3D data collected in real time and not yet analyzed or processed. In this example, the central control node can control the operation of a first 3D camera mounted on the collaborative robot to collect first initial 3D data from the collaborative robot's perspective, and control the operation of a second 3D camera not mounted on the collaborative robot to collect second initial 3D data corresponding to a third-party perspective outside the collaborative robot's perspective. Because the first 3D camera is mounted on the collaborative robot, its imaging range is limited by the collaborative robot's position. However, the second 3D camera is not mounted outside the collaborative robot, and its imaging range is not limited by the collaborative robot's position. Therefore, the second 3D camera can capture information about the external environment with a wider imaging range.

[0040] As an example, the central control node performs target detection and tracking on the initial 3D data and the material data corresponding to the target material to determine whether the collected initial 3D data includes the target material. If the target material is included, the central control node further determines the target material's position, posture, or other information, and determines the current shooting angle of the 3D camera capturing the target material. This current shooting angle is the angle at which the 3D camera captures the target object. In one example, the central control node may employ a 3D target detection algorithm, such as but not limited to the VoxelNet algorithm, the PointPillars algorithm, and the AVOD algorithm, to perform target detection and tracking on the initial 3D data and the material data corresponding to the target material. This can quickly determine whether the target material is within the 3D camera's capturing range, and if so, quickly determine the current shooting angle of the 3D camera capturing the target material. In another example, the central control node may also transmit the initial 3D data to the target VLA model, so that the target VLA model analyzes and processes the initial 3D data and the material data corresponding to the target material and outputs the current shooting angle of the 3D camera capturing the target material.

[0041] The preset angle range is a range corresponding to a preset optimal shooting angle.

[0042] As an example, after obtaining the current shooting angle of the 3D camera shooting the target material, the central control node compares the current shooting angle with the preset angle range. If the current shooting angle is within the preset angle range, it is determined that the current shooting angle of the 3D camera shooting the target material is within the preferred preset shooting angle range, and the initial 3D data can be directly determined as the measured 3D data; if the current shooting angle is not within the preset angle range, it is determined that the current shooting angle of the 3D camera shooting the target material is not within the preferred preset shooting angle range, and the initial 3D data can be directly determined as the measured 3D data. At this time, it is necessary to adjust the position and / or posture of the 3D camera to adjust the shooting angle of the 3D camera to the target material, and then repeat the control of the 3D camera to obtain the initial 3D data collected by the 3D camera and subsequent steps to ensure that the measured 3D data finally output is the data shot by the 3D camera at the preferred shooting angle, so that the measured 3D data can effectively reflect the external environment information of the target material, thereby ensuring the accuracy of subsequent position recognition.

[0043] In one embodiment, the composite robot includes a middle cabinet, a collaborative robot and an induction pan / tilt platform arranged on the middle cabinet; The collaborative robot includes a plurality of robotic arms that are rotatably connected in sequence, wherein two adjacent robotic arms are rotatably connected, the first robotic arm is mounted on the middle cabinet via a fixed base, and the last robotic arm is connected to the end effector via an adapter, wherein the adapter is provided with a first 3D camera and the 2D camera; The induction pan-tilt platform includes a lifting mechanism, a rotating mechanism and a swing mechanism. The lifting mechanism is installed on the middle cabinet, the rotating mechanism is installed on the lifting mechanism, the swing mechanism is installed on the rotating mechanism, and a second 3D camera is provided on the swing mechanism. Step S24, i.e., if the current shooting angle is not within the preset angle range, adjusting the position and / or posture of the 3D camera, includes: If the current shooting angle of the first 3D camera to the target material is not within a first preset angle range, controlling at least one of the plurality of robotic arms to rotate to adjust the position and / or posture of the first 3D camera; If the current shooting angle of the second 3D camera for the target material is not within a second preset angle range, at least one of the lifting mechanism, the rotating mechanism and the swing mechanism is controlled to operate to adjust the position and / or posture of the second 3D camera.

[0044] As an example, the composite robot includes a middle cabinet, a collaborative robot arranged on the middle cabinet, and an induction gimbal. The collaborative robot includes a plurality of robotic arms that are rotatably connected in sequence, and two adjacent robotic arms are rotatably connected. The first robotic arm is installed on the mobile chassis through a fixed seat, and the last robotic arm is connected to the end effector through an adapter seat, and the adapter seat is provided with the first 3D camera and the 2D camera. The induction gimbal specifically includes a lifting mechanism, a rotating mechanism, and a swinging mechanism; the lifting mechanism can be directly installed on the middle cabinet, or it can be installed on the middle cabinet through a base plate. When the lifting mechanism is installed on the base plate, the lifting mechanism can be lifted and lowered along the height direction of the collaborative robot; when the rotating mechanism is installed on the lifting mechanism, the rotating mechanism can be rotated around the height direction of the collaborative robot; when the swing mechanism is installed on the rotating mechanism, the swing mechanism can be swung around the height direction perpendicular to the collaborative robot.

[0045] As an example, the central control node analyzes and determines, based on the first initial 3D data collected by the first 3D camera, that the current shooting angle of the first 3D camera shooting the target material is not within the first preset angle range, and determines that the current shooting angle of the first 3D camera is not within the preferred preset shooting angle range, and that the position and / or posture of the first 3D camera needs to be adjusted. Specifically, this can be done by controlling at least one of the multiple robotic arms to rotate so that the position and / or posture of the adapter is adjusted, thereby adjusting the position and / or posture of the first 3D camera on the adapter, and then adjusting the shooting angle of the first 3D camera. The first preset angle range here is a pre-set range for evaluating whether the shooting angle of the first 3D camera is within the preferred shooting angle range. Since the first 3D camera is set on the collaborative robot, its shooting angle is limited by the position of the collaborative robot. In order to ensure that the first 3D camera can capture valid 3D data, the first preset angle range can be set larger.

[0046] As an example, the central control node analyzes and determines, based on the second initial 3D data collected by the second 3D camera, that the current shooting angle of the second 3D camera shooting the target material is not within a second preset angle range. The central control node determines that the current shooting angle of the second 3D camera is not within the preferred preset shooting angle range and needs to adjust the position and / or posture of the second 3D camera. Specifically, this can be done by controlling at least one of the lifting mechanism, the rotating mechanism, and the swing mechanism to change the position and / or posture of the second 3D camera disposed on the swing mechanism, thereby adjusting the shooting angle of the second 3D camera. The second preset angle range herein is a preset range used to assess whether the shooting angle of the second 3D camera is within a preferred shooting angle range. Since the second 3D camera is disposed outside the collaborative robot, its shooting angle is not restricted by the position of the collaborative robot. In order to ensure the shooting accuracy of the 3D data captured by the second 3D camera, the second preset angle range can be set to be smaller, that is, the second preset angle range is less than or equal to the first preset angle range.

[0047] In one embodiment, a movable chassis is provided below the middle cabinet; Before step S2, that is, before controlling the 3D camera to obtain the measured 3D data collected by the 3D camera, the composite robot control method further includes: Control the mobile chassis to move the middle cabinet to the working point where the target material is located; The working point where the target material is located is determined based on working scene data, and the working scene data is data obtained by photographing the working scene with a working scene camera.

[0048] As an example, before controlling the composite robot, the master control computer first controls a work scene camera installed in a work scene (e.g., a workshop) to capture the work scene, collect work scene data, and transmit this work scene data to the master control computer. The master control computer then performs target detection on the work scene data to determine the work point where the target material to be operated is located. Here, the work point is a point determined by pre-dividing the work scene. Based on the positioning data of at least one composite robot, the master control computer then determines the composite robot closest to the target material and in an idle state. The master control computer then transmits a control request to the composite robot, which carries the location information of the work point where the target material is located.

[0049] In this example, the central control node of each composite robot communicates with the main control machine, receives the control request sent by the main control machine, parses the control request, determines the working point where the target material is located, and then controls the mobile chassis to move the middle cabinet to the working point where the target material is located, so that the 3D camera can effectively shoot the working point and shoot the measured 3D data corresponding to the working point to increase the possibility of the measured 3D camera shooting the target material.

[0050] In one embodiment, a first sensing device is provided on the mobile chassis; The control of the mobile chassis to move the middle cabinet to the working point where the target material is located includes: Controlling the mobile chassis to move based on the target path, and acquiring first sensing data collected by the first sensing device in real time; performing obstacle detection based on the first sensing data, and determining an obstacle detection result; If the obstacle detection result indicates that an obstacle exists, a target obstacle avoidance strategy is executed to control the operation of the mobile chassis; If the obstacle detection result is that there is no obstacle, the mobile chassis is controlled to continue moving along the target path until the middle cabinet moves to the working point where the target material is located.

[0051] As an example, the mobile chassis is provided with a first sensing device, and the first sensing device here includes but is not limited to a laser radar and a 3D camera. In this example, when the central control node receives the control request from the main control machine, it needs to parse the control request and determine the target path corresponding to the mobile chassis. The target path is the path that needs to control the composite robot to move from the current position to the working point where the target material is located. Then, the mobile chassis is controlled to move based on the target path. During the movement of the mobile chassis, it is necessary to collect the first sensing data of the area in front of it in real time through the first sensing device, perform obstacle detection based on the first sensing data, and determine the obstacle detection result. When the obstacle detection result is that there is an obstacle, it is necessary to determine the obstacle type based on the first sensing data, determine the corresponding target obstacle avoidance strategy based on the obstacle type, execute the target obstacle avoidance strategy, and control the operation of the mobile chassis. Specifically, if the obstacle type is a moving object, the mobile chassis is controlled to stop moving within a preset time period, and then controlled to move after the preset time period. For example, if there is a person in the area in front of the mobile chassis, the mobile chassis will pause after a certain distance from the person and resume movement after the person leaves. If the obstacle type is a static object, the mobile chassis is controlled to execute an obstacle avoidance strategy, replan the target path, and control the mobile chassis to move based on the replanned target path. If the obstacle detection result is that there is no obstacle, the mobile chassis is controlled to continue moving along the target path until the middle cabinet moves to the working point where the target material is located, thereby realizing intelligent control of the mobile chassis.

[0052] In one embodiment, step S3, i.e., using the target VLA model to process the measured 3D data, the material data, and the instruction data to determine the initial position of the collaborative robot, includes: S31: converting the measured 3D data to determine target point cloud data; S32: Use the target VLA model to extract features of the target point cloud data, the material data and the instruction data, determine the point cloud features, material features and instruction features respectively, process the point cloud features, material features and instruction features, and determine the initial position of the collaborative robot.

[0053] As an example, the central control node may use, but is not limited to, OpenCV or other specialized depth perception libraries to convert the measured 3D data to target point cloud data that can be processed by a neural network model. The target point cloud data here is point cloud data for input into a target VLA model for recognition processing. The central control node may use the target VLA model to perform instruction extraction on the target point cloud data, the material data, and the instruction data input in multiple modal forms, respectively, to determine the point cloud features corresponding to the target point cloud data, the material features corresponding to the material data, and the instruction features corresponding to the instruction data; then, the point cloud features, material features, and instruction features are fused to complete the decoding and output corresponding to a specific task, and determine that the collaborative robot needs to be controlled to move to an initial position where the 2D camera has a better shooting angle. In this example, the target VLA model can perceive multimodal data such as images, text, and voice. Specifically, it can dynamically analyze and determine the first position of the target material at the first work point A based on the material characteristics and the point cloud characteristics, and dynamically analyze and determine that the target material needs to be moved to the second position of the second work point B based on the instruction characteristics and the point cloud characteristics. Based on the first position and the second position, a planned path between the two is determined. Based on the planned path, the position where the 2D camera set on the collaborative robot has a better shooting angle is determined, and this position is determined as the initial position of the collaborative robot.

[0054] In one embodiment, the measured 3D data includes first 3D data collected by a first 3D camera and second 3D data collected by a second 3D camera, wherein the first 3D camera is disposed on the collaborative robot and the second 3D camera is disposed outside the collaborative robot; Step S31, i.e., converting the measured 3D data to determine target point cloud data, includes: S311: Convert the first 3D data and the second 3D data to determine first point cloud data corresponding to the first 3D data and second point cloud data corresponding to the second 3D data; S312: Determining a first fusion coefficient corresponding to the first 3D camera and a second fusion coefficient corresponding to the second 3D camera based on the position and posture data corresponding to the first 3D camera and the position and posture data corresponding to the second 3D camera; S313: Determine target point cloud data based on the first point cloud data, the second point cloud data, the first fusion coefficient, and the second fusion coefficient.

[0055] As an example, the central control node may utilize, but is not limited to, OpenCV or other specialized depth perception libraries to convert the first 3D data and the second 3D data, respectively, to convert the first 3D data into first point cloud data and the second 3D data into second point cloud data. The first point cloud data includes a point cloud position and a first point cloud feature corresponding to the point cloud position, and the second point cloud data includes a point cloud position and a second point cloud feature corresponding to the point cloud position. The point cloud features herein include, but are not limited to, image depth.

[0056] As an example, the central control node may communicate with two 3D cameras (i.e., a first 3D camera and a second 3D camera) to determine the position and posture data corresponding to each of the two 3D cameras. Alternatively, the central control node may communicate with position and posture sensors mounted on each of the two 3D cameras to obtain the position and posture data corresponding to each 3D camera, respectively, collected by the two position and posture sensors. Based on the position and posture data corresponding to the two 3D cameras and the material position of the target material captured by them, the central control node determines the corresponding fusion coefficients for the two 3D cameras, namely, a first fusion coefficient corresponding to the first 3D camera and a second fusion coefficient corresponding to the second 3D camera. In this example, when the first and second 3D cameras are used to capture the same target material, the first, second, and target materials are spatially arranged in a triangle. Based on the position and posture data of the two 3D cameras and the material position of the target material, the central control node determines the corresponding fusion coefficient for each 3D camera.

[0057] As an example, the central control node can perform weighted fusion on the first point cloud features and the second point cloud features corresponding to the same point cloud position based on the first point cloud data and the first fusion coefficient corresponding to the first 3D camera, and the second point cloud data and the second fusion coefficient corresponding to the second 3D camera, to determine the target point cloud features corresponding to each point cloud position; based on the target point cloud features corresponding to all point cloud positions, the target point cloud data is determined to ensure that the target point cloud data feedback reflects the image features captured by the two 3D cameras at different angles, which helps to ensure the accuracy of subsequent recognition processing.

[0058] In one embodiment, step S2, i.e., controlling the collaborative robot to move to the initial position, includes: Controlling the movement of the collaborative robot based on the dynamic position output by the target VLA model, and determining measured movement data of the collaborative robot moving to the dynamic position; If the measured movement data does not meet the preset safety conditions, controlling the collaborative robot to stop working; The dynamic position is that the target VLA model processes the measured 3D data to determine the position to be moved at the next moment.

[0059] As an example, the target VLA model can also determine the dynamic position to which it needs to move next based on the measured 3D data collected in real time. This dynamic position is then sent to the central control node, allowing the central control node to control the movement of the collaborative robot based on the dynamic position and collect in real time the measured movement data of the collaborative robot from its current position to the dynamic position. This measured movement data includes, but is not limited to, measured speed, measured rotation angle, and other data. The measured movement data is then compared with preset safety conditions. If the measured movement data meets the preset safety conditions, the dynamic position output based on the target VLA model is repeatedly executed to control the movement of the collaborative robot and determine the measured movement data for the collaborative robot to move to the dynamic position. If the measured movement data does not meet the preset safety conditions, the collaborative robot is controlled to stop operating. This allows the target VLA model to dynamically drive the collaborative robot to operate, ensuring the safety of the composite robot's operation. The preset safety conditions herein include, but are not limited to, speed safety thresholds and angle safety thresholds. If the measured speed is not within the speed safety threshold or the measured rotation angle is not within the angle safety threshold, the preset safety conditions are determined to be unsatisfied, and the collaborative robot needs to be controlled to stop operating.

[0060] In one embodiment, the collaborative robot includes a plurality of robotic arms rotatably connected in sequence, each of the robotic arms being connected to a drive motor; The composite robot control method further includes: During the movement of the collaborative robot, obtaining measured motor data corresponding to a plurality of drive motors; When the measured motor data corresponding to at least one of the drive motors meets the motor risk condition, controlling the collaborative robot to stop working; Wherein, the measured motor data includes a measured current loop and a measured torque loop; The motor risk condition is that the measured current loop is greater than the current loop threshold, or the measured torque loop is greater than the torque loop threshold.

[0061] As an example, when controlling the movement of a collaborative robot, the central control node can obtain real-time measured motor data corresponding to multiple drive motors, specifically during the process of moving to an initial position or a target position. The measured motor data includes, but is not limited to, a measured current loop and a measured torque loop. The measured current loop is the current loop detected in real time by the drive motor; the measured torque loop is the torque loop detected in real time by the drive motor. Then, the measured motor data corresponding to at least one drive motor is compared with a pre-set motor risk condition, specifically comparing the measured current loop with a pre-set current loop threshold, and comparing the measured torque loop with a pre-set torque loop threshold. When the measured current loop is greater than the current loop threshold, or the measured torque loop is greater than the torque loop threshold, it is determined that the measured motor data of the drive motor meets the motor risk condition and determines that the drive motor is at risk. At this time, it is necessary to control the collaborative robot to stop working to ensure the safety of the composite robot. Conversely, when the measured current loop is not greater than the current loop threshold, or the measured torque loop is not greater than the torque loop threshold, it is determined that the measured motor data of the drive motor does not meet the motor risk condition, and it is determined that the drive motor is in a safe state, and the collaborative robot can continue to be controlled.

[0062] In one embodiment, the end effector is provided with a second sensing device; After controlling the end effector provided on the collaborative robot to perform the target operation corresponding to the instruction data on the target material, the composite robot control method further includes: Obtaining a measured sensing distance corresponding to a second sensing device, where the measured sensing distance is used to represent a distance between the end effector and the material placement frame; When the measured sensing distance is less than the preset safety distance, the end effector is controlled to stop working.

[0063] As an example, when the central control node controls the end effector provided on the collaborative robot and performs the target operation corresponding to the instruction data on the target material, it obtains the measured sensing distance corresponding to the second sensing device provided on the end effector in real time. The measured sensing distance is used to represent the distance between the end effector and the material placement frame. This distance can reflect whether the end effector will touch the material placement frame during movement. Then, the measured sensing distance is compared with the preset safety distance. When the measured sensing distance is less than the preset safety distance, it is determined that there is a high probability that the end effector will touch the material placement frame during movement, and there is a risk. Therefore, it is necessary to control the end effector to stop working. On the contrary, when the measured sensing distance is not less than the preset safety distance, it is determined that the end effector will not touch the material placement frame during movement, and there is no risk. The end effector can continue to be controlled to work. The material placement frame here refers to a limiting structure set at the working point for placing materials.

[0064] In this example, when it is necessary to perform a unloading operation on the target material, the clamping mechanism is specifically controlled to clamp the target material from the material placement frame, and the distance between the clamping mechanism and the material placement frame must be no less than the preset safety distance to ensure the smooth progress of the unloading operation; when it is necessary to perform a loading operation on the target material, the clamping mechanism can be controlled to be placed in the material placement frame, and the distance between the clamping mechanism and the material placement frame must be no less than the preset safety distance to ensure the smooth progress of the loading operation; during the loading operation or unloading operation, the clamping mechanism needs to be controlled to stop working only when it is detected that the distance between the clamping mechanism and the material placement frame is less than the preset safety distance.

[0065] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0066] In one embodiment, a control device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the control method of the composite robot in the above embodiment is implemented, for example Figure 4 To avoid repetition, S1-S6 are not described here again.

[0067] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the control method of the composite robot in the above embodiment is implemented, for example Figure 4 To avoid repetition, S1-S5 shown are not described here again.

[0068] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0069] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0070] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A composite robot control method, characterized in that: include: Obtaining a control request, wherein the control request includes material data and instruction data corresponding to a target material; In response to the control request, controlling the 3D camera to operate and obtaining measured 3D data collected by the 3D camera; Processing the measured 3D data, the material data, and the instruction data using a target VLA model to determine an initial position of the collaborative robot; Controlling the collaborative robot to move to the initial position, controlling a 2D camera provided on the collaborative robot to operate, and acquiring 2D data collected by the 2D camera; determining a target position of the collaborative robot based on the 2D data, and controlling the collaborative robot to move to the target position; The end effector provided on the collaborative robot is controlled to perform the target operation corresponding to the instruction data on the target material.

2. The composite robot control method according to claim 1, characterized in that: The controlling the 3D camera to obtain the measured 3D data collected by the 3D camera includes: Controlling the 3D camera to operate and obtain initial 3D data collected by the 3D camera; performing target tracking based on the initial 3D data and material data corresponding to the target material, and determining a current shooting angle of the 3D camera shooting the target material; If the current shooting angle is within a preset angle range, determining the initial 3D data as the measured 3D data; If the current shooting angle is not within the preset angle range, the position and / or posture of the 3D camera is adjusted, and the 3D camera control operation is repeatedly performed to obtain initial 3D data collected by the 3D camera.

3. The composite robot control method according to claim 2, characterized in that: The composite robot includes a middle cabinet, a collaborative robot and an induction pan-tilt platform arranged on the middle cabinet; The collaborative robot includes a plurality of robotic arms that are rotatably connected in sequence, wherein two adjacent robotic arms are rotatably connected, the first robotic arm is mounted on the middle cabinet via a fixed base, and the last robotic arm is connected to the end effector via an adapter, wherein the adapter is provided with a first 3D camera and the 2D camera; The induction pan-tilt platform includes a lifting mechanism, a rotating mechanism and a swing mechanism. The lifting mechanism is installed on the middle cabinet, the rotating mechanism is installed on the lifting mechanism, the swing mechanism is installed on the rotating mechanism, and a second 3D camera is provided on the swing mechanism. If the current shooting angle is not within the preset angle range, adjusting the position and / or posture of the 3D camera includes: If the current shooting angle of the first 3D camera to the target material is not within a first preset angle range, controlling at least one of the plurality of robotic arms to rotate to adjust the position and / or posture of the first 3D camera; If the current shooting angle of the second 3D camera for the target material is not within a second preset angle range, at least one of the lifting mechanism, the rotating mechanism and the swing mechanism is controlled to operate to adjust the position and / or posture of the second 3D camera.

4. The composite robot control method according to claim 3, characterized in that: A movable chassis is provided under the middle cabinet; Before controlling the 3D camera to operate and obtaining the measured 3D data collected by the 3D camera, the composite robot control method further includes: Control the mobile chassis to move the middle cabinet to the working point where the target material is located; The working point where the target material is located is determined based on working scene data, and the working scene data is data obtained by photographing the working scene with a working scene camera.

5. The composite robot control method according to claim 4, characterized in that: The mobile chassis is provided with a first sensing device; The control of the mobile chassis to move the middle cabinet to the working point where the target material is located includes: Controlling the mobile chassis to move based on the target path, and acquiring first sensing data collected by the first sensing device in real time; performing obstacle detection based on the first sensing data, and determining an obstacle detection result; If the obstacle detection result indicates that an obstacle exists, a target obstacle avoidance strategy is executed to control the operation of the mobile chassis; If the obstacle detection result is that there is no obstacle, the mobile chassis is controlled to continue moving along the target path until the middle cabinet moves to the working point where the target material is located.

6. The composite robot control method according to claim 1, characterized in that: The process of processing the measured 3D data, the material data, and the instruction data using the target VLA model to determine the initial position of the collaborative robot includes: Converting the measured 3D data to determine target point cloud data; A target VLA model is used to extract features of the target point cloud data, the material data and the instruction data, and point cloud features, material features and instruction features are determined respectively. The point cloud features, material features and instruction features are processed to determine the initial position of the collaborative robot.

7. The composite robot control method according to claim 6, characterized in that: The measured 3D data includes first 3D data collected by a first 3D camera and second 3D data collected by a second 3D camera, wherein the first 3D camera is disposed on the collaborative robot and the second 3D camera is disposed outside the collaborative robot; The converting the measured 3D data to determine target point cloud data includes: Converting the first 3D data and the second 3D data to determine first point cloud data corresponding to the first 3D data and second point cloud data corresponding to the second 3D data; determining, based on position and posture data corresponding to the first 3D camera and position and posture data corresponding to the second 3D camera, a first fusion coefficient corresponding to the first 3D camera and a second fusion coefficient corresponding to the second 3D camera; Target point cloud data is determined based on the first point cloud data, the second point cloud data, the first fusion coefficient, and the second fusion coefficient.

8. The composite robot control method according to claim 1, wherein: The controlling the collaborative robot to move to the initial position includes: Controlling the movement of the collaborative robot based on the dynamic position output by the target VLA model, and determining measured movement data of the collaborative robot moving to the dynamic position; If the measured movement data does not meet the preset safety conditions, controlling the collaborative robot to stop working; The dynamic position is that the target VLA model processes the measured 3D data to determine the position to be moved at the next moment.

9. The composite robot control method according to claim 1, characterized in that: The collaborative robot includes a plurality of mechanical arms that are rotatably connected in sequence, and each of the mechanical arms is connected to a drive motor; The composite robot control method further includes: During the movement of the collaborative robot, obtaining measured motor data corresponding to a plurality of drive motors; When the measured motor data corresponding to at least one of the drive motors meets the motor risk condition, controlling the collaborative robot to stop working; Wherein, the measured motor data includes a measured current loop and a measured torque loop; The motor risk condition is that the measured current loop is greater than the current loop threshold, or the measured torque loop is greater than the torque loop threshold.

10. The composite robot control method according to claim 1, characterized in that: The end effector is provided with a second sensing device; After controlling the end effector provided on the collaborative robot to perform the target operation corresponding to the instruction data on the target material, the composite robot control method further includes: Obtaining a measured sensing distance corresponding to a second sensing device, where the measured sensing distance is used to represent a distance between the end effector and the material placement frame; When the measured sensing distance is less than the preset safety distance, the end effector is controlled to stop working.

11. A control device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the composite robot control method according to any one of claims 1 to 10 is implemented.

12. A composite robot, characterized in that: Comprising the control device according to claim 11.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the composite robot control method according to any one of claims 1 to 10 is implemented.

Images