Control system and robot system

Abstract

The application discloses a control system and a robot system. The control system comprises a vision module configured to collect image data of a workpiece to be worked on; an edge computing module in communication connection with the vision module, configured to receive the image data sent by the vision module and to pre-process the image data, the pre-processing comprising at least one of cleaning, filtering and compressing the image data; and a cloud computing platform in communication connection with the edge computing module, configured to receive the pre-processed image data sent by the edge computing module and to generate a first work instruction of a robot according to the image data; wherein the edge computing module is further configured to receive the first work instruction sent by the cloud computing platform and to control the robot to perform a work task according to the first work instruction. The design of the application can realize demonstration-free of the robot.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a control system and a robot system. Background Technology

[0002] In industrial manufacturing, welding is a crucial process widely used in metal processing and assembly. However, traditional welding robot systems typically require operators to teach the welding trajectory, meaning the robot manually performs a series of actions to record the process, enabling it to execute the same welding task in repetitive production. This method has the following problems: 1. Low efficiency: Manual teaching is time-consuming, especially when dealing with complex shapes or mass production, making it difficult to meet the demands of modern manufacturing; 2. Limited accuracy: The accuracy of manual teaching depends on the operator's experience and skills, making it susceptible to human error and leading to inconsistent welding quality; 3. Difficulty adapting to complex workpieces: Manual teaching is difficult for workpieces with complex shapes and diverse dimensions, and it cannot handle subtle differences between workpieces, easily leading to welding defects; 4. High cost: Manual teaching requires highly skilled operators, increasing labor costs. Furthermore, potential errors during the teaching process can cause material and equipment waste. Utility Model Content

[0003] This application provides a control system and a robot system that enable robots to operate without teaching.

[0004] This application provides a control system comprising: a vision module for acquiring image data of a workpiece to be worked on; an edge computing module communicatively connected to the vision module for receiving the image data sent by the vision module and preprocessing the image data, the preprocessing including at least one of cleaning, filtering, and compressing the image data; and a cloud computing platform communicatively connected to the edge computing module for receiving the preprocessed image data sent by the edge computing module and generating a first work instruction for the robot based on the image data; wherein the edge computing module is further configured to receive the first work instruction sent by the cloud computing platform and control the robot to perform a work task based on the first work instruction.

[0005] The vision module includes a 3D camera, and the image data acquired by the vision module includes three-dimensional image data of the workpiece to be worked on, acquired by the 3D camera.

[0006] The cloud computing platform includes: an image processing unit for receiving image data sent by the edge computing module and identifying the work position on the workpiece; a trajectory planning unit connected to the image processing unit for planning the robot's motion trajectory based on the work position on the workpiece; and an instruction generation unit connected to the trajectory planning unit for generating the first operation instruction based on the motion trajectory.

[0007] The cloud computing platform further includes a deep learning module, which connects the trajectory planning unit and the instruction generation unit, for optimizing the motion trajectory planned by the trajectory planning unit using a deep learning algorithm, and sending the optimized motion trajectory to the instruction generation unit.

[0008] The cloud computing platform further includes a data storage unit connected to the deep learning module for storing the robot's historical motion data; wherein the deep learning module is also used to optimize the motion trajectory based on the historical motion data and using the deep learning algorithm.

[0009] The image processing unit is further configured to: identify the job type of the workpiece to be worked on based on the image data; the instruction generation unit is further configured to: generate the first job instruction based on the job type and the motion trajectory.

[0010] The cloud computing platform further includes: a real-time adjustment unit connected to the instruction generation unit, used to obtain real-time data of the robot performing the task through the edge computing module and dynamically adjust the task parameters according to the real-time data; the instruction generation unit is also used to: adjust the first task instruction in real time according to the task parameters.

[0011] The control system further includes a front-end control module connected to the edge computing module. The front-end control module displays the robot's operation process through a human-machine interface, receives a second operation instruction input by the user through the human-machine interface, and sends the second operation instruction to the edge computing module so that the edge computing module controls the robot to perform the operation task according to the second operation instruction.

[0012] The edge computing module establishes a communication connection with the front-end control module through an internal network; the edge computing module establishes a communication connection with the cloud computing platform through the public Internet.

[0013] This application also provides a robot system including a robot and a control system as described in any of the above embodiments.

[0014] Unlike existing technologies, the beneficial effects of this application are as follows: The control system of this application includes a vision module, an edge computing module, and a cloud computing platform. The vision module is used to collect image data of the workpiece to be operated. After transmitting the image data to the edge computing module, the edge computing module preprocesses the image data. The preprocessing includes at least one of cleaning, filtering, and compression of the image data. The preprocessed image data is then transmitted to the cloud computing platform. The cloud computing platform generates a first operation instruction based on the image data and then transmits the first operation instruction to the edge computing module. The edge computing module controls the robot to complete the operation task according to the first operation instruction. It can be seen that this application can improve the transmission speed by preprocessing the image data through the edge computing module, thereby improving the response speed of the cloud computing platform. At the same time, by generating the first operation instruction based on the image data through the cloud computing platform, there is no need for manual teaching of the robot, realizing robot teaching-free operation. In addition, the cloud computing platform can reduce local manpower and equipment costs and also help improve the working effect of the robot. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0016] Figure 1 This is a schematic diagram of one embodiment of the robot system of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] See Figure 1This application provides a control system, which includes a vision module 100, an edge computing module 200, and a cloud computing platform 300. The vision module 100 is used to acquire image data of the workpiece to be worked on. The edge computing module 200 is communicatively connected to the vision module 100 and is used to receive image data sent by the vision module 100 and preprocess the image data, including at least one of cleaning, filtering, and compression. The cloud computing platform 300 is communicatively connected to the edge computing module 200 and is used to receive the preprocessed image data sent by the edge computing module 200 and generate a first work instruction for the robot 400 based on the image data. The edge computing module 200 is also used to receive the first work instruction sent by the cloud computing platform 300 and control the robot 400 to perform the work task according to the first work instruction.

[0019] Specifically, the vision module 100 is used to acquire image data of the workpiece's working area. Optionally, the vision module 100 can also acquire image data of all areas of the workpiece. The image data includes at least one of point clouds, RGB images, and grayscale images. After acquiring the image data, the vision module 100 sends the image data to the edge computing module 200. The edge computing module 200 preprocesses the image data to reduce its volume. This is because, in subsequent processes, the edge computing module 200 needs to send the preprocessed image data to the cloud computing platform 300. Reducing the image data volume allows for faster data transmission to the cloud computing platform 300, while also reducing the data processing load on the cloud computing platform 300 and improving its response speed. Specifically, preprocessing can involve cleaning the image data (removing low-quality data and retaining high-quality data), filtering the image data (selecting features and retaining necessary features), or compressing the image data (simplifying the image data). The above preprocessing methods can be selected as one or more during the specific implementation process.

[0020] Furthermore, after receiving the preprocessed image data sent by the edge computing module 200, the cloud computing platform 300 processes the image data and generates the first work instruction for the robot 400. Then, it controls the robot 400 to execute the work task according to the first work instruction. It should be noted that the cloud computing platform 300 has powerful computing capabilities and can process large amounts of data. The advantage of the cloud computing platform 300 is that by having the image data analyzed and processed by the cloud computing platform 300, manual teaching is eliminated. After the image data is given to the cloud computing platform 300, it automatically generates the first work instruction based on the image data, which is then executed by the edge computing module 200. This improves work efficiency compared to manual teaching. Furthermore, manual teaching suffers from inconsistent accuracy, while the cloud computing platform 300 is far superior in accuracy. For workpieces with complex shapes, the cloud computing platform 300 has an even greater advantage. Secondly, the cloud computing platform 300 reduces the need for local manpower and local equipment computing power, thus reducing local labor and equipment costs. Moreover, it eliminates the need for manually importing drawings and models of the workpiece. In summary, compared with the prior art, this application uses a combination of vision module 100, edge computing module 200 and cloud computing platform 300 to control robot 400 to perform work tasks on the workpiece without teaching, while also improving the working efficiency of robot 400.

[0021] It should be noted that the workpiece to be operated can be a workpiece to be welded, and the cloud computing platform 300 generates corresponding welding operation instructions based on the image data. The workpiece to be operated can also be a workpiece to be sprayed, and the cloud computing platform 300 generates corresponding spraying operation instructions based on the image data. Of course, the workpiece to be operated can also be other workpieces.

[0022] The aforementioned edge computing module 200 can be integrated into an existing industrial control computer.

[0023] In one embodiment, the vision module 100 includes a 3D camera, and the image data acquired by the vision module 100 includes three-dimensional image data of the workpiece to be worked on, acquired by the 3D camera.

[0024] Specifically, the 3D camera can acquire three-dimensional image data of the workpiece to be worked on. Based on the three-dimensional image data, the cloud computing platform 300 can generate the first work instruction in three-dimensional space, thereby enabling the workpiece to be worked on in three-dimensional space. The 3D camera includes line laser 3D cameras and binocular structured light 3D cameras.

[0025] In one application scenario, the 3D camera collects point cloud data and uses 360-degree point cloud construction technology to construct a complete point cloud based on the size and orientation of the workpiece to be worked on. The number and location of the scanned points can be dynamically changed according to the different positions of the workpiece to be worked on.

[0026] In some other embodiments, the vision module 100 may also include a 2D camera, and the image data acquired by the vision module 100 includes two-dimensional image data of the workpiece to be processed acquired by the 2D camera, which is used to process planar workpieces.

[0027] In one embodiment, see Figure 1 The cloud computing platform 300 includes an image processing unit 310, a trajectory planning unit 320, and an instruction generation unit 330. The image processing unit 310 receives image data sent by the edge computing module 200 and identifies the work position on the workpiece to be worked on. The trajectory planning unit 320 is connected to the image processing unit 310 and is used to plan the motion trajectory of the robot 400 based on the work position on the workpiece to be worked on. The instruction generation unit 330 is connected to the trajectory planning unit 320 and is used to generate a first work instruction based on the motion trajectory.

[0028] Specifically, the image processing unit 310 can process image data. For example, during actual operation, image processing algorithms can be used to process image data. After processing, the position of the workpiece to be worked on can be identified. For example, if the workpiece is to be welded, the position is the welding position; if the workpiece is to be painted, the position is the painting position, and so on. When the trajectory planning unit 320 obtains the position of the workpiece to be worked on, it makes an overall plan for the work path, that is, plans the motion trajectory of the robot 400. Then, it sends the motion trajectory to the instruction generation unit 330. The instruction generation unit 330 is used to convert the motion trajectory into an instruction that the edge computing module 200 can execute, that is, the first work instruction. It can be seen that through the three units of image processing unit 310, trajectory planning unit 320 and instruction generation unit 330, image data can be automatically processed and finally converted into the first work instruction. The whole process automatically completes the planning of the work path of the workpiece to be worked on without manual intervention.

[0029] In one embodiment, the cloud computing platform 300 further includes a deep learning module 340, which is connected to the trajectory planning unit 320 and the instruction generation unit 330. The deep learning module 340 is used to optimize the motion trajectory planned by the trajectory planning unit 320 using a deep learning algorithm and send the optimized motion trajectory to the instruction generation unit 330.

[0030] Specifically, the deep learning module 340 can continuously train the motion trajectory based on deep learning algorithms, thereby continuously optimizing the motion trajectory. Specifically, each time the vision module 100 collects image data of the workpiece to be worked on, it sends it to the edge computing module 200. The image data is then transmitted from the edge computing module 200 to the cloud computing platform 300. The image data is then processed by the image processing unit 310, and the motion trajectory is generated by the trajectory planning unit 320. The motion trajectory is then provided to the deep learning module 340. Each collection of image data ultimately serves as a training sample for the deep learning module 340, and the motion trajectory is optimized through reinforcement learning during continuous training.

[0031] In one embodiment, the cloud computing platform 300 further includes a data storage unit 350. The data storage unit 350 is connected to the deep learning module 340 and is used to store historical motion data of the robot 400; wherein, the deep learning module 340 is also used to optimize the motion trajectory based on the historical motion data and using deep learning algorithms.

[0032] Specifically, the data storage unit 350 is used to store data, including the historical motion data of the robot 400. This historical motion data includes the motion trajectory optimized by the deep learning module 340 each time. The data storage unit 350 can store a large amount of historical motion data of the robot 400. This historical motion data provides a large number of training samples for the deep learning module 340, and it is continuously updated after the robot 400 performs its tasks, thereby continuously optimizing the motion trajectory. Furthermore, the deep learning module 340 can also provide a better motion trajectory based on the historical motion data even when the image data is poor (e.g., poor lighting conditions or poor shooting posture). Of course, it is understood that the data storage unit 350 can also store other data, such as image data.

[0033] Of course, in some other implementations, a data storage unit may not be provided. Instead, the deep learning module 340 can be provided with preset training samples for pre-training, and the optimized scheme can be applied to the optimization of the motion trajectory.

[0034] In one embodiment, the image processing unit 310 is further configured to: identify the type of workpiece to be worked on based on the image data.

[0035] Specifically, for example, when the workpiece to be welded is a workpiece to be welded, the job type includes flat welds and vertical welds. The image processing unit 310 identifies the planes on both sides of the weld in the image data, and then determines whether the normal vectors of the two planes are parallel or intersecting to determine whether it is a flat weld or a vertical weld. That is, if the normal vectors of the two planes are determined to be parallel, the weld is determined to be a flat weld; if the normal vectors of the two planes are determined to be intersecting, the weld is determined to be a vertical weld. The job type can also be for different materials; for example, steel and aluminum can use different process parameters. Of course, the job types in this application are not limited to the above situations.

[0036] Furthermore, the instruction generation unit 330 is also used to generate a first operation instruction based on the operation type and the motion trajectory.

[0037] Specifically, different job types require different process parameters. For example, the aforementioned flat welds and vertical welds require different processes: flat welds use a flat fillet weld process, and vertical welds use a vertical fillet weld process. By selecting different process parameters for different job types, the quality of the robot 400's work on various workpieces can be improved. Therefore, the instruction generation unit 330 can generate a first job instruction by combining the job type and motion trajectory.

[0038] In one embodiment, the cloud computing platform 300 further includes a real-time adjustment unit 360, which is connected to the instruction generation unit 330. The real-time adjustment unit 360 is used to obtain real-time data of the robot 400 performing the task through the edge computing module 200 and dynamically adjust the task parameters according to the real-time data. The instruction generation unit 330 is also used to adjust the first task instruction in real time according to the task parameters.

[0039] Specifically, when the robot 400 performs a task, it generates corresponding real-time data. This real-time data is sent to the edge computing module 200, which then sends it to the real-time adjustment unit 360. The real-time adjustment unit 360 analyzes the real-time data, which includes, for example, real-time path, real-time temperature, real-time speed, and real-time angle. To ensure consistent task performance, the real-time adjustment unit 360 dynamically adjusts the task parameters in real time (i.e., adjustments are made simultaneously with the task) to guarantee task quality and improve task consistency. Furthermore, the instruction generation unit 330 can also adjust and send a first task instruction to the edge computing module 200 based on the real-time data sent by the real-time adjustment unit 360.

[0040] Of course, in some other implementations, the real-time adjustment unit may not be used, which can save resources of the cloud computing platform, but its operation quality is slightly inferior to the above implementation.

[0041] In one embodiment, the control system further includes a front-end control module 500, which is connected to the edge computing module 200. The front-end control module 500 displays the operation process of the robot 400 through a human-machine interface, and receives a second operation instruction input by the user through the human-machine interface, and sends the second operation instruction to the edge computing module 200 so that the edge computing module 200 controls the robot 400 to perform the operation task according to the second operation instruction.

[0042] Specifically, robot 400 sends real-time data to front-end control module 500 via edge computing module 200. Users can view the robot 400's operation process through the human-machine interface of front-end control module 500. For example, users can view the real-time progress of the operation, real-time parameters, and any anomalies encountered. Through front-end control module 500, users can gain a comprehensive understanding of the robot 400's entire operation process, thus better monitoring its work. Users can also make adjustments based on the findings observed during robot 400's operation. They can send a second operation command to edge computing module 200 via front-end control module 500, and edge computing module 200 will then control robot 400 according to the second operation command. Furthermore, users can access cloud computing platform 300 through front-end control module 500. Generally, the second operation command has a higher priority than the first operation command. Front-end control module 500 facilitates real-time viewing of robot 400's operation process and allows users to issue operation commands promptly. Front-end control module 500 can be a tablet computer or a computer, etc.

[0043] In one embodiment, the edge computing module 200 establishes a communication connection with the front-end control module 500 through an internal network; the edge computing module 200 establishes a communication connection with the cloud computing platform 300 through the public Internet.

[0044] Specifically, placing the front-end control module 500 and the edge computing module 200 within an internal network enhances communication security. Generally, internal networks are equipped with firewalls or encryption technologies to ensure data security. Then, the edge computing module 200 is connected to a cloud computing platform 300 on the public internet. Typically, the computing power of the public internet cloud computing platform 300 far exceeds that of a privately deployed cloud computing platform, eliminating the need for user deployment.

[0045] See Figure 1 This application also provides a robot system 10, which includes a robot 400 and a control system as described in any of the above embodiments.

[0046] It should be noted that the detailed structure of the control system in the robot system 10 can be referred to the above-described embodiment of the control system, and will not be repeated here. The robot 400 can be a welding robot or a painting robot, etc. Since the above-described control system is used in the robot system 10 of this application, the embodiments of the robot system 10 of this application include all the technical solutions of all the embodiments of the above-described control system, and the technical effects achieved are exactly the same, and will not be repeated here.

[0047] In one application scenario, the robot system 10 of this application is used for welding BK (Ballast keel) plates in the shipbuilding process. A 3D camera is mounted and fixed to the hand of the robot 400 through a mechanical interface. The 3D camera is connected to the edge computing module 200 through a network interface and connected to an adapter power supply. Preprocessing algorithms and other units, as well as camera drivers, are deployed on the edge computing module 200. An internal network is deployed between the edge computing module 200 and the front-end control module 500. The robot 400 and the edge computing module 200 establish communication through the internal network. A communication connection is established between the cloud computing platform 300 and the edge computing module 200, and the cloud computing platform 300 is accessed through the front-end control module 500.

[0048] 1. Operation Procedure: Place the BK plate to be welded on the welding platform. After starting the robot system 10, the 3D camera automatically captures the 3D image data of the BK plate and sends it to the edge computing module 200. The edge computing module 200 preprocesses the 3D image data and uploads it to the cloud computing platform 300. The cloud computing platform 300 automatically generates a welding path based on the shape and material in the preprocessed 3D image data, and the instruction generation unit 330 generates a first operation instruction and sends it to the edge computing module 200. The edge computing module 200 then controls the robot 400 to execute the welding task.

[0049] 2. Practical Results: The robot system 10 of this application can automatically complete the welding task of BK boards without teaching, without any manual intervention. The quality of the welded BK boards is stable. Compared with the existing technology that requires manual teaching, the efficiency of the welding process is increased by more than 30%, the number of operators on the line is reduced by half, and labor costs are reduced.

[0050] Of course, it should be noted that this application is not limited to the above application scenarios. The robot system 10 of this application is applicable to components such as small components, small groups, and medium groups.

[0051] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A control system, characterized in that, The control system includes: The vision module is used to acquire image data of the workpiece to be processed; An edge computing module, communicatively connected to the vision module, is used to receive the image data sent by the vision module and preprocess the image data, wherein the preprocessing includes at least one of cleaning, filtering, and compressing the image data; A cloud computing platform, which is communicatively connected to the edge computing module, is used to receive the preprocessed image data sent by the edge computing module and generate the robot's first operation instruction based on the image data; The edge computing module is further configured to receive the first job instruction sent by the cloud computing platform and control the robot to perform job tasks according to the first job instruction.

2. The control system according to claim 1, characterized in that, The vision module includes a 3D camera, and the image data acquired by the vision module includes three-dimensional image data of the workpiece to be worked on, acquired by the 3D camera.

3. The control system according to claim 1, characterized in that, The cloud computing platform includes: An image processing unit is used to receive the image data sent by the edge computing module and identify the position to be worked on the workpiece. A trajectory planning unit, connected to the image processing unit, is used to plan the robot's motion trajectory based on the workpiece's position to be worked on. The instruction generation unit, connected to the trajectory planning unit, is used to generate the first operation instruction based on the motion trajectory.

4. The control system according to claim 3, characterized in that, The cloud computing platform also includes: A deep learning module, connected to the trajectory planning unit and the instruction generation unit, is used to optimize the motion trajectory planned by the trajectory planning unit using a deep learning algorithm, and send the optimized motion trajectory to the instruction generation unit.

5. The control system according to claim 4, characterized in that, The cloud computing platform also includes: A data storage unit, connected to the deep learning module, is used to store the robot's historical motion data; The deep learning module is further used to optimize the motion trajectory based on the historical motion data and using the deep learning algorithm.

6. The control system according to claim 3, characterized in that, The image processing unit is further configured to: identify the type of workpiece to be worked on based on the image data; The instruction generation unit is further configured to: generate the first operation instruction based on the operation type and the motion trajectory.

7. The control system according to claim 3, characterized in that, The cloud computing platform also includes: The real-time adjustment unit, connected to the instruction generation unit, is used to obtain real-time data of the robot performing the task through the edge computing module and dynamically adjust the task parameters according to the real-time data; the instruction generation unit is also used to adjust the first task instruction in real time according to the task parameters.

8. The control system according to claim 1, characterized in that, The control system further includes: A front-end control module is connected to the edge computing module. The front-end control module displays the robot's operation process through a human-machine interface, and receives a second operation instruction input by the user through the human-machine interface. The second operation instruction is then sent to the edge computing module so that the edge computing module controls the robot to perform the operation task according to the second operation instruction.

9. The control system according to claim 8, characterized in that, The edge computing module establishes a communication connection with the front-end control module through an internal network; The edge computing module establishes a communication connection with the cloud computing platform via the public Internet.

10. A robot system, characterized in that, This includes robots and control systems as described in any one of claims 1 to 9.

Images