Multifunctional vision system, control method thereof, computer device and storage medium

Abstract

The application discloses a kind of multifunctional vision system and its control method, computer equipment and storage medium, belong to intelligent manufacturing technical field.The multifunctional vision system includes: a server;Two robot control cabinets, each robot control cabinet is connected with the server communication;Two robots, each robot is connected with a robot control cabinet communication;Two 3D cameras, each 3D camera is connected with the server communication, each 3D camera is respectively set on the flange of a robot;The server is communicated with the robot and the 3D camera by default communication protocol, and the server is used to realize 3D vision positioning function, 3D vision trajectory rectification function or 2D image flexible mistake-proof detection function according to the instruction sent by the robot.The application improves the utilization of robot, camera, server and other equipment on the automobile production line, avoids resource waste.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing technology, and in particular to a multifunctional vision system and its control method, computer equipment and storage medium. Background Technology

[0002] With the rapid development of intelligent manufacturing technology, automobile manufacturing has gradually moved towards less manpower and unmanned operation, and the application of vision technology has further accelerated the process of unmanned automobile production lines.

[0003] However, most existing vision guidance systems consist of a single server equipped with a single camera, working in conjunction with a robot at the corresponding workstation to locate parts, resulting in limited functionality. Existing error-proofing systems typically use photoelectric sensors or 2D cameras to detect errors in parts, leading to numerous and complex sensor arrangements that do not meet the needs of flexible manufacturing. For automotive production lines, there are many parts loading and unloading stations, a wide variety of car models, and significant requirements for error-proofing. Using a single server at only one workstation and having a single camera with limited functionality results in low equipment utilization and wasted resources. Summary of the Invention

[0004] The main objective of this invention is to provide a multifunctional vision system and its control method, computer equipment and storage medium, aiming to solve the technical problem of how to improve the utilization rate of robots, cameras, servers and other equipment on automobile production lines.

[0005] To achieve the above objectives, the present invention provides a multifunctional vision system, the multifunctional vision system comprising:

[0006] One server;

[0007] Two robot control cabinets, each of which is communicatively connected to the server;

[0008] Two robots, each of which is communicatively connected to one of the robot control cabinets;

[0009] Two 3D cameras, each of which is communicatively connected to the server, and each of which is mounted on the flange of one of the robots;

[0010] The server communicates with the robot and the 3D camera through a preset communication protocol. The server is used to implement 3D visual positioning function, 3D visual trajectory correction function or 2D image flexible error prevention detection function according to the instructions sent by the robot.

[0011] Optionally, the 3D camera includes two 2D cameras and a light source device, and each of the 2D cameras and the light source device is communicatively connected to the server.

[0012] Furthermore, to achieve the above objectives, the present invention also provides a control method for a multifunctional vision system, wherein the control method is applied to the multifunctional vision system described above, and the control method includes the following steps:

[0013] Receive task instructions sent after the robot arrives in position;

[0014] Generate corresponding camera action instructions based on the task instructions;

[0015] The camera is controlled to acquire image information according to the camera action commands;

[0016] The image information is processed by a preset algorithm to obtain the processing result;

[0017] The processing result is returned to the robot so that the robot can complete the task instruction or stop running and issue an alarm based on the processing result.

[0018] Optionally, the task instructions include: positioning instructions and error prevention detection instructions; the camera action instructions include: 3D image acquisition instructions and 2D image acquisition instructions;

[0019] The step of generating corresponding camera motion commands based on the task commands includes:

[0020] When the task instruction is a positioning instruction, a 3D image acquisition instruction is generated;

[0021] When the task instruction is a fault-prevention detection instruction, a 2D image acquisition instruction is generated.

[0022] Optionally, the image information includes: 3D point cloud images and 2D images;

[0023] The step of controlling the camera to acquire image information according to the camera action command includes:

[0024] When the camera action command is a 3D image acquisition command, the 3D image acquisition command is sent to the 3D camera, and the 3D point cloud image acquired by the 3D camera according to the 3D image acquisition command is received;

[0025] When the camera action command is a 2D image acquisition command, the 2D image acquisition command is sent to the 2D camera, and the 2D image acquired by the 2D camera according to the 2D image acquisition command is received.

[0026] Optionally, before the step of invoking a preset algorithm to process the image information, the control method of the multifunctional vision system further includes:

[0027] Receive the part model number sent after the robot arrives in position;

[0028] A preset algorithm is selected based on the task instruction and the part model number.

[0029] Optionally, the preset algorithm includes: a 3D point cloud matching algorithm and a 2D image processing algorithm; the processing result includes: calculation result and detection result;

[0030] The step of calling a preset algorithm to process the image information and obtain the processing result includes:

[0031] When the preset algorithm is a 3D point cloud matching algorithm, the 3D point cloud matching algorithm is called to identify the 3D point cloud image and obtain the identification result;

[0032] When the preset algorithm is a 2D image processing algorithm, the 2D image is detected by calling the 2D image processing algorithm to obtain the detection result.

[0033] Optionally, the identification result includes: target point coordinates and error information; the detection result includes: success information and failure information;

[0034] The step of returning the processing result to the robot, so that the robot can complete the task instruction or stop running and issue an alarm based on the processing result, includes:

[0035] When recognition is successful, the recognition result is the target point coordinates, and the target point coordinates are returned to the robot so that the robot can complete the positioning command based on the target point coordinates;

[0036] When recognition fails, the recognition result is an error message, which is returned to the robot so that the robot stops running and issues an alarm based on the error message.

[0037] When the detection is successful, the detection result is a success message, which is returned to the robot so that the robot can complete the error prevention instruction and continue to run based on the success message;

[0038] When the detection fails, the detection result is a failure message, which is returned to the robot so that the robot stops running and issues an alarm based on the failure message.

[0039] In addition, to achieve the above objectives, the present invention also provides a computer device, the computer device comprising: a memory, a processor, and a control program for a multi-functional vision system stored in the memory and executable on the processor, wherein the control program for the multi-functional vision system, when executed by the processor, implements the steps of the control method for the multi-functional vision system as described above.

[0040] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a control program for a multifunctional vision system, wherein the control program for the multifunctional vision system, when executed by a processor, implements the steps of the control method for the multifunctional vision system as described above.

[0041] This invention proposes a multifunctional vision system and its control method, computer equipment, and storage medium. In this system, two 3D cameras are connected to a server, serving two workstations. Without tooling positioning, the system can guide robots for loading and unloading, welding, and gluing. While providing positioning guidance for the robots, it also enables flexible error-proofing detection of parts at each workstation. Utilizing the robot's flexibility, it can capture images of any part within its reach, achieving flexible error-proofing detection. Integrating part positioning, trajectory correction, and flexible error-proofing significantly improves equipment utilization. The 3D point cloud registration algorithm greatly enhances positioning accuracy. The system automatically calls up the corresponding function and model by sending corresponding instructions and model numbers to the robot, eliminating the need for manual model switching, making it convenient and fast. This improves the utilization rate of robots, cameras, servers, and other equipment on the automotive production line, avoids resource waste, and overcomes the technical shortcomings of existing technologies, such as limited camera functionality, numerous error-proofing sensors, and poor flexible production. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the structure of an embodiment of the multifunctional vision system of the present invention;

[0043] Figure 2 This is a schematic diagram of the structure of a 3D camera according to an embodiment of the multifunctional vision system of the present invention;

[0044] Figure 3 This is a flowchart illustrating an embodiment of the control method for the multifunctional vision system of the present invention;

[0045] Figure 4 This is a schematic diagram of the process of a multi-functional vision system performing a positioning task in one embodiment of the control method of the multi-functional vision system of the present invention;

[0046] Figure 5 This is a flowchart illustrating the execution of error prevention tasks by the multi-functional vision system in one embodiment of the control method for the multi-functional vision system of the present invention.

[0047] Figure 6 This is a schematic diagram of the structure of the computer device involved in the embodiment of the present invention.

[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0050] This invention provides a multifunctional vision system, referring to... Figure 1 , Figure 1 This is a schematic diagram of an embodiment of a multifunctional vision system according to the present invention.

[0051] In this embodiment, the multifunctional vision system includes:

[0052] One server 10;

[0053] Two robot control cabinets 20, each of which is communicatively connected to the server 10;

[0054] Two robots 30, each of which is communicatively connected to one of the robot control cabinets 20;

[0055] Two 3D cameras 40 are provided, each of which is communicatively connected to the server 10, and each of which is mounted on the flange of one of the robots 30.

[0056] The server 10 communicates with the robot 30 and the 3D camera 40 through a preset communication protocol. The server 10 is used to implement 3D visual positioning function, 3D visual trajectory correction function or 2D image flexible error prevention detection function according to the instructions sent by the robot 30.

[0057] It should be noted that in this embodiment, the system uses two robot systems, two 3D cameras 40 and one vision server 10. The 3D cameras 40 are mounted on the flange of the robot 30, and the robot 30 is mounted on both sides of the workstation. In this embodiment, a server 10 is equipped with two 3D (3-dimensional) cameras 40, each mounted on the flange of a robot 30. When the robot 30 moves to the top of a part, the multi-functional vision system can control the 3D cameras 40 through the server 10 to take pictures and generate 3D point cloud images. Then, the point cloud registration algorithm is used to locate the part at the workstation and correct the trajectory of the robot 30. This embodiment utilizes the flexibility of the robot 30 to acquire 2D images of the part at different positions and uses a 2D image model matching algorithm to achieve flexible error prevention detection of the part. The system integrates 3D vision positioning, 3D vision trajectory correction, and 2D image flexible error prevention detection into one system. The server 10, which supports machine vision algorithms, communicates with the robot 30 and the cameras via TCP / IP (Transmission Control Protocol / Internet Protocol). When the robot 30 sends different instructions, the server 10 automatically calls different functions without manual switching.

[0058] Furthermore, as a possible embodiment, refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of the 3D camera 40 in this embodiment.

[0059] In this embodiment, the 3D camera 40 includes two 2D cameras 41 and a light source device 42, and each of the 2D cameras 41 and the light source device 42 is communicatively connected to the server 10.

[0060] It should be noted that in this embodiment, two 2D cameras 41 and one light source device 42 are combined to form a 3D camera 40. Each of the 2D cameras 41 acquires grayscale images from different angles. Feature points of the two captured grayscale images are matched, and a depth map is generated based on the intrinsic and extrinsic parameters of the 2D cameras 41. The depth map is then converted into a point cloud to generate a 3D point cloud image. The light source device 42 is a small projector that projects blue striped structured light with alternating bright and dark areas. The brightness of the light source can be adjusted according to the actual shooting requirements.

[0061] This embodiment provides a multifunctional vision system that connects two 3D cameras to a single server, serving two workstations and increasing space utilization and equipment efficiency. Even without tooling positioning, the system can guide robots for loading and unloading, welding, and gluing. While providing positioning guidance for the robot, it also enables flexible error-proofing detection of parts at the workstation. Utilizing the robot's flexibility, it can capture images of any part within its reach, achieving flexible error-proofing detection. Integrating part positioning, trajectory correction, and flexible error-proofing into one system significantly improves equipment utilization. This embodiment features a self-developed 3D point cloud registration algorithm, achieving a part positioning / trajectory guidance accuracy of up to ±0.5mm, greatly improving positioning accuracy. The system automatically calls up the corresponding function and model by sending the corresponding command and model number to the robot, eliminating the need for manual model switching, making it convenient and fast. This improves the utilization rate of robots, cameras, servers, and other equipment on the automotive production line, avoids resource waste, and overcomes the technical shortcomings of existing technologies, such as limited camera functionality, numerous error-proofing sensors, and poor flexible production.

[0062] This invention provides a control method for a multifunctional vision system, referring to... Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of a control method for a multifunctional vision system according to the present invention.

[0063] In this embodiment, the control method of the multifunctional vision system includes:

[0064] Step S10: Receive the task instruction sent after the robot arrives in position;

[0065] It should be noted that in this embodiment, the executing entity is the server in the above-mentioned multi-functional vision system embodiment. In this multi-functional vision system, the robot's actions can be executed according to the preset automated program in the robot control cabinet, or they can be manually controlled by the operator by operating the robot control cabinet or by the controller that can manipulate the robot control cabinet. The robot's arrival means that the robot's robotic arm has moved to the part detection position specified by the automated program or manually specified by the operator. When the robot arrives in position, it will send a task instruction to the server through the robot control cabinet, so that the server controls the camera to perform machine vision-related tasks according to the task instruction.

[0066] Step S20: Generate corresponding camera action instructions based on the task instructions;

[0067] Furthermore, as a feasible embodiment, in this embodiment, the task instructions include: a positioning instruction and a fault-prevention detection instruction; the camera action instructions include: a 3D image acquisition instruction and a 2D image acquisition instruction; the above step S20 includes:

[0068] Step S21: When the task instruction is a positioning instruction, generate a 3D image acquisition instruction;

[0069] Step S22: When the task instruction is an error prevention detection instruction, generate a 2D image acquisition instruction.

[0070] It should be noted that in this embodiment, the multi-functional vision system can simultaneously support part inspection at two workstations. Both workstations are equipped with 3D visual positioning, 3D visual trajectory correction, and 2D error prevention detection functions. The robot at each workstation can invoke positioning commands to implement the 3D visual positioning function or 3D visual trajectory correction function in the above-mentioned multi-functional vision system embodiment. The robot at each workstation can invoke error prevention detection commands to implement the 2D image flexible error prevention detection function in the above-mentioned multi-functional vision system embodiment. The camera action commands are used to control the 3D camera to perform actions. The 3D image acquisition command corresponding to the positioning command is used to control the 3D camera to acquire 3D point cloud images. The 2D image acquisition command corresponding to the error prevention command is used to control the 2D camera that makes up the 3D camera to acquire 2D images.

[0071] Step S30: Control the camera to acquire image information according to the camera action command;

[0072] Furthermore, as a feasible embodiment, in this embodiment, the image information includes: a 3D point cloud image and a 2D image; the above step S30 includes:

[0073] Step S31: When the camera action command is a 3D image acquisition command, the 3D image acquisition command is sent to the 3D camera, and the 3D point cloud image acquired by the 3D camera according to the 3D image acquisition command is received.

[0074] Step S32: When the camera action command is a 2D image acquisition command, the 2D image acquisition command is sent to the 2D camera, and the 2D image acquired by the 2D camera according to the 2D image acquisition command is received.

[0075] It is understood that in this embodiment, the server cannot directly acquire image information. Instead, it first sends camera action commands to the corresponding camera, then acquires image information through the camera, and finally receives the image information acquired by the camera. When the camera action command is a 3D image acquisition command, the 3D image acquisition command is sent to the 3D camera. After the 3D camera completes image information acquisition according to the 3D image acquisition command, the server receives the 3D point cloud image acquired by the 3D camera. When the camera action command is a 2D image acquisition command, the 2D image acquisition command is sent to the 2D camera. After the 2D camera completes image information acquisition according to the 2D image acquisition command, the server receives the 2D image acquired by the 2D camera.

[0076] Step S40: Invoke a preset algorithm to process the image information and obtain the processing result;

[0077] Furthermore, as a feasible embodiment, in this embodiment, before step S40 above, the control method of the multifunctional vision system provided in this embodiment further includes:

[0078] Step A: Receive the part model number sent by the robot after it arrives in place;

[0079] Step B: Select a preset algorithm based on the task instruction and the part model number.

[0080] It should be noted that before testing at each workstation, standard templates are created for standard parts with different part model numbers, and these templates are stored for comparison during formal production. After the robot arrives, in addition to sending task instructions, the robot also sends the model number of the currently inspected part to the server through the robot controller. The server can find the standard template of the corresponding model of standard part based on the part model number, and select a preset algorithm based on the task instructions and the standard template to process the image information captured by the camera.

[0081] Furthermore, as a feasible embodiment, in this embodiment, the preset algorithm includes: a 3D point cloud matching algorithm and a 2D image processing algorithm; the processing result includes: a calculation result and a detection result; the above step S40 includes:

[0082] Step S41: When the preset algorithm is a 3D point cloud matching algorithm, the 3D point cloud matching algorithm is called to identify the 3D point cloud image and obtain the identification result.

[0083] Step S42: When the preset algorithm is a 2D image processing algorithm, the 2D image processing algorithm is called to detect the 2D image and obtain the detection result.

[0084] It is understood that in this embodiment, the preset algorithm corresponding to the 3D visual positioning function or the 3D visual trajectory correction function is a 3D point cloud matching algorithm. After the vision server loads the part model number and acquires the 3D point cloud image, it will call the 3D point cloud matching algorithm to recognize the 3D point cloud image. When the recognition is successful, the corresponding template can be called and matched with the current 3D point cloud to generate positioning coordinates, i.e., target point coordinates. When the recognition fails, an error message will be generated. The preset algorithm corresponding to the 2D image flexible error prevention detection function is a 2D image processing algorithm. After the vision server loads the part model number and acquires the 2D image, it will call the 2D image processing algorithm to load the corresponding error prevention template and match it with the current 2D image. When the matching is successful, an "OK" signal is generated. When the matching fails, an "NG" signal is generated.

[0085] Step S50: Return the processing result to the robot so that the robot can complete the task instruction or stop running and issue an alarm based on the processing result.

[0086] Furthermore, as a feasible embodiment, in this embodiment, the identification result includes: target point coordinates and error information; the detection result includes: success information and failure information; the above step S50 includes:

[0087] Step S51: When recognition is successful, the recognition result is the target point coordinates. The target point coordinates are returned to the robot so that the robot can complete the positioning command based on the target point coordinates.

[0088] Step S52: When recognition fails, the recognition result is an error message. The error message is returned to the robot so that the robot stops running and issues an alarm based on the error message.

[0089] Step S53: When the detection is successful, the detection result is a success message. The success message is returned to the robot so that the robot can complete the error prevention instruction and continue to run based on the success message.

[0090] Step S54: When the detection fails, the detection result is a failure message. The failure message is returned to the robot so that the robot stops running and issues an alarm based on the failure message.

[0091] As an example, in this embodiment, when the task instruction is a location instruction, it can be combined with... Figure 4 To understand, Figure 4 This is a flowchart illustrating the process of the multi-functional vision system performing the localization task in this embodiment. Figure 4 As can be seen, after the robot reaches the camera position, it sends a positioning command and the part model number to the vision server. The vision server loads the model number and acquires a 3D point cloud image, and calls a 3D point cloud matching algorithm to calculate the coordinates of the target point. Then it determines whether the recognition is successful. If the recognition is successful, the target point coordinates are returned to the robot, so that the robot can obtain the coordinates to realize the positioning / trajectory correction function. If the recognition is unsuccessful, an error message is returned to the robot, causing the robot to stop running and alarm.

[0092] As an example, in this embodiment, when the task instruction is a fault-prevention instruction, it can be combined with... Figure 5 To understand, Figure 5 This is a flowchart illustrating the process of the multi-functional vision system performing error prevention tasks in this embodiment. Figure 5As can be seen, after the robot reaches the camera position, it sends an error prevention detection command and the model number to the vision server. The vision server loads the model number, acquires a 2D image, and calls a 2D image processing algorithm to detect feature points. Then it determines whether the detection is successful. If the detection is successful, it returns OK to the robot, allowing the robot to continue running. If the detection is unsuccessful, it returns NG to the robot, causing the robot to stop running and trigger an alarm.

[0093] Compared to existing technologies, the traditional approach for loading automotive parts involves manual hoisting and error prevention using manual checks or fixed inductive switches. This approach fails to meet the flexible production requirements of production lines and is inefficient. This embodiment leverages the flexibility of robots, allowing them to capture images of any part within their reach, enabling flexible error prevention detection. It integrates part positioning, trajectory correction, and flexible error prevention, significantly improving equipment utilization. The 3D point cloud registration algorithm greatly enhances positioning accuracy. The robot automatically calls up the corresponding function and model by sending the appropriate command and model number, eliminating the need for manual model switching and providing convenience and speed.

[0094] This embodiment provides a control method for a multifunctional vision system. After the robot, carrying a 3D camera, moves above the part to be grasped, it sends a positioning command and the part model number to the vision server. The vision server calls the corresponding template and matches it with the current 3D point cloud to generate positioning coordinates. The vision server returns the positioning coordinates to the robot, and the robot moves to these coordinates to achieve the function of locating and grasping the part. After the robot, carrying a 3D camera, moves above the part to be grasped, it sends an error-proofing command and an error-proofing model number to the vision server. The vision server loads the corresponding error-proofing template and matches it with the current 2D image. If the match is successful, the robot returns an "OK" signal to the robot via the TCP / IP communication protocol, and the robot continues to operate. If the match fails, it returns an "NG" signal to the robot. Upon receiving the "NG" signal, the robot reports a part model error and stops working. For various different parts, the robot can capture images from multiple different positions to achieve a flexible error-proofing detection function.

[0095] Furthermore, embodiments of the present invention also propose a computer device, referring to... Figure 6 , Figure 6 This is a schematic diagram of the structure of the computer device involved in the embodiment of the present invention.

[0096] like Figure 6As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0097] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0098] like Figure 6 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a control program for a multi-functional vision system.

[0099] exist Figure 6 In the computer device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in this embodiment can be located in the computer device, and the computer device calls the control program of the multi-functional vision system stored in the memory 1005 through the processor 1001 and performs the following operations:

[0100] Receive task instructions sent after the robot arrives in position;

[0101] Generate corresponding camera action instructions based on the task instructions;

[0102] The camera is controlled to acquire image information according to the camera action commands;

[0103] The image information is processed by a preset algorithm to obtain the processing result;

[0104] The processing result is returned to the robot so that the robot can complete the task instruction or stop running and issue an alarm based on the processing result.

[0105] Furthermore, the task instructions include: positioning instructions and error prevention detection instructions; the camera action instructions include: 3D image acquisition instructions and 2D image acquisition instructions; the processor 1001 can call the control program of the multi-functional vision system stored in the memory 1005, and also perform the following operations:

[0106] When the task instruction is a positioning instruction, a 3D image acquisition instruction is generated;

[0107] When the task instruction is a fault-prevention detection instruction, a 2D image acquisition instruction is generated.

[0108] Furthermore, the image information includes: 3D point cloud images and 2D images; the processor 1001 can call the control program of the multi-functional vision system stored in the memory 1005, and also perform the following operations:

[0109] When the camera action command is a 3D image acquisition command, the 3D image acquisition command is sent to the 3D camera, and the 3D point cloud image acquired by the 3D camera according to the 3D image acquisition command is received;

[0110] When the camera action command is a 2D image acquisition command, the 2D image acquisition command is sent to the 2D camera, and the 2D image acquired by the 2D camera according to the 2D image acquisition command is received.

[0111] Furthermore, the processor 1001 can call the control program of the multi-functional vision system stored in the memory 1005, and also perform the following operations:

[0112] Receive the part model number sent after the robot arrives in position;

[0113] A preset algorithm is selected based on the task instruction and the part model number.

[0114] Furthermore, the preset algorithm includes: a 3D point cloud matching algorithm and a 2D image processing algorithm; the processing result includes: calculation result and detection result; the processor 1001 can call the control program of the multi-functional vision system stored in the memory 1005, and also perform the following operations:

[0115] When the preset algorithm is a 3D point cloud matching algorithm, the 3D point cloud matching algorithm is called to identify the 3D point cloud image and obtain the identification result;

[0116] When the preset algorithm is a 2D image processing algorithm, the 2D image is detected by calling the 2D image processing algorithm to obtain the detection result.

[0117] Furthermore, the recognition result includes: target point coordinates and error information; the detection result includes: success information and failure information; the processor 1001 can call the control program of the multi-functional vision system stored in the memory 1005, and also perform the following operations:

[0118] When recognition is successful, the recognition result is the target point coordinates, and the target point coordinates are returned to the robot so that the robot can complete the positioning command based on the target point coordinates;

[0119] When recognition fails, the recognition result is an error message, which is returned to the robot so that the robot stops running and issues an alarm based on the error message.

[0120] When the detection is successful, the detection result is a success message, which is returned to the robot so that the robot can complete the error prevention instruction and continue to run based on the success message;

[0121] When the detection fails, the detection result is a failure message, which is returned to the robot so that the robot stops running and issues an alarm based on the failure message.

[0122] Furthermore, embodiments of the present invention also propose a computer-readable storage medium for use in a computer. This computer-readable storage medium can be a non-volatile computer-readable storage medium, on which a control program for a multi-functional vision system is stored. When the control program for the multi-functional vision system is executed by a processor, it implements the steps of the control method for the multi-functional vision system of the present invention as described above.

[0123] The various embodiments of the computer device and computer-readable storage medium of the present invention can be referred to the various embodiments of the control method of the multifunctional vision system of the present invention, which will not be repeated here.

[0124] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0125] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0127] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A multifunctional vision system, characterized in that, The multifunctional vision system includes: One server; Two robot control cabinets, each of which is communicatively connected to the server; Two robots, each of which is communicatively connected to one of the robot control cabinets; Two 3D cameras are provided, each of which is communicatively connected to the server and is mounted on the flange of one of the robots. Each 3D camera includes two 2D cameras and a light source device, and each 2D camera and the light source device are communicatively connected to the server. The server communicates with the robot and the 3D camera via a preset communication protocol, and the server is used for: Receive the task instructions sent by the robot after it has reached its position, the task instructions including: positioning instructions and error prevention detection instructions; Generate corresponding camera action instructions based on the task instructions, including: when the task instructions are positioning instructions, generate 3D image acquisition instructions; when the task instructions are error prevention and detection instructions, generate 2D image acquisition instructions. The camera is controlled to acquire image information according to the camera action command, and the image information is processed by calling the preset algorithm corresponding to the task command, so as to realize at least one of the following functions: 3D visual positioning function, 3D visual trajectory correction function, and 2D image flexible error prevention detection function.

2. A control method for a multifunctional vision system, characterized in that, The control method for the multifunctional vision system is applied to the multifunctional vision system as described in claim 1, and the control method for the multifunctional vision system includes the following steps: Receive task instructions sent after the robot arrives in position; Generate corresponding camera action instructions based on the task instructions; The camera is controlled to acquire image information according to the camera action commands; The image information is processed by a preset algorithm to obtain the processing result; The processing result is returned to the robot so that the robot can complete the task instruction or stop running and issue an alarm based on the processing result.

3. The control method for the multifunctional vision system as described in claim 2, characterized in that, The task instructions include: positioning instructions and error prevention detection instructions; the camera action instructions include: 3D image acquisition instructions and 2D image acquisition instructions. The step of generating corresponding camera motion commands based on the task commands includes: When the task instruction is a positioning instruction, a 3D image acquisition instruction is generated; When the task instruction is a fault-prevention detection instruction, a 2D image acquisition instruction is generated.

4. The control method for the multifunctional vision system as described in claim 3, characterized in that, The image information includes: 3D point cloud images and 2D images; The step of controlling the camera to acquire image information according to the camera action command includes: When the camera action command is a 3D image acquisition command, the 3D image acquisition command is sent to the 3D camera, and the 3D point cloud image acquired by the 3D camera according to the 3D image acquisition command is received; When the camera action command is a 2D image acquisition command, the 2D image acquisition command is sent to the 2D camera, and the 2D image acquired by the 2D camera according to the 2D image acquisition command is received.

5. The control method for the multifunctional vision system as described in claim 4, characterized in that, Before the step of invoking a preset algorithm to process the image information, the control method of the multifunctional vision system further includes: Receive the part model number sent after the robot arrives in position; A preset algorithm is selected based on the task instruction and the part model number.

6. The control method for the multifunctional vision system as described in claim 5, characterized in that, The preset algorithm includes: a 3D point cloud matching algorithm and a 2D image processing algorithm; the processing result includes: calculation result and detection result; The step of calling a preset algorithm to process the image information and obtain the processing result includes: When the preset algorithm is a 3D point cloud matching algorithm, the 3D point cloud image is identified by calling the 3D point cloud matching algorithm to obtain the identification result; When the preset algorithm is a 2D image processing algorithm, the 2D image is detected by calling the 2D image processing algorithm to obtain the detection result.

7. The control method for the multifunctional vision system as described in claim 6, characterized in that, The identification results include: target point coordinates and error information; the detection results include: success information and failure information. The step of returning the processing result to the robot, so that the robot can complete the task instruction or stop running and issue an alarm based on the processing result, includes: When recognition is successful, the recognition result is the target point coordinates, and the target point coordinates are returned to the robot so that the robot can complete the positioning command based on the target point coordinates; When recognition fails, the recognition result is an error message, which is returned to the robot so that the robot stops running and issues an alarm based on the error message. When the detection is successful, the detection result is a success message, which is returned to the robot so that the robot can complete the error prevention detection instruction based on the success message and continue to run. When the detection fails, the detection result is a failure message, which is returned to the robot so that the robot stops running and issues an alarm based on the failure message.

8. A computer device, characterized in that, The computer device includes: a memory, a processor, and a control program for a multi-functional vision system stored in the memory and executable on the processor. When the control program for the multi-functional vision system is executed by the processor, it implements the steps of the control method for the multi-functional vision system as described in any one of claims 2 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a control program for a multi-functional vision system, which, when executed by a processor, implements the steps of the control method for the multi-functional vision system as described in any one of claims 2 to 7.

Images