Workpiece collision detection method based on image registration and spraying system

Abstract

The application provides a workpiece collision detection method based on image registration and a spraying system, which comprises the following steps: S1, selecting a detection area; S2, setting an entity coordinate system in the detection area; S3, capturing and generating a point cloud of the entity coordinate system by using a camera device; S4, fitting a reference coordinate system according to the captured point cloud; S5, comparing the reference coordinate system with the entity coordinate system, and if the error of the reference coordinate system relative to the entity coordinate system is less than a preset value, the fitting of the reference coordinate system is successful; the spraying system further comprises a spraying room, a chain conveying mechanism and a camera device, the camera device is installed on both sides of the spraying room, the chain conveying mechanism is located at the top of the spraying room, the shaking amount of a workpiece relative to the spraying room is detected before the workpiece enters the spraying room, and if the shaking amount of the workpiece exceeds a preset value, the chain conveying mechanism is stopped or a collision warning is sent; the technical scheme has the advantages that the workpiece can be subjected to real-time collision detection, and hierarchical alarm of different areas can be realized.

Description

Technical Field

[0001] This invention belongs to the field of spraying technology, specifically relating to a workpiece collision detection method and spraying system based on image registration. Background Technology

[0002] On a spray painting production line, workpieces awaiting painting are mostly suspended and move along a conveyor chain. In this suspended configuration, workpiece swaying is almost unavoidable, especially during the start and stop of the conveyor chain. Therefore, real-time monitoring of the workpiece swaying range, triggering alarms for swaying workpieces exceeding safe limits, and even immediately stopping the conveyor chain, has become a key focus of monitoring. Specifically, detecting workpiece swaying is to prevent workpieces with large amplitude of swaying from colliding with the spray booth or spray gun upon entering the painting booth, thereby damaging the equipment or affecting the painting quality.

[0003] In existing technologies, gratings and laser scanning are commonly used position detection methods. After being arranged in several directions such as width and height, they are used to detect workpieces on their working plane. However, their limitation is that the detection plane is fixed. If a workpiece that has passed through the working plane shakes, it cannot be continuously tracked and alarmed. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a workpiece collision detection method and spraying system based on image registration that can perform real-time collision detection on workpieces and realize graded alarms for different areas.

[0005] To solve the above-mentioned technical problems, the technical solution used in this invention is as follows:

[0006] A workpiece collision detection method based on image registration includes the following steps:

[0007] S1. Select the detection area;

[0008] S2. Set up an entity coordinate system within the detection area;

[0009] S3. Use a camera device to capture and generate a point cloud of the entity's coordinate system;

[0010] S4. Fit a reference coordinate system based on the captured point cloud;

[0011] S5. Compare the reference coordinate system with the entity coordinate system. If the error between the reference coordinate system and the entity coordinate system is less than a preset value, it indicates that the reference coordinate system has been successfully fitted. Otherwise, adjust the entity coordinate system and / or the camera device.

[0012] As a further improvement to the image registration-based workpiece collision detection method, the following steps are also included:

[0013] S6. Once the reference coordinate system is successfully fitted, the position of the camera device relative to the area to be detected is fixed.

[0014] S7. Convert the captured reference coordinate system into the world coordinate system;

[0015] S8. Set the parameters of the world coordinate system to determine the triggering conditions for collision warning.

[0016] As a further improvement to the image registration-based workpiece collision detection method, the following steps are also included:

[0017] S9. Once the reference coordinate system is successfully fitted, the entity coordinate system set in the area to be detected is removed.

[0018] As a further improvement to the image registration-based workpiece collision detection method, the entity coordinate system includes three spheres, which respectively form the endpoints of the X-axis, Y-axis and Z-axis of the entity coordinate system, and the spheres are the capture objects of the point cloud.

[0019] As a further improvement to the image registration-based workpiece collision detection method, the surface of the sphere is polished or brushed.

[0020] As a further improvement to the image registration-based workpiece collision detection method, the camera device captures point clouds of the areas where the three spheres are located and fits the reference coordinate system using the least squares method.

[0021] As a further improvement to the image registration-based workpiece collision detection method, step S7 further includes obtaining the parameter values ​​of the reference coordinate system and forming a Pc matrix, obtaining the parameter values ​​of the world coordinate system and forming a Pw matrix, and multiplying the Pc matrix by the Pw matrix to obtain the rotation offset matrix Tctw.

[0022] The Pw matrix is ​​represented as: Pw=(R+L 0 0 R+L / 3,0 R+L 0 R+L / 3,0 0 R+L R+L / 3,1 1 1 1);

[0023] The Pc matrix is ​​represented as: Pc = (P 1x P 2x P 3x P 1x +P 2x +P 3x / 3、P 1y P2y P 3y P 1y +P 2y +P 3y / 3、P 1z P 2z P 3z P 1z +P 2z +P 3z / 3、1 1 1 1).

[0024] As a further improvement to the image registration-based workpiece collision detection method, two camera devices are provided, located on both sides of the detection area, and the camera devices are depth cameras.

[0025] As a further improvement to the image registration-based workpiece collision detection method, the distance between the camera device and the detection area is 0.6-1.3M.

[0026] A spraying system includes a spray booth, a chain conveyor mechanism, and a camera device. The camera device is installed on both sides in front of the spray booth, and the chain conveyor mechanism is located on the top of the spray booth. The chain conveyor mechanism is used to feed workpieces into and out of the spray booth. Before the workpiece enters the spray booth, the camera device detects the amount of swaying of the workpiece relative to the spray booth using the workpiece collision detection method based on image registration as described above.

[0027] When the shaking of the workpiece exceeds a preset value, the chain conveyor mechanism stops or a collision warning is issued.

[0028] As a further improvement to the spraying system, a collision detection zone and a safety zone are set within the detection area covered by the camera device's field of view. The collision detection zone corresponds to the boundary of the spray booth, the safety zone corresponds to the movement area of ​​the workpiece, and the area outside the collision detection zone is a non-detection area.

[0029] Compared with existing technologies, the main advantages of this invention are as follows: This invention utilizes the wide working range of a structured light depth camera to perform real-time collision detection across the entire range from the side, and uses algorithms to implement graded alarms for different areas, making collision detection, or rather, the protection of spraying equipment, more targeted; furthermore, it explores the potential of camera devices (depth cameras) in collision detection (discovering that the point cloud quality obtained by a consumer-grade structured light camera on the surface of a metal wire ball within its field of view can meet the requirements of spherical fitting algorithms), and develops a complete set of algorithms to successfully solve the positioning problem of depth cameras, details the matrix construction calculation method for spatial coordinate transformation, and implements it in engineering applications; compared with commonly used gratings and line laser scanning, which can only work on their respective working planes, this invention can achieve real-time detection over a wide range, and is not limited to a specific working plane. Attached Figure Description

[0030] The above and other objects, features, and advantages of the invention will become clearer through a more detailed description of the preferred embodiments illustrated in the accompanying drawings. The same reference numerals denote the same parts throughout the drawings, and the drawings are not intentionally drawn to scale with actual dimensions; the focus is on illustrating the gist of the invention.

[0031] Figure 1 This refers to the point cloud obtained by the camera device from the solid coordinate system in this invention;

[0032] Figure 2 This is a schematic diagram of collision detection in this invention; Detailed Implementation

[0033] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In this embodiment, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the present invention and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0034] It should be noted that when one element is considered to be "connected" to another element, it can be directly connected to and integrated with the other element, or there may be an intervening element present. The terms "mounted," "one end," "the other end," and similar expressions used in this invention are for illustrative purposes only.

[0035] like Figure 1-2As shown, this embodiment provides a workpiece collision detection method based on image registration, which includes the following steps:

[0036] S1. Select the detection area; Specifically, after the workpiece enters the spray booth or before the corresponding workpiece is sprayed, it is necessary to ensure that the shaking amplitude of the workpiece is kept within a safe range, so as to avoid the workpiece from hitting the spray booth or spraying equipment due to shaking; Therefore, in order to effectively detect collisions of the workpiece, a detection area needs to be set. In this embodiment, the detection area is the boundary formed by the inner contour of the spray booth, or the corresponding area that the workpiece can pass through.

[0037] S2. Establish a solid coordinate system within the detection area. In this embodiment, three metal surface brushed balls of known radius R are welded together perpendicularly by connecting rods of known length L. The positioning device is mounted on a tripod, placed within the field of view of the depth camera, and leveled using a level. Although the algorithm in this invention does not restrict the position of the balls, as long as the imaging quality meets the requirements of spherical fitting, the balls can be placed arbitrarily to establish any coordinate system suitable for the field. However, the best practice for this project is to make the X-axis parallel to the direction of the conveyor chain, and the Y-axis and Z-axis in the horizontal and vertical directions respectively (the Y-axis direction can be towards the inside or outside of the powder chamber, with no essential difference). The connection point of the three connecting rods is placed flush with the vertical edge of the powder chamber. This allows the edge of the powder chamber to serve as a reference for setting alarm parameters, with the y-coordinate = 0, facilitating subsequent operations.

[0038] S3. Use a camera device to capture and generate a point cloud in a solid coordinate system. Figure 1 The image circled in red is a spherical point cloud captured and generated by a depth camera; in engineering, the set of point data on the surface of a product obtained using measuring instruments is also called a point cloud; that is, a point cloud refers to a massive set of points representing the surface characteristics of a target. Typically, the number of points obtained using a 3D coordinate measuring machine is relatively small, and the spacing between points is relatively large, called a sparse point cloud; while the point cloud obtained using a 3D laser scanner or photographic scanner has a large number of points and is relatively dense, called a dense point cloud.

[0039] S4. Fit a reference coordinate system based on the captured point cloud;

[0040] S5. Compare the reference coordinate system with the entity coordinate system. If the error between the reference coordinate system and the entity coordinate system is less than a preset value, the reference coordinate system is considered successfully fitted. Otherwise, adjust the entity coordinate system and / or the camera device. Specifically, in this embodiment, the camera device is a depth camera. Furthermore, the depth camera is used to capture point clouds in the areas where the three spheres (entity coordinate system) are located. The capture method is the sphere center radius method: using a point on the sphere as the center and radius R, the point cloud is cropped to ensure that the obtained point cloud contains only points on the sphere's surface, without other interference. The captured spherical point cloud is then fitted with the sphere center coordinates (camera coordinate system) and the sphere radius R0 using the least squares method. In this embodiment, a sphere with a radius of 75mm is used. If the radius error of the fitted sphere is within ±1mm, the fitting is considered successful. If the radius of the fitted sphere meets the requirements, the distance between the three sphere centers is calculated. If all distances satisfy √2L ± 3mm, the fitting is considered successful. If the fitted radius does not meet the error requirements, it is mostly due to two reasons: one is that the distance between the sphere and the camera exceeds the optimal shooting distance, and the other is strong light interference in the environment, resulting in incomplete point cloud capture. It is necessary to try two approaches: light blocking and moving the sphere. Usually, the position of the camera device is adjusted appropriately until the fitting is successful. Experimental results show that the optimal fitting distance for consumer-grade monocular structured light is 0.6m-1.3m, such as 0.7m, 0.8m, 0.9m, 1.0m, 1.1m, and 1.2m. Of course, if the budget allows, a camera with better parameters can be selected.

[0041] Furthermore, in a preferred embodiment, the following steps are also included:

[0042] S6. Once the reference coordinate system is successfully fitted, the position of the camera device relative to the area to be inspected is fixed. Specifically, different specifications of spray booths, different sizes of inspection areas, different installation environments, and different specifications of camera devices will all affect the fitting. Therefore, the specific position of the camera device can only be determined and fixed after the fitting is successful. That is, the position of the camera device is located in order to facilitate obtaining the rotation matrix of the depth camera relative to the spray booth.

[0043] S7. Convert the captured reference coordinate system to the world coordinate system. Specifically, converting the point coordinates obtained by the depth camera to the world coordinate system makes it easier to set and calculate alarm and detection parameters. To obtain the 4×4 rotation offset matrix Tctw (Transition camera to world), two 4×4 matrices Pw (points of world) and Pc (points of camera) need to be constructed using the coordinates obtained in the previous step to calculate Tctw. In the formula below, each column of Pw represents the coordinates of the center of the ball along the X-axis, Y-axis, and Z-axis, as well as the center of the equilateral triangle formed by the three points, in the world coordinate system. Each column of Pc represents the coordinates of the center of the three balls and the center of the equilateral triangle formed by them, in the camera coordinate system. The last row of both matrices is filled with 1s as an augmentation to facilitate inversion operations using a computer, obtaining the rotation offset matrix Tctw in one step.

[0044]

[0045] Because of the relationship Pw = Tctw * Pc, therefore Tctw = Pw * Pc - 1. Multiplying Tctw by the coordinates obtained in all camera coordinate systems yields the coordinates in the world coordinate system. This completes the calculation of the spatial coordinate system transformation matrix. Specifically, the Pc and Pw matrices, constructed from the values ​​of the three sphere centers in the camera coordinate system (reference coordinate system) and their values ​​in the world coordinate system, are multiplied to obtain the rotation matrix. The Pc matrix requires inverse operation. Furthermore, the value of Pw is known (the radius of the sphere + the length of the rod), and the value of Pc is generated from the coordinates of the three sphere centers fitted from the spherical point cloud. With the rotation matrix, the coordinates of the points measured by the camera can be corrected, allowing comparison with the actual spray booth, thereby achieving collision detection of the workpiece.

[0046] S8. Set the parameters of the world coordinate system to determine the triggering conditions for collision warning.

[0047] In a preferred embodiment, the following steps are also included:

[0048] S9. Once the reference coordinate system is successfully fitted, it indicates that the collision detection system has successfully established a reference coordinate system. At this point, the entity coordinate system set in the area to be detected can be removed.

[0049] In a preferred embodiment, the solid coordinate system includes three spheres, which respectively form the endpoints of the X-axis, Y-axis, and Z-axis of the solid coordinate system, and the spheres are the objects for capturing the point cloud. The surfaces of the spheres are polished or brushed to facilitate the camera device in acquiring and generating the point cloud.

[0050] like Figure 2 As shown, this embodiment also provides a spraying system, including a spray booth, a chain conveyor mechanism and a camera device. The camera device is installed on both sides in front of the spray booth, and the chain conveyor mechanism is located on the top of the spray booth. The chain conveyor mechanism is used to send the workpiece into and out of the spray booth. Before the workpiece enters the spray booth, the camera device detects the amount of shaking of the workpiece relative to the spray booth using the workpiece collision detection method based on image registration as described above.

[0051] When the workpiece's sway exceeds the preset value, the chain conveyor mechanism will stop or a collision warning will be issued.

[0052] like Figure 2 As shown, in a preferred embodiment, a collision detection zone and a safety zone are set within the detection area covered by the camera device's field of view. The collision detection zone corresponds to the boundary of the spray booth, the safety zone corresponds to the workpiece movement area, and the area outside the collision detection zone is a non-detection area. In this embodiment, the alarm range is set as follows: after setting and fixing the two cameras as described above, the world coordinate range of the collision detection is set according to the acceptable safety margin for on-site collision detection. To prevent false collision warnings caused by nearby personnel or other objects entering the camera's detection range, this invention sets three areas within the camera's coverage area: a green safety zone, a red collision detection zone, and a gray non-detection zone. The red area is the collision detection zone (spray booth boundary); the area within the red inner frame is the green safety zone (workpiece area); and the area outside the red outer frame is the gray non-detection zone (area outside the spray booth boundary). To prevent false alarms caused by chains and hangers passing through, a rectangular opening area is designed at the top of the red collision detection zone. The opening range is determined by four parameters: UYL, UYH, UZL, and UZH. The X range is the same as that of the red collision detection zone.

[0053] Compared with existing technologies, the main advantages of this invention are as follows: This invention utilizes the wide working range of a structured light depth camera to perform real-time collision detection across the entire range from the side, and uses algorithms to implement graded alarms for different areas, making collision detection, or rather, the protection of spraying equipment, more targeted; furthermore, it explores the potential of camera devices (depth cameras) in collision detection (discovering that the point cloud quality obtained by a consumer-grade structured light camera on the surface of a metal wire ball within its field of view can meet the requirements of spherical fitting algorithms), and develops a complete set of algorithms to successfully solve the positioning problem of depth cameras, details the matrix construction calculation method for spatial coordinate transformation, and implements it in engineering applications; compared with commonly used gratings and line laser scanning, which can only work on their respective working planes, this invention can achieve real-time detection over a wide range, and is not limited to a specific working plane.

[0054] In this specification, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0055] In the description of this specification, the references to terms such as "preferred embodiment," "another embodiment," "other embodiment," or "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0056] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A workpiece collision detection method based on image registration, characterized in that, Includes the following steps: S1. Select the detection area; S2. Set up an entity coordinate system within the detection area; S3. Use a camera device to capture and generate a point cloud of the entity's coordinate system; S4. Fit a reference coordinate system based on the captured point cloud; S5. Compare the reference coordinate system with the entity coordinate system. If the error between the reference coordinate system and the entity coordinate system is less than a preset value, it means that the reference coordinate system has been successfully fitted. Otherwise, adjust the camera device. S6. Once the reference coordinate system is successfully fitted, the position of the camera device relative to the area to be detected is fixed. S7. Once the reference coordinate system is successfully fitted, the entity coordinate system set in the area to be detected is removed. The entity coordinate system includes three spheres, which respectively form the endpoints of the X-axis, Y-axis and Z-axis of the entity coordinate system. The spheres are the objects to be captured by the point cloud, and the surfaces of the spheres are polished or brushed.

2. The workpiece collision detection method based on image registration according to claim 1, characterized in that, Step S6 also includes the following steps: S61. Convert the captured reference coordinate system into the world coordinate system; S62. Set the parameters of the world coordinate system to determine the triggering conditions for collision warning.

3. The workpiece collision detection method based on image registration according to claim 1 or 2, characterized in that, The camera device captures point clouds of the areas where the three spheres are located and fits the reference coordinate system using the least squares method.

4. The workpiece collision detection method based on image registration according to claim 1 or 2, characterized in that, Step S7 further includes obtaining the parameter values ​​of the reference coordinate system and forming a Pc matrix, obtaining the parameter values ​​of the world coordinate system and forming a Pw matrix, and multiplying the Pc matrix by the Pw matrix on the right to obtain the rotation offset matrix Tctw. The Pw matrix is ​​represented as: Pw=(R+L 0 0R+L / 3,0R+L 0R+L / 3,00R+L R+L / 3,1 1 1 1); The Pc matrix is ​​represented as: Pc = (P1x P2x P3x P1x+P2x+P3x / 3, P1y P2y P3y P1y+P2y+P3y / 3, P1z P2z P3z P1z+P2z+P3z / 3, 1 1 1 1).

5. The workpiece collision detection method based on image registration according to claim 1, characterized in that, The system has two cameras, which are located on opposite sides of the detection area, and each camera is a depth camera.

6. The workpiece collision detection method based on image registration according to claim 5, characterized in that, The distance between the camera device and the detection area is 0.6-1.3M.

7. A spraying system, characterized in that: The system includes a spray booth, a chain conveyor mechanism, and a camera device. The camera device is installed on both sides in front of the spray booth, and the chain conveyor mechanism is located on the top of the spray booth. The chain conveyor mechanism is used to feed workpieces into and out of the spray booth. Before the workpiece enters the spray booth, the camera device detects the amount of swaying of the workpiece relative to the spray booth using the workpiece collision detection method based on image registration as described in any one of claims 1-6. When the shaking of the workpiece exceeds a preset value, the chain conveyor mechanism stops or a collision warning is issued.

8. The spraying system according to claim 7, characterized in that: The detection area covered by the camera device is divided into a collision detection zone and a safety zone. The collision detection zone corresponds to the boundary of the spray booth, and the safety zone corresponds to the movement zone of the workpiece. The area outside the collision detection zone is a non-detection area.

Images