A machine vision-based method and system for locating sprayed areas

Abstract

This invention relates to the field of infrastructure engineering technology, specifically to a machine vision-based method and system for locating spraying areas. The method includes: using a depth camera to capture the original point cloud of the beam surface, segmenting and selecting target planes on the beam surface using the RANSAC algorithm; converting the image to two dimensions, extracting line features using Hough transform and edge detection, inversely mapping back to three dimensions, and selecting the uppermost point of the line; moving the camera upwards to capture the point cloud of the upper part of the beam, removing interference to extract two target planes; calculating the normal vectors of the two planes, comparing the point cloud to obtain the coordinates of the calibration points, constructing a new three-dimensional coordinate system and generating a rotation matrix; and planning the robot spraying path based on the beam surface design parameters. The system includes: a depth camera, point cloud segmentation, two-dimensional mapping and feature extraction, three-dimensional coordinate recovery and key point selection, spatial calibration, and path planning modules, which work together to implement the above method, eliminate point cloud blind spots, improve positioning accuracy, and meet the requirements for precise spraying of beam surfaces.

Description

Technical Field

[0001] This invention belongs to the field of infrastructure engineering technology, specifically relating to a method and system for locating sprayed areas based on machine vision. Background Technology

[0002] In infrastructure projects such as bridges and buildings, beam structures are core load-bearing components, and their surface coating (such as anti-corrosion coatings and protective coatings) is a crucial process for ensuring structural durability and safety. Beam surface coating must be strictly confined to a predetermined area. Excessive deviation in the positioning of the coating area will not only waste paint and increase construction costs, but may also lead to contamination of non-coated areas or insufficient protection of critical areas, thereby affecting the service life and safety performance of the beam structure. Therefore, accurate positioning of the beam surface coating area is a core prerequisite for ensuring the quality of the coating construction.

[0003] With the application of machine vision technology in industrial positioning, some solutions attempt to introduce vision devices (such as ordinary cameras) to achieve beam surface spraying area positioning. However, existing machine vision solutions still have obvious technical defects: On the one hand, most solutions rely only on two-dimensional images for feature extraction and positioning, without combining three-dimensional spatial information. In complex working environments (such as uneven lighting or background obstruction by clutter), the line features of two-dimensional images are prone to misidentification or omission, resulting in insufficient plane extraction accuracy and inability to accurately lock the target area on the beam surface. On the other hand, even if some solutions introduce depth cameras to collect point cloud data, they do not optimize the processing flow for the spatial structural characteristics of the beam surface, or do not supplement the collection of point cloud data on the upper part of the beam, resulting in blind spots in point cloud collection in key areas such as the top and upper part of the beam surface, making it impossible to obtain complete spatial information of the beam surface; or they do not build a dedicated three-dimensional coordinate system and coordinate transformation rotation matrix, relying only on the original camera coordinate system for positioning. This is prone to deviations in calibration point determination and insufficient path planning accuracy due to inconsistent coordinate references, ultimately affecting the accuracy of the spraying operation.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] To address the aforementioned technical problems in the existing technology, this invention provides a machine vision-based method and system for locating sprayed areas, which solves the problems of low positioning accuracy, poor efficiency, and weak adaptability in existing beam surface spraying area positioning technologies, making it difficult to meet the high-precision and high-efficiency construction requirements of beam surface spraying in infrastructure projects.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: Firstly, a machine vision-based method for locating sprayed areas includes: S1. Control the depth camera to capture the beam surface at a preset shooting position to obtain the original point cloud data of the beam surface; use the RANSAC algorithm to perform plane segmentation on the original point cloud data of the beam surface, set the plane filtering threshold according to the actual situation, and select the target plane of the beam surface from the segmented planes; S2. Call the intrinsic parameter data of the depth camera and the preset camera model to convert the three-dimensional point cloud data of the target plane of the beam surface into two-dimensional image coordinates to obtain a two-dimensional image of the beam surface; on the two-dimensional image of the beam surface, the Hough transform algorithm and the edge detection algorithm are used to detect and extract the continuous line features of the beam surface. S3. Based on the camera's intrinsic parameter data and the preset camera model, establish a mapping relationship between two-dimensional image coordinates and three-dimensional spatial coordinates, inversely map the two-dimensional image coordinates of the continuous line features to three-dimensional space, and recover the three-dimensional spatial points corresponding to the continuous line features; select the three-dimensional spatial points located at the line vertices. S4. Control the depth camera to move a preset distance directly upward from the shooting position in step S1, and re-acquire the point cloud data of the upper part of the beam; exclude other interfering planes in the point cloud data of the upper part of the beam, and extract the two target planes in the upper part; S5. Calculate the normal vectors of the two target planes in step S4; compare the point cloud data of the two target planes with the point cloud data obtained in step S1 to obtain the precise coordinates of the calibration point in space; calculate the normal vector of the third plane perpendicular to the normal vectors of the two planes, construct a new three-dimensional coordinate system with the three normal vectors, and generate a rotation matrix describing the transition from the original coordinate system to the new coordinate system. S6. Based on the design parameters of the beam surface, combined with the rotation matrix and calibration point coordinates calculated in step S5, path planning is performed to obtain the predetermined path for the robot to perform the spraying operation.

[0007] Furthermore, the preset shooting position is the position where the depth camera can completely collect point cloud data of the key structural area of ​​the beam surface, and the position of the point cloud data avoids the obstruction of the beam surface point cloud collection by background clutter.

[0008] Furthermore, the preset camera model is an imaging model adapted to the intrinsic parameter data of a depth camera; The edge detection algorithm, in conjunction with the Hough transform algorithm, is used to detect and extract continuous line features on the beam surface; the process of converting the three-dimensional point cloud data of the target plane of the beam surface into two-dimensional image coordinates is based on the intrinsic parameter data of the depth camera.

[0009] Furthermore, the vertex of the line is the highest point in three-dimensional space of the beam surface structural line.

[0010] Furthermore, the preset distance mentioned in step S4 is determined based on the actual height parameter of the beam. The preset distance ensures that the point cloud data of the upper part of the beam collected by the depth camera covers the area of ​​the upper part of the beam that is not covered by the original point cloud data in step S1.

[0011] Furthermore, in step S5, when calculating the normal vectors of the two target planes in step S4, the normal vectors are calculated based on the point cloud data of the two target planes through plane characteristic analysis. When comparing the point cloud data of the two target planes with the point cloud data obtained in step S1, the original point cloud data of the beam surface in step S1 is used as a reference. The two types of point cloud data are spatially aligned and then compared. At the same time, the precise coordinates of the calibration point are corrected by combining the line vertices. When constructing a new three-dimensional coordinate system with three normal vectors, the three normal vectors correspond to the three coordinate axis directions of the new three-dimensional coordinate system.

[0012] Secondly, a machine vision-based spraying area positioning system includes: Depth camera module: used to capture images of the beam surface at a preset shooting position to obtain the original point cloud data of the beam surface; and can receive control commands to move a preset distance directly upward from the preset shooting position to re-collect the point cloud data of the upper part of the beam; Point cloud segmentation module: Communicatively connected to the depth camera module, used to receive the original point cloud data of the beam surface and the point cloud data of the upper part of the beam, and to perform planar segmentation on the original point cloud data and the point cloud data of the upper part of the beam using the RANSAC algorithm respectively. The planar filtering threshold is set according to the actual situation, and the target plane of the beam surface is selected from the segmented planes, while other interfering planes in the point cloud data of the upper part of the beam are excluded, and two target planes of the upper part of the beam are extracted. Two-dimensional mapping and feature extraction module: It is communicatively connected to the depth camera module and the point cloud segmentation module, respectively. It is used to call the intrinsic parameter data of the depth camera and the preset camera model, convert the three-dimensional point cloud data of the target plane of the beam surface selected by the point cloud segmentation module into two-dimensional image coordinates to obtain a two-dimensional image of the beam surface; and on the two-dimensional image of the beam surface, the Hough transform algorithm and the edge detection algorithm are used to detect and extract the continuous line features of the beam surface. The 3D coordinate recovery and key point selection module is communicatively connected to the 2D mapping and feature extraction module and the depth camera module, respectively. It is used to establish a mapping relationship between 2D image coordinates and 3D spatial coordinates based on the intrinsic parameter data of the depth camera and the preset camera model. It inversely maps the 2D image coordinates of the continuous line features extracted by the 2D mapping and feature extraction module to 3D space to recover the 3D spatial points corresponding to the continuous line features; and selects the point located at the top of the line among the 3D spatial points. Spatial calibration module: Communicatively connected to the point cloud segmentation module and the 3D coordinate recovery and key point selection module, respectively, used to calculate the normal vectors of the two target planes on the upper part of the beam extracted by the point cloud segmentation module; compare the point cloud data of the two target planes with the original point cloud data of the beam surface acquired by the depth camera module to obtain the precise coordinates of the calibration point in space; also used to calculate the normal vector of a third plane perpendicular to the normal vectors of the two planes, construct a new 3D coordinate system with the three normal vectors, and generate a rotation matrix describing the transition from the original coordinate system to the new coordinate system; Path planning module: Communicatively connected to the spatial calibration module, used to obtain the design parameters of the beam surface, and combined with the rotation matrix and calibration point coordinates calculated by the spatial calibration module, to perform path planning and obtain the predetermined path for the robot to perform the spraying operation.

[0013] Furthermore, the preset distance is determined based on the actual height parameter of the beam; During the movement, the depth camera module maintains the reference direction for collecting point cloud data, and the point cloud data collected after the movement covers the beam surface structure area that was not collected by the preset shooting position.

[0014] Furthermore, the actual situation includes the material reflection characteristics of the beam surface and the degree of environmental interference when the depth camera collects point cloud data; The plane filtering threshold is set to accurately distinguish between the target plane on the beam surface and the background interference plane, and between the target plane on the upper part of the beam and the non-beam surface interference plane from the multiple planes after segmentation.

[0015] Furthermore, the spatial calibration module uses a plane fitting algorithm to calculate the normal vectors of the two target planes on the upper part of the beam. The plane fitting algorithm is the least squares method. When the spatial calibration module constructs a new three-dimensional coordinate system, the three normal vectors correspond to the three coordinate axes of the new three-dimensional coordinate system, and the three coordinate axes are perpendicular to each other; the original coordinate system is the reference coordinate system used by the depth camera module when acquiring point cloud data.

[0016] Compared with existing technologies, the present invention provides a machine vision-based method and system for locating spraying areas. The method includes: acquiring the original point cloud of the beam surface using a depth camera, segmenting and selecting the target plane of the beam surface using the RANSAC algorithm; converting it to a two-dimensional image, extracting line features using Hough transform and edge detection, inversely mapping it back to three dimensions and selecting the uppermost point of the line; moving the camera upwards to acquire the point cloud of the upper part of the beam, removing interference and extracting two target planes; calculating the normal vectors of the two planes, comparing the point cloud to obtain the coordinates of the calibration points, constructing a new three-dimensional coordinate system and generating a rotation matrix; and planning the robot spraying path in conjunction with the beam surface design parameters. The system includes: a depth camera, point cloud segmentation, two-dimensional mapping and feature extraction, three-dimensional coordinate recovery and key point selection, spatial calibration and path planning modules, which work together to implement the above method, eliminate point cloud blind spots, improve positioning accuracy, and meet the requirements for precise spraying of beam surfaces. Attached Figure Description

[0017] Figure 1 A flowchart of the spraying area positioning method provided in an embodiment of the present invention; Figure 2 This is an architectural diagram of the spraying area positioning system provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0019] It should be noted that, unless otherwise specifically stated, the relative arrangement and numerical expressions of the components and steps described in these embodiments should not be construed as limiting the scope of the invention.

[0020] The following description of exemplary embodiments is merely illustrative and is not intended to limit the invention or its application or use in any way. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail herein, but where applicable, such techniques, methods, and apparatus should be considered part of this specification.

[0021] Example 1 See Figure 1 , Figure 1 This is a flowchart of a machine vision-based spraying area localization method proposed in this invention. During the intelligent beam surface detection process, a depth camera first captures and acquires point cloud data of the beam surface at a suitable location. This point cloud data, after processing, can be used for accurate spatial modeling and analysis of the beam surface. Specific steps may include: S1. Control the depth camera to capture the beam surface at the preset shooting position and obtain the original point cloud data of the beam surface; use the RANSAC algorithm to perform plane segmentation on the original point cloud data, set the plane filtering threshold according to the actual situation, and select the target plane of the beam surface from the segmented planes; Specifically, the preset shooting position is the location where the depth camera can completely collect point cloud data of the key structural areas of the beam surface, and the position of the point cloud data avoids the obstruction of the beam surface point cloud collection by background clutter.

[0022] S2. Call the intrinsic parameter data of the depth camera and the preset camera model to convert the three-dimensional point cloud data of the target plane of the beam surface into two-dimensional image coordinates to obtain a two-dimensional image of the beam surface; on the two-dimensional image of the beam surface, the Hough transform algorithm and the edge detection algorithm are used to detect and extract the continuous line features of the beam surface. Specifically, the preset camera model is an imaging model adapted to the intrinsic parameter data of the depth camera; The edge detection algorithm, in conjunction with the Hough transform algorithm, is used to detect and extract continuous line features on the beam surface; the process of converting the three-dimensional point cloud data of the target plane of the beam surface into two-dimensional image coordinates is based on the intrinsic parameter data of the depth camera.

[0023] S3. Based on the camera's intrinsic parameter data and the preset camera model, establish the mapping relationship between two-dimensional image coordinates and three-dimensional spatial coordinates, reverse map the two-dimensional image coordinates of the line features to three-dimensional space, and recover the three-dimensional spatial points corresponding to the line features; select the three-dimensional spatial points located at the line vertices. Specifically, the vertex of the line is the highest point in three-dimensional space of the beam surface structural line.

[0024] S4. Control the depth camera to move a preset distance directly upward from the shooting position in step S1, and re-acquire the point cloud data of the upper part of the beam; exclude other interfering planes in the point cloud data of the upper part of the beam, and extract the two target planes in the upper part; Specifically, the preset distance is determined based on the actual height parameters of the beam. The preset distance ensures that the point cloud data of the upper part of the beam collected by the depth camera covers the area of ​​the upper part of the beam that is not covered by the original point cloud data in step S1.

[0025] S5. Calculate the normal vectors of the two target planes in step S4; compare the point cloud data of the two target planes with the point cloud data obtained in step S1 to obtain the precise coordinates of the calibration point in space; calculate the normal vector of the third plane perpendicular to the normal vectors of the two planes, construct a new three-dimensional coordinate system with the three normal vectors, and generate a rotation matrix describing the transition from the original coordinate system to the new coordinate system.

[0026] Specifically, when calculating the normal vectors of the two target planes in step S4, the normal vectors are obtained by analyzing the plane characteristics based on the point cloud data of the two target planes.

[0027] When comparing the point cloud data of the two target planes with the point cloud data obtained in step S1, the original point cloud data of the beam surface in step S1 is used as the reference. The two types of point cloud data are spatially aligned and then compared. At the same time, the precise coordinates of the calibration point are corrected by combining the line vertices. When constructing a new three-dimensional coordinate system with three normal vectors, the three normal vectors correspond to the three coordinate axis directions of the new three-dimensional coordinate system.

[0028] S6. Based on the design parameters of the beam surface, and combined with the rotation matrix and calibration point coordinates calculated in step S5, path planning is performed to obtain the predetermined path for the robot to perform the spraying operation. This step includes precise processing of each point cloud data to ensure that the robot can perform intelligent spraying, roughening, or other operations according to the predetermined path.

[0029] Example 2 See Figure 2 , Figure 2 This is an architecture diagram of a machine vision-based spraying area localization system proposed in this invention. The base coordinate system (J) serves as the reference for the movement of the robot base and the depth camera (determining the initial shooting position and movement direction of the camera); the camera coordinate system (C) provides the initial spatial coordinates for the depth camera to acquire and process point clouds (all point cloud data is stored based on this coordinate system); the calibration coordinate system (B) is a dedicated reference for the calibration point coordinates and spatial calibration (constructed by two upper plane normal vectors + a third vertical normal vector); and the tool coordinate system (S) serves as the attitude and path reference for the robot's spraying tool (such as the spray head) (the final path must adapt to this coordinate system). Specifically, it includes: M1, Depth Camera Module: Used to capture images of the beam surface at a preset shooting position to obtain the original point cloud data of the beam surface; and can receive control commands to move a preset distance directly upward from the preset shooting position to re-collect the point cloud data of the upper part of the beam.

[0030] Specifically, referring to the spatial positioning of the base coordinate system (J), the depth camera is placed at a suitable position that can completely capture the key areas of the beam surface. The beam surface is photographed at this position to obtain and output the original point cloud data of the beam surface in the camera coordinate system (C). Then, a control command is received to move the camera a preset distance (the distance is determined according to the actual height parameters of the beam) from the above-mentioned suitable shooting position along the direction directly above the base coordinate system (J). The beam surface is photographed again at the moved position to obtain and output the point cloud data of the upper part of the beam in the camera coordinate system (C).

[0031] M2, Point Cloud Segmentation Module: Communicates with the depth camera module to receive the original point cloud data of the beam surface and the point cloud data of the upper part of the beam. It uses the RANSAC algorithm to perform planar segmentation on the original point cloud data and the point cloud data of the upper part of the beam, respectively. The planar filtering threshold is set according to the actual situation. The target plane of the beam surface is selected from the segmented planes, and other interfering planes in the point cloud data of the upper part of the beam are excluded. The two target planes of the upper part of the beam are extracted.

[0032] Specifically, the system receives the original point cloud data of the beam surface and the point cloud data of the upper part of the beam in the camera coordinate system (C) output by the depth camera module. The RANSAC algorithm is used to perform planar segmentation on the two types of point cloud data respectively. The planar filtering threshold is set according to the actual situation of the beam surface (such as material and environmental interference). The target plane of the beam surface in the camera coordinate system (C) is selected from the segmentation results of the original point cloud data of the beam surface. At the same time, the interference planes that are not beam surfaces are excluded from the segmentation results of the upper part of the beam point cloud data. The two target planes of the upper part of the beam in the camera coordinate system (C) are extracted. Finally, the selected target plane data of the beam surface and the two target plane data of the upper part of the beam are output to the subsequent modules.

[0033] M3, 2D Mapping and Feature Extraction Module: Communicates with the depth camera module and the point cloud segmentation module respectively. It is used to call the intrinsic parameter data of the depth camera and the preset camera model, convert the 3D point cloud data of the target plane of the beam surface selected by the point cloud segmentation module into 2D image coordinates, and obtain the 2D image of the beam surface. On the 2D image of the beam surface, the Hough transform algorithm and the edge detection algorithm are used to detect and extract the continuous line features of the beam surface.

[0034] Specifically, the intrinsic parameter data and preset camera model of the depth camera are obtained from the depth camera module, and the target plane data of the beam surface in the camera coordinate system (C) is obtained from the point cloud segmentation module. Based on the obtained intrinsic parameter data and preset camera model, the three-dimensional point cloud data of the target plane of the beam surface in the camera coordinate system (C) is projected onto a plane perpendicular to the Z_c axis of the camera coordinate system (C) and converted into two-dimensional image coordinates to generate a two-dimensional image of the beam surface. On the generated two-dimensional image of the beam surface, the Hough transform algorithm and the edge detection algorithm are used to jointly detect and extract the continuous line features of the beam surface, and the two-dimensional image coordinates corresponding to the extracted line features are output to the next module.

[0035] M4, 3D Coordinate Recovery and Keypoint Selection Module: Communicates with the 2D Mapping and Feature Extraction Module and the Depth Camera Module respectively. It is used to establish a mapping relationship between 2D image coordinates and 3D spatial coordinates based on the intrinsic parameter data of the depth camera and the preset camera model. It inversely maps the 2D image coordinates of the line features extracted by the 2D Mapping and Feature Extraction Module to 3D space, recovers the 3D spatial points corresponding to the line features, and selects the point located at the top of the line in the 3D spatial points.

[0036] Specifically, the camera intrinsic data and preset camera model are obtained from the depth camera module, and the two-dimensional image coordinates of the line features are obtained from the two-dimensional mapping and feature extraction module. Based on the camera intrinsic data and preset camera model, a mapping relationship between the two-dimensional image coordinates and the three-dimensional coordinates of the camera coordinate system (C) is established. Through this mapping relationship, the two-dimensional image coordinates of the line features are inversely mapped to the camera coordinate system (C) to recover the three-dimensional spatial points corresponding to the line features. The point located at the top of the line is selected from the recovered three-dimensional spatial points, and the coordinates of the point in the camera coordinate system (C) are output to the spatial calibration module.

[0037] M5, Spatial Calibration Module: Communicates with the point cloud segmentation module and the 3D coordinate recovery and key point selection module respectively. It is used to calculate the normal vectors of the two target planes on the upper part of the beam extracted by the point cloud segmentation module; compares the point cloud data of the two target planes with the original point cloud data of the beam surface acquired by the depth camera module to obtain the precise coordinates of the calibration point in space; it is also used to calculate the normal vector of the third plane perpendicular to the normal vectors of the two planes, constructs a new 3D coordinate system with the three normal vectors, and generates a rotation matrix describing the transition from the original coordinate system to the new coordinate system.

[0038] Specifically, the system obtains the target plane data of the two beams in the camera coordinate system (C) from the point cloud segmentation module and calculates the normal vectors of these two target planes; it obtains the coordinates of the uppermost point of the line from the 3D coordinate recovery and key point selection module, and simultaneously retrieves the original point cloud data of the beam surface in the camera coordinate system (C) collected by the depth camera module. The point cloud data of the two beams' upper target planes and the original point cloud data of the beam surface are spatially aligned and compared in the camera coordinate system (C). The deviation is corrected by combining the coordinates of the uppermost point of the line, and the accurate coordinates of the calibration point in space are obtained; the normal vector of the third plane that is perpendicular to the normal vectors of the two beams' upper target planes is calculated, and the calibration coordinate system (B) in the attached figure is constructed with the three normal vectors as the coordinate axes. A rotation matrix describing the transition from the original coordinate system (including the camera coordinate system (C) and the base coordinate system (J)) to the calibration coordinate system (B) is generated. Finally, the accurate coordinates of the calibration point and the rotation matrix are output to the path planning module.

[0039] M6, Path Planning Module: Communicates with the spatial calibration module to obtain the design parameters of the beam surface. Combined with the rotation matrix and calibration point coordinates calculated by the spatial calibration module, it performs path planning to obtain the predetermined path for the robot to perform the spraying operation.

[0040] Specifically, the rotation matrix and precise coordinates of the calibration points are obtained from the spatial calibration module, while the design parameters of the beam surface (such as the spraying area range and spraying accuracy requirements) are obtained from the outside. Based on the obtained rotation matrix, a spatial relationship is established between the calibration coordinate system (B) and the base coordinate system (J). Then, combined with the fixed relationship between the robot spraying tool and the base coordinate system (J), the calibration coordinate system (B) is further associated with the tool coordinate system (S) in the attached figure. Using the precise coordinates of the calibration points as a spatial reference, and combined with the beam surface design parameters, an initial spraying path is planned in the calibration coordinate system (B). The initial path is transformed into the tool coordinate system (S) through the rotation matrix to obtain the predetermined path for the robot to use for spraying operations. Finally, the predetermined path is output to the robot control system to guide the spraying operation.

[0041] In summary, this invention can ensure the accuracy and efficiency of beam surface detection and robot path planning. By combining point cloud segmentation, image processing, normal vector calculation and spatial calibration, it can provide high-precision measurement and operation basis for beam structures in complex environments.

[0042] The above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for locating sprayed areas based on machine vision, characterized in that, include: S1. Control the depth camera to capture the beam surface at a preset shooting position to obtain the original point cloud data of the beam surface; use the RANSAC algorithm to perform plane segmentation on the original point cloud data of the beam surface, set the plane filtering threshold according to the actual situation, and select the target plane of the beam surface from the segmented planes; S2. Call the intrinsic parameter data of the depth camera and the preset camera model to convert the three-dimensional point cloud data of the target plane of the beam surface into two-dimensional image coordinates to obtain a two-dimensional image of the beam surface; on the two-dimensional image of the beam surface, the Hough transform algorithm and the edge detection algorithm are used to detect and extract the continuous line features of the beam surface. S3. Based on the camera's intrinsic parameter data and the preset camera model, establish a mapping relationship between two-dimensional image coordinates and three-dimensional spatial coordinates, and inversely map the two-dimensional image coordinates of the continuous line features to three-dimensional space to recover the three-dimensional spatial points corresponding to the continuous line features. Select the point located at the vertex of the line in the three-dimensional space; S4. Control the depth camera to move a preset distance directly upward from the shooting position in step S1, and re-collect the point cloud data of the upper part of the beam; Exclude other interfering planes in the point cloud data of the upper part of the beam and extract the two target planes in the upper part; S5. Calculate the normal vectors of the two target planes in step S4; compare the point cloud data of the two target planes with the point cloud data obtained in step S1 to obtain the precise coordinates of the calibration point in space; calculate the normal vector of the third plane perpendicular to the normal vectors of the two planes, construct a new three-dimensional coordinate system with the three normal vectors, and generate a rotation matrix describing the transition from the original coordinate system to the new coordinate system. S6. Based on the design parameters of the beam surface, combined with the rotation matrix and calibration point coordinates calculated in step S5, path planning is performed to obtain the predetermined path for the robot to perform the spraying operation.

2. The machine vision-based spraying area positioning method according to claim 1, characterized in that, The preset shooting position is the position where the depth camera can completely collect point cloud data of the key structural area of ​​the beam surface, and the position of the point cloud data avoids the obstruction of the beam surface point cloud collection by background clutter.

3. The machine vision-based spraying area positioning method according to claim 1, characterized in that, The preset camera model is an imaging model adapted to the intrinsic parameter data of a depth camera. The edge detection algorithm, in conjunction with the Hough transform algorithm, is used to detect and extract continuous line features on the beam surface; The process of converting the three-dimensional point cloud data of the target plane of the beam surface into two-dimensional image coordinates is based on the intrinsic parameter data of the depth camera.

4. The machine vision-based spraying area positioning method according to claim 1, characterized in that, The vertex of the line is the highest point in three-dimensional space of the beam surface structural line.

5. The machine vision-based spraying area positioning method according to claim 1, characterized in that, The preset distance mentioned in step S4 is determined based on the actual height parameters of the beam. The preset distance ensures that the point cloud data of the upper part of the beam collected by the depth camera covers the area of ​​the upper part of the beam that was not covered by the original point cloud data in step S1.

6. The machine vision-based spraying area positioning method according to claim 1, characterized in that, In step S5, when calculating the normal vectors of the two target planes in step S4, the normal vectors are calculated based on the point cloud data of the two target planes through plane characteristic analysis. When comparing the point cloud data of the two target planes with the point cloud data obtained in step S1, the original point cloud data of the beam surface in step S1 is used as a reference. The two types of point cloud data are spatially aligned and then compared. At the same time, the precise coordinates of the calibration point are corrected by combining the line vertices. When constructing a new three-dimensional coordinate system with three normal vectors, the three normal vectors correspond to the three coordinate axis directions of the new three-dimensional coordinate system.

7. A machine vision-based spraying area positioning system, characterized in that, include: Depth camera module: used to capture images of the beam surface at preset shooting positions and obtain raw point cloud data of the beam surface; It can also receive control commands to move a preset distance upward from the preset shooting position and re-collect point cloud data of the upper part of the beam; Point cloud segmentation module: Communicatively connected to the depth camera module, used to receive the original point cloud data of the beam surface and the point cloud data of the upper part of the beam, and to perform planar segmentation on the original point cloud data and the point cloud data of the upper part of the beam using the RANSAC algorithm respectively. The planar filtering threshold is set according to the actual situation, and the target plane of the beam surface is selected from the segmented planes, while other interfering planes in the point cloud data of the upper part of the beam are excluded, and two target planes of the upper part of the beam are extracted. Two-dimensional mapping and feature extraction module: It is communicatively connected to the depth camera module and the point cloud segmentation module, respectively. It is used to call the intrinsic parameter data of the depth camera and the preset camera model, convert the three-dimensional point cloud data of the target plane of the beam surface selected by the point cloud segmentation module into two-dimensional image coordinates to obtain a two-dimensional image of the beam surface; and on the two-dimensional image of the beam surface, the Hough transform algorithm and the edge detection algorithm are used to detect and extract the continuous line features of the beam surface. The 3D coordinate recovery and key point selection module is communicatively connected to the 2D mapping and feature extraction module and the depth camera module, respectively. It is used to establish a mapping relationship between 2D image coordinates and 3D spatial coordinates based on the intrinsic parameter data of the depth camera and the preset camera model. It inversely maps the 2D image coordinates of the continuous line features extracted by the 2D mapping and feature extraction module to 3D space to recover the 3D spatial points corresponding to the continuous line features; and selects the point located at the top of the line among the 3D spatial points. Spatial calibration module: Communicatively connected to the point cloud segmentation module and the 3D coordinate recovery and key point selection module, respectively, used to calculate the normal vectors of the two target planes on the upper part of the beam extracted by the point cloud segmentation module; compare the point cloud data of the two target planes with the original point cloud data of the beam surface acquired by the depth camera module to obtain the precise coordinates of the calibration point in space; also used to calculate the normal vector of a third plane perpendicular to the normal vectors of the two planes, construct a new 3D coordinate system with the three normal vectors, and generate a rotation matrix describing the transition from the original coordinate system to the new coordinate system; Path planning module: Communicatively connected to the spatial calibration module, used to obtain the design parameters of the beam surface, and combined with the rotation matrix and calibration point coordinates calculated by the spatial calibration module, to perform path planning and obtain the predetermined path for the robot to perform the spraying operation.

8. The machine vision-based spraying area positioning system according to claim 7, characterized in that, The preset distance is determined based on the actual height parameters of the beam; During the movement, the depth camera module maintains the reference direction for collecting point cloud data, and the point cloud data collected after the movement covers the beam surface structure area that was not collected by the preset shooting position.

9. The machine vision-based spraying area positioning system according to claim 7, characterized in that, The actual situation includes the material reflection characteristics of the beam surface and the degree of environmental interference when the depth camera collects point cloud data; The plane filtering threshold is set to accurately distinguish between the target plane on the beam surface and the background interference plane, and between the target plane on the upper part of the beam and the non-beam surface interference plane from the multiple planes after segmentation.

10. The machine vision-based spraying area positioning system according to claim 7, characterized in that, The spatial calibration module uses a plane fitting algorithm to calculate the normal vectors of the two target planes on the upper part of the beam. The plane fitting algorithm is the least squares method. When the spatial calibration module constructs a new three-dimensional coordinate system, the three normal vectors correspond to the three coordinate axes of the new three-dimensional coordinate system, and the three coordinate axes are perpendicular to each other; the original coordinate system is the reference coordinate system used by the depth camera module when acquiring point cloud data.

Images