A multi-scan based object detection method, device and medium

Abstract

The application provides a multi-scan-based object detection method, device and medium, and relates to the technical field of object measurement. The method comprises the following steps: acquiring first point cloud data of a public mark point and second point cloud data of a first region of an object to be scanned, which are collected by a first scanner; the first scanner is a scanner carried by a first robot; the public mark point is arranged in a public region; the public region is a repeated scanning region of the first scanner and a second scanner; acquiring third point cloud data of the public mark point and fourth point cloud data of a second region of the object to be scanned, which are collected by the second scanner; the second scanner is a scanner carried by a second robot; and detecting the object to be scanned according to the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data and a digital model of the object to be scanned. The application improves the overall detection efficiency and realizes a high-precision and high-efficiency detection process.

Description

Technical Field

[0001] This application relates to the field of object measurement technology, and more specifically, to an object detection method, device, and medium based on multiple scanners. Background Technology

[0002] In modern high-end manufacturing industries, such as aerospace, shipbuilding, and large-scale energy equipment production, the demand for high-precision three-dimensional measurement of large-sized workpieces is increasing. Large-sized workpieces are bulky and complex in structure, making it difficult to guarantee measurement accuracy using traditional manual inspection methods.

[0003] Currently, before scanning begins, the relative position and orientation between the two scanners are calibrated once using a dedicated calibration tool or common marker points to obtain a fixed coordinate transformation relationship. Subsequently, throughout the scanning process, all acquired data is stitched together based on this initial calibration result. After scanning is complete, a point cloud registration algorithm is used to align the point cloud data with the computer-aided design (CAD) data.

[0004] However, pre-calibration is a static operation. After the robot leaves the initial calibration area and continues to move, factors such as minor deformations of the mechanical structure, external environmental disturbances, and cumulative drift of the positioning data will gradually reduce the stitching accuracy of subsequent data acquisitions. The resulting cumulative error directly affects the reliability of the final measurement results. Alignment operations can only be initiated after all scanning tasks are completed, making it a post-processing step. This prolongs the total time required for a single inspection and makes it prone to failure in areas with missing point cloud data or relatively simple surface geometry, affecting the stability of the overall process. This makes it impossible for existing technologies to achieve mutual error correction, limiting overall operational efficiency and anti-interference capabilities. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a multi-scanner-based object detection method, device, and medium to improve overall detection efficiency and achieve a high-precision and high-efficiency detection process.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, one embodiment of this application provides an object detection method based on multiple scanners, the method comprising: The system acquires first point cloud data of common marker points collected by a first scanner and second point cloud data of a first region of the object to be scanned, wherein the first scanner is a scanner mounted on a first robot; the common marker points are set in a common area, and the common area is the area repeatedly scanned by the first scanner and the second scanner; The third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned are obtained, wherein the second scanner is a scanner mounted on the second robot; The object to be scanned is detected based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned.

[0007] Optionally, detecting the object to be scanned based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned includes: Based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned, determine the alignment matrix and the transformation matrix; The transformation matrix is ​​used to transform the fourth point cloud data from the second coordinate system of the second scanner to the first coordinate system of the first scanner to obtain the fifth point cloud data. Using the alignment matrix, the second point cloud data and the fifth point cloud data are transformed from the first coordinate system to the model coordinate system to obtain the scan data; The object to be scanned is detected based on the scan data and the digital model.

[0008] Optionally, the step of determining the alignment matrix and transformation matrix based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned includes: Align the second point cloud data with the digital model to obtain the alignment matrix; The first point cloud data and the third point cloud data are subjected to consistency constraints on common marker points to obtain the transformation matrix.

[0009] Optionally, the transformation matrix includes: a rotation matrix and a translation vector; the step of applying consistency constraints on common marker points to the first point cloud data and the third point cloud data to obtain the transformation matrix includes: Based on the first point cloud data, the third point cloud data, the initial rotation matrix, and the initial translation vector, construct the objective function; The objective function is solved using the least squares method to obtain the rotation matrix and the translation vector. The transformation matrix is ​​constructed based on the rotation matrix and the translation vector.

[0010] Optionally, acquiring the first point cloud data of the common marker points collected by the first scanner and the second point cloud data of the first region of the object to be scanned includes: Acquire the first initial point cloud data of the first public marker point collected by the first scanner in the initial public area; Control the first robot to move along a direction away from the initial public area to obtain a second public marker point corresponding to a new public area, and acquire the second initial point cloud data of the second public marker point; The first point cloud data is determined based on the first initial point cloud data and the second initial point cloud data.

[0011] Optionally, acquiring the third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned includes: Acquire the third initial point cloud data of the first public marker point collected by the second scanner in the initial public area; Control the second robot to move along a direction away from the initial common area to obtain the second common marker point, and acquire the fourth initial point cloud data of the second common marker point; The second point cloud data is determined based on the third initial point cloud data and the fourth initial point cloud data.

[0012] Optionally, detecting the object to be scanned based on the scan data and the digital model includes: Generate the minimum distance, color difference map, and key dimensions from the scanned data to the digital model; The object to be scanned is detected based on the minimum distance, the color difference map, and the key dimensions.

[0013] Optionally, aligning the first point cloud data with the digital model to obtain the alignment matrix includes: The singular value decomposition method is used to align the first point cloud data with the digital model to obtain the alignment matrix.

[0014] Secondly, another embodiment of this application provides an object detection device based on multiple scanners, the device comprising: The first acquisition module is used to acquire first point cloud data of common marker points collected by the first scanner and second point cloud data of a first region of the object to be scanned, wherein the first scanner is a scanner mounted on the first robot; the common marker points are set in a common area, and the common area is the area repeatedly scanned by the first scanner and the second scanner; The second acquisition module is used to acquire the third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned, wherein the second scanner is a scanner mounted on the second robot; The detection module is used to detect the object to be scanned based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned.

[0015] Thirdly, another embodiment of this application provides a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any of the methods described in the first aspect above.

[0016] Fourthly, another embodiment of this application provides a storage medium storing a computer program, which, when executed by a processor, performs the steps of any of the methods described in the first aspect above.

[0017] The beneficial effects of this application are: This application provides a method, device, and medium for object detection based on multiple scanners. It acquires first point cloud data of common marker points collected by a first scanner and second point cloud data of a first region of the object to be scanned. It then acquires third point cloud data of common marker points collected by a second scanner and fourth point cloud data of a second region of the object to be scanned. Based on the first, second, third, and fourth point cloud data, and a digital model of the object to be scanned, the object is detected. This application utilizes collaborative scanning by two robots, leveraging the common marker points collected by each robot and the point cloud data of the object's region, combined with a pre-built digital model. By establishing a unified coordinate reference through shared common marker points, it suppresses accumulated errors in large-scale scanning and improves the accuracy of the coordinate system. Simultaneously, by aligning the digital model beforehand, it avoids complex point cloud registration calculations after scanning, allowing the scanned data to be directly used for deviation analysis. This shortens the detection cycle, improves overall detection efficiency and automation, and achieves a high-precision, high-efficiency detection process. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A scene diagram of object detection based on multiple scanners provided in an embodiment of this application; Figure 2 A flowchart illustrating a multi-scanner-based object detection method provided in this application; Figure 3 This is a schematic diagram illustrating the process of detecting an object to be scanned in a multi-scanner-based object detection method, as provided in an embodiment of this application. Figure 4 A schematic diagram illustrating the process of determining a matrix in object detection based on multiple scanners, provided for an embodiment of this application; Figure 5 A schematic diagram illustrating the process of determining the transformation matrix in object detection based on multiple scanners, provided for an embodiment of this application; Figure 6 A schematic diagram illustrating the process of determining a matrix in object detection based on multiple scanners, provided for an embodiment of this application; Figure 7 A schematic diagram illustrating another process for determining a matrix in object detection based on multiple scanners, provided for an embodiment of this application; Figure 8 A schematic diagram of the object detection process in a multi-scanner-based object detection system provided in this application embodiment; Figure 9 A schematic diagram of the structure of an object detection device based on multiple scanners provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0021] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0023] As industrial manufacturing moves towards larger scale and greater precision, the demand for high-precision 3D measurement and inspection of large-sized components is becoming increasingly urgent. Traditional single-robot scanning systems, when measuring large objects, suffer from limited field of view due to the limited movement of a single scanner, requiring multiple movements and multi-view data stitching, which easily leads to accumulated errors and is inefficient. Therefore, multi-robot collaborative 3D measurement is typically achieved through calibration between the robots. However, the lack of a correction mechanism during the scanning process results in complex and time-consuming calculations required for alignment with the digital model of the scanned object during subsequent inspection, impacting inspection efficiency.

[0024] Therefore, this application provides an object detection method based on multiple scanners. By setting common marker points to align the data collected by multiple scanners, it obtains first point cloud data of the common marker points collected by the first scanner and second point cloud data of the first region of the object to be scanned. It also obtains third point cloud data of the common marker points collected by the second scanner and fourth point cloud data of the second region of the object to be scanned. Based on the first, second, third, and fourth point cloud data and the digital model of the object to be scanned, the object is detected. This method aligns the data scanned by the first and second scanners during the object scanning process, thereby improving the efficiency of object detection.

[0025] The method will be explained below with reference to a scene diagram. Figure 1 A scene diagram for object detection based on multiple scanners provided in this application embodiment, such as... Figure 1As shown, the object to be scanned is scanned by a first scanner and a second scanner. The scanning range of the first scanner is a first region, and the scanning range of the second scanner is a second region. The overlapping area of ​​the first and second regions of the first and second scanners is a common region, which contains common marker points. Both the first and second scanners can scan these common marker points. Since the object to be scanned is a large object, neither the first nor the second scanner can scan the entire object. Therefore, the data scanned by the first and second scanners are aligned using the common marker points to detect the object. Simultaneously, the first and second scanners can be moved by corresponding first and second robots, thereby changing their respective scanning ranges. The first robot moves the first scanner away from the second scanner, and the second robot moves the second scanner away from the first scanner, increasing the common region between the first and second scanners and thus improving the alignment accuracy. It is worth noting that this application only uses two scanners as an example; in actual measurement processes, more scanners can be used, each with a corresponding robot that moves the scanner.

[0026] To clearly describe the method provided in the embodiments of this application, the method is explained below with reference to the flowchart. Figure 2 A flowchart illustrating a multi-scanner-based object detection method provided in this application is shown below. Figure 2 As shown, the method includes: Step 201: Obtain the first point cloud data of the common marker points collected by the first scanner and the second point cloud data of the first region of the object to be scanned.

[0027] In this embodiment, the first area is the area that the first scanner can scan, and the first scanner is a scanner mounted on the first robot. The robot can be a humanoid robot or other types of robots. This application does not limit this. The first scanner is used to collect three-dimensional point cloud data of the surface of the object to be scanned and the surrounding area of ​​the object. Common marker points are set in a common area, which is the area where the first scanner and the second scanner scan repeatedly. The common marker points are artificial marker points that are pre-pasted at fixed positions around the object to be scanned and can be jointly identified by the first scanner and the second scanner. The common marker points can also be pre-pasted on the object to be scanned.

[0028] Optionally, the first region is scanned by a first scanner to obtain first point cloud data of common marker points collected by the first scanner and second point cloud data of the first region of the object to be scanned.

[0029] Step 202: Obtain the third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned.

[0030] The second region is the area that the second scanner can scan, and the second scanner is a scanner mounted on a second robot. The robot can be a humanoid robot or other types of robots. This application embodiment does not limit this. The second scanner is used to collect three-dimensional point cloud data of the surface of the object to be scanned and the surrounding area of ​​the object to be scanned.

[0031] Optionally, the second region is scanned by a second scanner to obtain the third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned.

[0032] Step 203: Detect the object to be scanned based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned.

[0033] The digital model of the object to be scanned is its CAD model, which is the theoretical 3D model generated during the design phase. The detection process is used to determine the differences between the measured point cloud data and the digital model of the object to be scanned, thereby judging whether the object is qualified.

[0034] Optionally, the second and fourth point cloud data are aligned based on the first and third point cloud data to obtain the measurement point cloud data of the object to be scanned. The object to be scanned is then detected based on the measurement point cloud data and the digital model of the object.

[0035] In this embodiment, first point cloud data of common marker points collected by a first scanner and second point cloud data of a first region of the object to be scanned are acquired. Third point cloud data of common marker points collected by a second scanner and fourth point cloud data of a second region of the object to be scanned are also acquired. Based on the first, second, third, and fourth point cloud data, and the digital model of the object to be scanned, the object is detected. This application utilizes dual-robot collaborative scanning, leveraging the common marker points collected by each robot and the point cloud data of the object's region, combined with a pre-built digital model. By establishing a unified coordinate reference through shared common marker points, accumulated errors in large-scale scanning are suppressed, improving the accuracy of the coordinate system. Simultaneously, aligning the digital model beforehand avoids complex point cloud registration calculations after scanning, allowing the scanned data to be directly used for deviation analysis. This shortens the detection cycle, improves overall detection efficiency and automation, and achieves a high-precision, high-efficiency detection process.

[0036] Based on the above embodiments, this application also provides a process for detecting objects to be scanned in multi-scanner-based object detection. Figure 3 This application provides a schematic diagram of the detection process for an object to be scanned in a multi-scanner-based object detection system, as shown in the embodiments below. Figure 3 As shown, in step 203 above, the object to be scanned is detected based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned, including: Step 301: Based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned, determine the alignment matrix and the transformation matrix.

[0037] The alignment matrix is ​​used to align the second point cloud data of the object to be scanned obtained by the first scanner, the fourth point cloud data of the object to be scanned obtained by the second scanner, and the digital model of the object to be scanned. The transformation matrix is ​​used to transform the data in the second coordinate system corresponding to the second scanner to the first coordinate system corresponding to the first scanner. The origin of the first coordinate system corresponding to the first scanner is the position at the time when the first scanner starts scanning, and the first and second point cloud data obtained by the first scanner are data in the first coordinate system. The origin of the second coordinate system corresponding to the second scanner is the position at the time when the second scanner starts scanning, and the third and fourth point cloud data obtained by the second scanner are data in the second coordinate system. The digital model of the object to be scanned is data in the CAD coordinate system.

[0038] Optionally, a transformation matrix is ​​obtained by processing the first point cloud data and the third point cloud data, and an alignment matrix is ​​determined based on the alignment of the second point cloud data and the digital model of the object to be scanned.

[0039] Step 302: Use a transformation matrix to transform the fourth point cloud data from the second coordinate system of the second scanner to the first coordinate system of the first scanner to obtain the fifth point cloud data.

[0040] Among them, the fifth point cloud data is the coordinate of the fourth point cloud data in the first coordinate system after transformation by the transformation matrix.

[0041] Optionally, a transformation matrix is ​​used to transform the fourth point cloud data to obtain the fifth point cloud data in the first coordinate system. If multiple second scanners exist, the fourth point cloud data from all the second scanners are transformed to the first coordinate system using a transformation matrix to obtain the fifth point cloud data.

[0042] Step 303: Using an alignment matrix, transform the second point cloud data and the fifth point cloud data from the first coordinate system to the model coordinate system to obtain the scan data.

[0043] The model coordinate system is the CAD coordinate system corresponding to the digital model, and the scanned data is the three-dimensional data obtained by converting the second point cloud data and the fifth point cloud data into the model coordinate system after removing duplicate data.

[0044] Optionally, since the first and second scanners share a common area, only one duplicate data point is retained from the second and fifth point cloud data to obtain merged point cloud data. An alignment matrix is ​​then used to transform the merged point cloud data to the model coordinate system to obtain the scan data.

[0045] Step 304: Detect the object to be scanned based on the scan data and digital model.

[0046] Optionally, the scanned data and each three-dimensional coordinate point in the digital model are compared to identify redundant three-dimensional points and valid three-dimensional points in the scanned data, thereby detecting the object to be scanned and obtaining the detection result.

[0047] In this embodiment, the point cloud data of the second scanner is accurately converted to the coordinate system of the first scanner by a transformation matrix, eliminating the coordinate deviation between the two devices and ensuring the accuracy of data stitching; then, the fused scan data is directly mapped to the model coordinate system by an alignment matrix, so that the scan results are aligned with the design model, avoiding the time-consuming post-processing registration in traditional detection, and improving detection efficiency and automation.

[0048] Based on the above embodiments, this application also provides a process for determining a matrix in object detection based on multiple scanners. Figure 4 This application provides a schematic diagram of a process for determining a matrix in object detection based on multiple scanners, as illustrated in the embodiments of this application. Figure 4 As shown, in step 301 above, based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned, the alignment matrix and the transformation matrix are determined, including: Step 401: Align the second point cloud data with the digital model to obtain the alignment matrix.

[0049] The alignment matrix is ​​a 4×4 homogeneous transformation matrix, containing rotation R and translation t to transform the first point cloud data actually scanned by the first scanner from the first coordinate system to the model coordinate system. Since the first robot corresponding to the first scanner is in motion, the second point cloud data is updated as the first scanner moves, and the corresponding alignment matrix is ​​updated as the second point cloud data is updated.

[0050] Optionally, the second point cloud data at each time step is aligned with the digital model to determine the correspondence between the second point cloud data at each time step and the points in the digital model, thereby obtaining the alignment matrix at each time step.

[0051] Step 402: Apply consistency constraints on common marker points to the first point cloud data and the third point cloud data to obtain the transformation matrix.

[0052] The transformation matrix is ​​a 4×4 homogeneous transformation matrix used to transform the data in the second coordinate system to the first coordinate system. Since the first robot corresponding to the first scanner is in motion, the scanning range increases and the number of common marker points increases as the first scanner moves, thus updating the second point cloud data to obtain the updated second point cloud data. Similarly, the scanning range increases and the number of common marker points increases as the second scanner moves, thus updating the fourth point cloud data to obtain the updated fourth point cloud data.

[0053] Optionally, consistency constraints on common marker points are applied to the first point cloud data and the third point cloud data at each time step to ensure that the first point cloud data and the third point cloud data are consistent at each time step, thereby obtaining the transformation matrix at each time step.

[0054] In one possible implementation, the first point cloud data of the common marker points collected by the first scanner is imported into the second scanner. The coordinate origin of the second scanner is constrained to be consistent with the coordinate origin of the first scanner, so that the second coordinate system of the second scanner is aligned with the first coordinate system of the first scanner. At this time, the third point cloud data of the common coordinate points collected by the second scanner and the fourth point cloud data of the object to be scanned are both in the first coordinate system.

[0055] In this embodiment, the alignment matrix is ​​determined in advance, avoiding the complex point cloud model registration process in the later stage, improving detection efficiency. The transformation matrix is ​​dynamically solved by using the consistency constraint of common marker points, which suppresses the cumulative error in large-scale scanning, ensuring the accuracy and stability of multi-robot collaborative scanning, and realizing efficient and high-precision detection that can be used immediately after scanning.

[0056] Based on the above embodiments, the transformation matrix includes a rotation matrix and a translation vector. This application also provides a process for determining the transformation matrix in multi-scanner-based object detection. Figure 5 This application provides a schematic diagram of a process for determining the transformation matrix in object detection based on multiple scanners, as illustrated in the embodiments of this application. Figure 5 As shown, in step 402 above, consistency constraints are applied to the common marker points of the first point cloud data and the third point cloud data to obtain the transformation matrix, including: Step 501: Construct the objective function based on the first point cloud data, the third point cloud data, the initial rotation matrix, and the initial translation vector.

[0057] The initial rotation matrix is ​​a 3×3 matrix used to indicate the directional difference between the first and second coordinate systems, and the initial translation vector is the displacement between the origins of the first and second coordinate systems.

[0058] Optionally, based on the first point cloud data Third point: cloud data Initial rotation matrix and the initial translation vector Construct the objective function Among them, the first point is cloud data. Third point: cloud data .

[0059] Step 502: Solve the objective function using the least squares method to obtain the rotation matrix and translation vector.

[0060] The least squares method solves for the optimal parameters of the initial rotation matrix and the initial translation vector by minimizing the sum of squares of the differences between the observed and predicted values, thus obtaining the rotation matrix and the translation vector.

[0061] Step 503: Construct the transformation matrix based on the rotation matrix and the translation vector.

[0062] Optionally, according to the rotation matrix and translation vector Construct a 4×4 transformation matrix [R21,t21;0,1].

[0063] In this embodiment, by constructing an objective function based on common marker points and using the least squares method for optimization, the rotation matrix and translation vector between the two scanners can be calculated with high precision, eliminating coordinate system deviation, improving the accuracy of coordinate unification, and using initial values ​​to guide the optimization process, accelerating the convergence speed, and improving the stability of the algorithm, thereby improving the accuracy and efficiency of three-dimensional measurement of large-size workpieces.

[0064] Based on the above embodiments, common marker points are set in a common area, which is the area repeatedly scanned by the first and second scanners. This application also provides a process for determining point cloud data in object detection based on multiple scanners. Figure 6 This application provides a schematic diagram of a process for determining a matrix in object detection based on multiple scanners, as illustrated in the embodiments of this application. Figure 6 As shown, in step 201 above, acquiring the first point cloud data of the common marker points collected by the first scanner and the second point cloud data of the first region of the object to be scanned includes: Step 601: Obtain the first initial point cloud data of the first public marker point collected by the first scanner in the initial public area.

[0065] The initial public area is the area repeatedly scanned by the first and second scanners at the first moment of their movement, including a portion of the object to be scanned and the surrounding environmental data of the object to be scanned. The initial public area includes multiple first public markers, which are markers pre-set in the initial public area.

[0066] Optionally, a first scanner is used to scan within the initial public area to obtain the first initial point cloud data of the first public marker point collected by the first scanner within the initial public area.

[0067] Step 602: Control the first robot to move along the direction away from the initial public area to obtain the second public marker point corresponding to the new public area, and obtain the second initial point cloud data of the second public marker point.

[0068] The new public area is the area repeatedly scanned after the first and second scanners move. Multiple repeated public markers exist in the new public area and the initial public area. When only the first robot moves and the second robot does not, the new public area is the public area after the first robot moves; when both the first and second robots move, the new public area is the sum of the public areas after the first robot moves and the public areas after the second robot moves. Multiple second public markers are pre-set in the new public area.

[0069] Optionally, a first scanner is used to scan within the new public area to obtain second initial point cloud data of the second public marker points collected by the first scanner within the new public area.

[0070] Step 603: Determine the first point cloud data based on the first initial point cloud data and the second initial point cloud data.

[0071] Optionally, the set of the first initial point cloud data and the second initial point cloud data is used as the first point cloud data, and duplicate data is deleted.

[0072] In this embodiment, by retaining and fusing marker point data from multiple stages, the coverage of the measurement reference is expanded, the cumulative error caused by a single calibration in large-size scanning is suppressed, and the continuity of common marker points is used to achieve accurate coordinate transformation, thereby improving the accuracy and stability of multi-region point cloud stitching and ensuring the high accuracy and traceability of the entire scanning process.

[0073] Based on the above embodiments, this application also provides another process for determining point cloud data in object detection based on multiple scanners. Figure 7 This application provides another flowchart illustrating the determination of a matrix in object detection based on multiple scanners, as shown in the embodiments below. Figure 7As shown, in step 202 above, acquiring the third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned includes: Step 701: Obtain the third initial point cloud data of the first public marker point collected by the second scanner in the starting public area.

[0074] Optionally, the first scanner shown in the figure is used to scan within the initial public area to obtain the third initial point cloud data of the first public marker point collected by the first scanner in the initial public area.

[0075] Step 702: Control the second robot to move along the direction away from the initial common area to obtain the second common marker point, and obtain the fourth initial point cloud data of the second common marker point.

[0076] Optionally, a second scanner is used to scan within the new public area to obtain fourth initial point cloud data of the second public marker points collected by the second scanner within the new public area.

[0077] Step 703: Determine the second point cloud data based on the third and fourth initial point cloud data.

[0078] Optionally, the set of the third initial point cloud data and the fourth initial point cloud data is used as the first point cloud data, and duplicate data is deleted.

[0079] In this embodiment, by establishing a high-precision initial benchmark in the starting region and continuing to collect common marker points in the newly added region after moving outward, the common marker points in the overlapping region can be used to perform segmented coordinate alignment and transformation matrix update, avoiding the cumulative error caused by traditional single calibration; at the same time, dynamically expanding the marker point coverage enhances the robustness and accuracy stability of the system in large-size scanning.

[0080] Based on the above embodiments, this application also provides a process for object detection in multi-scanner-based object detection. Figure 8 This application provides a schematic diagram of the object detection process in a multi-scanner-based object detection system, as illustrated in the embodiments of this application. Figure 8 As shown, step 304 above describes detecting the object to be scanned based on the scan data and digital model, including: Step 801: Generate the minimum distance from the scanned data to the digital model, the color difference map, and the key dimensions.

[0081] The minimum distance is the shortest Euclidean distance from each 3D coordinate point in the scanned data to the corresponding point on the surface of the digital model. The minimum distance reflects the magnitude of the deviation between the actual manufactured part and the design model; a positive minimum distance indicates a convexity, and a negative minimum distance indicates a concaveness. The color difference map is a visual image that maps the minimum distance results onto the model or point cloud in a color-coded format. Critical dimensions are key parameters used in object design, such as hole spacing, wall thickness, flange height, and assembly clearance, obtained from the scanned data.

[0082] Optionally, a feature registration algorithm is used to spatially rigidify the 3D data in the scanned data with the 3D data in the digital model. For each 3D data point, the nearest Euclidean distance to the model surface is calculated, and these distance values ​​are mapped to pseudo-color bands to generate a color difference map. Key dimensional indicators such as average deviation, standard deviation, and maximum and minimum deviations are calculated based on the scanned data. For specific geometric features, a segmentation algorithm is used to extract the point cloud of the corresponding region, and the actual size is calculated using fitting. The specific geometric features can be aperture, cylinder diameter, flatness, or the distance between two features, etc. This embodiment of the application does not limit this.

[0083] Step 802: Detect the object to be scanned based on the minimum distance, color difference map, and key dimensions.

[0084] Optionally, the deviation values ​​in the minimum distance and color difference map are used to determine whether the overall surface shape is out of tolerance, and the measured values ​​of the key dimensions are used to determine whether the local features are qualified. By comprehensively considering whether all out-of-tolerance items are within the allowable range, a qualified / unqualified inspection conclusion is output, and the out-of-tolerance areas are highlighted on the color difference map to assist in locating defects.

[0085] In this embodiment, by generating the minimum distance from the scanned data to the digital model, the color difference map, and the key dimensions, high-precision and efficient detection of large-sized objects to be scanned is achieved, improving the accuracy and readability of the detection results, reducing manual interpretation and post-processing time, and improving the overall detection efficiency and automation level.

[0086] Based on the above embodiments, this application also provides a process for determining an alignment matrix in object detection based on multiple scanners. In step 301 above, the first point cloud data is aligned with the digital model to obtain the alignment matrix, including: The singular value decomposition method is used to align the first point cloud data with the digital model to obtain the alignment matrix.

[0087] Singular value decomposition decomposes a matrix into the product of three specific matrices: the left singular vector matrix, the diagonalized singular value matrix, and the transpose of the right singular vector matrix.

[0088] Optionally, a set of corresponding feature points are extracted from the first point cloud data and the digital model, respectively. Then, a least squares optimization problem is constructed to determine the rotation matrix and translation vector. The centroids of the two sets of point sets are calculated and the point sets are decentered. Then, a covariance matrix is ​​constructed, and singular value decomposition is performed on the matrix to obtain the rotation matrix and translation vector. Finally, the rotation matrix and translation vector are constructed into a 4×4 homogeneous transformation matrix. The alignment is completed by left-multiplying the matrix by the homogeneous coordinates of the original point cloud.

[0089] In this embodiment, the optimal spatial transformation parameters are accurately solved by extracting the actual marker coordinates from the first point cloud data and the theoretical marker coordinates from the digital model. This avoids iteration errors and improves the accuracy and reliability of subsequent comparisons between scan data and the design model.

[0090] Based on the same inventive concept, this application also provides an object detection device based on multiple scanners, which corresponds to the object detection method based on multiple scanners. Since the principle of the device in this application is similar to the object detection method based on multiple scanners described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0091] Figure 9 This application provides a schematic diagram of the structure of an object detection device based on multiple scanners, as shown in the embodiments. Figure 9 As shown, the device includes: The first acquisition module 901 is used to acquire the first point cloud data of common marker points collected by the first scanner and the second point cloud data of the first area of ​​the object to be scanned. The first scanner is a scanner mounted on the first robot. The common marker points are set in a common area, which is the area scanned repeatedly by the first scanner and the second scanner. The second acquisition module 902 is used to acquire the third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned, wherein the second scanner is a scanner mounted on the second robot; The detection module 903 is used to detect the object to be scanned based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned.

[0092] In one possible implementation, the detection module 903 is specifically used to: determine the alignment matrix and the transformation matrix based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data and the digital model of the object to be scanned; The fourth point cloud data is transformed from the second coordinate system of the second scanner to the first coordinate system of the first scanner using a transformation matrix to obtain the fifth point cloud data; Using an alignment matrix, the second and fifth point cloud data are transformed from the first coordinate system to the model coordinate system to obtain the scan data; The object to be scanned is detected based on the scan data and digital model.

[0093] In one possible implementation, the detection module 903 is specifically used to: align the second point cloud data with the digital model to obtain an alignment matrix; Consistency constraints on common marker points are applied to the first and third point cloud data to obtain the transformation matrix.

[0094] In one possible implementation, the detection module 903 is specifically used to: construct an objective function based on the first point cloud data, the third point cloud data, the initial rotation matrix, and the initial translation vector; The objective function is solved using the least squares method, yielding the rotation matrix and translation vector. Construct the transformation matrix based on the rotation matrix and the translation vector.

[0095] In one possible implementation, the first acquisition module 901 is specifically used to: acquire the first initial point cloud data of the first public marker point collected by the first scanner in the initial public area; Control the first robot to move along a direction away from the initial public area to obtain the second public marker point corresponding to the new public area, and obtain the second initial point cloud data of the second public marker point; The first point cloud data is determined based on the first initial point cloud data and the second initial point cloud data.

[0096] In one possible implementation, the second acquisition module 901 is specifically used to: acquire the third initial point cloud data of the first public marker point collected by the second scanner in the initial public area; Control the second robot to move along a direction away from the initial public area to obtain the second public marker point, and acquire the fourth initial point cloud data of the second public marker point; The second point cloud data is determined based on the third and fourth initial point cloud data.

[0097] In one possible implementation, the detection module 903 is specifically used to: generate the minimum distance from the scanned data to the digital model, a color difference map, and key dimensions; The object to be scanned is detected based on the minimum distance, color difference map, and key dimensions.

[0098] In one possible implementation, the detection module 903 is specifically used to: align the first point cloud data with the digital model using singular value decomposition to obtain an alignment matrix.

[0099] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0100] This application also provides a computer device. Figure 10 This application provides a schematic diagram of the structure of a computer device, as shown in the embodiment of the present application. Figure 10 As shown, the computer device includes a processor 1001 and a memory 1002, and optionally, a bus 1003. The memory 1002 stores machine-readable instructions executable by the processor 1001. When the computer device is running, the processor 1001 and the memory 1002 communicate via the bus 1003. When the machine-readable instructions are executed by the processor 1001, the steps of the aforementioned multi-scanner-based object detection method are performed. The computer device is also connected to a corresponding first robot and a second robot for controlling their movement. The computer device is also connected to a first scanner and a second scanner for acquiring scan data obtained by the first and second scanners.

[0101] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described multi-scanner-based object detection method.

[0102] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0104] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for object detection based on multiple scanners, characterized in that, The method includes: The system acquires first point cloud data of common marker points collected by a first scanner and second point cloud data of a first region of the object to be scanned, wherein the first scanner is a scanner mounted on a first robot; the common marker points are set in a common area, and the common area is the area repeatedly scanned by the first scanner and the second scanner; The third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned are obtained, wherein the second scanner is a scanner mounted on the second robot; The object to be scanned is detected based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned.

2. The method according to claim 1, characterized in that, The step of detecting the object to be scanned based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned includes: Based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned, determine the alignment matrix and the transformation matrix; The transformation matrix is ​​used to transform the fourth point cloud data from the second coordinate system of the second scanner to the first coordinate system of the first scanner to obtain the fifth point cloud data. Using the alignment matrix, the second point cloud data and the fifth point cloud data are transformed from the first coordinate system to the model coordinate system to obtain the scan data; The object to be scanned is detected based on the scan data and the digital model.

3. The method according to claim 2, characterized in that, The step of determining the alignment matrix and transformation matrix based on the first point cloud data, the second point cloud data, the third point cloud data, the fourth point cloud data, and the digital model of the object to be scanned includes: Align the second point cloud data with the digital model to obtain the alignment matrix; The first point cloud data and the third point cloud data are subjected to consistency constraints on common marker points to obtain the transformation matrix.

4. The method according to claim 3, characterized in that, The transformation matrix includes a rotation matrix and a translation vector; the process of applying consistency constraints on common marker points to the first point cloud data and the third point cloud data to obtain the transformation matrix includes: Based on the first point cloud data, the third point cloud data, the initial rotation matrix, and the initial translation vector, construct the objective function; The objective function is solved using the least squares method to obtain the rotation matrix and the translation vector. The transformation matrix is ​​constructed based on the rotation matrix and the translation vector.

5. The method according to claim 1, characterized in that, The acquisition of the first point cloud data of the common marker points collected by the first scanner and the second point cloud data of the first region of the object to be scanned includes: Acquire the first initial point cloud data of the first public marker point collected by the first scanner in the initial public area; Control the first robot to move along a direction away from the initial public area to obtain a second public marker point corresponding to a new public area, and acquire the second initial point cloud data of the second public marker point; The first point cloud data is determined based on the first initial point cloud data and the second initial point cloud data.

6. The method according to claim 3, characterized in that, The acquisition of the third point cloud data of the common marker points collected by the second scanner and the fourth point cloud data of the second region of the object to be scanned includes: Acquire the third initial point cloud data of the first public marker point collected by the second scanner in the initial public area; Control the second robot to move along a direction away from the initial common area to obtain the second common marker point, and acquire the fourth initial point cloud data of the second common marker point; The second point cloud data is determined based on the third initial point cloud data and the fourth initial point cloud data.

7. The method according to claim 2, characterized in that, The step of detecting the object to be scanned based on the scan data and the digital model includes: Generate the minimum distance, color difference map, and key dimensions from the scanned data to the digital model; The object to be scanned is detected based on the minimum distance, the color difference map, and the key dimensions.

8. The method according to claim 3, characterized in that, The step of aligning the first point cloud data with the digital model to obtain the alignment matrix includes: The singular value decomposition method is used to align the first point cloud data with the digital model to obtain the alignment matrix.

9. A computer device, characterized in that, include: The processor and memory, the memory storing machine-readable instructions executable by the processor, which, when the computer device is running, are executed by the processor to perform the steps of the multi-scanner-based object detection method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the multi-scanner-based object detection method as described in any one of claims 1 to 8.

Images