A fusion method, device and detection system for defect points of a workpiece

Abstract

The embodiment of the application provides a workpiece defect point fusion method, device and detection system, which can uniformly convert defect points from different acquisition devices to a target coordinate system of a workpiece model, accurately calculate the distance between each defect point, and intelligently fuse the points with similar distances by using a preset screening threshold to obtain target defect points, thereby reducing redundant data, retaining defect information with real detection significance, reducing the complexity of data processing and the false report rate, making the finally determined target defect points more accurate and reliable, and further providing strong support for subsequent defect analysis, repair and quality control, and significantly improving production efficiency and product quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a workpiece defect point fusion method and device and detection system. BACKGROUND

[0002] Traditional manual detection methods are inefficient and easily affected by subjective factors, making it difficult to ensure the accuracy and consistency of detection results, and have been difficult to meet the needs of modern manufacturing. With the rise of machine vision technology, automated detection systems based on a single camera have emerged and are widely used in the automotive and electronics manufacturing industries. However, when faced with complex production environments and multi-faceted or irregular-shaped parts, single-camera detection has blind spots and misjudgment problems.

[0003] To break through this bottleneck, a multi-camera collaborative detection system has been developed. This technology uses multiple cameras to capture parts from different angles, achieving comprehensive coverage of the detection area and deep mining of image information, significantly improving the comprehensiveness and accuracy of detection. However, the multi-camera system places higher demands on software development, configuration, deployment, and maintenance, and multiple cameras may capture the same defect from different angles, resulting in a large amount of repeated defect information in the detection results, which further leads to low accuracy and efficiency of defect detection, affecting production efficiency and product quality. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a workpiece defect point fusion method, device and detection system to improve the accuracy and efficiency of workpiece defect detection positioning, and thus improve production efficiency and product quality. The specific technical solutions are as follows:

[0005] In a first aspect of the present application, a workpiece defect point fusion method is provided, the method comprising:

[0006] Projecting the initial defect points obtained by each acquisition device collecting a target workpiece into a target coordinate system to which a workpiece model of the target workpiece belongs, based on a target conversion relationship corresponding to each acquisition device, to obtain defect points to be screened;

[0007] Calculate the distance between each of the defect points to be screened;

[0008] According to the distance, fuse the defect points to be screened whose distance is less than a preset screening threshold to obtain target defect points.

[0009] In a possible implementation, the acquisition device includes a first acquisition device and a plurality of second acquisition devices, and the method further comprises:

[0010] The calibration obtains a conversion relationship between a first device coordinate system of the first acquisition device and the target coordinate system as a first conversion relationship corresponding to the first acquisition device;

[0011] According to the first position relationship and the first conversion relationship, a conversion relationship between each second acquisition device and the target coordinate system is calculated as a second conversion relationship corresponding to each second acquisition device, wherein the first position relationship is a position relationship between the first acquisition device and the second acquisition device.

[0012] In a possible implementation, the acquisition device is arranged on a mechanical arm, and the method further comprises:

[0013] A position relationship between each acquisition device and the mechanical arm is obtained through hand-eye calibration;

[0014] According to the position relationship between the first acquisition device and the mechanical arm and the position relationship between the second acquisition device and the mechanical arm, a position relationship between the first acquisition device and the second acquisition device is calculated as the first position relationship.

[0015] In a possible implementation, the calibration obtains a conversion relationship between a first device coordinate system of the first acquisition device and the target coordinate system as a first conversion relationship corresponding to the first acquisition device, comprising:

[0016] The initial image data obtained by the first acquisition device shooting the target workpiece and the workpiece model are displayed;

[0017] A plurality of first calibration points input in the initial image data and a plurality of second calibration points input in the workpiece model are obtained, wherein each second calibration point corresponds to a first calibration point;

[0018] A conversion relationship between the plurality of first calibration points and the plurality of second calibration points is calculated as a first conversion relationship.

[0019] In a possible implementation, the method further comprises:

[0020] A third calibration point input in the initial image data and a fourth calibration point input in the workpiece model are obtained, wherein the fourth calibration point corresponds to the third calibration point;

[0021] The third calibration point is projected to the target coordinate system based on the first conversion relationship to obtain a projected calibration point;

[0022] An error between the projected calibration point and the fourth calibration point is calculated.

[0023] The conversion relationship between each second acquisition device and the target coordinate system is calculated according to the first position relationship and the first conversion relationship as the second conversion relationship corresponding to each second acquisition device.

[0024] If the error satisfies the preset error condition, the conversion relationship between each second acquisition device and the target coordinate system is calculated according to the first position relationship and the first conversion relationship as the second conversion relationship corresponding to each second acquisition device.

[0025] In a possible implementation, the conversion relationship between the plurality of first calibration points and the plurality of second calibration points is calculated as the first conversion relationship, including:

[0026] The conversion relationship between the plurality of first calibration points and the plurality of second calibration points is calculated as the first conversion relationship by a PNP algorithm.

[0027] In a possible implementation, the acquisition device is arranged on a mechanical arm, and the mechanical arm is deployed on an automobile production line, and the method further includes:

[0028] A model is established in advance for each component in the automobile on the automobile production line;

[0029] The type of the target workpiece is obtained, and a corresponding model in the model is searched as a workpiece model of the target workpiece.

[0030] In the second aspect of the present application, a workpiece defect point fusion device is provided, and the device includes:

[0031] A defect projection module is configured to project initial defect points obtained by each acquisition device from a target workpiece into a target coordinate system to which a workpiece model of the target workpiece belongs, based on a target conversion relationship corresponding to each acquisition device, to obtain to-be-screened defect points.

[0032] A distance calculation module is configured to calculate distances between the to-be-screened defect points.

[0033] A defect fusion module is configured to fuse defect points in the to-be-screened defect points with distances less than a preset screening threshold according to the distances, to obtain target defect points.

[0034] In a possible implementation, the acquisition device includes a first acquisition device and a plurality of second acquisition devices, and the device further includes:

[0035] The first calibration module is configured to calibrate a conversion relationship between a first device coordinate system of the first acquisition device and the target coordinate system as a first conversion relationship corresponding to the first acquisition device.

[0036] The second calibration module is configured to calculate conversion relationships between the second acquisition devices and the target coordinate system as second conversion relationships corresponding to the second acquisition devices respectively according to the first position relationship and the first conversion relationship, wherein the first position relationship is a position relationship between the first acquisition device and the second acquisition device.

[0037] The acquisition device is arranged on a mechanical arm, and the device further comprises:

[0038] The third calibration module is configured to obtain a position relationship between the acquisition device and the mechanical arm through hand-eye calibration.

[0039] The fourth calibration module is configured to calculate a position relationship between the first acquisition device and the second acquisition device as the first position relationship according to the position relationship between the first acquisition device and the mechanical arm and the position relationship between the second acquisition device and the mechanical arm.

[0040] The first calibration module comprises:

[0041] The calibration first submodule is configured to display initial image data obtained by the first acquisition device in shooting the target workpiece and the workpiece model.

[0042] The calibration second submodule is configured to obtain a plurality of first calibration points input in the initial image data and a plurality of second calibration points input in the workpiece model, wherein each second calibration point corresponds to a first calibration point.

[0043] The calibration third submodule is configured to calculate a conversion relationship between the plurality of first calibration points and the plurality of second calibration points as a first conversion relationship.

[0044] The device further comprises:

[0045] The fifth calibration module is configured to obtain a third calibration point input in the initial image data and a fourth calibration point input in the workpiece model, wherein the fourth calibration point corresponds to the third calibration point.

[0046] The sixth calibration module is configured to project the third calibration point to the target coordinate system based on the first conversion relationship to obtain a projected calibration point.

[0047] A seventh calibration module is configured to calculate an error between the projection calibration point and the fourth calibration point;

[0048] The second calibration module comprises:

[0049] A fourth calibration sub-module is configured to, if the error satisfies a preset error condition, calculate a conversion relationship between each second acquisition device and the target coordinate system according to the first position relationship and the first conversion relationship, as a second conversion relationship corresponding to each second acquisition device.

[0050] The third calibration sub-module comprises:

[0051] A first calibration sub-unit is configured to calculate a conversion relationship between the plurality of first calibration points and the plurality of second calibration points as a first conversion relationship by using a PNP algorithm.

[0052] The acquisition device is arranged on a mechanical arm, and the mechanical arm is deployed on an automobile production line.

[0053] A model establishing module is configured to pre-establish a model for each component in the automobile on the automobile production line.

[0054] A model searching module is configured to acquire a type of the target workpiece, search for a model of the corresponding type in the model as a workpiece model of the target workpiece.

[0055] In the third aspect of the present application, a detection system is provided, which comprises a mechanical arm, an acquisition device, a control device, and a display device.

[0056] The mechanical arm is configured to fix the acquisition device.

[0057] The acquisition device is configured to acquire an initial defect point of a target workpiece and send the acquired initial defect point to the control device.

[0058] The control device is configured to implement the fusion method of the workpiece defect point as described above.

[0059] In the fourth aspect of the present application, an electronic device is provided, which comprises:

[0060] A memory is configured to store a computer program.

[0061] A processor is configured to execute the program stored in the memory, and implement the fusion method of the workpiece defect point as described above.

[0062] The embodiments of the present application have the following beneficial effects:

[0063] The workpiece defect point fusion method, device and detection system provided by the embodiments of the present application can uniformly convert defect points from different acquisition devices to a target coordinate system of a workpiece model, accurately calculate the distances between the defect points, and intelligently fuse the points with similar distances by using a preset screening threshold to obtain target defect points, thereby reducing redundant data, retaining defect information that is truly meaningful for detection, reducing the complexity of data processing and the false positive rate, making the finally determined target defect points more accurate and reliable, and further providing strong support for subsequent defect analysis, repair and quality control, and significantly improving production efficiency and product quality.

[0064] Of course, implementing any product or method of the present application does not necessarily require achieving all the advantages described above at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art based on these drawings.

[0066] Figure 1 A scene diagram of workpiece defect detection positioning is provided for the embodiments of the present application.

[0067] Figure 2 A flowchart of a workpiece defect point fusion method is provided for the embodiments of the present application.

[0068] Figure 3 A first conversion relationship calibration first schematic diagram is provided for the embodiments of the present application.

[0069] Figure 4 A second conversion relationship calibration schematic diagram is provided for the embodiments of the present application.

[0070] Figure 5 A third conversion relationship calibration schematic diagram is provided for the embodiments of the present application.

[0071] Figure 6 A fourth conversion relationship calibration schematic diagram is provided for the embodiments of the present application.

[0072] Figure 7 A fifth conversion relationship calibration schematic diagram is provided for the embodiments of the present application.

[0073] Figure 8 A sixth conversion relationship calibration schematic diagram is provided for the embodiments of the present application.

[0074] Figure 9The seventh schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0075] Figure 10 The eighth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0076] Figure 11 The ninth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0077] Figure 12 The tenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0078] Figure 13 The eleventh schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0079] Figure 14 The twelfth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0080] Figure 15 The thirteenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0081] Figure 16 The fourteenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0082] Figure 17 The fifteenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0083] Figure 18 The sixteenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0084] Figure 19 The seventeenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application;

[0085] Figure 20 The structure schematic diagram of the detection system provided by the embodiment of the present application;

[0086] Figure 21 The structure schematic diagram of the fusion device of the workpiece defect point provided by the embodiment of the present application;

[0087] Figure 22 The electronic device schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0088] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application are within the scope of protection of the present application.

[0089] In order to more clearly describe the positioning method for workpiece defect detection provided in the present application, a possible application scenario of the positioning method for workpiece defect detection provided in the present application will be exemplarily described below. It can be understood that the following example is only a possible application scenario of the positioning method for workpiece defect detection provided in the present application, and in other possible embodiments, the positioning method for workpiece defect detection provided in the present application can also be applied to other possible application scenarios, and the following example does not limit the positioning method for workpiece defect detection provided in the present application in any way.

[0090] Taking an automobile production line as an example, the positioning of workpiece defect detection is an important link on the automobile production line. Referring to Figure 1 , Figure 1 The scene diagram of the positioning of workpiece defect detection provided in the embodiments of the present application is to detect whether the automobile workpiece has defects and to obtain the specific position of the defects.

[0091] In order to detect whether the automobile workpiece has defects and to obtain the position of the defects, the present application provides a fusion method for workpiece defect point position, which is described below Figure 2 , Figure 2 The flowchart of the fusion method for workpiece defect point position provided in the embodiments of the present application includes the following steps.

[0092] S201, respectively based on the target conversion relationship corresponding to each acquisition device, project the initial defect point positions obtained by each acquisition device from the target workpiece to the target coordinate system to which the workpiece model of the target workpiece belongs, to obtain the to-be-screened defect point positions.

[0093] S202, calculate the distances between the to-be-screened defect point positions;

[0094] S203, according to the distances, fuse the defect point positions with distances less than a preset screening threshold in the to-be-screened defect point positions, to obtain the target defect point positions.

[0095] By applying the above embodiments, the defect point positions from different acquisition devices can be uniformly converted to the target coordinate system of the workpiece model. By accurately calculating the distances between the defect point positions and using a pre-set screening threshold, the points with similar distances can be intelligently fused to obtain target defect point positions, reducing redundant data and retaining defect information that is truly meaningful for detection, thereby reducing the complexity of data processing and the false positive rate, making the final determined target defect point positions more accurate and reliable, and further providing strong support for subsequent defect analysis, repair, and quality control, significantly improving production efficiency and product quality.

[0096] The above S201-S203 will be exemplarily described as follows:

[0097] In S201, the acquisition device is integrated with a visual algorithm for detecting defects of a workpiece to obtain defect point positions, such as FCN (Fully Convolutional Networks), YOLO algorithm (You Only Look Once, target detection model), etc.

[0098] The acquisition device can be an infrared camera, a visible light camera, or other devices with image acquisition functions. In some scenarios, only infrared cameras or only visible light cameras can be provided on the production line, or both infrared cameras and visible light cameras can be provided.

[0099] In one possible embodiment, the acquisition device can be arranged on a wall or an immovable support. However, since the detection object in the automobile production line is usually large, and needs to be compatible with multiple vehicle models and multiple different specifications of parts, more shooting positions are needed to cover the entire detection area.

[0100] Therefore, in another possible embodiment, the acquisition device can be arranged on a mechanical arm. The mechanical arm is deployed on the automobile production line, and the mechanical arm is a six-axis robot. Due to the multi-degree-of-freedom characteristics of the six-axis robot, the shooting position can be flexibly adjusted, and different parts of the detection object can be accurately aligned, thereby achieving seamless coverage of the entire detection area. This arrangement not only significantly improves the comprehensiveness and accuracy of detection, but also greatly enhances the compatibility and flexibility of the production line. By controlling the six-axis robot, the detection requirements of different vehicle models and parts can be quickly adapted, reducing manual intervention and debugging time, and improving production efficiency and automation level.

[0101] A plurality of acquisition devices are arranged on each mechanical arm. Still referring to Figure 1 , only two acquisition devices, i.e., an infrared camera 2 and a visible light camera 3, are arranged on the mechanical arm 1, and the automobile workpiece 4 is placed on the work flow line 5.

[0102] The target conversion relationship is a coordinate conversion relationship between a device coordinate system to which each acquisition device belongs and a target coordinate system to which the workpiece model belongs. In a possible embodiment, the target conversion relationship corresponding to each acquisition device can be obtained by calibrating each acquisition device. However, in this way, the calibration process is too cumbersome, and the efficiency of defect detection is reduced.

[0103] Based on this, in another possible embodiment, the acquisition devices are divided into first acquisition devices and second acquisition devices. The first acquisition devices are devices that are convenient for accurate calibration and can achieve a high calibration accuracy in actual application. For example, the acquisition device installed at the top end of the mechanical arm, because its position is fixed and easy to accurately calibrate through technical means. All acquisition devices that do not belong to the first acquisition device category are classified as second acquisition devices.

[0104] The target conversion relationship includes a conversion relationship between a coordinate system to which the first acquisition device belongs and the target coordinate system, and a conversion relationship between a coordinate system to which the second acquisition device belongs and the target coordinate system. For convenience of description, the conversion relationship between the coordinate system to which the first acquisition device belongs and the target coordinate system is referred to as the first conversion relationship, and the conversion relationship between the coordinate system to which the second acquisition device belongs and the target coordinate system is referred to as the second conversion relationship.

[0105] First, the first conversion relationship can be calibrated by the following method:

[0106] The initial image data obtained by the first acquisition device shooting the target workpiece and the workpiece model are displayed.

[0107] A plurality of first calibration points input in the initial image data and a plurality of second calibration points input in the workpiece model are obtained. The second calibration points are one-to-one corresponding to the first calibration points.

[0108] The conversion relationship between the plurality of first calibration points and the plurality of second calibration points is calculated as the first conversion relationship.

[0109] In order to more clearly explain how to obtain the first conversion relationship in the calibration process, the following is described by taking the target workpiece as the left rear door inner panel of a part code 14 and the acquisition device as a camera at the top end of an R3 mechanical arm at No. 1 position as an example:

[0110] Referring to Figure 3 , Figure 3 the first conversion relationship calibration of the first schematic diagram provided by the embodiment of the present application, Figure 3The point mapping interface of the stamping defect detection fusion client is shown in the middle. When the target workpiece is the left rear door inner plate with part code 14 and defect detection is required, click the add control in the upper left corner, enter the part code 14 in the new part configuration pop-up box, and click the add control in the lower right corner. Save the part information.

[0111] After saving the part information, refer to Figure 4 , Figure 4 The first conversion relationship calibration provided by the embodiment of the application is a second schematic diagram. The newly added part information box displays the number of conversion points (i.e., the number of positions) and the number of cameras (i.e., the number of acquisition devices). Click the configuration control in the newly added part information box to pop up the basic configuration interface.

[0112] Refer to Figure 5 , Figure 5 The third schematic diagram of the first conversion relationship calibration provided by the embodiment of the application is shown in the figure. Figure 5 The page shown in the figure is a basic configuration page in the site configuration interface. In addition to the basic configuration page, the site configuration interface also includes a camera configuration page, a robot arm coordinate configuration page, and a matrix configuration page.

[0113] The basic configuration page includes a plurality of configuration controls directly related to calibration, including a part code configuration control, a model storage path configuration control, a point cloud storage path configuration control, a main camera (i.e., a first acquisition device) name configuration control, and an added conversion control. The part code configuration control is used to input the unique identification code of the target workpiece, i.e., the part code, which is set to 14 here, representing the left rear door inner plate. The model storage path configuration control and the point cloud storage path configuration control are used to configure the storage location of the workpiece model and the workpiece model point cloud data of the target workpiece. The main camera name control is used to explicitly specify the first acquisition device participating in calibration. In the following, the name of the first acquisition device is taken as an example to illustrate the TopCamDetect (top detection camera).

[0114] After the user clicks the added conversion control, the point configuration page is opened, as shown in Figure 6 , Figure 6 The fourth schematic diagram of the first conversion relationship calibration provided by the embodiment of the application is shown in the figure. This page aims to establish an accurate coordinate conversion relationship between the point in the 2D image of the target workpiece obtained by the first acquisition device (the top camera of the R3 robot arm) and the corresponding point in the 3D workpiece model of the target workpiece through user interaction or automated means.

[0115] In a possible embodiment, the 3D workpiece model can be retrieved by accessing a public database, which may face problems such as insufficient model accuracy, copyright restrictions, or update lag, thereby resulting in low accuracy and efficiency of workpiece defect detection and positioning, and high cost.

[0116] Therefore, in order to improve the accuracy and efficiency of workpiece defect detection and positioning, ensure product quality, enhance production flexibility, and reduce production cost, in another possible embodiment, a model can be established in advance for each component in the automobile on the automobile production line, when it is necessary to detect defects of the target workpiece, the type of the target workpiece is acquired, the corresponding type of model is searched in the model, and then the model is used as the workpiece model of the target workpiece. For example, the user clicks the model selection control on the top of the point configuration page, selects the model file of the left rear door inner panel in all model files, clicks the open control, and waits for the workpiece model of the target workpiece to be loaded. All model files include the left rear door, the left rear door inner panel, the left front door, the left front door inner panel, and the left wing panel.

[0117] After the model is loaded, the user can click the import camera parameter control in the lower left corner of the point configuration page, as shown in Figure 7 , Figure 7 The fifth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application shows selectable camera parameters, including HandEye_K90018446_4-0508.mfa, HandEye_K90801468_5-camera.mfa, HandEye_K90018456_2-camera.mfa, HandEye_K90018458_1-camera.mfa, and HandEye_L00878254_3-camera.mfa. In all selectable camera parameters, the calibration file HandEye_K90801468_5-camera.mfa of the first acquisition device with the camera name TopCamDetect is selected, the open control is clicked, and the internal parameter data of the first acquisition device is loaded.

[0118] Referring to Figure 8 , Figure 8This is a sixth schematic diagram illustrating the first transformation relationship calibration provided in this application embodiment. Users can input the robotic arm name, top camera name (i.e., the name of the first acquisition device), and the specific location of the robotic arm by using the robotic arm configuration control, camera name configuration control, and point (location) information configuration control on the right side of the camera intrinsic parameter control. For example, the robotic arm name is R3, the name of the first acquisition device is TopCamDetect, and the location of the robotic arm is position 1.

[0119] After configuring the above settings, the user clicks the image selection control in the upper right corner of the location configuration page, see [link / reference]. Figure 9 , Figure 9 This is a seventh schematic diagram illustrating the first transformation relationship calibration provided in this application embodiment. After clicking the image selection control, the point configuration page displays 2D images of all workpieces acquired by the first acquisition device of the R3 robotic arm located at position 1, as well as the corresponding 3D workpiece models. The 2D images of the workpieces, i.e. Figure 9 The images shown are production images 1, 2, and 3, and only include the 3D model of the workpiece. Figure 9 The teaching images in the image only show teaching image 1, teaching image 2, and teaching image 3. The user selects a 2D image of the target workpiece to... Figure 9 Taking production image 1 as an example, production image 1 refers to the aforementioned initial image data. The user also needs to select a 3D workpiece model of the target workpiece. Figure 9 Taking teaching image 1 as an example, after selecting both production image 1 and teaching image 1, click to open the control. This will display the initial image data and workpiece model obtained by the first acquisition device from the target workpiece. Locate the area corresponding to the initial image data on the workpiece model. You can use the scroll wheel to zoom in on the workpiece model, paying attention to the front and back of the workpiece model. See also... Figure 10 , Figure 10 This is the eighth schematic diagram illustrating the first transformation relationship labeling provided in the embodiments of this application. Figure 10 The image shown in the left half is a 3D model of the target workpiece, and the image shown in the right half is the initial image data of the target workpiece obtained by the first acquisition device.

[0120] against Figure 10 The initial image data and 3D workpiece model shown in the image data allow users to input multiple first calibration points in the initial image data and second calibration points in the 3D workpiece model that correspond one-to-one with each of the first calibration points. To improve the accuracy of the first transformation relationship obtained through calibration, residual or previously input calibration points need to be cleared before inputting them. For example, see [link to example]. Figure 11 , Figure 11The ninth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application can click the delete control in the selected point list below the 3D workpiece model and the initial image data to clear the residual or the last input of each calibration point position.

[0121] In the input of each calibration point position, see Figure 12 , Figure 12 The tenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application, first, in the appropriate position of the initial image data, a single click is made by using the left mouse button to select the first calibration point position, and then the system automatically or manually records the coordinate position of the first calibration point position in the initial image data. Subsequently, in the 3D workpiece model, a single click is made by using the left mouse button at the model position corresponding to the first calibration point position in the initial image data to select the second calibration point position, and similarly, the coordinate position of the second calibration point position in the 3D workpiece model is recorded.

[0122] After obtaining the corresponding calibration point position, see Figure 13 , Figure 13 The eleventh schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application, the user can record the selected corresponding calibration point position to the selected point list by clicking the record point control. The coordinates of each calibration point position are recorded in the selected point list according to the serial number of the calibration point position, and the delete control, the up control and the down control are included to the right of the coordinates of each calibration point position. The user can delete the corresponding calibration point position by clicking the delete control, and adjust the order of each calibration point position by clicking the up control or the down control.

[0123] Using the same method, multiple first calibration point positions and second calibration point positions can be selected, and as an example, four such corresponding points can be selected. When selecting these calibration point positions, in order to ensure accuracy and easy to identify, those positions with obvious features should be selected, such as one corner of a square hole, one side of an elliptical contour, or a clear and identifiable point on the edge of a circular hole. Such selection helps to improve positioning accuracy and reduce errors.

[0124] Specifically, see Figure 14 , Figure 14 The twelfth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application shows an example of selecting an edge of an ellipse as a calibration point. Such a point is easy to identify in the initial image data and is also convenient to locate in the 3D workpiece model. Similarly, see Figure 15 , Figure 15 The thirteenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application, the selected calibration point position is located at a right angle of the model, see Figure 16 , Figure 16The fourteenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application shows a case where an obvious point easily identified is selected as a calibration point at the edge of a circular hole.

[0125] After the plurality of corresponding calibration point positions are selected, a conversion relationship between the plurality of first calibration point positions and the plurality of second calibration point positions is calculated as the first conversion relationship. For example, refer to Figure 17 , Figure 17 The fifteenth schematic diagram of the first conversion relationship calibration provided by the embodiment of the present application is that the user can trigger the calculation process automatically by clicking the matrix calculation control in the lower left corner of the point position configuration page. This process not only outputs the camera result matrix, which represents the first conversion relationship corresponding to the first acquisition device, but also gives the error value of the matrix calculation to evaluate the accuracy of the conversion relationship.

[0126] In order to improve the accuracy and calculation efficiency of the first conversion relationship, and further improve the efficiency of workpiece defect detection and positioning, PNP (Perspective-n-Point) algorithm can be used in the calculation process. This algorithm uses the corresponding relationship between a plurality of space points and their projection points on the image to solve the optimal coordinate conversion parameter, i.e. the camera result matrix, which directly corresponds to the first conversion relationship. In other embodiments, any algorithm that can calculate the camera result matrix can also be used.

[0127] After the camera result matrix is calculated by the above algorithm, if the camera result matrix is 0, it indicates that the selected calibration point position may be incorrect, the calibration file of the first acquisition device is not correctly configured (such as missing intrinsic data), the dongle is not effective, etc. At this time, the relevant settings should be rechecked and adjusted.

[0128] After the above first conversion relationship is obtained, in order to improve the accuracy of the first conversion relationship and further improve the accuracy of workpiece defect detection and positioning, the first conversion relationship can be verified. In one possible embodiment, if the matrix calculation error is within an acceptable range, for example, the error value is less than 10, it indicates that the selected calibration point position is relatively accurate, and the obtained first conversion relationship is also relatively accurate.

[0129] In another possible embodiment, a third calibration point position input in the initial image data and a fourth calibration point position input in the 3D workpiece model can be obtained, wherein the fourth calibration point position corresponds to the third calibration point position. Then, based on the first conversion relationship, the third calibration point position is projected into the target coordinate system corresponding to the 3D workpiece model to obtain a projection calibration point position. The error between the projection calibration point position and the fourth calibration point position is calculated. If the calculated error meets the preset error condition, the first conversion relationship is relatively accurate.

[0130] If a third calibration point and a fourth calibration point are used to verify the first conversion relationship, the error between the third calibration point and the fourth calibration point can be calculated. If the calculated error meets the preset error condition, the first conversion relationship is relatively accurate.

[0131] If multiple third calibration points and multiple fourth calibration points are used to verify the first conversion relationship, the error value between each third calibration point and the corresponding fourth calibration point is first calculated. Subsequently, statistical analysis can be performed based on these error values, such as calculating the average value, median of the error, or the proportion of points whose error values exceed a certain preset threshold. If any of these statistical indicators meets the preset error condition (such as the average value or median being less than a certain threshold, or the proportion of exceeding the threshold being lower than a certain proportion), it can be considered that the first conversion relationship is relatively accurate overall.

[0132] After completing the above verification process, if it is confirmed that the first conversion relationship is accurate, it should be saved for subsequent use. For example, see Figure 18 , Figure 18 The sixteenth exemplary diagram of the first conversion relationship calibration provided by the embodiments of the present application, the user can save the camera result matrix (i.e. the first conversion relationship) to the point information by clicking the setting matrix control in the point configuration page. See Figure 19 , Figure 19 The seventeenth exemplary diagram of the first conversion relationship calibration provided by the embodiments of the present application, the user can view the set camera result matrix (i.e. the first conversion relationship) in the matrix configuration page.

[0133] After obtaining the first conversion relationship corresponding to the first acquisition device on the R3 robot arm at position 1, the first conversion relationship corresponding to the first acquisition device on the R3 robot arm at other positions can also be calibrated in the same way. After obtaining the first conversion relationship corresponding to the current first acquisition device, the calibration file of the first acquisition device on other robot arms can be imported, and the robot arm name, position information, etc. can be modified. If the workpiece is a one-mold double-piece, the first conversion relationship corresponding to the left and right first acquisition devices needs to be obtained respectively. For example, if a one-mold single-piece part corresponds to 20 different positions of the first acquisition device, a total of 20 camera result matrices, i.e. 20 first conversion relationships, are needed. If a one-mold double-piece part corresponds to 8 different positions of the first acquisition device, a total of 8x2=16 camera result matrices, i.e. 16 first conversion relationships, are needed.

[0134] After obtaining the first conversion relationship corresponding to each first acquisition device, click the save control at the bottom right of the software to prevent information loss after closing the point configuration page.

[0135] By applying the above embodiment, by simultaneously displaying the initial image data acquired by the first acquisition device and the detailed workpiece model, the operator can intuitively determine the corresponding calibration point in two dimensions. By accurately acquiring and matching a plurality of first calibration points in the initial image and a plurality of second calibration points in the workpiece model, the accuracy and reliability of the calibration are enhanced, further, the first conversion relationship calculated based on these calibration points can ensure that the data collected by the first acquisition device can be accurately mapped into the target coordinate system, improving the accuracy and efficiency of workpiece defect detection positioning, providing a solid data foundation for subsequent workpiece defect detection positioning, thereby greatly improving production efficiency and product quality.

[0136] The above describes how to obtain the first conversion relationship, and the following describes how to obtain the second conversion relationship. The second conversion relationship can be obtained by the following method:

[0137] According to the first position relationship and the first conversion relationship, the conversion relationship between each second acquisition device and the target coordinate system is calculated as the second conversion relationship corresponding to each second acquisition device.

[0138] The first position relationship is the position relationship between the first acquisition device and each second acquisition device on the same mechanical arm. Specifically, the position relationship between the first acquisition device and the mechanical arm and the position relationship between each second acquisition device and the mechanical arm can be obtained through hand-eye calibration.

[0139] Hand-eye calibration is to determine the position relationship between the mechanical arm end coordinate system and the device coordinate system corresponding to the acquisition device, which generally includes the following key steps:

[0140] Prepare the calibration scene, which needs to have a to-be-moved object (such as a calibration chessboard), a mechanical arm, and at least one acquisition device.

[0141] Acquisition device calibration, which is the process of determining the camera internal and external parameters, including camera lens focal length, distortion parameters, camera optical center, etc., which is usually performed by using a chessboard calibration method or a multi-angle calibration method.

[0142] Mechanical arm calibration, which is performed by using the calibration tool provided by the mechanical arm or an external calibration tool, and requires the mechanical arm end effector to align with the calibration object, and the joint angle and the pose of the end effector are calculated by using the inverse kinematics algorithm.

[0143] Data acquisition, which is to acquire a series of pose data between the mechanical arm end effector and the acquisition device. These data can be measured by the joint angle and the pose of the end effector, or acquired by using an external measuring device.

[0144] Select a calibration method, select a suitable hand-eye calibration method according to the actual situation, such as point-to-point method, voxel method, direct solution method, least squares method, etc.

[0145] Calculate the conversion relationship, calculate the relative position and attitude relationship between the robot arm and the vision sensor through the selected calibration method.

[0146] The relative position and attitude relationship between the robot arm and the vision sensor obtained by the above hand-eye calibration method, that is, the position relationship between each collection device and the mechanical arm. According to the position relationship between each collection device and the mechanical arm, the relative position relationship between each collection device can be obtained, that is, the position relationship between the first collection device and the second collection device. The position relationship between the first collection device and the second collection device is taken as the first position relationship.

[0147] By applying the above embodiment, the collection device is directly arranged on the mechanical arm, and the hand-eye calibration technology is combined to realize the rapid acquisition of the accurate position relationship between multiple collection devices. Not only does it simplify the cumbersome operation steps in the traditional calibration process, greatly improving the overall performance and flexibility of workpiece defect detection positioning, but also greatly improves the calculation accuracy and efficiency of the position relationship, thereby improving the accuracy and efficiency of workpiece defect detection positioning. Provide a solid data foundation for subsequent workpiece defect detection positioning, thereby greatly improving production efficiency and product quality.

[0148] After obtaining the first position relationship, according to the first position relationship and the aforementioned first conversion relationship corresponding to the first collection device, the conversion relationship between each second collection device and the target coordinate system can be calculated, that is, the second conversion relationship corresponding to each second collection device.

[0149] By applying the above embodiment, by constructing a conversion system with the first collection device as the core reference point, the calculation process of the complex conversion relationship between multiple collection devices and the target coordinate system is greatly simplified. Specifically, first, the conversion relationship between the first device coordinate system of the first collection device and the target coordinate system (i.e. the first conversion relationship) is calibrated. Then, by using the known position relationship between the first collection device and multiple second collection devices (i.e. the first position relationship), combined with the first conversion relationship, the conversion relationship between each second collection device and the target coordinate system (i.e. each second conversion relationship) can be efficiently and accurately derived. This method not only reduces the workload of repeated calibration, but also significantly improves the calculation accuracy and efficiency of the conversion relationship, thereby improving the accuracy and efficiency of workpiece defect detection positioning. Provide strong support for subsequent defect analysis, repair and quality control, significantly improve production efficiency and product quality.

[0150] The first conversion relationship and the second conversion relationship are target conversion relationships. The initial defect points in the 2D images collected by the acquisition devices are projected into a target coordinate system to which the 3D workpiece model of the target workpiece belongs based on the target conversion relationship corresponding to each acquisition device, to obtain the to-be-screened defect points.

[0151] In S202, the distance between each to-be-screened defect point can be calculated by the Euclidean distance formula, or can be calculated by the vector method, or can be calculated by other methods that can obtain the distance between two points.

[0152] In S203, it can be understood that, without considering scaling, the distance between each defect point is the same in different coordinate systems, so the distance between each initial defect point can be calculated as the distance between each to-be-screened defect point before the initial defect points are projected into the target coordinate system to which the workpiece model of the target workpiece belongs. Then, the defect points with a distance less than a preset screening threshold are fused according to the distance, and then projected into the target coordinate system.

[0153] Specifically, when the distance between multiple to-be-screened defect points is less than the preset screening threshold, there are two main fusion methods that can be applied to these points to obtain a target defect point.

[0154] The first method is to select one of the to-be-screened defect points as a representative, and fuse all other defect points with a distance less than the threshold to this representative point, thereby forming a more comprehensive or prominent target defect point. For example, the geometric center, weighted average position or the most representative point of these points can be selected as the fusion result.

[0155] The second method is to create a new defect point, which is not based on any of the existing to-be-screened defect points, but is calculated based on the position information and other possible attributes (such as defect type, size, severity, etc.) of all points with a distance less than the threshold. This newly generated defect point will be used as the target defect point for subsequent defect analysis or processing. When creating a new point, methods such as the center point of the minimum bounding box, weighted centroid or other more complex algorithms can be used to determine its specific position.

[0156] In a possible implementation, after the target defect points are fused, the target defect points can be displayed, specifically, the target defect points can be superimposed on the workpiece model for display. When the workpiece model with the superimposed target defect points is output, in order to ensure that the user can intuitively and conveniently observe each target defect point, thereby helping the user to more efficiently perform defect detection and processing, the target defect points can be enhanced and displayed or marked.

[0157] First, the most intuitive way is to visually highlight the target defect points, and use a color that is in sharp contrast with the main color of the workpiece model to mark the points. For example, if the workpiece model is mainly in gray or silver, a bright color such as red, yellow or blue can be selected to mark the defect points.

[0158] In addition to color marking, icon or symbol marking can also be used, for example, a specific icon or symbol such as a circle, a cross or an arrow is superimposed at the defect point. These icons not only indicate the location of the defect, but also convey additional information (such as defect type, severity, etc.) through their shape or color. Text annotation can also be used, for example, a short text description is added near the defect point to directly explain the nature of the defect or matters needing attention. The text can be set to the same or similar font color as the marking color to ensure the overall visual effect is coordinated.

[0159] The embodiments of the present application also provide a detection system, see Figure 20 , Figure 20 The structure diagram of the detection system provided by the embodiments of the present application is shown in FIG. 2. The detection system 2000 includes a mechanical arm 2001, a collection device 2002, and a control device 2003.

[0160] The mechanical arm 2001 is used to fix the collection device.

[0161] The collection device 2002 is used to collect initial defect points of a target workpiece, and send the collected initial defect points to the control device.

[0162] The control device 2003 is used to execute any of the fusion methods of the workpiece defect points.

[0163] For the steps performed by the mechanical arm 2001, the collection device 2002 and the control device 2003, refer to the descriptions in the foregoing steps 201-203, which will not be repeated here.

[0164] The collection device 2002 and the control device 2003 are in communication connection.

[0165] By applying the above embodiment, the defect point positions from different acquisition devices can be uniformly converted to the target coordinate system of the workpiece model, the distances between the defect point positions are accurately calculated, and the preset screening threshold is used to intelligently fuse the point positions with similar distances to obtain target defect point positions, thereby reducing redundant data, retaining defect information with real detection significance, reducing the complexity of data processing and the false positive rate, making the finally determined target defect point positions more accurate and reliable, providing strong support for subsequent defect analysis, repair and quality control, and significantly improving production efficiency and product quality.

[0166] Corresponding to the positioning method for workpiece defect detection, the embodiment of the present application further provides a positioning device for workpiece defect detection, which is described below with reference to Figure 21 , Figure 21 The structure diagram of the fusion device for workpiece defect point positions provided by the embodiment of the present application includes:

[0167] The defect projection module 2101 is configured to project the initial defect point positions obtained by each acquisition device from the target workpiece into the target coordinate system to which the workpiece model belongs based on the target conversion relationship corresponding to each acquisition device, to obtain defect point positions to be screened.

[0168] The distance calculation module 2102 is configured to calculate the distances between the defect point positions to be screened.

[0169] The defect fusion module 2103 is configured to fuse the defect point positions to be screened whose distances are less than the preset screening threshold according to the distances, to obtain target defect point positions.

[0170] By applying the above embodiment, the defect point positions from different acquisition devices can be uniformly converted to the target coordinate system of the workpiece model, the distances between the defect point positions are accurately calculated, and the preset screening threshold is used to intelligently fuse the point positions with similar distances to obtain target defect point positions, thereby reducing redundant data, retaining defect information with real detection significance, reducing the complexity of data processing and the false positive rate, making the finally determined target defect point positions more accurate and reliable, providing strong support for subsequent defect analysis, repair and quality control, and significantly improving production efficiency and product quality.

[0171] In a possible implementation, the acquisition device includes a first acquisition device and a plurality of second acquisition devices, and the device further includes:

[0172] The first calibration module is configured to calibrate the conversion relationship between the first device coordinate system of the first acquisition device and the target coordinate system as the first conversion relationship corresponding to the first acquisition device.

[0173] a second calibration module, configured to calculate a conversion relationship between each of the second acquisition devices and the target coordinate system as a second conversion relationship corresponding to each of the second acquisition devices according to the first position relationship and the first conversion relationship, wherein the first position relationship is a position relationship between the first acquisition device and the second acquisition device;

[0174] The acquisition device is arranged on a mechanical arm, and the device further comprises:

[0175] a third calibration module, configured to obtain a position relationship between each of the acquisition devices and the mechanical arm through hand-eye calibration;

[0176] a fourth calibration module, configured to calculate a position relationship between the first acquisition device and the second acquisition device as the first position relationship according to the position relationship between the first acquisition device and the mechanical arm and the position relationship between the second acquisition device and the mechanical arm;

[0177] The first calibration module comprises:

[0178] a calibration first submodule, configured to display initial image data obtained by the first acquisition device in shooting the target workpiece and the workpiece model;

[0179] a calibration second submodule, configured to acquire a plurality of first calibration points input in the initial image data and a plurality of second calibration points input in the workpiece model, wherein each of the second calibration points corresponds to one of the first calibration points;

[0180] a calibration third submodule, configured to calculate a conversion relationship between the plurality of first calibration points and the plurality of second calibration points as the first conversion relationship;

[0181] The device further comprises:

[0182] a fifth calibration module, configured to acquire a third calibration point input in the initial image data and a fourth calibration point input in the workpiece model, wherein the fourth calibration point corresponds to the third calibration point;

[0183] a sixth calibration module, configured to project the third calibration point to the target coordinate system based on the first conversion relationship to obtain a projected calibration point;

[0184] a seventh calibration module, configured to calculate an error between the projected calibration point and the fourth calibration point;

[0185] The second calibration module comprises:

[0186] The calibration fourth submodule is configured to, if the error satisfies a preset error condition, calculate a conversion relationship between each second acquisition device and the target coordinate system according to the first position relationship and the first conversion relationship, as a second conversion relationship corresponding to each second acquisition device respectively.

[0187] The calibration third submodule comprises:

[0188] The calibration first subunit is configured to calculate a conversion relationship between the plurality of first calibration points and the plurality of second calibration points by a PNP algorithm, as a first conversion relationship.

[0189] The acquisition device is arranged on a mechanical arm, and the mechanical arm is deployed on an automobile production line.

[0190] The model establishment module is configured to pre-establish a model for each component in the automobile on the automobile production line.

[0191] The model searching module is configured to acquire a type of the target workpiece, search for a model corresponding to the type in the model, and take the model as a workpiece model of the target workpiece.

[0192] The embodiment of the application further provides an electronic device, such as Figure 22 as shown, comprising:

[0193] The memory 2201 is configured to store a computer program.

[0194] The processor 2202 is configured to execute the program stored in the memory 2201, and realize the following steps:

[0195] Project initial defect points of the target workpiece acquired by each acquisition device into a target coordinate system to which a workpiece model of the target workpiece belongs, based on a target conversion relationship corresponding to each acquisition device, to obtain to-be-screened defect points;

[0196] Calculate distances between the to-be-screened defect points.

[0197] Fuse defect points in the to-be-screened defect points with a distance less than a preset screening threshold, to obtain target defect points.

[0198] The electronic device can further comprise a communication bus and / or a communication interface, and the processor 2202, the communication interface, and the memory 2201 can communicate with each other through the communication bus.

[0199] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0200] The communication interface is used for communication between the above electronic device and other devices.

[0201] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0202] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0203] In another embodiment provided in the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the workpiece defect point fusion method in any of the above embodiments.

[0204] In another embodiment provided in the present application, a computer program product containing instructions is also provided, and when the computer program product is run on a computer, the computer is caused to execute the workpiece defect point fusion method in any of the above embodiments.

[0205] In the embodiments described above, all or some of the steps can be implemented by using software, hardware, firmware or any combination thereof. When implemented by using software, all or some of the steps can be implemented in the form of one or more computer programs. The computer program is stored in a computer readable medium, and can be executed by a computer to perform all or some of the steps described above. The computer readable medium can be a computer program product in the form of a memory, such as a read-only memory (ROM), a flash memory, a random access memory (RAM), a programmable read-only memory (PROM) or an electrically programmable read-only memory (EPROM). The computer readable medium can also be a removable storage medium, such as a floppy disk, a flexible disk, an optical disk, a magnetic disk, a memory card or a memory stick. The computer readable medium can also be a server computer, a database, a network or the like.

[0206] It should be noted that, in the present document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Also, the terms "comprising", "containing" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not expressly listed or other elements inherent to such process, method, article or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0207] Each of the embodiments in the present document is described in a related manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0208] The above only describes the preferred embodiments of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for fusing defect points in a workpiece, characterized in that, The method comprises: Projecting initial defect points obtained by each acquisition device on a target workpiece into a target coordinate system to which a workpiece model of the target workpiece belongs, based on a target conversion relationship corresponding to each acquisition device, to obtain to-be-screened defect points; wherein the target workpiece is placed on a work flow line; each acquisition device comprises an infrared camera and / or a visible light camera; Calculating distances between the to-be-screened defect points; Fusing defect points in the to-be-screened defect points that are less than a preset screening threshold in distance, to obtain target defect points; The acquisition device is arranged on a mechanical arm, and a plurality of acquisition devices are arranged on each mechanical arm; the acquisition device comprises a first acquisition device and a plurality of second acquisition devices; the method further comprises: Obtaining a positional relationship between each acquisition device and the mechanical arm through hand-eye calibration; Calculating a positional relationship between the first acquisition device and the second acquisition device as a first positional relationship, based on the positional relationship between the first acquisition device and the mechanical arm and the positional relationship between the second acquisition device and the mechanical arm; Displaying initial image data obtained by the first acquisition device shooting the target workpiece and the workpiece model; Obtaining a plurality of first calibration points input in the initial image data and a plurality of second calibration points input in the workpiece model, wherein each second calibration point corresponds to a first calibration point; Calculating a conversion relationship between the plurality of first calibration points and the plurality of second calibration points as a first conversion relationship through a PNP algorithm; Calculating a conversion relationship between each second acquisition device and the target coordinate system as a second conversion relationship corresponding to each second acquisition device, based on the first positional relationship and the first conversion relationship.

2. The method of claim 1, wherein, The method further comprises: Obtaining a third calibration point input in the initial image data and a fourth calibration point input in the workpiece model, wherein the fourth calibration point corresponds to the third calibration point; Projecting the third calibration point to the target coordinate system to obtain a projected calibration point, based on the first conversion relationship; Calculating an error between the projected calibration point and the fourth calibration point; The calculating a conversion relationship between each second acquisition device and the target coordinate system as a second conversion relationship corresponding to each second acquisition device, based on the first positional relationship and the first conversion relationship, comprises: If the error meets a preset error condition, calculating a conversion relationship between each second acquisition device and the target coordinate system as a second conversion relationship corresponding to each second acquisition device, based on the first positional relationship and the first conversion relationship.

3. The method of claim 1, wherein, The method further comprises: A model is established in advance for each component in a vehicle on a vehicle production line; Obtaining a type of the target workpiece, and searching for a model of the corresponding type in the model as a workpiece model of the target workpiece.

4. A fusion device for defect locations of a workpiece, characterized by, The device comprises: A defect projection module is configured to project initial defect points obtained by each of the acquisition devices from the target workpiece to a target coordinate system to which a workpiece model of the target workpiece belongs, based on a target conversion relationship corresponding to each of the acquisition devices, to obtain to-be-screened defect points. A distance calculation module is configured to calculate distances between the to-be-screened defect points. A defect fusion module is configured to fuse defect points in the to-be-screened defect points that have a distance less than a preset screening threshold, based on the distances, to obtain target defect points. The acquisition devices are arranged on the mechanical arms, and each of the mechanical arms is provided with a plurality of the acquisition devices. The acquisition devices include a first acquisition device and a plurality of second acquisition devices. The device further includes: A third calibration module is configured to obtain a positional relationship between each of the acquisition devices and the mechanical arms through hand-eye calibration. A fourth calibration module is configured to calculate a positional relationship between the first acquisition device and the second acquisition devices based on the positional relationship between the first acquisition device and the mechanical arms and the positional relationship between the second acquisition devices and the mechanical arms, as a first positional relationship. A first calibration module is configured to calibrate a conversion relationship between a first device coordinate system of the first acquisition device and the target coordinate system, as a first conversion relationship corresponding to the first acquisition device. The first calibration module includes: A calibration first sub-module is configured to display initial image data obtained by the first acquisition device from the target workpiece and the workpiece model. A calibration second sub-module is configured to obtain a plurality of first calibration points input in the initial image data and a plurality of second calibration points input in the workpiece model, wherein each of the second calibration points corresponds to one of the first calibration points. A calibration third sub-module is configured to calculate a conversion relationship between the plurality of first calibration points and the plurality of second calibration points, as a first conversion relationship. The calibration third sub-module includes: A calibration first sub-unit is configured to calculate a conversion relationship between the plurality of first calibration points and the plurality of second calibration points, as a first conversion relationship, through a PNP algorithm. A second calibration module is configured to calculate a conversion relationship between each of the second acquisition devices and the target coordinate system based on the first positional relationship and the first conversion relationship, as a second conversion relationship corresponding to each of the second acquisition devices.

5. The apparatus of claim 4, wherein, The device further includes: A fifth calibration module is configured to obtain a third calibration point input in the initial image data and a fourth calibration point input in the workpiece model, wherein the fourth calibration point corresponds to the third calibration point. A sixth calibration module is configured to project the third calibration point to the target coordinate system based on the first conversion relationship, to obtain a projected calibration point. A seventh calibration module is configured to calculate an error between the projected calibration point and the fourth calibration point. The second calibration module includes: The fourth calibration sub-module is configured to, if the error satisfies a preset error condition, calculate a conversion relationship between each second acquisition device and the target coordinate system according to the first position relationship and the first conversion relationship, and take the conversion relationship as a second conversion relationship corresponding to each second acquisition device. The device further comprises: The model establishing module is configured to pre-establish a model for each part in the automobile on the automobile production line; The model searching module is configured to acquire the type of the target workpiece, search for a model of the corresponding type in the model, and take the model as the workpiece model of the target workpiece.

6. A detection system characterized by, The system comprises a mechanical arm, an acquisition device, and a control device. The mechanical arm is configured to fix the acquisition device. The acquisition device is configured to acquire initial defect points of a target workpiece and send the acquired initial defect points to the control device. The control device is configured to implement the method in any one of claims 1-3.

Images