Workpiece detection system, method and device, computer equipment and storage medium

Abstract

The invention relates to a workpiece detection system, method and device, computer equipment, a storage medium and a computer program product. The system comprises a conveying device used for conveying a target workpiece to a detection station, an image acquisition device used for performing image acquisition on local areas of the target workpiece to obtain local images of the local areas, and a control device used for controlling the local images of the local areas based on the relative position relation between the local areas. The local images of the local areas are fused to obtain a global plane image of the target workpiece, and the configuration control device is further used for reconstructing a global three-dimensional point cloud of the target workpiece. By adopting the system, the configuration control equipment is also used for determining the expected anomaly type matched with the anomaly position in the global plane image, and detecting the global three-dimensional point cloud by taking the expected anomaly type as a detection target to obtain a detection result of the target workpiece, so that the global three-dimensional point cloud does not need to be traversed, and the detection efficiency of the target workpiece is improved.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and in particular to a workpiece detection system, method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] In the automobile and parts manufacturing industry, workpieces, as structural components and appearance components, have a significant impact on the overall performance and appearance quality of the product. Therefore, the need for defect detection on the surface appearance of workpieces is urgent and important.

[0003] In traditional technology, a three-dimensional camera is used to scan the workpiece to obtain workpiece point cloud data. Based on the workpiece point cloud data, defect detection is performed on the workpiece. Although the demand for defect detection of the workpiece can be met, the problem of low detection efficiency still exists. Summary of the invention

[0004] Based on this, it is necessary to provide a workpiece detection method, device, computer equipment, computer-readable storage medium and computer program product that can improve detection efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a workpiece detection system. The method comprises:

[0006] A conveying device is used to convey the target workpiece to the inspection station; the inspection station is provided with a plurality of image acquisition devices; and the image acquisition postures of any two of the image acquisition devices are different;

[0007] The image acquisition device is used to acquire an image of a local area of ​​the target workpiece to obtain a local image of the local area;

[0008] A control device is used to fuse the local images of each local area based on the relative position relationship between the local areas to obtain a global plane image of the target workpiece; reconstruct the global three-dimensional point cloud of the target workpiece according to the image acquisition posture of each image acquisition device and each local image; determine the expected abnormality type that matches the abnormal position in the global plane image; and detect the global three-dimensional point cloud with the expected abnormality type as the detection target to obtain the detection result of the target workpiece.

[0009] In one of the embodiments, the image acquisition device is specifically used to acquire images of a local area of ​​the target workpiece under different lighting conditions to obtain local images of the local area under each lighting condition; the control device is also used to: for each of the local areas, extract depth information of the local area according to image features of each local image corresponding to the local area; superimpose the depth information of each local area onto the global plane image to obtain a 2.5D image of the target workpiece; perform anomaly analysis on the global plane image and the 2.5D image to determine the abnormal position in the global plane image.

[0010] In one of the embodiments, when the control device determines the expected abnormality type that matches the abnormal position in the global plane image, it is configured to: acquire a standard plane image of a standard workpiece; determine the abnormal position in the global plane image based on the texture feature matching result between the global plane image and the standard plane image; and determine the expected abnormality type that matches the abnormal position.

[0011] In one of the embodiments, when the control device determines the expected abnormality type that matches the abnormal position, it is configured to: obtain reference abnormality types configured for each of the multiple image areas in the standard plane image; in each of the image areas, screen the target area to which the abnormal position belongs; and determine the reference abnormality type of the target area as the expected abnormality type that matches the abnormal position.

[0012] In one of the embodiments, when the control device detects the global three-dimensional point cloud with the expected abnormality type as the detection target and obtains the detection result of the target workpiece, it is configured to: obtain the standard point cloud of the target workpiece; extract feature points from the global three-dimensional point cloud and the standard point cloud respectively to obtain the three-dimensional coordinate points of the global three-dimensional point cloud and the standard coordinate points of the standard point cloud; perform point cloud registration on the global three-dimensional point cloud and the standard point cloud based on the relative position and posture matching the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points to obtain the difference point cloud of the global three-dimensional point cloud relative to the standard point cloud; detect the difference point cloud with the expected abnormality type as the detection target to obtain the detection result of the target workpiece.

[0013] In one embodiment, the system also includes a line laser camera; when the control device detects the global three-dimensional point cloud with the expected abnormality type as the detection target and obtains the detection result of the target workpiece, it is configured to: control the line laser camera to be directed to the spatial position corresponding to the abnormal position, and receive the supplementary point cloud collected by the line laser camera; detect the supplementary point cloud with the expected abnormality type as the detection target to obtain the detection result of the target workpiece.

[0014] In a second aspect, the present application provides a workpiece detection method. The method comprises:

[0015] Based on the relative position relationship between each of the local areas, the local images of each of the local areas are fused to obtain a global plane image of the target workpiece; according to the image acquisition posture of each of the image acquisition devices and each of the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; an expected abnormality type that matches the abnormal position in the global plane image is determined; taking the expected abnormality type as the detection target, the global three-dimensional point cloud is detected to obtain a detection result of the target workpiece.

[0016] In a third aspect, the present application also provides a workpiece detection device. The device comprises:

[0017] A fusion module is used to fuse the local images of each of the local areas based on the relative position relationship between the local areas to obtain a global plane image of the target workpiece; a reconstruction module is used to reconstruct the global three-dimensional point cloud of the target workpiece according to the image acquisition posture of each of the image acquisition devices and the local images; a type determination module is used to determine the expected abnormal type that matches the abnormal position in the global plane image; a detection module is used to detect the global three-dimensional point cloud with the expected abnormal type as the detection target to obtain the detection result of the target workpiece.

[0018] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0019] Based on the relative position relationship between each of the local areas, the local images of each of the local areas are fused to obtain a global plane image of the target workpiece; according to the image acquisition posture of each of the image acquisition devices and each of the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; an expected abnormality type that matches the abnormal position in the global plane image is determined; taking the expected abnormality type as the detection target, the global three-dimensional point cloud is detected to obtain a detection result of the target workpiece.

[0020] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0021] Based on the relative position relationship between each of the local areas, the local images of each of the local areas are fused to obtain a global plane image of the target workpiece; according to the image acquisition posture of each of the image acquisition devices and each of the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; an expected abnormality type that matches the abnormal position in the global plane image is determined; taking the expected abnormality type as the detection target, the global three-dimensional point cloud is detected to obtain a detection result of the target workpiece.

[0022] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0023] Based on the relative position relationship between each of the local areas, the local images of each of the local areas are fused to obtain a global plane image of the target workpiece; according to the image acquisition posture of each of the image acquisition devices and each of the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; an expected abnormality type that matches the abnormal position in the global plane image is determined; taking the expected abnormality type as the detection target, the global three-dimensional point cloud is detected to obtain a detection result of the target workpiece.

[0024] The above-mentioned workpiece detection system, method, device, computer equipment, storage medium and computer program product are configured with a conveying device for conveying the target workpiece to the detection station. The detection station is provided with multiple image acquisition devices. The image acquisition postures of any two image acquisition devices are different. The image acquisition devices are configured to perform image acquisition on the local area of ​​the target workpiece to obtain the local image of the local area. The target workpiece is imaged from multiple angles, which improves the accuracy of image acquisition. The control device is configured to fuse the local images of each local area based on the relative position relationship between the local areas to obtain the global plane image of the target workpiece. The control device is also configured to reconstruct the global three-dimensional point cloud of the target workpiece according to the image acquisition posture of each image acquisition device and each local image. Using the above-mentioned system, the control device is also configured to determine the expected abnormal type that matches the abnormal position in the global plane image, and quickly determine the expected abnormal type in the global plane image with lower data complexity. Furthermore, the configuration control device is also used to detect the global three-dimensional point cloud with the expected abnormality type as the detection target, and obtain the detection result of the target workpiece without traversing the global three-dimensional point cloud, which greatly reduces the processing time of the global three-dimensional point cloud and improves the detection efficiency of the target workpiece. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is an application environment diagram of a workpiece detection method in an embodiment;

[0026] Figure 2 is a schematic flow chart of a workpiece detection method in one embodiment;

[0027] Figure 3 A schematic diagram of a process for detecting the surface of a large-sized stamping part in one embodiment;

[0028] Figure 4 It is a structural schematic diagram of a surface detection system for medium and large-sized stamping parts in one embodiment;

[0029] Figure 5 A flowchart of stamping parts inspection in one embodiment;

[0030] Figure 6 is a structural block diagram of a workpiece detection device in one embodiment;

[0031] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In one embodiment, Figure 1 As shown, an embodiment of the present application provides a workpiece detection system, including: a conveying device 110 , an image acquisition device 120 and a control device 130 .

[0034] The conveying device 110 is used to convey the target workpiece to the inspection station.

[0035] The target workpiece can be a specific part in the manufacturing industry. The target workpiece can be a workpiece that needs to be inspected after processing. The target workpiece can also be any part that needs to be inspected. The target workpiece can include stamping parts, welded parts, cut parts, etc. according to different processing techniques.

[0036] The conveying device may be a device for carrying the target workpiece. For example, the conveying device may be a conveying tray or a conveyor belt light for a worker and a vehicle to be tested. There are many types of connection relationships between the conveying device and the target workpiece, for example, the conveying device is fixedly connected to the target workpiece. For another example, the conveying device may support the target workpiece.

[0037] The inspection station can refer to the station used to inspect the workpiece in the workpiece inspection line. Various inspection-related equipment are arranged around the inspection station. For example, a visual camera, a grayscale camera, a proximity switch, etc. Furthermore, in order to improve the optical recognition accuracy, the inspection station can also be arranged with a light shield.

[0038] Specifically, the conveying device is used to transfer the target workpiece to the inspection station.

[0039] In one of the embodiments, the conveying device is further used to send a workpiece arrival signal to the control device when the workpiece arrives at the inspection station.

[0040] In one embodiment, the conveying device can also be used to send an abnormal arrival signal when the workpiece does not reach the inspection station. The conveying device can send the abnormal arrival signal to the control device. The abnormal arrival signal can be used to indicate that the workpiece has not reached the inspection station or the conveying device itself is abnormal.

[0041] In one of the embodiments, the conveying device can also be used to convey the target workpiece to the inspection station and convey the target workpiece away from the inspection station according to the working rhythm of the inspection station.

[0042] In one embodiment, the conveying device can determine the working rhythm of the inspection station according to the working rhythm of the previous station and the length of the target workpiece. The working rhythm of the inspection station can represent the moving time, the staying time and the conveying speed of the conveying device.

[0043] The image acquisition device 120 is used to acquire an image of a local area of ​​the target workpiece to obtain a local image of the local area.

[0044] The image acquisition device is connected to the control device, and the image acquisition device can acquire images toward the target workpiece. The inspection station is provided with multiple image acquisition devices, and the image acquisition postures of any two image acquisition devices are different.

[0045] An image acquisition device may be a device with image acquisition capabilities. The main task of an image acquisition device is to optically image the characteristic information of a target object (such as a target workpiece or other object), convert the optical signal into an electrical signal, and then transmit it to a computer or image processing system. For example, an image acquisition device may be an industrial camera, a scanner, a visual camera, a CCD camera, etc.

[0046] The size of the target workpiece can be large or small. For example, the target workpiece can be a medium-large size stamping part. Medium-large size can mean that either the length or the width of the stamping part is greater than or equal to 50 cm when it is laid flat. The height of the stamping part is generally much less than 50 cm.

[0047] The local area of ​​the target workpiece may represent a plurality of areas divided by the moving direction of the target workpiece. For example, the target workpiece moves forward on the conveying device, and the target workpiece may be divided into a plurality of local areas according to the moving direction. It is understandable that the image acquisition device may be fixedly set, and the target workpiece moves forward on the conveying device, which is conducive to the image acquisition device to acquire the partial images of each of the plurality of local areas.

[0048] The local image may refer to an image acquired by an image acquisition device with respect to a local area of ​​the target workpiece.

[0049] Specifically, the image acquisition device is used to acquire images of a local area of ​​the target workpiece to obtain a local image of the local area. The image acquisition device may be an array composed of multiple visual cameras.

[0050] The control device 130 is used to fuse the local images of each local area based on the relative position relationship between the local areas to obtain a global plane image of the target workpiece.

[0051] The control device can be connected to the image acquisition device and the transmission device respectively.

[0052] The control device may be a device with image processing capabilities. The control device may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Specifically, the control device may be an industrial computer.

[0053] The relative position relationship may represent the position relationship between the local areas. The local area may represent a front area, and the local area may represent a back area. The multiple local areas include a front area, a middle area, and a back area. The concept of front and back may be determined depending on the moving direction of the conveying device.

[0054] The global plane image may represent a complete plane image of the target workpiece.

[0055] Specifically, the control device may fuse the local images of the local areas based on the relative positional relationship between the local areas to obtain a global plane image of the target workpiece.

[0056] In one embodiment, the control device may determine the fusion order of each local area based on the relative position relationship between the local areas. The control device may fuse the local images of each local area based on the fusion order to obtain the global plane image of the target workpiece. For example, the local images at the front are fused first.

[0057] In one embodiment, the control device may determine the fusion weight of each local area based on the relative position relationship between the local areas. The control device may perform weighted fusion on each local area based on each fusion weight to obtain a global plane image of the target workpiece.

[0058] The control device 130 is further used to reconstruct a global three-dimensional point cloud of the target workpiece according to the image acquisition postures of the image acquisition devices and the local images;

[0059] Image acquisition pose can represent the position and posture of the image acquisition device in three-dimensional space, usually including translation vector and rotation matrix. For example, the translation vector and rotation matrix corresponding to the visual camera.

[0060] A global 3D point cloud may refer to a set of multiple points in a 3D space, each point having a 3D coordinate (x, y, z) and may contain other additional information such as color, intensity value, etc.

[0061] Specifically, the control device can reconstruct the global three-dimensional point cloud of the target workpiece according to the image acquisition posture of each image acquisition device and each local image.

[0062] Exemplarily, taking the image acquisition device as a camera, the reconstruction of the point cloud includes: feature point matching, camera pose estimation, triangulation, point cloud generation, global optimization, and dense reconstruction. Specifically, the control device can extract feature points from multiple local images and match the feature points of the target workpiece at different viewing angles. The control device can calculate the camera pose (including rotation and translation) corresponding to each photo through known camera parameters and matched feature points. The control device can use the principle of multi-view geometry to triangulate the pixel coordinates of the feature points at multiple viewing angles to obtain the initial three-dimensional coordinates of these feature points. The control device can calculate the three-dimensional coordinates of each matching point to form a set of sparse three-dimensional point cloud data. The control device can use global optimization technology to jointly optimize the camera pose and spatial point position through nonlinear least squares method, so that the error is evenly distributed to each image. The control device can use multi-view stereo (MVS) technology to further increase the density of the point cloud through the stereo matching algorithm, thereby generating a more complete global three-dimensional point cloud of the target workpiece.

[0063] The control device 130 is further used to determine the expected abnormality type that matches the abnormality position in the global plane image.

[0064] The abnormal position may represent a position in the global plane image, and the local area corresponding to the abnormal position may have an abnormality.

[0065] The expected abnormality type can indicate the abnormality type that usually occurs at the abnormal position. For example, cracks, wrinkles, deformations, and other abnormalities often occur in the functional area of ​​a stamped part. For another example, the appearance area also needs to detect abnormalities that affect the appearance, such as necking, concave surfaces, bumps, scratches, etc. For another example, in the bending area of ​​a stamped part, in addition to the defects to be detected on the entire appearance surface, slip line abnormalities may also occur.

[0066] Specifically, the control device may perform abnormality recognition on the global plane image to obtain the abnormal position of the global plane image. The abnormality recognition may be performed through an abnormality recognition model. In one embodiment, the control device may use the abnormality recognition model to recognize the global plane image to obtain the abnormal position of the global plane image.

[0067] In one embodiment, the control device may also obtain a standard plane image of the standard workpiece. The control device may determine the abnormal position in the global plane image based on the texture feature matching result between the global plane image and the standard plane image. The control device may determine the expected abnormal type matching the abnormal position.

[0068] In one embodiment, the control device may obtain reference abnormality types configured for each of the plurality of image regions in the standard plane image. The control device may screen the target region to which the abnormal position belongs in each image region. The control device may determine the reference abnormality type of the target region as the expected abnormality type matching the abnormal position.

[0069] The control device 130 is also used to detect the global three-dimensional point cloud with the expected abnormality type as the detection target to obtain the detection result of the target workpiece.

[0070] The detection target can be the target of interest in the detection task. Specifically, the detection can be the target of interest in the target detection task. Target detection usually includes two key subtasks: target localization and target classification. First, the position of the target in the image is determined by target localization, and then the specific category of the target is determined by target classification. The position information of the target detection can generally be expressed in polar coordinates or center point coordinates.

[0071] The detection result can indicate the abnormality type, specific information of the abnormality, and confidence of the abnormality type of the target workpiece. The abnormality type can indicate the specific abnormality type that occurs in the target workpiece. For example, cross-section cracking and missing hole abnormalities. The abnormality specific information includes the number of abnormalities and the number of abnormalities. The confidence of the abnormality type can indicate the detection accuracy corresponding to the abnormality type. For example, the accuracy of end face cracking is 80%, and the accuracy of missing hole abnormality is 25%. The abnormality specific information can also be detailed information of the abnormality type. For example, the crack area of ​​end face cracking, etc.

[0072] Specifically, the control device can detect the global three-dimensional point cloud with the expected abnormal type as the target of interest, and obtain the detection result of the target workpiece.

[0073] In one embodiment, the detection result includes multiple abnormality types and the accuracy of each abnormality type. The control device can select a target abnormality type with the highest accuracy from each abnormality type.

[0074] In one embodiment, the control device may increase the weight of the expected abnormality type and recalculate each accuracy to obtain a new accuracy corresponding to each accuracy. The control device may select a target abnormality type with the highest new accuracy from each abnormality type.

[0075] Specifically, the control device can also segment the global three-dimensional point cloud to obtain a segmented point cloud. The control device can detect the segmented point cloud with the expected abnormal type as the detection target to obtain the detection result of the target workpiece.

[0076] Specifically, the control device can also rescan the target workpiece to obtain a new three-dimensional point cloud. The control device can detect the new three-dimensional point cloud with the expected abnormal type as the detection target to obtain the detection result of the target workpiece.

[0077] In the above-mentioned workpiece detection system, a transmission device is configured to transmit the target workpiece to the detection station. The detection station is provided with multiple image acquisition devices. The image acquisition postures of any two image acquisition devices are different. The image acquisition device is configured to perform image acquisition on the local area of ​​the target workpiece to obtain the local image of the local area. The target workpiece is imaged from multiple angles, thereby improving the accuracy of image acquisition. The control device is configured to fuse the local images of each local area based on the relative position relationship between the local areas to obtain the global plane image of the target workpiece. The control device is also configured to reconstruct the global three-dimensional point cloud of the target workpiece according to the image acquisition postures of each image acquisition device and the local images. With the above-mentioned system, the control device is also configured to determine the expected abnormal type that matches the abnormal position in the global plane image, and quickly determine the expected abnormal type in the global plane image with lower data complexity. Further, the control device is also configured to detect the global three-dimensional point cloud with the expected abnormal type as the detection target, and obtain the detection result of the target workpiece without traversing the global three-dimensional point cloud, which greatly reduces the processing time of the global three-dimensional point cloud and improves the detection efficiency of the target workpiece.

[0078] In one embodiment, the image acquisition device is specifically used to acquire images of local areas of the target workpiece under different lighting conditions to obtain local images of the local areas under each lighting condition. The control device is also used to extract depth information of each local area according to image features of each local image corresponding to the local area, superimpose the depth information of each local area onto the global plane image to obtain a 2.5D image of the target workpiece, perform abnormality analysis on the global plane image and the 2.5D image, and determine the abnormal position in the global plane image.

[0079] The lighting condition may refer to a condition set for the image acquisition device. For example, multiple light sources may be configured for each image acquisition device. Multiple light sources may also be configured for each image acquisition device. Each light source has a different light source orientation angle.

[0080] Specifically, the image acquisition device can perform image acquisition on local areas of the target workpiece when the light sources are sequentially turned on, to obtain local images of the local areas under different lighting conditions.

[0081] Depth information may refer to information indicating the distance or relative position between each pixel in an image and an observer.

[0082] A 2.5D image is an image in between two-dimensional (2D) and three-dimensional (3D) form, which retains certain three-dimensional features but does not fully represent a true three-dimensional space. This type of image usually simulates a three-dimensional effect by adding depth or perspective effects to a two-dimensional plane.

[0083] 2.5D images carry depth information, including grayscale information, color information, and pseudo three-dimensional information. Grayscale information can represent the use of grayscale images to represent depth information, where the grayscale value of each pixel represents its depth value. For example, brighter pixels represent closer objects, while darker pixels represent farther objects. Color information can represent the use of colors to represent depth information, usually red and black represent near points, and blue and dark colors represent far points. Pseudo three-dimensional information can represent the reconstruction of scenes by simulating different perspectives, thereby providing a visual three-dimensional effect.

[0084] Specifically, the control device can extract the depth information of each local area according to the image features of each local image corresponding to the local area. The control device can superimpose the depth information of each local area onto the global plane image to obtain a 2.5D image of the target workpiece. The control device can perform anomaly analysis on the global plane image and the 2.5D image to determine the abnormal position in the global plane image.

[0085] In one of the embodiments, the control device may extract grayscale information, color information, and pseudo three-dimensional information from the local image as the depth information of the local area.

[0086] In one embodiment, the control device may sequentially superimpose the depth information of each local area onto the global plane image. Specifically, the control device may sequentially fuse the depth information of each local area according to the superposition weight of each local area, and superimpose it onto the global plane image to obtain a 2.5D image of the target workpiece.

[0087] In one of the embodiments, the control device may use an anomaly analysis model based on 2D and 2.5D deep learning algorithms to perform anomaly analysis on the global plane image and the 2.5D image to determine the abnormal position in the global plane image.

[0088] In this embodiment, the image acquisition device is configured to perform image acquisition on a local area of ​​the target workpiece under different lighting conditions to obtain local images of the local area under each lighting condition. The control device is also configured to extract the depth information of each local area according to the image features of each local image corresponding to the local area, superimpose the depth information of each local area on the global plane image to obtain a 2.5D image of the target workpiece, perform abnormality analysis on the global plane image and the 2.5D image, determine the abnormal position in the global plane image, and quickly determine the abnormal position in the global plane image, thereby improving the detection efficiency of the target workpiece.

[0089] In one embodiment, when the control device determines the expected abnormality type that matches the abnormal position in the global plane image, it is configured to: obtain a standard plane image of a standard workpiece, determine the abnormal position in the global plane image based on the texture feature matching results between the global plane image and the standard plane image, and determine the expected abnormality type that matches the abnormal position.

[0090] Among them, the standard workpiece does not have a defective workpiece. The model of the standard workpiece is the same as that of the target workpiece.

[0091] The standard plane image may be a global plane image of a standard workpiece.

[0092] The texture feature matching result can represent the match between the texture features of the global plane image and the texture features of the standard plane image. The texture feature matching result includes the abnormal position and the number of abnormalities.

[0093] Specifically, the control device can perform model recognition on the target workpiece to obtain the target workpiece and the model. The control device can obtain a standard plane image of a standard workpiece of a corresponding model.

[0094] In one embodiment, the control device may perform texture feature extraction on the global plane image to obtain the texture feature of the global plane image. The control device may perform texture feature extraction on the standard plane image to obtain the texture feature of the standard plane image. The control device may perform texture feature matching on the global plane image and the standard plane image to obtain a texture feature matching result.

[0095] Specifically, the control device may obtain reference abnormality types configured for each of the plurality of image regions in the standard plane image. The control device may screen the target region to which the abnormal position belongs in each image region. The control device may determine the reference abnormality type of the target region as the expected abnormality type matching the abnormal position.

[0096] In this embodiment, a standard plane image of a standard workpiece is obtained, and the abnormal position in the global plane image is determined based on the texture feature matching results between the global plane image and the standard plane image, and the expected abnormal type matching the abnormal position is determined. The abnormal position is determined only based on the texture features, thereby improving the detection efficiency of the target workpiece.

[0097] In one embodiment, when the control device determines the expected abnormality type that matches the abnormal position, it is configured to: obtain the reference abnormality type configured for each of the multiple image areas in the standard plane image, and in each image area, screen the target area to which the abnormal position belongs, and determine the reference abnormality type of the target area as the expected abnormality type that matches the abnormal position.

[0098] The image region may represent a region in the image corresponding to a local region in the standard workpiece.

[0099] The reference anomaly type can indicate the anomaly type corresponding to the local area. The reference anomaly type includes anomaly types that can indicate the anomaly type that usually occurs at the anomaly location. For example, cracks, wrinkles, deformations, and other anomalies often occur in the functional area of ​​a stamping. For another example, the appearance area also needs to detect anomalies that affect the appearance, such as necking, concave surfaces, bumps, scratches, and the like. For another example, for the bending area of ​​a stamping, in addition to the defects to be detected on the entire appearance surface, slip line anomalies may also occur.

[0100] The target region may represent an image region in a standard planar image.

[0101] Specifically, the control device may first determine the corresponding position of the abnormal position in the standard plane image. The control device may determine the image area to which the corresponding position belongs. The control device may determine the image area as the target area to which the abnormal position belongs. The control device may determine the reference abnormality type of the target area as the expected abnormality type matching the abnormal position.

[0102] Exemplarily, the standard plane image may be a standard plane image of a stamping part. The reference abnormality types configured in each image region include a reference abnormality type in an appearance region, a reference abnormality type in a functional region, and a reference abnormality type in a bending region.

[0103] In this embodiment, reference abnormality types configured for each of the plurality of image regions in the standard plane image are obtained, and in each image region, the target region to which the abnormal position belongs is screened, and the reference abnormality type of the target region is determined to be the expected abnormality type matching the abnormal position, and the target region to which the abnormal position belongs is screened, and then the expected abnormality type matching the abnormal position is determined, thereby reducing the number of subsequent items to be detected, and thereby improving the detection efficiency of the target workpiece.

[0104] In one embodiment, the control device detects the global three-dimensional point cloud with the expected abnormality type as the detection target, and when obtaining the detection result of the target workpiece, it is configured to: obtain the standard point cloud of the target workpiece, extract feature points from the global three-dimensional point cloud and the standard point cloud respectively, obtain the three-dimensional coordinate points of the global three-dimensional point cloud, and the standard coordinate points of the standard point cloud, perform point cloud registration on the global three-dimensional point cloud and the standard point cloud based on the relative position and posture matching of the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, obtain the difference point cloud of the global three-dimensional point cloud relative to the standard point cloud, detect the difference point cloud with the expected abnormality type as the detection target, and obtain the detection result of the target workpiece.

[0105] The standard point cloud may be a point cloud of a standard workpiece corresponding to the target workpiece model.

[0106] The 3D coordinate point can represent the feature point coordinates of the global 3D point cloud. The standard coordinate point can represent the feature point coordinates of the standard point cloud.

[0107] The difference point cloud may represent a point cloud corresponding to a difference structure of the target workpiece relative to the standard workpiece.

[0108] Specifically, the control device can directly obtain the standard point cloud of the standard workpiece. The control device can use methods such as RANSAC to extract feature points from the global three-dimensional point cloud and the standard point cloud respectively, and obtain the three-dimensional coordinate points of the global three-dimensional point cloud and the standard coordinate points of the standard point cloud. The control device can use methods such as SVD to solve the conversion relationship between the two groups of point clouds, that is, the relative position and posture. The control device can perform point cloud registration on the global three-dimensional point cloud and the standard point cloud according to the position and posture, and obtain the difference point cloud of the global three-dimensional point cloud relative to the standard point cloud.

[0109] Exemplarily, the control device may also compare the two sets of point clouds to identify whether there are redundant planar point clouds (difference point clouds) in the current target workpiece, and if so, determine that there is a hole defect at that position.

[0110] In this embodiment, a standard point cloud of a target workpiece is obtained, feature points are extracted from the global three-dimensional point cloud and the standard point cloud respectively, and three-dimensional coordinate points of the global three-dimensional point cloud and standard coordinate points of the standard point cloud are obtained. Based on the relative position and posture matching of the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, point cloud registration is performed on the global three-dimensional point cloud and the standard point cloud to obtain a difference point cloud of the global three-dimensional point cloud relative to the standard point cloud. The difference point cloud is detected with the expected abnormality type as the detection target to obtain the detection result of the target workpiece. The difference point cloud is detected, which reduces the amount of data to be processed, thereby improving the detection efficiency of the target workpiece.

[0111] In one embodiment, the system also includes a line laser camera, and the control device detects the global three-dimensional point cloud with the expected abnormality type as the detection target, and when obtaining the detection result of the target workpiece, it is configured to: control the line laser camera to be directed to the spatial position corresponding to the abnormal position, and receive the supplementary point cloud collected by the line laser camera, detect the supplementary point cloud with the expected abnormality type as the detection target, and obtain the detection result of the target workpiece.

[0112] Among them, the line laser camera can be a three-dimensional vision technology device based on the principle of triangulation. It projects a laser line onto the surface of an object through a laser generator, and uses an image sensor to capture the reflected laser line information, thereby reconstructing the three-dimensional contour of the object.

[0113] The spatial position may represent the position of the abnormal position in the spatial coordinate system.

[0114] The supplementary point cloud may be a point cloud constructed from point cloud data collected by a line laser camera.

[0115] Specifically, the control device can control the line laser camera to face the spatial position corresponding to the abnormal position, and receive the high-quality point cloud collected by the line laser camera. The control device can detect the supplementary point cloud with the expected abnormal type as the detection target to obtain the detection result of the target workpiece.

[0116] In one embodiment, the control device can identify the feature values ​​that need to be measured in the high-quality point cloud, and calculate the feature values ​​(eg, defect length, defect area, defect depth) using methods such as RANSAC.

[0117] In this embodiment, the line laser camera is controlled to be directed toward the spatial position corresponding to the abnormal position, and the supplementary point cloud collected by the line laser camera is received. The expected abnormality type is used as the detection target, and the supplementary point cloud is detected to obtain the detection result of the target workpiece. The supplementary point cloud is detected, which improves the point cloud quality and the accuracy of workpiece detection.

[0118] In one embodiment, Figure 2 As shown, a workpiece detection method is provided, which is applied to Figure 1 The control device 130 in FIG. 1 is used as an example to illustrate, including:

[0119] S202 , based on the relative positional relationship between the local regions, the local images of the local regions are fused to obtain a global plane image of the target workpiece.

[0120] S204, reconstructing a global three-dimensional point cloud of the target workpiece according to the image acquisition posture of each image acquisition device and each local image.

[0121] S206, determining an expected abnormality type that matches the abnormal position in the global plane image.

[0122] S208, taking the expected abnormality type as the detection target, detecting the global three-dimensional point cloud to obtain the detection result of the target workpiece.

[0123] In the above workpiece detection method, based on the relative position relationship between each local area, the local images of each local area are fused to obtain the global plane image of the target workpiece, and the global three-dimensional point cloud of the target workpiece is reconstructed according to the image acquisition posture of each image acquisition device and each local image, and the expected abnormal type matching the abnormal position in the global plane image is determined, and the expected abnormal type is quickly determined in the global plane image with lower data complexity. Furthermore, the global three-dimensional point cloud is detected with the expected abnormal type as the detection target to obtain the detection result of the target workpiece without traversing the global three-dimensional point cloud, which greatly reduces the processing time of the global three-dimensional point cloud and improves the detection efficiency of the target workpiece.

[0124] In one embodiment, the workpiece detection method further includes: for each local area, extracting depth information of the local area according to image features of each local image corresponding to the local area; superimposing the depth information of each local area onto the global plane image to obtain a 2.5D image of the target workpiece; performing anomaly analysis on the global plane image and the 2.5D image to determine the abnormal position in the global plane image.

[0125] In one embodiment, S206 includes acquiring a standard plane image of a standard workpiece; determining an abnormal position in the global plane image according to a texture feature matching result between the global plane image and the standard plane image; and determining an expected abnormal type matching the abnormal position.

[0126] In one embodiment, S206 includes obtaining reference abnormality types configured for each of multiple image regions in the standard plane image; in each image region, screening a target region to which the abnormal position belongs; and determining the reference abnormality type of the target region as an expected abnormality type matching the abnormal position.

[0127] In one embodiment, S208 includes obtaining a standard point cloud of the target workpiece; extracting feature points from the global three-dimensional point cloud and the standard point cloud respectively to obtain three-dimensional coordinate points of the global three-dimensional point cloud and standard coordinate points of the standard point cloud; performing point cloud registration on the global three-dimensional point cloud and the standard point cloud based on the relative position and posture matching of the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points to obtain a difference point cloud of the global three-dimensional point cloud relative to the standard point cloud; taking the expected abnormality type as the detection target, the difference point cloud is detected to obtain the detection result of the target workpiece.

[0128] In one embodiment, S206 includes controlling the line laser camera to face the spatial position corresponding to the abnormal position, and receiving the supplementary point cloud collected by the line laser camera; taking the expected abnormality type as the detection target, detecting the supplementary point cloud, and obtaining the detection result of the target workpiece.

[0129] In one embodiment, Figure 3 As shown in the figure, a surface inspection method for medium and large size stamping parts is provided, and the method is applied to Figure 4 The structural diagram of a surface inspection system for medium and large-sized stamping parts is shown as follows:

[0130] S301 , based on the relative positional relationship between the local areas, the local images of the local areas are fused to obtain a global plane image of the target workpiece.

[0131] The target workpiece may be a stamping part. For example, Figure 4 As shown, the surface inspection system for medium and large-sized stamping parts includes: a conveyor belt 1, a tunnel vision system 2, a control device 3, an electrical control device 4, a 3D structured light camera 5, a photometric stereo camera array 6, and a laser line scan camera 7.

[0132] The stamping parts to be inspected are placed on a conveyor belt 1 , and the device 1 is controlled by an electrical control device 4 and performs signal connection with the preceding stamping processing device.

[0133] The moving speed of the conveyor belt 1 is calculated based on the size of the stamped part being measured and the production cycle of a single workpiece of the preceding stamping equipment.

[0134] After the 3D structured light camera 5 is installed, the entire structured light camera is calibrated to the same coordinate system through the feature calibration component, so that the multi-view local point clouds collected by the 3D structured light camera 5 can be spliced ​​into the overall surface point cloud of the stamping part being tested.

[0135] The control device 3 pre-sets some positioning features, such as round holes or grooves, according to the style of the stamping part being detected. At the same time, a set of template point clouds of stamping parts of the same style are pre-made, and the template point clouds and the collected overall point clouds of the current stamping part surface are respectively extracted using RANSAC and other methods to fit the feature point coordinates.

[0136] After the control device 3 extracts two sets of feature point coordinates of the template point cloud and the current stamping part respectively, it uses methods such as SVD to solve the conversion relationship between the two sets of point clouds, that is, the relative position and posture of the current stamping part and the template stamping part.

[0137] The control device 3 aligns the complete surface point cloud of the current stamping part with the template point cloud according to the relative position and posture of the current stamping part and the template stamping part. The two sets of point clouds are compared to identify whether there are redundant surface point clouds in the punching area of ​​the current stamping part, and if so, it is determined that there is a missing hole defect at this position.

[0138] The photometric stereo camera 6 is composed of 4 or more independent light sources arranged at multiple angles around a 2D array camera. Multiple light sources are illuminated in time, and only one light source works at a time. At the same time, the central 2D array camera collects a local image of the stamping part being tested when each light source works.

[0139] Since the light source is irradiated on the corresponding local surface in time-sharing, it is easily disturbed by the ambient light during the time-sharing image acquisition process. Therefore, 5 shading plates are arranged outside the photometric stereo camera 6 to form a darkroom structure.

[0140] After the photometric stereo camera 6 collects 2D images corresponding to the number of light sources, it calculates the global illumination map and 2.5D image, including the curvature map and the normal vector map, through the imaging algorithm; at the same time, based on the relationship between the pre-calibrated light source and the 2D camera, the local 3D point cloud of the surface is reconstructed; there is a mapping relationship between the pixels in the 2.5D image and the coordinate points in the 3D point cloud.

[0141] The control device 3 identifies and locates the appearance surface area and functional surface area of ​​the stamping part according to the current stamping part point cloud after the aforementioned process and the template point cloud registration, and accurately locates the bending area in the appearance surface. After the stamping part surface is divided into different areas, different defect types are detected in different areas:

[0142] a) For functional surfaces, detect cracks, wrinkles, deformation and other defects that affect the function; b) For appearance surfaces, in addition to the defects to be inspected on the functional surfaces, it is also necessary to detect defects that affect the appearance, such as necking, concave surfaces, bumps and bumps, scratches, etc.; c) For the bending area of ​​the appearance surface, in addition to the defects to be inspected on the entire appearance surface, it is also necessary to detect slip lines.

[0143] The control device 3 uses the 2D global illumination map and the 2.5D curvature map and normal vector map to filter out all suspected defect locations through a 2D deep learning algorithm; separates the 3D point clouds at the locations of various defects, calculates the point cloud morphology of the local area, and determines whether the suspected defects are actual defects;

[0144] The control device 3 outputs the corresponding position and type of the detected defect in the digital model of the stamping part based on the current stamping part position and posture obtained by the above calculation and in conjunction with the pre-calibrated photometric stereo camera external parameters.

[0145] The laser line scan camera 7 continuously scans the laser line across the surface of the stamping part by relative movement with the stamping part to obtain high-quality point clouds of key areas of the stamping part to be measured; the eigenvalues ​​to be measured are identified in the obtained point clouds, and the eigenvalues ​​are calculated using methods such as RANSAC.

[0146] It should be noted that 1. 3D structured light camera 5 can represent multiple 3D structured light cameras installed in parallel, and the complete point cloud of the stamping part under test is obtained after splicing. It can be replaced by a single large-viewing field structured light camera or a single or multiple 2D array cameras to directly obtain the complete point cloud or image for position and posture calculation.

[0147] 2. The photometric stereo camera array is a plurality of photometric stereo cameras, which can be replaced by a multi-angle light source array and a multi-angle 2D camera array.

[0148] 3. The relative positions of the 3D structured light camera, the photometric stereo camera array and the laser line scan camera 7 can be interchanged.

[0149] 4. The 3D structured light camera, photometric stereo camera array and laser line scan camera 7 all include multiple imaging units, which can be replaced by corresponding imaging modules mounted on the end of the robotic arm and moved point by point for imaging acquisition.

[0150] 5. After the data collected by the photometric stereo camera array is calculated, the imaging results obtained include but are not limited to single illumination map, global illumination map, curvature map, normal vector map, depth map, and 3D point cloud.

[0151] 6. The secondary judgment process can be replaced by a multimodal deep learning detection model.

[0152] 7. Equipment 1 transmits the stamped parts to be tested to each testing position of equipment 2, which can be replaced by other motion mechanisms to drive the stamped parts to move, such as multiple robotic arms with movable ground rails or motion modules.

[0153] S302: reconstructing a global three-dimensional point cloud of the target workpiece according to the image acquisition posture of each image acquisition device and each local image.

[0154] S303, obtaining a standard point cloud of the target workpiece.

[0155] S304, extracting feature points from the global three-dimensional point cloud and the standard point cloud respectively, to obtain three-dimensional coordinate points of the global three-dimensional point cloud and standard coordinate points of the standard point cloud.

[0156] S305, based on the relative position and posture of the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, the global three-dimensional point cloud and the standard point cloud are registered to obtain a difference point cloud between the global three-dimensional point cloud and the standard point cloud.

[0157] S306 , for each local area, extracting depth information of the local area according to image features of each local image corresponding to the local area.

[0158] S307 , superimposing the depth information of each local area onto the global plane image to obtain a 2.5D image of the target workpiece.

[0159] S308, performing anomaly analysis on the global plane image and the 2.5D image to determine the abnormal position in the global plane image.

[0160] S309, acquiring a standard plane image of the standard workpiece.

[0161] S310, determining an abnormal position in the global plane image according to a texture feature matching result between the global plane image and the standard plane image.

[0162] S311, obtaining reference anomaly types respectively configured for a plurality of image regions in a standard plane image.

[0163] S312, in each image region, screening the target region to which the abnormal position belongs.

[0164] S313, determining the reference anomaly type of the target area as the expected anomaly type matching the anomaly position.

[0165] S314, taking the expected abnormal type as the detection target, the difference point cloud is detected to obtain the detection result of the target workpiece.

[0166] S315, controlling the line laser camera to face the spatial position corresponding to the abnormal position, and receiving the supplementary point cloud collected by the line laser camera.

[0167] S316, taking the expected abnormality type as the detection target, detecting the supplementary point cloud to obtain the detection result of the target workpiece.

[0168] For example, Figure 5 The stamping parts inspection flow chart shown includes: For ease of description, refer to Figure 4 The conveyor belt 1, the tunnel vision system 2, the control device 3, the electrical control device 4, the 3D structured light camera 5, the photometric stereo camera array 6, and the laser line scan camera 7 are respectively referred to as device 1, device 2, device 3, device 4, device 5, device 6, and device 7.

[0169] From the beginning of the test, the stamped part to be tested is placed at the beginning of device 1. After the stamped part to be tested reaches device 5 and pauses, the structured light camera takes pictures and generates multiple local point clouds during the pause period. The point cloud is sent to device 3 for splicing and compared with the template point cloud to calculate the current position and posture of the stamped part. Device 3 identifies whether there is a missing punch hole and records the recognition result.

[0170] After the pause time is over, the stamping part moves to the first workpiece position of device 6, and multiple photometric stereo cameras of device 6 are placed at multiple consecutive workpiece positions. Pause at the first workpiece position for a fixed time, during which several photometric stereo cameras corresponding to the workpiece position collect segmented illumination images and send them to device 3. The built-in algorithm of device 3 calculates and images several groups of illumination images respectively to obtain 2D global illumination map (reflecting texture information), 2.5D curvature map and normal vector map (reflecting relative surface fluctuations and morphological differences), and 3D point cloud (reflecting three-dimensional absolute information of the surface). Search for suspected defect positions in the 2D global illumination map, 2.5D curvature map and normal vector map, segment the 3D point cloud of the suspected defect position and perform secondary defect determination. Determine whether the current workpiece position is the last workpiece position of device 6. If so, proceed to the next step; otherwise, repeat the above process until the last workpiece position of device 6 is reached and the cycle ends. Device 3 records the determination results of all workpiece positions of device 6.

[0171] When the stamping part to be tested is being transferred from the last workpiece position of device 6, device 7 scans the local key area of ​​the stamping part and sends the point cloud data to device 3. Device 3 calculates the surface characteristic parameters of the stamping part, compares them with the standard parameter values ​​and records the results. Device 3 determines the status of the current stamping part based on the test results recorded in the entire process mentioned above, and displays the type, quantity and location of the defects. At this point, the entire inspection process is completed.

[0172] In this embodiment, based on the relative positional relationship between each local area, the local images of each local area are fused to obtain the global plane image of the target workpiece, and the global three-dimensional point cloud of the target workpiece is reconstructed according to the image acquisition posture of each image acquisition device and each local image, and the expected abnormal type matching the abnormal position in the global plane image is determined, and the expected abnormal type is quickly determined in the global plane image with lower data complexity. Furthermore, the global three-dimensional point cloud is detected with the expected abnormal type as the detection target to obtain the detection result of the target workpiece without traversing the global three-dimensional point cloud, which greatly reduces the processing time of the global three-dimensional point cloud and improves the detection efficiency of the target workpiece.

[0173] The above method can also bring the following beneficial effects: The current mainstream manual inspection method in the industry relies on the experience and sensory sensitivity of quality inspectors, which is highly subjective. Different quality inspectors may have different judgment criteria for the appearance defects of the same stamping part, resulting in the inability to standardize the inspection process. Compared with the manual inspection method, an automatic inspection process is realized, and no human participation is required after deployment, ensuring that the inspection standards for stamping parts are consistent, and eliminating the inspection deviation caused by human subjective factors. Since skilled and highly skilled quality inspectors need to undergo long-term training and practice, and in order to meet the rhythm requirements of online inspection, multiple quality inspectors are required to work together. This embodiment replaces on-site quality inspectors, reduces the workload of personnel, and greatly saves labor costs. For feature items that require precise measurement of dimensions, such as the aperture of punching, the accuracy of manual inspection is low and cannot meet the actual accuracy requirements of the production process. The measurement accuracy of this embodiment is high and can meet the needs of the scene.

[0174] The existing 2D visual inspection method has a fast inspection speed, but without depth information, it is difficult to ensure that the undulating defects or surface defects on the surface of the stamping parts can be stably detected, such as convex and concave defects. At the same time, 2D visual inspection is greatly affected by ambient lighting and the shape of the stamping parts being tested, and its defect detection stability and compatibility with different types of stamping parts are limited. Compared with the existing 2D visual inspection technology, this embodiment additionally uses 2.5D images and 3D point clouds as data sources to obtain the depth information of the stamping parts being tested, so that the system of this embodiment has very effective detection means for the undulating defects and surface defects on the surface of the stamping parts, ensuring the detection rate and false detection rate. In addition, the system uses a photometric stereo camera to output a global illumination map to obtain the surface texture information of the stamping parts being tested. Since the global illumination map is calculated from multiple 2D images obtained from the same area under different angles of light, it is insensitive to the environment and the surface shape of the stamping parts. In addition, this embodiment designs a darkroom for the photometric stereo camera, which further ensures the quality and stability of the global illumination map and achieves a high detection accuracy for various stamping parts.

[0175] The existing 3D visual inspection method can accurately obtain the three-dimensional information of the surface of the stamped parts to be tested, but due to the hardware level of the structured light camera, the point cloud resolution of a single structured light camera is difficult to improve. Since the scale of the defects to be inspected in stamped parts is small, the coverage field of a single structured light camera is also correspondingly small to ensure the representation degree of the features of the point cloud. Therefore, the 3D visual inspection technology of medium and large-sized stamped parts uses a large number of structured light cameras for inspection. In order not to miss the surface area of ​​the stamped parts, the 3D inspection technology needs to reserve a certain area of ​​common field of view at the edge of the field of view of two adjacent 3D cameras. Since multiple structured light cameras with a common field of view project active light sources respectively, they will interfere with each other if they work at the same time. Therefore, 3D visual inspection can only select adjacent structured light cameras to work in time-sharing at the same workpiece stop position, or work in different areas at the positions where multiple workpieces stop. Limited by the high beat requirements of online quality inspection of stamped parts and the relatively long single-frame imaging time of structured light cameras, the time-sharing working method at the same position cannot meet the online inspection requirements. Therefore, 3D visual inspection technology often uses imaging at multiple workpiece stop positions, which makes this technology more expensive and occupies a large area. Compared with 3D visual inspection technology, the hardware cost of this embodiment is lower, and the tunnel-type system form occupies less space. Because all suspected defects are located in the image domain first, and then the point cloud of the corresponding area is judged twice. This means that less point cloud data needs to be processed, and point cloud computing is slower than image processing. Therefore, the overall processing speed is faster.

[0176] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0177] Based on the same inventive concept, the embodiment of the present application also provides a workpiece detection device for implementing the workpiece detection method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more workpiece detection device embodiments provided below can refer to the limitations on the workpiece detection method above, and will not be repeated here.

[0178] In one embodiment, Figure 6As shown, a workpiece detection device is provided, including: a fusion module 601, a reconstruction module 602, a type determination module 603 and a detection module 604, wherein:

[0179] A fusion module 601 is used to fuse the local images of each local area based on the relative position relationship between the local areas to obtain a global plane image of the target workpiece;

[0180] A reconstruction module 602 is used to reconstruct a global three-dimensional point cloud of the target workpiece according to the image acquisition postures of each image acquisition device and each local image;

[0181] A type determination module 603 is used to determine an expected anomaly type that matches the anomaly position in the global planar image;

[0182] The detection module 604 is used to detect the global three-dimensional point cloud with the expected abnormal type as the detection target to obtain the detection result of the target workpiece.

[0183] In one of the embodiments, the workpiece detection device also includes: an abnormality analysis module, which is used to extract the depth information of each local area according to the image features of each local image corresponding to the local area; superimpose the depth information of each local area on the global plane image to obtain a 2.5D image of the target workpiece; perform abnormality analysis on the global plane image and the 2.5D image to determine the abnormal position in the global plane image.

[0184] In one of the embodiments, the type determination module 603 is further used to obtain a standard plane image of a standard workpiece; determine the abnormal position in the global plane image based on the texture feature matching result between the global plane image and the standard plane image; and determine the expected abnormal type matching the abnormal position.

[0185] In one of the embodiments, the type determination module 603 is further used to obtain reference abnormality types configured for each of multiple image regions in the standard plane image; in each image region, screen the target region to which the abnormal position belongs; and determine the reference abnormality type of the target region as the expected abnormality type that matches the abnormal position.

[0186] In one of the embodiments, the detection module 604 is also used to extract feature points from the global three-dimensional point cloud and the standard point cloud respectively, to obtain the three-dimensional coordinate points of the global three-dimensional point cloud and the standard coordinate points of the standard point cloud; based on the relative position and posture matching of the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, the global three-dimensional point cloud and the standard point cloud are aligned to obtain the difference point cloud of the global three-dimensional point cloud relative to the standard point cloud; with the expected abnormality type as the detection target, the difference point cloud is detected to obtain the detection result of the target workpiece.

[0187] In one of the embodiments, the type determination module 603 is also used to control the line laser camera to face the spatial position corresponding to the abnormal position, and receive the supplementary point cloud collected by the line laser camera; the supplementary point cloud is detected with the expected abnormal type as the detection target to obtain the detection result of the target workpiece.

[0188] Each module in the above workpiece detection device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0189] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a workpiece detection method is implemented.

[0190] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0191] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method steps are implemented when the processor executes the computer program.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method steps are implemented.

[0193] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above method steps when executed by a processor.

[0194] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0195] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A workpiece detection system, characterized in that: The system comprises: A conveying device is used to convey the target workpiece to the inspection station; the inspection station is provided with a plurality of image acquisition devices; and the image acquisition postures of any two of the image acquisition devices are different; The image acquisition device is used to acquire an image of a local area of ​​the target workpiece to obtain a local image of the local area; Control equipment for: Based on the relative positional relationship between the local areas, the local images of the local areas are fused to obtain a global plane image of the target workpiece; Reconstructing a global three-dimensional point cloud of the target workpiece according to the image acquisition postures of the image acquisition devices and the local images; determining an expected anomaly type that matches the anomaly location in the global planar image; The expected abnormality type is used as a detection target, the global three-dimensional point cloud is detected, and a detection result of the target workpiece is obtained.

2. The system according to claim 1, characterized in that: The image acquisition device is specifically used to acquire images of a local area of ​​the target workpiece under different lighting conditions to obtain respective local images of the local area under each lighting condition; The control device is also used for: For each of the local areas, extracting depth information of the local area according to image features of each of the local images corresponding to the local area; Superimposing the depth information of each of the local areas onto the global plane image to obtain a 2.5D image of the target workpiece; Anomaly analysis is performed on the global planar image and the 2.5D image to determine an abnormal position in the global planar image.

3. The system according to claim 1, characterized in that When the control device determines the expected abnormality type matching the abnormal position in the global planar image, the control device is configured to: Acquire a standard plane image of a standard workpiece; Determining an abnormal position in the global plane image according to a texture feature matching result between the global plane image and the standard plane image; An expected anomaly type matching the anomaly location is determined.

4. The system according to claim 3, characterized in that When the control device determines the expected abnormality type matching the abnormal location, the control device is configured to: Acquire reference anomaly types respectively configured for a plurality of image regions in the standard plane image; In each of the image regions, screening a target region to which the abnormal position belongs; The reference anomaly type of the target area is determined as the expected anomaly type matching the anomaly position.

5. The system according to claim 1, characterized in that When the control device detects the global three-dimensional point cloud with the expected abnormality type as the detection target and obtains the detection result of the target workpiece, the control device is configured as follows: Acquire a standard point cloud of the target workpiece; Extracting feature points from the global three-dimensional point cloud and the standard point cloud respectively to obtain three-dimensional coordinate points of the global three-dimensional point cloud and standard coordinate points of the standard point cloud; Based on the relative position and posture of the coordinate transformation relationship between the three-dimensional coordinate point and the standard coordinate point, the global three-dimensional point cloud and the standard point cloud are registered to obtain a difference point cloud of the global three-dimensional point cloud relative to the standard point cloud; The expected abnormality type is used as a detection target, and the difference point cloud is detected to obtain a detection result of the target workpiece.

6. The system according to claim 1, characterized in that The system also includes a line laser camera; When the control device detects the global three-dimensional point cloud with the expected abnormality type as the detection target and obtains the detection result of the target workpiece, the control device is configured as follows: Based on a preset orientation of the line laser camera, receiving a supplementary point cloud collected by the line laser camera; The expected abnormality type is used as a detection target, the supplementary point cloud is detected, and a detection result of the target workpiece is obtained.

7. A workpiece detection method, characterized in that: The method comprises: Based on the relative positional relationship between the local areas, the local images of the local areas are fused to obtain a global plane image of the target workpiece; Reconstructing a global three-dimensional point cloud of the target workpiece according to the image acquisition postures of the image acquisition devices and the local images; determining an expected anomaly type that matches the anomaly location in the global planar image; The expected abnormality type is used as a detection target, the global three-dimensional point cloud is detected, and a detection result of the target workpiece is obtained.

8. A workpiece detection device, characterized in that: The device comprises: A fusion module, used for fusing the local images of the local areas based on the relative position relationship between the local areas to obtain a global plane image of the target workpiece; A reconstruction module, used to reconstruct a global three-dimensional point cloud of the target workpiece according to the image acquisition posture of each image acquisition device and each local image; a type determination module, configured to determine an expected anomaly type that matches an anomaly position in the global planar image; The detection module is used to detect the global three-dimensional point cloud with the expected abnormality type as the detection target to obtain the detection result of the target workpiece.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images