Workpiece inspection system, method, apparatus, computer device and storage medium

Abstract

The application relates to a workpiece detection system, method, device, computer equipment, storage medium and computer program product. The system comprises: a configuration conveying device configured to convey a target workpiece to a detection station; an image acquisition device configured to acquire images of local regions of the target workpiece; a control device configured to fuse the local images of the local regions based on the relative positional relationship between the local regions to obtain a global planar image of the target workpiece; and the control device is further configured to reconstruct a global three-dimensional point cloud of the target workpiece. The control device is further configured to determine an expected abnormal type matched with an abnormal position in the global planar image, detect the global three-dimensional point cloud based on the expected abnormal type as a detection target, and obtain a detection result of the target workpiece, thereby improving the detection efficiency of the target workpiece without traversing the global three-dimensional point cloud.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a workpiece inspection system, method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] In the automotive and auto parts manufacturing industry, workpieces, as structural components and appearance parts, have a significant impact on the overall performance and appearance quality of products. Therefore, the need for defect detection of workpiece surface appearance is both urgent and important.

[0003] In traditional technology, a 3D camera is used to scan the workpiece to obtain point cloud data. Based on the point cloud data, defect detection is performed on the workpiece. Although this can meet the defect detection requirements, it still suffers from low detection efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a workpiece inspection method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve inspection efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a workpiece inspection system. The method includes:

[0006] A conveying device is used to convey the target workpiece to the inspection station; the inspection station is equipped with multiple image acquisition devices; the image acquisition poses of any two of the image acquisition devices are different.

[0007] The image acquisition device is used to acquire images of a local area of ​​the target workpiece to obtain a local image of the local area;

[0008] A control device is configured to: fuse local images of each local region based on the relative positional relationship between the local regions to obtain a global planar image of the target workpiece; reconstruct a global three-dimensional point cloud of the target workpiece based on the image acquisition pose of each image acquisition device and the local images; determine the expected anomaly type that matches the abnormal position in the global planar image; and detect the global three-dimensional point cloud using the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0009] In one embodiment, the image acquisition device is specifically used to acquire images of a local area of ​​the target workpiece under different lighting conditions, obtaining local images of the local area under each lighting condition; the control device is further used to: extract depth information of each local area based on the image features of each local image corresponding to the local area; superimpose the depth information of each local area onto the global planar image to obtain a 2.5D image of the target workpiece; and perform anomaly analysis on the global planar image and the 2.5D image to determine the abnormal locations in the global planar image.

[0010] In one embodiment, when the control device determines an expected anomaly type that matches an anomaly location in the global planar image, it is configured to: acquire a standard planar image of a standard workpiece; determine an anomaly location in the global planar image based on a texture feature matching result between the global planar image and the standard planar image; and determine an expected anomaly type that matches the anomaly location.

[0011] In one embodiment, when the control device determines the expected anomaly type that matches the anomaly location, it is configured to: acquire reference anomaly types configured for each of multiple image regions in the standard planar image; filter the target region to which the anomaly location belongs in each of the image regions; and determine the reference anomaly type of the target region as the expected anomaly type that matches the anomaly location.

[0012] In one embodiment, when the control device detects the global 3D point cloud with the expected anomaly type as the detection target and obtains the detection result of the target workpiece, it is configured to: acquire a standard point cloud of the target workpiece; extract feature points from the global 3D point cloud and the standard point cloud respectively to obtain the 3D coordinate points of the global 3D point cloud and the standard coordinate points of the standard point cloud; perform point cloud registration on the global 3D point cloud and the standard point cloud based on the relative position and pose matched by the coordinate transformation relationship between the 3D coordinate points and the standard coordinate points to obtain the difference point cloud of the global 3D point cloud relative to the standard point cloud; and detect the difference point cloud with the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0013] In one embodiment, the system further includes a line laser camera; when the control device detects the global three-dimensional point cloud with the expected anomaly type as the detection target and obtains the detection result of the target workpiece, it is configured to: control the line laser camera to face the spatial position corresponding to the anomaly position, and receive the supplementary point cloud collected by the line laser camera; detect the supplementary point cloud with the expected anomaly type as the detection target, and obtain the detection result of the target workpiece.

[0014] Secondly, this application provides a workpiece inspection method. The method includes:

[0015] Based on the relative positional relationship between the local regions, the local images of each local region are fused to obtain a global planar image of the target workpiece; according to the image acquisition pose of each image acquisition device and the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; the expected anomaly type matching the abnormal position in the global planar image is determined; the global three-dimensional point cloud is detected using the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0016] Thirdly, this application also provides a workpiece inspection device. The device includes:

[0017] The fusion module is used to fuse the local images of each local region based on the relative positional relationship between the local regions to obtain a global planar image of the target workpiece; the reconstruction module is used to reconstruct a global three-dimensional point cloud of the target workpiece according to the image acquisition pose of each image acquisition device and the local images; the type determination module is used to determine the expected anomaly type that matches the abnormal position in the global planar image; and the detection module is used to detect the global three-dimensional point cloud with the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0018] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0019] Based on the relative positional relationship between the local regions, the local images of each local region are fused to obtain a global planar image of the target workpiece; according to the image acquisition pose of each image acquisition device and the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; the expected anomaly type matching the abnormal position in the global planar image is determined; the global three-dimensional point cloud is detected using the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0020] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0021] Based on the relative positional relationship between the local regions, the local images of each local region are fused to obtain a global planar image of the target workpiece; according to the image acquisition pose of each image acquisition device and the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; the expected anomaly type matching the abnormal position in the global planar image is determined; the global three-dimensional point cloud is detected using the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0022] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0023] Based on the relative positional relationship between the local regions, the local images of each local region are fused to obtain a global planar image of the target workpiece; according to the image acquisition pose of each image acquisition device and the local images, a global three-dimensional point cloud of the target workpiece is reconstructed; the expected anomaly type matching the abnormal position in the global planar image is determined; the global three-dimensional point cloud is detected using the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0024] The aforementioned workpiece inspection system, method, apparatus, computer equipment, storage medium, and computer program products include a conveying device for transporting the target workpiece to the inspection station. The inspection station is equipped with multiple image acquisition devices, each with a different image acquisition pose. These image acquisition devices are configured to acquire images of local areas of the target workpiece, obtaining local images of those areas. This multi-angle image acquisition improves the accuracy of image acquisition. A control device is configured to fuse the local images of each local area based on their relative positional relationships, obtaining a global planar image of the target workpiece. The control device also reconstructs a global 3D point cloud of the target workpiece based on the image acquisition poses of each image acquisition device and the local images. Furthermore, the control device is used to determine the expected anomaly type matching the anomaly location in the global planar image, enabling rapid determination of the expected anomaly type in the less complex global planar image. Furthermore, the configuration control equipment is also used to detect the global 3D point cloud with the expected anomaly type as the detection target, and obtain the detection result of the target workpiece. It does not require traversing the global 3D point cloud, which greatly reduces the processing time of the global 3D point cloud and improves the detection efficiency of the target workpiece. Attached Figure Description

[0025] Figure 1 This is a diagram illustrating the application environment of a workpiece inspection method in one embodiment;

[0026] Figure 2 This is a flowchart illustrating a workpiece inspection method in one embodiment;

[0027] Figure 3 This is a flowchart illustrating a method for surface inspection of medium and large-sized stamped parts in one embodiment;

[0028] Figure 4 This is a schematic diagram of the surface inspection system for medium and large-sized stamped parts in one embodiment;

[0029] Figure 5 Here is a flowchart of the stamping part inspection process in one embodiment;

[0030] Figure 6 This is a structural block diagram of a workpiece inspection device in one embodiment;

[0031] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] In one embodiment, such as Figure 1 As shown in the figure, this application provides a workpiece inspection system, including: a conveying device 110, an image acquisition device 120, and a control device 130.

[0034] Among them, the conveying device 110 is used to convey the target workpiece to the inspection station.

[0035] The target workpiece can be a specific part in the manufacturing industry. The target workpiece can be a workpiece that needs to be inspected after processing. The target workpiece can also be any part that needs to be inspected. Depending on the processing technology, the target workpiece can include: stamped parts, welded parts, cut parts, etc.

[0036] Conveying equipment can be a device used to carry a target workpiece. For example, conveying equipment can be a conveyor tray or conveyor belt carrying a vehicle to be tested. The connection between the conveying equipment and the target workpiece can be varied; for example, the conveying equipment can be fixedly connected to the target workpiece. Or, for example, the conveying equipment can support the target workpiece.

[0037] An inspection station can refer to a station in a workpiece inspection production line used for inspecting workpieces. Various inspection-related equipment is arranged around the inspection station, such as vision cameras, grayscale cameras, and proximity switches. Furthermore, to improve optical recognition accuracy, a light shield can also be installed at the inspection station.

[0038] Specifically, the conveying equipment is used to transfer the target workpiece to the inspection station.

[0039] In one embodiment, the conveying device is also used to send a workpiece arrival signal to the control device when the workpiece arrives at the inspection station.

[0040] In one embodiment, the conveying device can also be used to send an abnormal arrival signal when the workpiece has not reached the inspection station. The conveying device can send the abnormal arrival signal to the control device. The abnormal arrival signal can be used to indicate that the workpiece has not reached the inspection station or that the conveying device itself is malfunctioning.

[0041] In one embodiment, the conveying device can also be used to deliver the target workpiece to the inspection station and to deliver the target workpiece away from the inspection station according to the working rhythm of the inspection station.

[0042] In one embodiment, the conveying device can determine the working cycle of the inspection station based on the working cycle of the previous station and the length of the target workpiece. The working cycle of the inspection station can represent the moving time, dwell time, and conveying speed of the conveying device.

[0043] The image acquisition device 120 is used to acquire images of a local area of ​​the target workpiece to obtain a local image of the local area.

[0044] The image acquisition device is connected to the control device, and can be oriented towards the target workpiece to acquire images. Multiple image acquisition devices are set up at the inspection station, and the image acquisition poses of any two image acquisition devices are different.

[0045] Image acquisition equipment can be any device capable of acquiring images. Its main task is to optically image the feature information of a target object (such as a workpiece or other object), convert the optical signals into electrical signals, and then transmit them to a computer or image processing system. Examples of image acquisition equipment include industrial cameras, scanners, vision cameras, and CCD cameras.

[0046] The target workpiece can vary in size. For example, the target workpiece can be a medium-to-large-sized stamped part. Medium-to-large size means that either the length or width of the stamped part when laid flat is greater than or equal to 50cm. The height of the stamped part is generally much less than 50cm.

[0047] A local region of a target workpiece can represent multiple regions divided along the direction of its movement. For example, as the target workpiece moves forward on a conveyor, it can be divided into multiple local regions according to its direction of movement. Understandably, the image acquisition device can be fixed in place, allowing it to capture local images of each of the multiple local regions as the target workpiece moves forward on the conveyor.

[0048] A local image can represent an image captured by an image acquisition device for a local area of ​​a target workpiece.

[0049] Specifically, the image acquisition device is used to acquire images of a local area of ​​a target workpiece, obtaining a local image of that area. The image acquisition device can be an array composed of multiple vision cameras.

[0050] The control device 130 is used to fuse the local images of each local region based on the relative positional relationship between each local region to obtain a global planar image of the target workpiece.

[0051] The control device can be connected to the image acquisition device and the transmission device respectively.

[0052] The control device can be a device with image processing capabilities. Control devices can be, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Specifically, the control device can be an industrial control computer.

[0053] Relative positional relationships can represent the positional relationships between various local areas. A local area can refer to a region further forward or further back. Multiple local areas include forward, intermediate, and backward areas. The concepts of forward and backward can be determined based on the direction of movement of the conveying equipment.

[0054] A global planar image can represent a complete planar image of the target workpiece.

[0055] Specifically, the control device can fuse the local images of each local area based on the relative positional relationship between the local areas to obtain a global planar image of the target workpiece.

[0056] In one embodiment, the control device can determine the fusion order of each local region based on the relative positional relationship between them. The control device can then fuse the local images of each local region according to this fusion order to obtain a global planar image of the target workpiece. For example, the earlier local images may be fused first.

[0057] In one embodiment, the control device can determine the fusion weight of each local region based on the relative positional relationship between them. The control device can then perform weighted fusion of the local regions based on these fusion weights to obtain a global planar image of the target workpiece.

[0058] Among them, the control device 130 is also used to reconstruct the global three-dimensional point cloud of the target workpiece based on the image acquisition pose of each image acquisition device and each local image.

[0059] Image acquisition pose can represent the position and orientation of an image acquisition device in three-dimensional space, typically including translation vectors and rotation matrices. For example, the translation vector and rotation matrix corresponding to a vision camera.

[0060] A global 3D point cloud can refer to a collection of multiple points in 3D space, each point having 3D coordinates (x, y, z) and possibly containing other additional information such as color and intensity values.

[0061] Specifically, the control equipment can reconstruct the global three-dimensional point cloud of the target workpiece based on the image acquisition pose of each image acquisition device and the local images.

[0062] For example, taking an image acquisition device as a camera, the point cloud reconstruction includes: feature point matching, camera pose estimation, triangulation, point cloud generation, global optimization, and dense reconstruction. Specifically, the control device can extract feature points from multiple local images and match the feature points of the target workpiece from different viewpoints. The control device can calculate the camera pose (including rotation and translation) corresponding to each image using known camera parameters and matched feature points. The control device can utilize multi-view geometry principles to triangulate the pixel coordinates of feature points in multiple viewpoints to obtain the initial 3D coordinates of these feature points. The control device can calculate the 3D coordinates of each matched point, forming a set of sparse 3D point cloud data. The control device can use global optimization techniques, employing nonlinear least squares to jointly optimize the camera pose and spatial point positions, ensuring that the error is evenly distributed across all images. The control device can use multi-view stereo (MVS) technology to further increase the density of the point cloud through stereo matching algorithms, thereby generating a more complete global 3D point cloud of the target workpiece.

[0063] The control device 130 is also used to determine the expected anomaly type that matches the anomaly location in the global planar image.

[0064] Anomalies can be represented by their location within a global planar image. The local region corresponding to an anomaly may contain anomalies.

[0065] The expected anomaly type indicates the type of anomaly that typically occurs at that location. For example, functional areas of stamped parts often exhibit anomalies such as cracking, wrinkling, and deformation. Similarly, for appearance areas, it's necessary to inspect for anomalies affecting appearance, such as necking, surface concavity, bumps, and scratches. Furthermore, for bent areas of stamped parts, in addition to all defects to be inspected on the surface, slip lines may also appear.

[0066] Specifically, the control device can perform anomaly identification on the global planar image to determine the location of anomalies. Anomaly identification can be performed using an anomaly identification model. In one embodiment, the control device can use an anomaly identification model to identify the location of anomalies in the global planar image.

[0067] In one embodiment, the control device can also acquire a standard planar image of a standard workpiece. The control device can determine the location of anomalies in the global planar image based on the texture feature matching results between the global planar image and the standard planar image. The control device can then determine the expected anomaly type matching the anomaly location.

[0068] In one embodiment, the control device can acquire reference anomaly types configured for each of multiple image regions in a standard planar image. The control device can then filter the target region to which the anomaly location belongs within each image region. The control device can then determine the reference anomaly type of the target region as the expected anomaly type matching the anomaly location.

[0069] Among them, the control device 130 is also used to detect the global three-dimensional point cloud with the expected anomaly type as the detection target, and obtain the detection result of the target workpiece.

[0070] The target to be detected can be the object of interest in a detection task. Specifically, detection can be the object of interest in an object detection task. Object detection typically includes two key subtasks: object localization and object classification. First, object localization determines the location of the object in the image, and then object classification determines the specific category of the object. The location information of the object can generally be represented by polar coordinates or center point coordinates.

[0071] The detection results can indicate the anomaly type, specific information about the anomaly, and the confidence level of the anomaly type in the target workpiece. The anomaly type indicates the specific type of anomaly found in the target workpiece. For example, a cracked surface or a missing hole. Specific anomaly information includes the number and quantity of anomalies. The confidence level of the anomaly type indicates the detection accuracy corresponding to that anomaly type. For example, the accuracy for a cracked surface is 80%, and the accuracy for a missing hole is 25%. Specific anomaly information can also be detailed information about the anomaly type. For example, the crack area of ​​a cracked surface.

[0072] Specifically, the control equipment can use the expected anomaly type as the target of interest to detect the global 3D point cloud and obtain the detection result of the target workpiece.

[0073] In one embodiment, the detection results include multiple anomaly types and the accuracy of each anomaly type. The control device can then filter for the target anomaly type with the highest accuracy among the anomaly types.

[0074] In one embodiment, the control device can increase the weight of the expected anomaly type and recalculate the accuracy to obtain a new accuracy for each type. The control device can then select the target anomaly type with the highest new accuracy from among the anomaly types.

[0075] Specifically, the control equipment can also segment the global 3D point cloud to obtain a segmented point cloud. The control equipment can then detect the segmented point cloud with the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0076] Specifically, the control equipment can also rescan the target workpiece to obtain a new 3D point cloud. The control equipment can then use the expected anomaly type as the detection target to detect the new 3D point cloud and obtain the detection results for the target workpiece.

[0077] In the aforementioned workpiece inspection system, a conveying device is configured to transport the target workpiece to the inspection station. The inspection station is equipped with multiple image acquisition devices, each with a different image acquisition pose. These image acquisition devices are used to acquire images of local areas of the target workpiece, obtaining local images of those areas. This multi-angle image acquisition improves the accuracy of image acquisition. A control device is configured to fuse the local images of each local area based on their relative positions, obtaining a global planar image of the target workpiece. The control device also reconstructs a global 3D point cloud of the target workpiece based on the image acquisition poses of each image acquisition device and the local images. Furthermore, the control device is used to determine the expected anomaly type matching the anomaly location in the global planar image, quickly identifying the expected anomaly type in the less complex global planar image. Further, the control device is used to detect the global 3D point cloud using the expected anomaly type as the detection target, obtaining the detection result of the target workpiece without traversing the entire 3D point cloud, significantly reducing the processing time and improving the detection efficiency of the target workpiece.

[0078] In one embodiment, the image acquisition device is specifically used to acquire images of a local area of ​​a target workpiece under different lighting conditions, obtaining local images of the local area under each lighting condition. The control device is also used to extract the depth information of each local area based on the image features of each local image corresponding to the local area, and to superimpose the depth information of each local area onto a global planar image to obtain a 2.5D image of the target workpiece. Anomaly analysis is performed on the global planar image and the 2.5D image to determine the abnormal location in the global planar image.

[0079] The lighting conditions refer to the conditions set for the image acquisition devices. For example, multiple light sources can be configured for each image acquisition device. Alternatively, multiple light sources can be configured for each image acquisition device. Each light source has a different orientation angle.

[0080] Specifically, the image acquisition device can acquire images of local areas of the target workpiece at the times when each light source is lit in sequence, and obtain local images of the local areas under each illumination condition.

[0081] Depth information can represent information in an image that indicates the distance or relative position of each pixel to the observer.

[0082] 2.5D images can be a form of image that lies between two-dimensional (2D) and three-dimensional (3D). They retain some three-dimensional features but do not fully represent true three-dimensional space. Such images are usually simulated to achieve a three-dimensional effect by adding depth or perspective effects to a two-dimensional plane.

[0083] 2.5D images carry depth information, specifically including grayscale information, color information, and pseudo-3D information. Grayscale information represents depth information using a grayscale image, where the grayscale value of each pixel represents its depth value. For example, brighter pixels represent closer objects, while darker pixels represent farther objects. Color information represents depth information using colors; typically, red and black represent near points, and blue and dark colors represent far points. Pseudo-3D information represents a scene reconstructed by simulating different viewpoints, thus providing a visual 3D effect.

[0084] Specifically, the control device can extract depth information for each local region based on the image features of its corresponding local images. The control device can then overlay the depth information of each local region onto the global planar image to obtain a 2.5D image of the target workpiece. The control device can then perform anomaly analysis on the global planar image and the 2.5D image to determine the locations of anomalies in the global planar image.

[0085] In one embodiment, the control device can extract grayscale information, color information, and pseudo-3D information from a local image as depth information for the local region.

[0086] In one embodiment, the control device can sequentially overlay the depth information of each local region onto a global planar image. Specifically, the control device can sequentially fuse the depth information of each local region according to its respective overlay weight, and then overlay it onto the global planar image to obtain a 2.5D image of the target workpiece.

[0087] In one embodiment, the control device can use an anomaly analysis model based on 2D and 2.5D deep learning algorithms to perform anomaly analysis on the global planar image and the 2.5D image to determine the location of anomalies in the global planar image.

[0088] In this embodiment, the image acquisition device is specifically configured to acquire images of local areas of the target workpiece under different lighting conditions, obtaining local images of each local area under each lighting condition. The control device is also configured to, for each local area, extract depth information based on the image features of each local image, and superimpose the depth information of each local area onto the global planar image to obtain a 2.5D image of the target workpiece. Anomaly analysis is then performed on the global planar image and the 2.5D image to determine the anomaly locations in the global planar image, thereby improving the detection efficiency of the target workpiece by quickly identifying the anomaly locations in the global planar image.

[0089] In one embodiment, when the control device determines the expected anomaly type that matches the anomaly location in the global planar image, it is configured to: acquire a standard planar image of a standard workpiece, determine the anomaly location in the global planar image based on the texture feature matching result between the global planar image and the standard planar image, and determine the expected anomaly type that matches the anomaly location.

[0090] Among them, the standard workpiece is a workpiece without defects. The standard workpiece and the target workpiece have the same model number.

[0091] A standard planar image can be a global planar image of a standard workpiece.

[0092] Texture feature matching results can represent the matching between the texture features of a global planar image and the texture features of a standard planar image. Texture feature matching results include the location and number of anomalies.

[0093] Specifically, the control equipment can identify the target workpiece by its model to obtain the target workpiece and its model number. The control equipment can also acquire a standard planar image of a standard workpiece of the corresponding model number.

[0094] In one embodiment, the control device can extract texture features from a global planar image to obtain the texture features of the global planar image. The control device can also extract texture features from a standard planar image to obtain the texture features of the standard planar image. Finally, the control device performs texture feature matching between the global planar image and the standard planar image to obtain a texture feature matching result.

[0095] Specifically, the control device can acquire reference anomaly types configured for each of multiple image regions in a standard planar image. Within each image region, the control device can filter for the target region to which the anomaly location belongs. The control device can then determine the reference anomaly type of the target region as the expected anomaly type matching the anomaly location.

[0096] In this embodiment, a standard planar image of a standard workpiece is obtained. Based on the texture feature matching results between the global planar image and the standard planar image, the abnormal location in the global planar image is determined. The expected abnormal type matching the abnormal location is determined. The abnormal location is determined only based on texture features, thereby improving the detection efficiency of the target workpiece.

[0097] In one embodiment, when the control device determines the expected anomaly type that matches the anomaly location, it is configured to: obtain reference anomaly types configured for each of multiple image regions in a standard planar image; filter the target region to which the anomaly location belongs in each image region; and determine the reference anomaly type of the target region as the expected anomaly type that matches the anomaly location.

[0098] The image region can represent the region in the image corresponding to a local area in a standard workpiece.

[0099] A reference anomaly type indicates the type of anomaly corresponding to a specific local area. Reference anomaly types include those that typically occur at that location. For example, functional areas of stamped parts often exhibit anomalies such as cracking, wrinkling, and deformation. Similarly, for appearance areas, it's necessary to inspect for anomalies affecting appearance, such as necking, surface concavity, bumps, and scratches. Furthermore, for bent areas of stamped parts, in addition to all defects to be inspected on the surface, slip lines may also appear.

[0100] The target region can represent an image region in a standard planar image.

[0101] Specifically, the control device can first determine the corresponding location of the anomaly in a standard planar image. The control device can then determine the image region to which the corresponding location belongs. The control device can then designate this image region as the target region to which the anomaly belongs. Finally, the control device can determine the reference anomaly type of the target region as the expected anomaly type that matches the anomaly location.

[0102] For example, the standard planar image can be a standard planar image of a stamped part. The reference anomaly types configured for each image region include reference anomaly types for appearance regions, reference anomaly types for functional regions, and reference anomaly types for bending regions.

[0103] In this embodiment, reference anomaly types are obtained for each of the multiple image regions in a standard planar image. In each image region, the target region to which the anomaly location belongs is filtered, and the reference anomaly type of the target region is determined as the expected anomaly type that matches the anomaly location. The target region to which the anomaly location belongs is filtered, and the expected anomaly type that matches the anomaly location is determined. This reduces the number of items to be detected in the subsequent process, thereby improving the detection efficiency of the target workpiece.

[0104] In one embodiment, when the control device detects the global 3D point cloud with the expected anomaly type as the detection target and obtains the detection result of the target workpiece, it is configured to: acquire the standard point cloud of the target workpiece; extract feature points from the global 3D point cloud and the standard point cloud respectively to obtain the 3D coordinate points of the global 3D point cloud and the standard coordinate points of the standard point cloud; perform point cloud registration on the global 3D point cloud and the standard point cloud based on the relative position and pose matched by the coordinate transformation relationship between the 3D coordinate points and the standard coordinate points to obtain the difference point cloud of the global 3D point cloud relative to the standard point cloud; and detect the difference point cloud with the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0105] Among them, the standard point cloud can be the point cloud of the standard workpiece corresponding to the target workpiece model.

[0106] Three-dimensional coordinate points can represent the coordinates of feature points in a global three-dimensional point cloud. Standard coordinate points can represent the coordinates of feature points in a standard point cloud.

[0107] Difference point cloud can represent the point cloud corresponding to the difference structure of the target workpiece relative to the standard workpiece.

[0108] Specifically, the control equipment can directly acquire the standard point cloud of a standard workpiece. The control equipment can use methods such as RANSAC to extract feature points from both the global 3D point cloud and the standard point cloud, obtaining the 3D coordinates of the global 3D point cloud and the standard coordinates of the standard point cloud. The control equipment can use methods such as SVD to solve for the transformation relationship between the two sets of point clouds, i.e., the relative position and attitude. Based on the position and attitude, the control equipment can perform point cloud registration between the global 3D point cloud and the standard point cloud, obtaining the difference point cloud between the global 3D point cloud and the standard point cloud.

[0109] For example, the control device can also compare two sets of point clouds to identify whether there are extra planar point clouds (difference point clouds) in the current target workpiece. If so, it is determined that there is a missing hole defect at that location.

[0110] In this embodiment, a standard point cloud of the target workpiece is acquired. Feature points are extracted from both the global 3D point cloud and the standard point cloud to obtain the 3D coordinates of the global 3D point cloud and the standard coordinates of the standard point cloud. Based on the coordinate transformation relationship between the 3D coordinates and the standard coordinates, the relative position and pose are matched, and point cloud registration is performed between the global 3D point cloud and the standard point cloud to obtain the difference point cloud between the global 3D point cloud and the standard point cloud. The difference point cloud is detected with the expected anomaly type as the detection target to obtain the detection result of the target workpiece. Detecting the difference point cloud reduces the amount of data that needs to be processed, thereby improving the detection efficiency of the target workpiece.

[0111] In one embodiment, the system further includes a line laser camera. When the control device detects the global three-dimensional point cloud with the expected anomaly type as the detection target and obtains the detection result of the target workpiece, it is configured to: control the line laser camera to face the spatial position corresponding to the anomaly position, and receive the supplementary point cloud collected by the line laser camera, detect the supplementary point cloud with the expected anomaly type as the detection target, and obtain the detection result of the target workpiece.

[0112] Among them, a line laser camera can be a three-dimensional vision technology device based on the principle of triangulation. It projects a laser line onto the surface of an object through a laser generator and uses an image sensor to capture the information of the reflected laser line, thereby reconstructing the three-dimensional contour of the object.

[0113] Spatial location can represent the position of an anomaly in a spatial coordinate system.

[0114] Supplementary point clouds can be point clouds constructed from point cloud data collected by line laser cameras.

[0115] Specifically, the control equipment can direct the line laser camera toward the spatial position corresponding to the abnormal location and receive the high-quality point cloud acquired by the line laser camera. The control equipment can then use the expected type of abnormality as the detection target to detect the supplementary point cloud and obtain the detection result for the target workpiece.

[0116] In one embodiment, the control device can identify the feature values ​​to be measured in a high-quality point cloud and calculate the feature values ​​(e.g., defect length, defect area, defect depth) using methods such as RANSAC.

[0117] In this embodiment, the line laser camera is controlled to face the spatial position corresponding to the abnormal position, and the supplementary point cloud collected by the line laser camera is received. The supplementary point cloud is detected with the expected abnormal type as the detection target to obtain the detection result of the target workpiece. Detecting the supplementary point cloud improves the point cloud quality and the accuracy of workpiece detection.

[0118] In one embodiment, such as Figure 2 As shown, a workpiece inspection method is provided, which is applied to... Figure 1 The following explanation uses the control device 130 as an example, including:

[0119] S202, based on the relative positional relationship between each local region, fuse the local images of each local region to obtain a global planar image of the target workpiece.

[0120] S204. Based on the image acquisition poses of each image acquisition device and the local images, reconstruct the global three-dimensional point cloud of the target workpiece.

[0121] S206, Determine the expected anomaly type that matches the anomaly location in the global planar image.

[0122] S208 uses the expected anomaly type as the detection target to detect the global 3D point cloud and obtain the detection result of the target workpiece.

[0123] In the aforementioned workpiece inspection method, based on the relative positional relationships between local regions, the local images of each region are fused to obtain a global planar image of the target workpiece. Then, based on the image acquisition poses of each image acquisition device and the local images, a global 3D point cloud of the target workpiece is reconstructed. The expected anomaly type matching the anomaly location in the global planar image is determined, allowing for rapid identification of the expected anomaly type in the less complex global planar image. Furthermore, using the expected anomaly type as the detection target, the global 3D point cloud is inspected to obtain the detection result of the target workpiece. This eliminates the need to traverse the entire global 3D point cloud, significantly reducing processing time and improving the detection efficiency of the target workpiece.

[0124] In one embodiment, the workpiece detection method further includes: for each local region, extracting depth information of the local region based on the image features of each local image corresponding to the local region; superimposing the depth information of each local region onto a global planar image to obtain a 2.5D image of the target workpiece; and performing anomaly analysis on the global planar image and the 2.5D image to determine the abnormal locations in the global planar image.

[0125] In one embodiment, S206 includes acquiring a standard planar image of a standard workpiece; determining an anomaly location in the global planar image based on the texture feature matching result between the global planar image and the standard planar image; and determining the expected anomaly type that matches the anomaly location.

[0126] In one embodiment, S206 includes obtaining reference anomaly types configured for each of multiple image regions in a standard planar image; filtering the target region to which the anomaly location belongs in each image region; and determining the reference anomaly type of the target region as the expected anomaly type that matches the anomaly location.

[0127] In one embodiment, S208 includes acquiring a standard point cloud of the target workpiece; extracting feature points from the global 3D point cloud and the standard point cloud respectively to obtain the 3D coordinate points of the global 3D point cloud and the standard coordinate points of the standard point cloud; performing point cloud registration on the global 3D point cloud and the standard point cloud based on the relative position and pose matching the coordinate transformation relationship between the 3D coordinate points and the standard coordinate points to obtain the difference point cloud of the global 3D point cloud relative to the standard point cloud; and detecting the difference point cloud with the expected anomaly type as the detection target to obtain the detection result of the target workpiece.

[0128] In one embodiment, S206 includes controlling the orientation of the line laser camera to the spatial position corresponding to the abnormal position, and receiving the supplementary point cloud acquired by the line laser camera; using the expected abnormal type as the detection target, detecting the supplementary point cloud to obtain the detection result of the target workpiece.

[0129] In one embodiment, such as Figure 3 As shown, a surface inspection method for medium and large-sized stamped parts is provided, which can be applied to, for example... Figure 4 The following is a schematic diagram illustrating the structure of a surface inspection system for medium and large-sized stamped parts:

[0130] S301, based on the relative positional relationship between each local region, fuse the local images of each local region to obtain a global planar image of the target workpiece.

[0131] The target workpiece can be a stamped part. For example, such as... Figure 4 As shown, the surface inspection system for medium and large-sized stamped parts includes: 1. Conveyor belt, 2. Tunnel vision system, 3. Control equipment, 4. Electrical control equipment, 5. 3D structured light camera, 6. Photometric stereo camera array, and 7. Laser line scanning camera.

[0132] The stamped parts to be inspected are placed on conveyor belt 1. Equipment 1 is controlled by electrical control equipment 4 and communicates with the preceding stamping equipment.

[0133] The speed of conveyor belt 1 is calculated from the size of the stamped part being measured and the production cycle of a single workpiece of the preceding stamping equipment.

[0134] After the 3D structured light camera 5 is installed, all the structured light cameras are calibrated to the same coordinate system through the feature calibration component, so that the multi-view local point clouds collected by the 3D structured light camera 5 can be stitched together to form the overall surface point cloud of the stamped part under test.

[0135] The control device 3 pre-sets some positioning features, such as round holes or grooves, according to the style of the stamped part being inspected. At the same time, it pre-creates a set of template point clouds of stamped parts with the same style. The template point clouds and the overall point cloud of the current stamped part surface are respectively used to extract corresponding features and fit feature point coordinates using methods such as RANSAC.

[0136] After the control device 3 extracts the coordinates of two sets of feature points of the template point cloud and the current stamping part, it uses methods such as SVD to solve the transformation relationship between the two sets of point clouds, which is the relative position and attitude of the current stamping part and the template stamping part.

[0137] The control device 3 registers the point cloud of the complete surface of the current stamped part with the point cloud of the template based on the relative position and posture of the current stamped part and the template stamped part. By comparing the two sets of point clouds, it identifies whether there are extra planar point clouds in the punching area of ​​the current stamped part. If so, it determines that there is a missing hole defect at that location.

[0138] The photometric stereo camera 6 consists of four or more independent light sources arranged around a 2D area array camera at multiple angles. The multiple light sources provide time-sharing illumination, with only one light source working at a time. Simultaneously, the central 2D area array camera acquires a partial image of the stamped part under test when each light source is working.

[0139] Since the light source illuminates the corresponding local surface in a time-division manner, it is easily affected by ambient light during the time-division image acquisition process. Therefore, five light-shielding plates are set outside the photometric stereo camera 6 to form a darkroom structure.

[0140] After the photometric stereo camera 6 acquires 2D images corresponding to the number of light sources, it calculates the global illumination map and 2.5D image, including curvature map and normal vector map, through imaging algorithm; at the same time, based on the pre-calibrated relationship between the light source and the 2D camera, it reconstructs the local 3D point cloud of the surface; there is a mapping relationship between the pixels in the 2.5D image and the coordinate points in the 3D point cloud.

[0141] Control device 3, based on the aforementioned process and the current stamped part point cloud registered with the template point cloud, identifies and locates the appearance surface area and functional surface area of ​​the stamped part, and accurately locates the bending area within the appearance surface. After dividing the stamped part surface into sections, different defect types are detected in different areas:

[0142] a) For functional surfaces, detect defects that affect function, such as cracks, wrinkles, and deformation; b) For appearance surfaces, in addition to the defects to be inspected on functional surfaces, also detect defects that affect appearance, such as necking, surface concavity, bumps, and scratches; c) For the bending areas of appearance surfaces, in addition to all defects to be inspected on appearance surfaces, also detect slip lines.

[0143] The control device 3 uses a 2D global illumination map and a 2.5D curvature map and normal vector map. Through a 2D deep learning algorithm, it filters out all suspected defect locations. It then separates the 3D point cloud at each defect location, calculates the local point cloud morphology, and makes a secondary determination as to whether the suspected defect is an actual defect.

[0144] Based on the current position and orientation of the stamped part calculated above, and in conjunction with the pre-calibrated extrinsic parameters of the photometric stereo camera, the control device 3 outputs the corresponding position and type of the detected defects in the digital model of the stamped part.

[0145] The laser line scanning camera 7 continuously scans the surface of the stamping part with a laser line by moving relative to the stamping part under test, thereby acquiring a high-quality point cloud of the key area of ​​the stamping part under test; the feature values ​​to be measured are identified in the obtained point cloud, and the feature values ​​are calculated using methods such as RANSAC.

[0146] It should be noted that 1. 3D structured light camera 5 can represent multiple 3D structured light cameras installed in parallel, which are stitched together to obtain a complete point cloud of the stamped part under test. It can be replaced by a single large-field-of-view structured light camera or a single or multiple 2D area array cameras to directly acquire the complete point cloud or image for position and attitude calculation.

[0147] 2. The photometric stereo camera array consists of multiple photometric stereo cameras, which can be replaced by a multi-angle light source array and a multi-angle 2D camera array.

[0148] 3. The relative positions of the 3D structured light camera, the photometric stereo camera array, and the laser line scan camera 7 can be interchanged.

[0149] 4. The 3D structured light camera, photometric stereo camera array, and laser line scan camera 7 each contain multiple imaging units, which can be replaced by corresponding imaging modules mounted on the end of the robotic arm, moving point by point to collect images.

[0150] 5. The imaging results obtained after the data collected by the photometric stereo camera array are processed include, but are not limited to, single illumination map, global illumination map, curvature map, normal vector map, depth map, and 3D point cloud.

[0151] 6. The secondary judgment process can be replaced by a multimodal deep learning detection model.

[0152] 7. Equipment 1 transmits the stamped part to be tested to various testing positions of Equipment 2. It can be replaced by other motion mechanisms to drive the stamped part to move, such as multiple robotic arms working with movable ground rails or motion modules.

[0153] S302, based on the image acquisition poses of each image acquisition device and each local image, reconstructs the global three-dimensional point cloud of the target workpiece.

[0154] S303, Obtain the standard point cloud of the target workpiece.

[0155] S304 extracts feature points from the global 3D point cloud and the standard point cloud respectively, to obtain the 3D coordinates of the global 3D point cloud and the standard coordinates of the standard point cloud.

[0156] S305, based on the relative position and pose matching of the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, performs point cloud registration on the global three-dimensional point cloud and the standard point cloud, and obtains the difference point cloud between the global three-dimensional point cloud and the standard point cloud.

[0157] S306, For each local region, extract the depth information of the local region based on the image features of each local image corresponding to the local region.

[0158] S307: The depth information of each local area is superimposed onto the global planar image to obtain a 2.5D image of the target workpiece.

[0159] S308 performs anomaly analysis on the global planar image and the 2.5D image to determine the location of anomalies in the global planar image.

[0160] S309, Obtain a standard planar image of a standard workpiece.

[0161] S310, determine the abnormal location in the global planar image based on the texture feature matching result between the global planar image and the standard planar image.

[0162] S311, obtain the reference anomaly type configured for each of the multiple image regions in the standard planar image.

[0163] S312, in each image region, filter the target region to which the abnormal location belongs.

[0164] S313, determine the reference anomaly type of the target area as the expected anomaly type that matches the anomaly location.

[0165] S314 uses the expected anomaly type as the detection target and performs detection on the difference point cloud to obtain the detection result of the target workpiece.

[0166] S315 controls the orientation of the line laser camera to the spatial position corresponding to the abnormal location and receives the supplementary point cloud acquired by the line laser camera.

[0167] S316, using the expected anomaly type as the detection target, detects the supplementary point cloud to obtain the detection result of the target workpiece.

[0168] For example, such as Figure 5 The flowchart for stamping part inspection shown includes: (For ease of description, please refer to...) Figure 4 The conveyor belt 1, tunnel vision system 2, control equipment 3, electrical control equipment 4, 3D structured light camera 5, photometric stereo camera array 6, and laser line scan camera 7 are respectively referred to as equipment 1, equipment 2, equipment 3, equipment 4, equipment 5, equipment 6, and equipment 7.

[0169] At the start of the inspection, the stamped part under test is placed at the initial stage of device 1. After the stamped part stops at device 5, a structured light camera captures and generates multiple local point clouds during the pause period. The point clouds are sent to device 3 for stitching and compared with the template point cloud to calculate the current position and orientation of the stamped part. Device 3 identifies whether there are missing punches and records the identification results.

[0170] After the pause time, the stamped part moves to the first workpiece position of device 6. Multiple photometric stereo cameras on device 6 are placed at several consecutive workpiece positions. The stamped part pauses at the first workpiece position for a fixed time. During this period, the corresponding photometric stereo cameras at that workpiece position acquire photometric images at different times and send them to device 3. Device 3's built-in algorithm performs calculations and imaging on these photometric images, obtaining a 2D global illumination map (reflecting texture information), a 2.5D curvature map and normal vector map (reflecting relative surface undulations and morphological differences), and a 3D point cloud (reflecting the surface's three-dimensional absolute information). Suspected defect locations are searched in the 2D global illumination map, 2.5D curvature map, and normal vector map. The 3D point cloud of the suspected defect location is segmented, and a second defect determination is performed. It is determined whether the current workpiece position is the last workpiece position on device 6. If so, proceed to the next step; otherwise, repeat the above process until the last workpiece position on device 6 is reached, ending the loop. Device 3 records the determination results for all workpiece positions on device 6.

[0171] During the transfer of the stamped part from the last workpiece position on device 6, device 7 scans key areas of the stamped part and sends the point cloud data to device 3. Device 3 calculates the surface feature parameters of the stamped part, compares them with standard parameter values, and records the results. Based on the detection results recorded throughout the entire process, device 3 determines the current state of the stamped part and displays the type, quantity, and location of defects. At this point, the entire inspection process is complete.

[0172] In this embodiment, based on the relative positional relationships between local regions, the local images of each local region are fused to obtain a global planar image of the target workpiece. Based on the image acquisition poses of each image acquisition device and the local images, a global 3D point cloud of the target workpiece is reconstructed. The expected anomaly type matching the anomaly location in the global planar image is determined, allowing for rapid identification of the expected anomaly type in the less complex global planar image. Furthermore, using the expected anomaly type as the detection target, the global 3D point cloud is detected to obtain the detection result of the target workpiece. This eliminates the need to traverse the entire global 3D point cloud, significantly reducing processing time and improving the detection efficiency of the target workpiece.

[0173] The above method also offers the following benefits: Current mainstream manual inspection methods rely on the experience and sensory sensitivity of quality inspectors, which are highly subjective. Different inspectors may have different judgment standards for the same stamped part's appearance defects, leading to a lack of standardization in the inspection process. Compared to manual inspection, this method achieves an automated inspection process, eliminating the need for human intervention after deployment. This ensures consistent inspection standards for stamped parts and eliminates inspection deviations caused by subjective human factors. Since skilled quality inspectors require extensive training and practice, and multiple inspectors are needed to meet the cycle time requirements of online inspection, this embodiment replaces on-site quality inspectors, reducing workload and significantly saving labor costs. For features requiring precise dimensional measurement, such as the diameter of punched holes, manual inspection has low accuracy and cannot meet the actual accuracy requirements of the production process. This embodiment, however, offers higher measurement accuracy, meeting the needs of the scenario.

[0174] Existing 2D vision inspection methods offer fast detection speeds but lack depth information, making it difficult to reliably detect surface defects or undulations in stamped parts, such as bumps and concave surfaces. Furthermore, 2D vision inspection is significantly affected by ambient lighting and the shape of the stamped part, limiting its stability and compatibility with different types of stamped parts. Compared to existing 2D vision inspection technologies, this embodiment additionally uses 2.5D images and 3D point clouds as data sources to acquire depth information of the stamped part. This makes the system highly effective in detecting surface defects and undulations in stamped parts, ensuring both high detection and low false positive rates. Additionally, this system uses a photometric stereo camera to output a global illumination map to obtain surface texture information of the stamped part. Since the global illumination map is calculated from multiple 2D images of the same area under different angle light sources, it is insensitive to environmental factors and the surface shape of the stamped part. Furthermore, this embodiment features a darkroom designed for the photometric stereo camera, which further ensures the quality and stability of the global illumination image, enabling high detection accuracy for various stamped parts.

[0175] Existing 3D vision inspection methods can accurately acquire the three-dimensional information of the surface of the stamped parts under test. However, due to the limitations of structured light camera hardware, the point cloud resolution of a single structured light camera is difficult to improve. Because the defects to be inspected on the stamped parts are relatively small, the field of view of a single structured light camera is also correspondingly small to ensure the point cloud's representation of the features. Therefore, 3D vision inspection technology for medium and large-sized stamped parts uses a large number of structured light cameras. To avoid missing any areas on the stamped part surface, 3D inspection technology needs to reserve a certain area of ​​common field of view at the edge of the field of view of adjacent 3D cameras. Multiple structured light cameras with a common field of view will interfere with each other if they work simultaneously, as they project active light sources separately. Therefore, 3D vision inspection can only choose to have adjacent structured light cameras work at the same workpiece location in a time-sharing manner, or work in different areas at multiple workpiece locations. Due to the high cycle time requirements of online quality inspection of stamped parts and the relatively long single-frame imaging time of structured light cameras, the time-sharing method at the same location cannot meet the requirements of online inspection. Therefore, 3D vision inspection technology often employs imaging at multiple workpiece resting positions, resulting in high costs and large space requirements. In contrast, this embodiment offers lower hardware costs and a smaller footprint due to its tunnel-style system. Because it first locates all suspected defects in the image domain and then performs secondary judgment on the point cloud of the corresponding area, it requires less point cloud data to process. Furthermore, point cloud computing is slower than image processing. Therefore, the overall processing speed is faster.

[0176] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0177] Based on the same inventive concept, this application also provides a workpiece inspection apparatus for implementing the workpiece inspection method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more workpiece inspection apparatus embodiments provided below can be found in the limitations of the workpiece inspection method described above, and will not be repeated here.

[0178] In one embodiment, such as Figure 6As shown, a workpiece inspection device is provided, including: a fusion module 601, a reconstruction module 602, a type determination module 603, and a detection module 604, wherein:

[0179] The fusion module 601 is used to fuse the local images of each local region based on the relative positional relationship between each local region to obtain a global planar image of the target workpiece.

[0180] The reconstruction module 602 is used to reconstruct the global three-dimensional point cloud of the target workpiece based on the image acquisition pose of each image acquisition device and each local image.

[0181] Type determination module 603 is used to determine the expected anomaly type that matches the anomaly location in the global planar image;

[0182] The detection module 604 is used to detect the global three-dimensional point cloud with the expected anomaly type as the detection target, and obtain the detection result of the target workpiece.

[0183] In one embodiment, the workpiece inspection device further includes: an anomaly analysis module, used to extract depth information of each local area based on the image features of each local image corresponding to the local area; to superimpose the depth information of each local area onto the global planar image to obtain a 2.5D image of the target workpiece; and to perform anomaly analysis on the global planar image and the 2.5D image to determine the anomaly location in the global planar image.

[0184] In one embodiment, the type determination module 603 is further configured to acquire a standard planar image of a standard workpiece; determine an abnormal location in the global planar image based on the texture feature matching result between the global planar image and the standard planar image; and determine the expected abnormal type that matches the abnormal location.

[0185] In one embodiment, the type determination module 603 is further configured to obtain reference anomaly types configured for each of multiple image regions in a standard planar image; filter the target region to which the anomaly location belongs in each image region; and determine the reference anomaly type of the target region as the expected anomaly type that matches the anomaly location.

[0186] In one embodiment, the detection module 604 is further configured to extract feature points from the global 3D point cloud and the standard point cloud respectively, to obtain the 3D coordinate points of the global 3D point cloud and the standard coordinate points of the standard point cloud; based on the relative position and pose matching the coordinate transformation relationship between the 3D coordinate points and the standard coordinate points, the global 3D point cloud and the standard point cloud are registered to obtain the difference point cloud between the global 3D point cloud and the standard point cloud; with the expected anomaly type as the detection target, the difference point cloud is detected to obtain the detection result of the target workpiece.

[0187] In one embodiment, the type determination module 603 is further configured to control the orientation of the line laser camera to the spatial position corresponding to the abnormal position, and receive the supplementary point cloud acquired by the line laser camera; with the expected abnormal type as the detection target, the supplementary point cloud is detected to obtain the detection result of the target workpiece.

[0188] Each module in the aforementioned workpiece inspection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0189] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores image data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a workpiece inspection method.

[0190] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0191] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method steps.

[0192] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method steps.

[0193] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method steps.

[0194] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0195] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A workpiece inspection system, characterized in that, The system includes: A conveying device is used to convey the target workpiece to the inspection station; the inspection station is equipped with multiple image acquisition devices; the image acquisition poses of any two of the image acquisition devices are different. The image acquisition device is used to acquire images of a local area of ​​the target workpiece to obtain a local image of the local area; Control equipment, used for: Based on the relative positional relationship between the local regions, the local images of each local region are fused to obtain a global planar image of the target workpiece; Based on the image acquisition pose of each of the image acquisition devices and the local images, the global three-dimensional point cloud of the target workpiece is reconstructed. Obtain a standard planar image of a standard workpiece; Based on the texture feature matching results between the global planar image and the standard planar image, the abnormal locations in the global planar image are determined; Obtain reference anomaly types configured for each of the multiple image regions in the standard planar image; wherein, the reference anomaly types include anomalies such as cracking, wrinkling, and deformation that often occur in the functional areas of the stamped part; anomalies such as necking, surface concavity, bumps, and scratches that affect the appearance of the appearance area; and anomalies such as bending areas of the stamped part. Within each of the image regions, filter the target region to which the abnormal location belongs; The reference anomaly type of the target area is determined as the expected anomaly type that matches the anomaly location; Using the expected anomaly type as the detection target, the global three-dimensional point cloud is detected to obtain the detection result of the target workpiece; When the control device detects the global 3D point cloud using the expected anomaly type as the detection target and obtains the detection result for the target workpiece, it is configured as follows: Obtain the standard point cloud of the target workpiece; Feature points are extracted from the global 3D point cloud and the standard point cloud respectively to obtain the 3D coordinate points of the global 3D point cloud and the standard coordinate points of the standard point cloud; Based on the relative position and pose matching the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, point cloud registration is performed on the global three-dimensional point cloud and the standard point cloud to obtain the difference point cloud between the global three-dimensional point cloud and the standard point cloud. Using the expected anomaly type as the detection target, the difference point cloud is detected to obtain the detection result of the target workpiece.

2. The system according to claim 1, characterized in that: The image acquisition device is specifically used to acquire images of a local area of ​​the target workpiece under different lighting conditions, so as to obtain local images of the local area under each lighting condition. The control device is also used for: For each local region, depth information of the local region is extracted based on the image features of each local image corresponding to the local region. The depth information of each of the local regions is superimposed onto the global planar image to obtain a 2.5D image of the target workpiece; Anomaly analysis is performed on the global planar image and the 2.5D image to determine the anomaly locations in the global planar image.

3. The system according to claim 1, characterized in that, The system also includes a line laser camera; When the control device detects the global 3D point cloud using the expected anomaly type as the detection target and obtains the detection result for the target workpiece, it is also configured to: Based on the preset orientation of the line laser camera, receive supplementary point cloud data acquired by the line laser camera; Using the expected anomaly type as the detection target, the supplementary point cloud is detected to obtain the detection result of the target workpiece.

4. A workpiece inspection method, characterized in that, The method includes: A partial image of a target workpiece is obtained by image acquisition devices capturing images of a local area of ​​that area. Multiple image acquisition devices are used, all positioned at a detection station. The image acquisition poses of any two image acquisition devices are different. The target workpiece is transported to the detection station via a conveying device. Based on the relative positional relationship between the local regions, the local images of each local region are fused to obtain a global planar image of the target workpiece; Based on the image acquisition pose of each of the image acquisition devices and the local images, the global three-dimensional point cloud of the target workpiece is reconstructed. Obtain a standard planar image of a standard workpiece; Based on the texture feature matching results between the global planar image and the standard planar image, the abnormal locations in the global planar image are determined; Obtain reference anomaly types configured for each of the multiple image regions in the standard planar image; wherein, the reference anomaly types include anomalies such as cracking, wrinkling, and deformation that often occur in the functional areas of the stamped part; anomalies such as necking, surface concavity, bumps, and scratches that affect the appearance of the appearance area; and anomalies such as bending areas of the stamped part. Within each of the image regions, filter the target region to which the abnormal location belongs; The reference anomaly type of the target area is determined as the expected anomaly type that matches the anomaly location; Using the expected anomaly type as the detection target, the global three-dimensional point cloud is detected to obtain the detection result of the target workpiece; The step of detecting the global 3D point cloud using the expected anomaly type as the detection target to obtain the detection result of the target workpiece includes: Obtain the standard point cloud of the target workpiece; Feature points are extracted from the global 3D point cloud and the standard point cloud respectively to obtain the 3D coordinate points of the global 3D point cloud and the standard coordinate points of the standard point cloud; Based on the relative position and pose matching the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, point cloud registration is performed on the global three-dimensional point cloud and the standard point cloud to obtain the difference point cloud between the global three-dimensional point cloud and the standard point cloud. Using the expected anomaly type as the detection target, the difference point cloud is detected to obtain the detection result of the target workpiece.

5. A workpiece inspection device, characterized in that, The device includes: A fusion module is used to acquire local images of a local area of ​​a target workpiece obtained by image acquisition devices. Multiple image acquisition devices are used, all located at a detection station. The image acquisition poses of any two image acquisition devices are different. The target workpiece is transported to the detection station via a conveying device. Based on the relative positional relationship between the local areas, the local images of each local area are fused to obtain a global planar image of the target workpiece. The reconstruction module is used to reconstruct the global three-dimensional point cloud of the target workpiece based on the image acquisition pose of each of the image acquisition devices and the local images. The type determination module is used to obtain standard planar images of standard workpieces. Based on the texture feature matching results between the global planar image and the standard planar image, the abnormal locations in the global planar image are determined; Obtain reference anomaly types configured for each of the multiple image regions in the standard planar image; wherein, the reference anomaly types include anomalies such as cracking, wrinkling, and deformation that often occur in the functional areas of the stamped part; anomalies such as necking, surface concavity, bumps, and scratches that affect the appearance of the appearance area; and anomalies such as bending areas of the stamped part. Within each of the image regions, filter the target region to which the abnormal location belongs; The reference anomaly type of the target area is determined as the expected anomaly type that matches the anomaly location; The detection module is used to detect the global three-dimensional point cloud with the expected anomaly type as the detection target, and obtain the detection result of the target workpiece; The detection module is specifically used to acquire the standard point cloud of the target workpiece; Feature points are extracted from the global 3D point cloud and the standard point cloud respectively to obtain the 3D coordinate points of the global 3D point cloud and the standard coordinate points of the standard point cloud; Based on the relative position and pose matching the coordinate transformation relationship between the three-dimensional coordinate points and the standard coordinate points, point cloud registration is performed on the global three-dimensional point cloud and the standard point cloud to obtain the difference point cloud between the global three-dimensional point cloud and the standard point cloud. Using the expected anomaly type as the detection target, the difference point cloud is detected to obtain the detection result of the target workpiece.

6. The apparatus according to claim 5, characterized in that, The device also includes an anomaly analysis module for: For each local region, depth information of the local region is extracted based on the image features of each local image corresponding to the local region. The depth information of each of the local regions is superimposed onto the global planar image to obtain a 2.5D image of the target workpiece; Anomaly analysis is performed on the global planar image and the 2.5D image to determine the anomaly locations in the global planar image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in claim 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in claim 4.

Images