A detection system for a workpiece

Abstract

The embodiment of the application provides a kind of detection system to work piece, it is related to machine vision technical field;Among them, a kind of detection system to work piece, control subsystem, for controlling flexible mobile device motion to target pose;And, after flexible mobile device motion to target pose, control target acquisition equipment carries out image acquisition;Flexible mobile device, for under the control of control subsystem, motion is carried out;Target acquisition equipment, for under the control of control subsystem, image acquisition is carried out, and the target image collected is reported to control subsystem;Target light source device, for in the process of the image acquisition, according to predetermined light compensation control mode, executes the lighting control of light source component;Control subsystem is further used for based on the target image received, carries out defect detection to work piece area, obtains the defect detection result of work piece area.It can be seen that the present scheme can take into account the efficiency and accuracy of defect detection for work piece.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a workpiece inspection system. Background Technology

[0002] Defect detection of workpieces is a crucial step in the mechanical manufacturing process. For example, the body-in-white (BIW) is the foundation of an automobile, and its quality directly affects the overall quality and performance of the vehicle. Therefore, defect detection of the workpiece is an essential part of automobile manufacturing. Here, "workpiece" refers to the bare body structure (also known as a semi-finished automobile product) that has completed the main structural manufacturing processes such as stamping and welding but has not yet undergone painting, assembly, or other processes.

[0003] In related technologies, defect detection of workpieces mainly relies on manual labor, which is not only inefficient, but also may miss some small but critical defects (such as shallow scratches, small dents, etc.) due to the limitations of the human eye, resulting in poor detection accuracy.

[0004] Therefore, there is an urgent need for a workpiece inspection system that can balance the efficiency and accuracy of defect detection. Summary of the Invention

[0005] The purpose of this application is to provide a workpiece inspection system that balances efficiency and accuracy in defect detection. The specific technical solution is as follows:

[0006] This application provides a workpiece inspection system, including: a control subsystem, a flexible moving device, and a target acquisition device and a target light source device respectively deployed on the flexible moving device; the target light source device is provided with a light source assembly;

[0007] The control subsystem is used to control the flexible moving device to move to the target pose; and, after the flexible moving device moves to the target pose, to control the target acquisition device to perform image acquisition; wherein, the target pose is: a pose that enables the workpiece area of ​​the target workpiece to be within the field of view of the target acquisition device;

[0008] The flexible moving device is used to move under the control of the control subsystem;

[0009] The target acquisition device is used to acquire images under the control of the control subsystem and report the acquired target images to the control subsystem.

[0010] The target light source device is used to control the illumination of the light source components according to a predetermined supplementary lighting control method during the image acquisition process; wherein, the supplementary lighting control method is used to provide supplementary lighting to the workpiece area in different predetermined directions;

[0011] The control subsystem is also used to perform defect detection on the workpiece area based on the received target image, and obtain the defect detection result of the workpiece area.

[0012] Beneficial effects of the embodiments in this application:

[0013] In the workpiece inspection system provided in this application embodiment, the control subsystem can control a flexible moving device to move to a target pose, so that the workpiece area of ​​the target workpiece is within the field of view of the target acquisition device. After the flexible moving device moves to the target pose, the control subsystem controls the target acquisition device to acquire an image and report the acquired target image to the control subsystem. During the image acquisition process, the target light source device executes the lighting control of the light source component according to a predetermined supplementary lighting control method, so as to supplement the workpiece area with light in different predetermined directions through the supplementary lighting control method. Furthermore, the control subsystem can perform defect detection on the workpiece area based on the received target image and obtain the defect detection result of the workpiece area. It is evident that by cooperating with the control subsystem, flexible moving device, target acquisition equipment, and target light source device, defect detection of workpieces can be achieved without human intervention, thereby improving the efficiency of defect detection. Furthermore, during image acquisition, the target light source device can execute the illumination control of the light source components according to a predetermined supplementary lighting control method to provide supplementary lighting to the workpiece area in different predetermined directions. This allows the target acquisition equipment to acquire target images with high-quality illumination effects, providing clear images for defect detection of the workpiece area and improving the accuracy of defect detection. Therefore, the detection system of this application can balance the efficiency and accuracy of defect detection of workpieces.

[0014] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0016] Figure 1 A schematic diagram of a workpiece detection system provided in an embodiment of this application;

[0017] Figure 2 This is a schematic diagram of a target light source device and a target acquisition device provided in an embodiment of this application;

[0018] Figure 3 A schematic diagram of another workpiece detection system provided in this application embodiment;

[0019] Figure 4 A schematic diagram of another workpiece detection system provided in this application embodiment;

[0020] Figure 5(a) is a schematic diagram of a scenario for defect detection of a workpiece provided in an embodiment of this application;

[0021] Figure 5(b) is a schematic diagram of the process for defect detection of workpieces provided in the embodiments of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0023] First, a brief introduction to some technical terms used in the embodiments of this application:

[0024] Body-in-white: An important term in the automobile manufacturing process, referring to the bare body structure that has completed the main structural manufacturing such as stamping and welding, but has not yet undergone processes such as painting and assembly (such as installing engines, interiors, and electronic equipment).

[0025] Photometric stereo technology employs multi-angle lighting or dome lighting, combined with a sequential exposure mechanism, to complete image acquisition and fusion. Its core principle is to precisely control the sequential illumination of light sources from different directions, leveraging the complementarity of multi-directional lighting to actively eliminate interference problems such as shadow occlusion and specular reflection that are easily caused by single-direction illumination. Ultimately, it fuses these sources to generate a high-quality image with uniform illumination and clear details, providing reliable data support for subsequent stereoscopic vision processing (such as feature matching and 3D reconstruction).

[0026] Structured light technology: a classic non-contact measurement technology in the field of 3D vision inspection. Its core principle is to project a pre-designed "structured light" (such as stripes, checkerboard, random speckle) onto the object being measured, use a camera to capture the distorted image of the structured light on the object's surface, and then use algorithms such as triangulation and phase calculation to restore the object's 3D shape (coordinates, depth, contour).

[0027] Laser galvanometer technology: a precision optical deflection control technology that is widely used in laser processing and other fields.

[0028] Feature contrast: Differences in brightness, color, or texture detail between different areas of an image.

[0029] Exposure time: also known as shutter speed, is a key parameter in photography that controls the length of time that the camera's sensor (or film) receives light. It not only determines the brightness of the photo, but also has a decisive impact on the dynamic effects and sharpness of the image.

[0030] Hand-eye calibration: a key technology in the field of robotics used to establish the coordinate transformation relationship between the robot's end effector and the vision sensor. Its core objective is to enable the robot to accurately perceive and manipulate target objects through visual information.

[0031] PNP (Perspective-n-Point): A core algorithm in computer vision and robotics used to solve the rigid transformation relationship between the camera coordinate system and the target coordinate system.

[0032] Vehicle body digital model: also known as vehicle body digital model, is a three-dimensional model that digitally represents the vehicle body structure, appearance and internal parts in the automotive research and development and manufacturing process. It is the core digital carrier of the automotive industry from design to mass production.

[0033] Secondly, to better understand this solution, taking a car body as an example, we will introduce the methods for defect detection of workpieces in related technologies:

[0034] In related technology 1, defect detection of the body-in-white mainly relies on manual labor. However, manual defect detection of the body-in-white has the following problems:

[0035] (1) Inconsistent judgment criteria (i.e., Subjectivity & Inconsistency).

[0036] The criteria for defect judgment rely on the subjective experience of quality inspectors, and different inspectors may have significantly different criteria (for defects such as scratch depth, length, and visibility of dents). For example, the same scratch may be judged as acceptable by inspector A but unacceptable by inspector B. Furthermore, even the same inspector's judgment criteria can be affected by factors such as fatigue, emotional state, and work pressure, leading to fluctuations in judgment results at different times and making it difficult to establish stable and consistent standards. In addition, quality inspectors can only visually identify whether a defect "exists" or "does not exist," and cannot accurately quantify parameters such as the size, depth, and area of ​​the defect, which is detrimental to quality control and statistical analysis of defect distribution in the vehicle production process.

[0037] (2) Low efficiency;

[0038] Due to the large size and complex structure of the body-in-white, quality inspectors need to conduct a comprehensive and meticulous inspection from multiple angles, which is time-consuming. In high-speed assembly line scenarios (such as a rate of 1 body-in-white per minute), the low efficiency of manual inspection conflicts with the high turnover requirements of the production line, easily becoming a bottleneck restricting overall production efficiency. In addition, to ensure the quality of inspection, a large number of quality inspectors need to be deployed in a multi-shift work mode, which involves high investment and constitutes a continuous and heavy labor cost burden in the production process.

[0039] (3) Fatigue and limited attention.

[0040] Quality inspectors often experience eye strain when staring at repetitive, highly reflective metal surfaces under specific lighting conditions for extended periods. This leads to decreased attention span and inspection accuracy, causing the missed inspection rate to rise sharply with working hours. Furthermore, human visual perception has inherent limitations in white body inspection. For example, hidden areas such as the inside of door frames, pillar connections, and corners are easily overlooked, and subtle defects with weak contrast to the background, such as shallow scratches and tiny dents, are difficult to accurately identify with the human eye.

[0041] (4) Environmental and Capability Limitations.

[0042] Quality inspectors rely heavily on lighting conditions for visual inspection. Even with specialized inspection lights, the angle, intensity, and uniformity of the light can affect the visibility of defects. Some defects are only visible under specific lighting angles, significantly increasing the probability of missed defects. Human visual inspection can only cover the surface of the vehicle body, identifying superficial defects such as scratches, dents, weld spatter, and rust. However, it lacks effective means of observing internal defects in the body-in-white, such as weld quality, weld strength, and the integrity of adhesive strip application. Furthermore, limited by the physical resolution of the human eye, manual inspection is poor at detecting micron-level defects and fine hairline scratches, far inferior to optical sensors and machine vision systems.

[0043] (5) Poor integration and traceability.

[0044] Quality inspectors typically record their findings on paper, making it difficult to create digital, traceable quality archives. When a batch of body-in-white products has problems, it is impossible to quickly and accurately trace the source of the problem and the production batch. Furthermore, purely manual inspection cannot be seamlessly integrated with digital management systems such as MES (Manufacturing Execution System) and SPC (Statistical Process Control), hindering real-time data uploading and analysis. This impedes the establishment of an intelligent manufacturing system and the implementation of closed-loop quality control throughout the entire process.

[0045] In related technology 2, defect detection of the body-in-white using structured light has the following problems:

[0046] Traditional methods of acquiring images of the white body using structured light, laser galvanometers, or line-scan lasers all suffer from slow inspection cycles. The image processing time for these techniques is approximately 2 seconds, and continuous image output is not possible. Furthermore, current 3D scanning technology cannot detect defects as small as 0.1 millimeters, resulting in a large number of missed defects.

[0047] In related technology 3, defect detection of the body-in-white using ordinary light sources / programmed stripe light has the following problems:

[0048] (1) For traditional ordinary light sources, it is impossible to distinguish whether a defect is planar or three-dimensional. For example, the only difference between concave and convex points in an image and other areas is color. Concave points are darker than other areas, and it is impossible to distinguish whether the defect is a stain or a concave point in the image. In addition, there is external light interference in automobile factories, and there are changes in sunlight source from morning to evening. Traditional light sources are not robust to such changes and are easily affected by external light.

[0049] (2) To detect defects in the body-in-white by striped light, the body-in-white needs to have a certain mirror reflection. However, the metal surface of the body-in-white has been oiled and polished, and the surface reflection is poor. The sinusoidal and pulse wave light sources projected by the programmable striped light (programmable striped structured light) are difficult to clearly highlight surface defects and cannot meet the detection requirements.

[0050] Based on the problems described above, this application provides a detection system for workpieces.

[0051] The workpiece detection system provided in this application embodiment may include a control subsystem, a flexible moving device, and a target acquisition device and a target light source device respectively deployed on the flexible moving device, wherein the target light source device is provided with a light source component.

[0052] The control subsystem can be considered as a subsystem with control functions, capable of controlling some or all of the flexible moving device, target acquisition equipment, and target light source device.

[0053] Furthermore, the flexible moving device can move to drive the target acquisition device and the target light source device to move. For example, the flexible moving device can be a robotic arm with a moving function, but it is not limited to the form of a robotic arm. Any device that can drive the image acquisition device and the target light source device to move can be used as the flexible moving device in this application. It is understood that in one implementation, a moving device is provided at the bottom of the flexible moving device, which allows the entire flexible moving device to move around the workpiece.

[0054] Furthermore, both the target acquisition device and the target light source device are mounted on a flexible moving device, and their specific locations may include, but are not limited to, the end of the flexible moving device. The target acquisition device is an acquisition device capable of image acquisition; for example, it can be a camera with any frame rate specification. The target light source device is equipped with one or more light source components; for example, the light source components can be light-emitting diode (LED) lamp groups. During image acquisition, the target light source device can control all or part of the light source components to be lit (e.g., ordered lighting), thereby providing illumination in at least one direction (i.e., supplemental lighting in at least one predetermined direction). It is understood that the target image acquired by the target acquisition device can be a YUV format image, where "Y" represents luminance, and "U" and "V" represent chrominance, and "U" and "V" can also describe the color and saturation of the image. Of course, the target is not limited to a YUV format image; it can also be an RGB (red, green, blue) format image. This application embodiment does not specifically limit the image format of the target image.

[0055] Furthermore, the target workpiece involved in this application can be any workpiece that has a defect detection requirement; for example, the target workpiece can be a workpiece in various mechanical manufacturing scenarios, such as: in the automobile manufacturing scenario, the target workpiece can be a white body, or a frame, door, etc.; in addition, various workpieces in CNC machining, metal stamping and other scenarios can all fall within the scope of workpieces applicable to this application.

[0056] Specifically, in the workpiece detection system provided in this application, the control subsystem is used to control the flexible moving device to move to a target pose; and, after the flexible moving device moves to the target pose, to control the target acquisition device to perform image acquisition; wherein, the target pose is: a pose that enables the workpiece area of ​​the target workpiece to be within the field of view of the target acquisition device;

[0057] The flexible moving device is used to move under the control of the control subsystem;

[0058] The target acquisition device is used to acquire images under the control of the control subsystem and report the acquired target images to the control subsystem.

[0059] The target light source device is used to control the illumination of the light source components according to a predetermined supplementary lighting control method during the image acquisition process; wherein, the supplementary lighting control method is used to provide supplementary lighting to the workpiece area in different predetermined directions;

[0060] The control subsystem is also used to perform defect detection on the workpiece area based on the received target image, and obtain the defect detection result of the workpiece area.

[0061] In the workpiece inspection system provided in this application embodiment, the control subsystem can control a flexible moving device to move to a target pose, so that the workpiece area of ​​the target workpiece is within the field of view of the target acquisition device. After the flexible moving device moves to the target pose, the control subsystem controls the target acquisition device to acquire an image and report the acquired target image to the control subsystem. During the image acquisition process, the target light source device executes the lighting control of the light source component according to a predetermined supplementary lighting control method, so as to supplement the workpiece area with light in different predetermined directions through the supplementary lighting control method. Furthermore, the control subsystem can perform defect detection on the workpiece area based on the received target image and obtain the defect detection result of the workpiece area. It is evident that by cooperating with the control subsystem, flexible moving device, target acquisition equipment, and target light source device, defect detection of workpieces can be achieved without human intervention, thereby improving the efficiency of defect detection. Furthermore, during image acquisition, the target light source device can execute the illumination control of the light source components according to a predetermined supplementary lighting control method to provide supplementary lighting to the workpiece area in different predetermined directions. This allows the target acquisition equipment to acquire target images with high-quality illumination effects, providing clear images for defect detection of the workpiece area and improving the accuracy of defect detection. Therefore, the detection system of this application can balance the efficiency and accuracy of defect detection of workpieces.

[0062] Furthermore, since the defect detection process described in this application embodiment does not require human intervention, all problems arising from human intervention can be avoided. Moreover, the target images acquired by the target acquisition device all have excellent lighting effects, and the defects in the target images are all clear. Based on the target images, the defects can be clearly seen, thereby improving the accuracy of defect detection for workpieces. Subsequently, the staff can optimize the workpiece based on the detected defects, laying a solid foundation for subsequent processes of the workpiece (e.g., for a white body, the subsequent process is painting).

[0063] The workpiece inspection system provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0064] like Figure 1 As shown, this application provides a workpiece detection system, including: a control subsystem 100, a flexible moving device 110, and a target acquisition device 120 and a target light source device 130 respectively deployed on the flexible moving device; the target light source device 130 is provided with a light source assembly;

[0065] A control subsystem 100 is used to control the flexible moving device to move to a target pose; and, after the flexible moving device moves to the target pose, to control the target acquisition device to perform image acquisition; wherein, the target pose is: a pose that enables the workpiece area of ​​the target workpiece to be within the field of view of the target acquisition device;

[0066] The flexible moving device 110 is used to move under the control of the control subsystem 100;

[0067] The target acquisition device 120 is used to acquire images under the control of the control subsystem and report the acquired target images to the control subsystem 100.

[0068] The target light source device 130 is used to control the illumination of the light source component according to a predetermined supplementary lighting control method during the image acquisition process; wherein, the supplementary lighting control method is used to provide supplementary lighting to the workpiece area in different predetermined directions;

[0069] The control subsystem 100 is also used to perform defect detection on the workpiece area based on the received target image, and obtain the defect detection result of the workpiece area.

[0070] It is understood that when the flexible moving device 110 is in a target pose, the field of view of the target acquisition device 120 includes the workpiece area of ​​the target workpiece. The workpiece area of ​​the target workpiece can be a region of a predetermined point on the target workpiece. The same target workpiece can have one or more regions of predetermined points. For example, taking a car body as an example, the aforementioned workpiece area is the car body region of a predetermined point on the car body. Furthermore, the predetermined points can correspond to the target pose; different predetermined points correspond to different target poses. That is, different workpiece areas can correspond to different target poses. For example, if a predetermined point 1 of the target workpiece matches a target pose, when the flexible moving device 110 moves to that target pose, the field of view of the target acquisition device 120 includes the workpiece area represented by the predetermined point 1. It is understood that when the target acquisition device 120 and the target light source device 130 are provided at the end of the flexible moving device 110, the flexible moving device 110 being in a target pose can mean that the end of the flexible moving device 110 is in a target pose.

[0071] Furthermore, the control subsystem 100 can control the flexible motion device 110 to move to the target pose, and after moving to the target pose, the flexible motion device 110 can send information representing that the target pose has been reached to the control subsystem 100, so that after the control subsystem 100 determines that the flexible motion device 110 has moved to the target pose, it controls the target acquisition device to perform image acquisition, for example, by issuing an acquisition command to the target acquisition device 120; accordingly, under the control of the control subsystem 100 (for example, after receiving the acquisition command), the target acquisition device 120 performs image acquisition and reports the acquired target image to the control subsystem 100.

[0072] During image acquisition, the target light source device 130 can execute the illumination control of the light source component according to a predetermined supplementary lighting control method, so as to provide supplementary lighting to the workpiece area in different predetermined directions through the supplementary lighting control method. There are multiple ways to trigger the target light source device 130 to execute the illumination control of the light source component according to the predetermined supplementary lighting control method. For example, in one implementation, the target acquisition device 120 can respond to the acquisition command sent by the control subsystem 100 and send a predetermined control command to the target light source device 130 (e.g., a control command corresponding to the workpiece area, or a control command corresponding to a predetermined point), so that the target light source device 130 can execute the illumination control of the light source component according to the predetermined supplementary lighting control method; in other implementations, the control subsystem 110 can send the predetermined control command to the target light source device 130 while controlling the target acquisition device 120 to perform image acquisition, which is also reasonable.

[0073] It should be emphasized that when there are multiple predetermined points (i.e., when there are multiple workpiece areas), when the flexible moving device 110 moves to the target pose that matches other predetermined points, the target light source device 130 can receive the control command corresponding to the other predetermined point again, that is, receive a new predetermined control command; that is, the target acquisition device 120 or the control subsystem 100 can be considered to be sending predetermined control commands to the target light source device 130 in a cyclic manner.

[0074] The implementation of controlling the illumination of the light source component according to the predetermined supplementary lighting control method can be varied. For example, in one implementation, controlling the illumination of the light source component according to the predetermined supplementary lighting control method may include:

[0075] Based on a pre-defined combination of light source components, the lighting control of the light source components is executed; wherein, the illumination direction after different combinations of light source components are lit is different predetermined direction.

[0076] In this implementation, multiple light source component combinations are pre-set for the workpiece area. For example, the workpiece area can be equipped with three light source component combinations, including light source component combination a, light source component combination b, and light source component combination c. Thus, by controlling the illumination of each light source component combination separately (e.g., controlling light source component combinations a, b, and c in an orderly manner), supplementary lighting in different predetermined directions can be provided to the workpiece area. Furthermore, it should be noted that each light source component combination set in the workpiece area, when illuminated, can also have its own illumination intensity and illumination duration. Moreover, if there are multiple predetermined points, i.e., multiple workpiece areas, multiple light source component combinations can be set for each predetermined point, that is, multiple light source component combinations can be set for each workpiece area. For example, predetermined point 1 is equipped with three light source component combinations, including light source component combination a, light source component combination b, and light source component combination c; predetermined point 2 is equipped with two light source component combinations, including light source component combination b and light source component combination c. The three light source component combinations can provide multiple illumination directions, multiple illumination intensities, and multiple illumination durations.

[0077] Furthermore, it is understood that each light source component combination includes at least one light source component from a plurality of light source components in the target light source device. For example, light source component combination a includes 5 light source components, light source component combination b includes 3 light source components, and light source component combination c includes 1 light source component. Accordingly, the target light source device 130 can control at least one light source component in each light source component combination to illuminate the light source component combination. The lighting and extinguishing times of each light source component combination are preset. For example, for the 3 light source component combinations in the workpiece area, the 5 light source components included in light source component combination a can be lit first, then the 3 light source components included in light source component combination b can be lit, and finally the 1 light source component included in light source component combination c can be lit.

[0078] It is important to emphasize that the target light source device 130 can independently control the illumination of each light source component combination, ensuring that the exposure process of each frame of the target image is within a supplementary lighting period directed towards a specific direction (i.e., a predetermined direction). In other words, each frame of the target image is exposed within a unidirectional supplementary lighting period. Furthermore, the light intensity and uniformity of each light source component combination are also pre-set, thereby ensuring that each frame of target image data acquired by the target acquisition device 120 has excellent illumination. Additionally, the control of the target light source device 130 here can be considered to utilize photometric stereoscopic technology.

[0079] It is understood that the workpiece area may or may not have defects, and defects may or may not be detected in the workpiece area. Therefore, if a defect is detected in the workpiece area, the defect detection result may, for example, include the defect type and image location information of the target defect, and the number of target defects may be one or more. If no defect is detected in the workpiece area, the defect detection result may be a detection result indicating the absence of defects.

[0080] Furthermore, there are multiple ways for the control subsystem 100 to perform defect detection on the workpiece area based on the received target image. For example, in one implementation, the control subsystem 100 can perform defect detection on the workpiece area of ​​the received target image separately, and then fuse the obtained defect detection results to obtain the defect detection result for the workpiece area.

[0081] For example, in other implementations, the control subsystem 100 can perform image fusion on the received target image and perform defect detection on the workpiece area based on the fused image to obtain the defect detection result of the workpiece area. That is, for the received target image, an image fusion algorithm is used to synthesize a fused image, and the fused image is used to perform defect detection on the workpiece area to obtain the defect detection result of the workpiece area. It can be understood that since the multiple frames of target images acquired by the target acquisition device 120 are images of multiple lighting directions for the same predetermined point (i.e., the same workpiece area), when performing image fusion on the multiple frames of target images, shadows (the position of shadows is different in different target images, and shadows in one direction will be illuminated by light from another direction), specular reflections (highlights are in different positions in different target images, which can be effectively eliminated) can be eliminated, and feature contrast can be enhanced, so that the fused image is a "perfect" image: an image with fewer shadows, fewer highlights, and high feature contrast.

[0082] Furthermore, the control subsystem 100 can perform defect detection on the workpiece region of any image (such as a target image or a fused image) in various ways. For example, in one implementation, the control subsystem 100 performs defect detection on the workpiece region of any image (such as a target image or a fused image) including:

[0083] According to the target method, defect detection is performed on the image to obtain the defect detection results of the workpiece area;

[0084] The target methods include: deep learning detection methods, or instance segmentation methods.

[0085] The deep learning detection method can accurately locate target defects in the image, mark each target defect with a bounding box, and provide a label representing the type of each target defect. The instance segmentation detection method can identify the type of each target defect in the image, locate its position, and accurately distinguish the pixel-level contour of each target defect (segmenting the specific shape of each target defect, rather than just marking it with a bounding box). Of course, both of the above detection methods are applicable to the embodiments of this application, and the embodiments of this application do not specifically limit them. In this implementation method, defect detection is performed on the image according to the target method to obtain the defect detection result of the workpiece area, thereby realizing defect detection of the image and providing a basis for defect detection of the target workpiece.

[0086] For example, in one implementation, the target acquisition device 120 may be an area array camera with a frame rate higher than a predetermined threshold, and / or the target light source device 130 may be a device in the form of a ring light or a device in the form of a dome light.

[0087] It is understood that in this implementation, the target acquisition device 120 can be an area scan camera, and the acquisition frame rate of the area scan camera is higher than a predetermined threshold, enabling continuous acquisition of multiple frames of images; the target light source device 130 can be a ring-shaped device or a dome-shaped device; among them, the ring-shaped light source device has the advantage of its unique ring structure, which can create a soft, uniform and directional light source, especially suitable for scenes that need to highlight the details and three-dimensionality of the subject, while the dome-shaped light source device can simulate a uniform, shadowless diffused light environment, which is more suitable for scenes that need natural and soft overall lighting; different forms of light source devices can be selected based on different needs; this application embodiment does not specifically limit this.

[0088] To better understand the above content, the following will combine... Figure 2 To introduce, such as Figure 2 As shown:

[0089] The target acquisition device 120 is an area array camera, and a target light source device 130 with a ring light structure is surrounded on the outside of the area array camera. The target light source device 130 is the shadow area.

[0090] As can be seen, the target acquisition device in this implementation can be an area array camera, and the target light source device 130 can be a ring light or a dome light. Different types of target light source devices can be applied to different environments / scenes, thereby improving the versatility of defect detection for workpieces.

[0091] In the workpiece inspection system provided in this application embodiment, the control subsystem can control a flexible moving device to move to a target pose, so that the workpiece area of ​​the target workpiece is within the field of view of the target acquisition device. After the flexible moving device moves to the target pose, the control subsystem controls the target acquisition device to acquire an image and report the acquired target image to the control subsystem. During the image acquisition process, the target light source device executes the lighting control of the light source component according to a predetermined supplementary lighting control method, so as to supplement the workpiece area with light in different predetermined directions through the supplementary lighting control method. Furthermore, the control subsystem can perform defect detection on the workpiece area based on the received target image and obtain the defect detection result of the workpiece area. It is evident that by cooperating with the control subsystem, flexible moving device, target acquisition equipment, and target light source device, defect detection of workpieces can be achieved without human intervention, thereby improving the efficiency of defect detection. Furthermore, during image acquisition, the target light source device can execute the illumination control of the light source components according to a predetermined supplementary lighting control method to provide supplementary lighting to the workpiece area in different predetermined directions. This allows the target acquisition equipment to acquire target images with high-quality illumination effects, providing clear images for defect detection of the workpiece area and improving the accuracy of defect detection. Therefore, the detection system of this application can balance the efficiency and accuracy of defect detection of workpieces.

[0092] Furthermore, since the defect detection process described in this application embodiment does not require human intervention, all problems arising from human intervention can be avoided. Moreover, the target images acquired by the target acquisition device all have excellent lighting effects, and the defects in the target images are all clear. Based on the target images, the defects can be clearly seen, thereby improving the accuracy of defect detection for workpieces. Subsequently, the staff can optimize the workpiece based on the detected defects, laying a solid foundation for subsequent processes of the workpiece (e.g., for a white body, the subsequent process is painting).

[0093] Optionally, in one possible embodiment, the supplementary lighting control method is further configured to ensure that the light source assembly is lit no later than a predetermined start time and is turned off no earlier than a predetermined end time.

[0094] In the first optional approach, the predetermined start time is the exposure start time of a single frame image, and the corresponding predetermined end time is the exposure end time of that single frame image;

[0095] In the second alternative, the predetermined start time is the exposure start time of the consecutive frames, and the corresponding predetermined end time is the exposure end time of the consecutive frames.

[0096] For the first alternative approach, the moment when each light source component combination is lit is no later than the start moment of the exposure of a frame of image, and the moment when the light source component combination is turned off is no earlier than the end moment of the exposure of that frame of image.

[0097] For the second alternative approach, the time when each light source component combination is lit is no later than the start time of the corresponding exposure of the consecutive frames, and the time when the light source component combination is turned off is no earlier than the end time of the corresponding exposure of the consecutive frames.

[0098] The exposure start time of the consecutive multiple frames is the exposure start time of the first exposed frame in the consecutive multiple frames, and the exposure end time of the consecutive multiple frames is the exposure end time of the last exposed frame in the consecutive multiple frames.

[0099] Specifically, in the first optional method, the time when each light source component combination is lit is no later than the start time of the exposure of a frame of image, that is, the light source component combination has been lit before the start time of the exposure of a frame of image, and the time when the light source component combination is turned off is no earlier than the end time of the exposure of a frame of image, that is, the light source component combination can be turned off only after the end time of the exposure of a frame of image; the exposure period of the frame of image (also known as the exposure time interval) is located within the lighting period of the light source component combination, thereby ensuring that the light source component combination is lit when the frame of image is acquired, so that each frame of image (i.e. the acquired target image) acquired by the target acquisition device 120 has a high-quality lighting effect. Furthermore, the first optional method can be considered as the situation where the target acquisition device 120 can acquire one frame of image (i.e., one frame of target image) when the light source component in a set of light source component combinations is lit. Correspondingly, during the image acquisition process, the light source component combination is constantly changing, and the light source component combination corresponding to any two adjacent target images can be different. The target light source device 130 is constantly changing the lit light source component, which improves the accuracy of defect detection for workpieces.

[0100] Furthermore, in the second optional method, the illumination time of each light source component combination is no later than the exposure start time of the corresponding consecutive multiple frames, that is, before the exposure start time of the first exposed frame in the consecutive multiple frames, the light source component combination has been illuminated; and the extinguishing time of the light source component combination is no earlier than the exposure end time of the corresponding consecutive multiple frames, that is, after the exposure end time of the last exposed frame in the consecutive multiple frames, the light source component combination can be extinguished. The exposure period of the consecutive multiple frames is within the illumination duration of the light source component combination, thereby ensuring that the light source component combination is illuminated when the consecutive multiple frames are acquired, so that each frame acquired by the target acquisition device has a high-quality illumination effect. Furthermore, this second optional method can be considered as the situation where the target acquisition device can acquire consecutive multiple frames of images when the light source components in a group of light source component combinations are illuminated. Correspondingly, during the image acquisition process, the light source component combinations corresponding to the consecutive multiple frames are all the same, thereby reducing the consumption of the light source device.

[0101] As can be seen, the embodiments of this application provide two methods. The first method is to change the combination of light source components for each frame of image acquisition, and the combination of light source components corresponding to each frame of image is different, which improves the accuracy of defect detection for workpieces. The second method is to change the combination of light source components for each consecutive multiple frames of image acquisition, and the combination of light source components corresponding to the consecutive multiple frames of image is the same. The target light source device does not need to frequently change the light source components that are lit, which reduces the consumption of the target light source device.

[0102] Optionally, in one possible embodiment, the detection result of the workpiece area includes the defect type and image location information of the target defect;

[0103] Correspondingly, the control subsystem is also used for:

[0104] Based on the image location information of the target defect, the mapping position of the target defect in the workpiece digital model of the target workpiece is determined, and the target defect is mapped to the workpiece digital model of the workpiece according to the mapping position and the defect type.

[0105] It is understood that multiple defect types are predetermined in this embodiment, such as stains, dents, bumps, scratches, etc. When a defect is detected in a workpiece area, the defect detection result of the workpiece area may include the defect type and image location information of the target defect; wherein, the image location information is the two-dimensional location information of the target defect in the image, and the number of target defects may be one or more. Accordingly, since the image location information of the target defect is two-dimensional location information, it is not possible to clearly know the location information of the target defect on the target workpiece. Therefore, the image location information of the target defect can be mapped onto the workpiece digital model of the target workpiece (e.g., the body digital model of the white body) to determine the mapping position of the target defect on the workpiece digital model. And, after determining the mapping position of the target defect on the workpiece digital model, the target defect of the workpiece area can be mapped onto the workpiece digital model of the target workpiece according to the mapping position and defect type of the target defect, so that the mapping position and defect type of the target defect in the workpiece area can be viewed on the workpiece digital model. It is important to emphasize that after mapping the target defects in the workpiece area of ​​the target workpiece to the workpiece's digital model, the workpiece's digital model can be displayed. Workers can view the mapped location and defect type of the target defects in the workpiece area (i.e., the area at the predetermined point) on this digital model, enabling subsequent repair / adjustment of defects on the target workpiece. It is understood that the aforementioned workpiece digital model refers to a three-dimensional model that digitally represents the workpiece's structure, shape, and internal components; it is the core digital carrier for workpiece design and mass production.

[0106] As can be seen, the control subsystem in this embodiment can also determine the mapping position of the target defect on the workpiece digital model of the target workpiece based on the image location information of the target defect, and map the target defect in the workpiece area to the workpiece digital model of the target workpiece according to the mapping position and defect type of the target defect; thus, the operator can view the mapping position and defect type of the target defect in the workpiece area on the workpiece digital model of the target workpiece, providing a data basis for the defect repair of the target workpiece.

[0107] Optionally, in one possible embodiment, the control subsystem 100 determines the mapping position of the target defect in the workpiece digital model of the target workpiece based on the image location information of the target defect, which may include:

[0108] Based on the predetermined mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device 120, the image position information of the target defect is transformed by coordinate transformation to obtain the mapping position of the target defect in the workpiece digital model of the target workpiece.

[0109] It is understandable that the image location information of the target defect can be considered as the pixel location information of the target defect in the pixel coordinate system. Therefore, based on the predetermined mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device (such as the device coordinate system, i.e., the camera coordinate system), the image location information of the target defect can be transformed to obtain the mapping position of the target defect in the body digital model of the target workpiece.

[0110] For example, based on a predetermined mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device, a back projection algorithm is used to transform the image position information of the target defect into coordinates, thereby obtaining the mapped position of the target defect in the workpiece digital model of the target workpiece. The back projection algorithm may include: establishing a mapping relationship between the pixel coordinate system and the device coordinate system of the target acquisition device using Zhang Zhengyou intrinsic parameter calibration; using the optical principle of pinhole imaging, converting the pixel coordinates (image position information) of a defect into a ray emanating from the optical center in the target coordinate system of the target acquisition device; based on the mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device, mapping the 3D point cloud of the workpiece digital model from the workpiece digital model coordinate system to the target coordinate system of the target acquisition device, so that the ray and the 3D point cloud of the workpiece digital model are in the same coordinate system (the target coordinate system of the target acquisition device); and calculating the intersection point of the ray and the 3D point cloud of the workpiece digital model as the mapped position of the target defect in the workpiece digital model of the target workpiece. Furthermore, the mapping relationship between the workpiece digital model coordinate system and the target acquisition device's equipment coordinate system essentially involves solving the spatial pose relationship between the workpiece digital model's three-dimensional design coordinate system and the acquisition device's three-dimensional coordinate system. The core is to use feature information from the 2D image, combined with the geometric parameters of the workpiece digital model, to complete the pose mapping from two-dimensional pixel space to three-dimensional design space. Of course, the above description is merely illustrative; any method capable of mapping image position information to the workpiece digital model is applicable to the embodiments of this application.

[0111] For example, in one implementation, taking a flexible moving device as a robotic arm and a target acquisition device located at the end of the robotic arm, the method for determining the mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device includes methods A11-A12:

[0112] Method A11: Determine the mapping relationship between the workpiece digital model coordinate system and the base coordinate system of the robotic arm, the mapping relationship between the target coordinate system of the target acquisition device and the end-effector coordinate system of the robotic arm, and the mapping relationship between the base coordinate system of the robotic arm and the end-effector coordinate system of the robotic arm.

[0113] It is understandable that by using the tip-point calibration of the robotic arm, the mapping relationship between the workpiece's digital model coordinate system and the robotic arm's base coordinate system can be determined. For example, by physically touching eight actual feature points on the vehicle body (such as positioning holes, edges, and corners) with the tip of the robotic arm, the coordinates of these eight feature points in the workpiece's digital model coordinate system can be obtained. The robotic arm can also record the coordinates of these eight feature points in its base coordinate system. Using these eight sets of corresponding coordinates, the mapping relationship between the workpiece's digital model coordinate system and the robotic arm's base coordinate system can be calculated. Similarly, by using hand-eye calibration, the mapping relationship between the target coordinate system of the target acquisition device and the coordinate system of the robotic arm's end effector can be determined. For example, first, Zhang Zhengyou's 15-point calibration is performed to determine the intrinsic parameters of the target acquisition device, and then hand-eye calibration is performed to calculate the mapping relationship between the target coordinate system of the target acquisition device and the coordinate system of the robotic arm's end effector. Furthermore, the mapping relationship between the robot arm's base coordinate system and its end effector coordinate system can be considered as the inverse matrix of the end effector pose of the target acquisition device. Inverting the pose matrix of the target acquisition device relative to the robot arm's base coordinate system yields the pose mapping matrix of the robot arm's base coordinate system in the target acquisition device's coordinate system. In other words, when the target acquisition device is mounted on the robot arm's end effector, a fixed mapping relationship exists between the robot arm's base coordinate system and the end effector coordinate system; this mapping matrix is ​​the pose matrix of the end effector relative to the base coordinate system. Since the target acquisition device is fixed to the robot arm's end effector, inverting its pose matrix relative to the robot arm's base coordinate system yields the pose mapping matrix of the robot arm's base coordinate system in the target acquisition device's coordinate system. Additionally, the PNP calibration algorithm can typically be used in the hand-eye calibration process, and this application does not specifically limit this approach.

[0114] Of course, the above-described methods for determining the pose are merely illustrative and do not limit the scope of the embodiments in this application.

[0115] Method A12: Based on the mapping relationship between the target coordinate system of the target acquisition device and the coordinate system of the robotic arm end effector, the mapping relationship between the base coordinate system of the robotic arm and the coordinate system of the robotic arm end effector, and the mapping relationship between the workpiece digital model coordinate system and the base coordinate system of the robotic arm, calculate the mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device.

[0116] Understandably, the mapping relationship between the target coordinate system of the target acquisition device and the coordinate system of the robotic arm's end effector, the mapping relationship between the base coordinate system of the robotic arm and the coordinate system of the robotic arm's end effector, and the mapping relationship between the workpiece digital model coordinate system and the base coordinate system of the robotic arm can be calculated as a product to obtain the mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device. This process can be considered as starting from the target acquisition device and, through the link of target acquisition device - robotic arm end effector - robotic arm's base coordinate system - workpiece digital model, inversely deducing the mapping relationship between the workpiece digital model and the target acquisition device.

[0117] This implementation method allows for the precise association of physical point coordinates acquired by the target acquisition device with the workpiece digital model, enabling precise collaborative operation between the robotic arm and vision.

[0118] As can be seen, based on the predetermined mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device, the image position information of the target defect is transformed to obtain the mapping position of the target defect on the workpiece digital model of the target workpiece. This allows the position information of the target defect in the pixel coordinate system to be mapped onto the workpiece digital model, achieving accurate 3D positioning of the defect. Subsequently, staff can view the mapping position and defect type of the target defect in the workpiece area on the workpiece digital model of the target workpiece, providing a data foundation for defect repair of the target workpiece.

[0119] Optionally, in one possible embodiment, the control subsystem is further configured to detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, and obtain a pose detection result.

[0120] Based on the pose detection results, the target pose is determined according to the predetermined analysis method.

[0121] The predetermined analysis methods include methods B1-B2:

[0122] Method B1: If the pose detection result indicates that there is no pose deviation, the second teaching pose in the teaching process is determined as the target pose; wherein, the second teaching pose is: a pose used to ensure that the workpiece area is included within the field of view of the target acquisition device.

[0123] Method B2: If the pose detection result indicates a pose deviation, a target pose is generated according to a predetermined generation method; wherein, the predetermined generation method is a pose generation method based on the current pose of the target workpiece or the pose deviation.

[0124] Understandably, the target workpiece can be located in a defect detection station for defect detection. This station can be considered a testing workshop. A conveyor belt is installed at the bottom of the station to move the target workpiece. Photoelectric sensors are installed within the station to detect the entry of the target workpiece and send information indicating its entry to the control subsystem. Flexible motion devices are also located within the station. Furthermore, the control subsystem can detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, obtaining a pose detection result. The first taught pose is the pose for the target workpiece. Based on different pose detection results, the target pose can be determined according to different predetermined analysis methods, which can be divided into method B1 and method B2.

[0125] For method B1, which represents a situation where there is no pose deviation in the pose detection result, the second taught pose during the teaching process can be directly determined as the target pose. It can be understood that a second taught pose is matched for each predetermined point (i.e., each workpiece region), and the second taught pose can also be considered as the pose of the end effector of the flexible moving device; furthermore, the second taught pose can be considered as the pose located in the base coordinate system of the robotic arm.

[0126] Regarding method B2, which addresses the situation where the detection results indicate a pose deviation, a target pose can be generated according to a predetermined generation method. This predetermined generation method is based on the current pose or pose deviation of the target workpiece. There are various ways to determine the pose deviation between the current pose and the first taught pose of the target workpiece. For example, multiple 3D cameras can be pre-set, and image analysis can be performed on the images acquired by these cameras to determine the pose deviation. Furthermore, the current pose and the first taught pose of the target workpiece can be poses in the base coordinate system of the robotic arm or poses in the coordinate system of the 3D cameras. When a pose deviation exists between the current pose and the first taught pose of the target workpiece, the target pose can be determined based on this deviation. Under the target pose, the workpiece area is included within the field of view of the target acquisition device deployed on the flexible moving device. Additionally, the process in method B2 can also be referred to as pose compensation for the flexible moving device.

[0127] As can be seen, the control subsystem can detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, obtain the pose detection result, and determine the target pose based on the pose detection result according to a predetermined analysis method. If the pose detection result indicates that there is a pose deviation, the target pose can be generated according to a predetermined generation method. Thus, even when there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, an accurate target pose can still be generated, ensuring that the field of view of the target acquisition device deployed by the flexible moving device includes the workpiece area, thereby improving the accuracy of defect detection for the workpiece.

[0128] Optionally, in one possible embodiment, the detection system further includes: a 3D camera; the current pose of the target workpiece and the first teaching pose are both poses in the coordinate system of the 3D camera, and the second teaching pose is the pose in the base coordinate system of the flexible moving device.

[0129] Correspondingly, the control subsystem detects whether there is a pose deviation between the current pose of the target workpiece and the first taught pose, which may include steps C1-C2:

[0130] Step C1: Control the 3D camera to acquire images of the target workpiece to obtain a point cloud image acquired by the 3D camera;

[0131] Step C2: Based on the point cloud image, analyze whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process;

[0132] Accordingly, the target pose is generated according to the predetermined generation method, including step D1, or step D2:

[0133] Step D1: Obtain the relative pose relationship between the first teaching pose and the specified pose corresponding to the second teaching pose; determine the pose with a relative pose relationship to the current pose of the target workpiece in the coordinate system of the 3D camera to obtain the first pose; and map the first pose to the base coordinate system of the flexible moving device according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device to obtain the target pose; wherein, the specified pose corresponding to the second teaching pose is: the pose obtained after mapping the second teaching pose to the coordinate system of the 3D camera according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device.

[0134] Step D2: According to the pose deviation, the specified pose corresponding to the second teaching pose is offset to obtain the second pose; according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device, the second pose is mapped to the base coordinate system of the flexible moving device to obtain the target pose.

[0135] Regarding step C1, the 3D camera is a camera set at a fixed position in the defect detection station, for example: 3D cameras 1-4 are set at a fixed position in the defect detection station; the control subsystem can control the 3D camera to perform image acquisition to obtain point cloud images acquired by the 3D camera. In addition, the 3D camera can also be called a positioning camera.

[0136] Regarding step C2, the first teaching pose is the teaching pose for the workpiece. Furthermore, by acquiring point cloud images using a 3D camera, the position information of the workpiece in the point cloud image can be determined, and based on this position information, it can be analyzed whether there is a pose deviation between the current pose of the workpiece and the first teaching pose during the teaching process. There are various ways to analyze point cloud images, and this embodiment does not specifically limit this method.

[0137] Regarding step D1, step D1 involves generating a pose based on the current pose of the target workpiece. Specifically, the relative pose relationship between the first teaching pose and the specified pose corresponding to the second teaching pose can be obtained. The specified pose corresponding to the second teaching pose is obtained by mapping the second teaching pose to the coordinate system of the 3D camera according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device. Since the specified poses corresponding to the first and second teaching poses are in the same coordinate system (the coordinate system of the 3D camera), the relative pose relationship can be determined. Subsequently, the pose with a relative pose relationship to the current pose of the target workpiece in the coordinate system of the 3D camera is determined, resulting in the first pose. The first pose can be considered as the pose of the end of the flexible moving device (the end of the target acquisition device) in the coordinate system of the 3D camera. Furthermore, according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device, the first pose is mapped to the base coordinate system of the flexible moving device to obtain the target pose. In addition, the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device can be obtained by pre-calibration. There are various calibration methods, and this application embodiment does not specifically limit them.

[0138] Regarding step D2, step D2 is a pose generation method based on pose deviation. Specifically, according to the pose deviation, the specified pose corresponding to the second taught pose can be offset to obtain the second pose. The second pose is obtained by offsetting the pose obtained after mapping the second taught pose to the coordinate system of the 3D camera based on the obtained pose deviation. Furthermore, according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device, the second pose can be mapped to the base coordinate system of the flexible moving device to obtain the target pose.

[0139] As can be seen, the system in this embodiment may further include a 3D camera, control the 3D camera to acquire images to obtain point cloud images acquired by the 3D camera, and analyze whether there is a pose deviation between the current pose of the target workpiece and the first taught pose based on the pose information of the target workpiece in the point cloud image acquired by the 3D camera. Furthermore, two methods for generating the target pose are provided to ensure the accuracy of the target pose generation, thereby improving the accuracy of defect detection for the workpiece.

[0140] Optionally, in one possible embodiment, the control subsystem includes: a vision processing module;

[0141] The vision processing module is used to control the flexible moving device to move to the target pose, and after the flexible moving device moves to the target pose, control the target acquisition device to acquire images; and, based on the received target image, perform defect detection on the workpiece area to obtain the defect detection result of the workpiece area.

[0142] The control subsystem may further include:

[0143] Host computer module and / or workpiece positioning module;

[0144] The host computer module is used to determine the mapping position of the target defect in the workpiece digital model of the target workpiece based on the image position information of the target defect when the defect detection result of the workpiece area includes the defect type and image position information of the target defect, and to map the target defect to the workpiece digital model of the target workpiece according to the mapping position and the defect type.

[0145] The workpiece positioning module is used to detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, and obtain a pose detection result; based on the pose detection result, the target pose is determined according to a predetermined analysis method.

[0146] When the control subsystem includes both a vision processing module and a host computer module, such as Figure 3 As shown, the detection system for the workpiece includes: a control subsystem 100, a flexible moving device 110, a target acquisition device 120, and a target light source device 130;

[0147] The control subsystem 100 includes: a vision processing module 101 and a host computer module 102;

[0148] The vision processing module 101 is used to control the flexible moving device to move to the target pose, and after the flexible moving device moves to the target pose, control the target acquisition device to perform image acquisition; and, based on the received target image, perform defect detection on the workpiece area to obtain the defect detection result of the workpiece area.

[0149] The host computer module 102 is used to determine the mapping position of the target defect in the workpiece digital model of the target workpiece based on the image location information of the target defect when the defect detection result of the workpiece area includes the defect type and image location information of the target defect, and to map the target defect to the workpiece digital model of the target workpiece according to the mapping position and the defect type.

[0150] As can be seen, when the control subsystem includes both a vision processing module and a host computer module, the vision processing module 101 in the control subsystem 100 can control some or all of the flexible moving device 110, the target acquisition device 120, and the target light source device 130, and the vision processing module 101 can determine the defect detection result of the workpiece area based on the received target image. Furthermore, the host computer module 102 in the control subsystem 100 can map the target defect onto the workpiece digital model of the target workpiece. It should be emphasized that the specific implementation methods of the vision processing module 101 and the host computer module 102 have been described in the above embodiments and will not be repeated here.

[0151] Therefore, defect detection of the target workpiece can be performed without manual intervention. By controlling some or all of the flexible moving devices, target acquisition equipment, and target light source devices through the vision processing module in the control subsystem, defect detection can be achieved, thus improving the efficiency of workpiece defect detection. Furthermore, the host computer module can map the target defects in the defect detection results to the workpiece's digital model, providing a data foundation for defect repair.

[0152] When the control subsystem includes both a vision processing module and a workpiece positioning module, such as Figure 4 As shown, the detection system for the workpiece includes: a control subsystem 100, a flexible moving device 110, a target acquisition device 120, and a target light source device 130;

[0153] The control subsystem 100 includes: a vision processing module 101 and a workpiece positioning module 103;

[0154] The vision processing module 101 is used to control the flexible moving device to move to the target pose, and after the flexible moving device moves to the target pose, control the target acquisition device to perform image acquisition; and, based on the received target image, perform defect detection on the workpiece area to obtain the defect detection result of the workpiece area.

[0155] The workpiece positioning module 103 is used to detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, and obtain the pose detection result; according to a predetermined analysis method, the target pose is determined based on the pose detection result.

[0156] As can be seen, when the control subsystem includes both a vision processing module and a workpiece positioning module, the vision processing module 101 in the control subsystem 100 can control all or part of the flexible moving device 110, the target acquisition device 120, and the target light source device 130. Furthermore, the vision processing module 101 can determine the defect detection result of the workpiece area based on the received target image. Additionally, the workpiece positioning module 103 in the control subsystem 100 can detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose in response to the target workpiece entering the defect detection station, and obtain the pose detection result. It should be emphasized that the specific implementation methods of the vision processing module 101 and the workpiece positioning module 103 have been described in the above embodiments and will not be repeated here.

[0157] To better understand the above content, the following explanation will take the target workpiece as a car body and the flexible moving device as a robotic arm as an example, and will be based on Figures 5(a) and 5(b):

[0158] Figure 5(a) is a schematic diagram of a scene for defect detection of a white body; four target robotic arms 510 are set around the white body, four 3D cameras 540 are set in fixed positions, all four target robotic arms 510 communicate with the vision processing module 501, and the target acquisition device and target light source device at the end of the target robotic arms also communicate with the vision processing module 501, and all four 3D cameras 540 communicate with the body positioning module 503.

[0159] Figure 5(b) is a schematic diagram of the defect detection process for the body-in-white, which includes:

[0160] start;

[0161] S201, via conveyor belt, controls the white body to enter the defect inspection station;

[0162] Among them, the defect detection station can also be called a workstation or a work station.

[0163] S202, the defect detection station uses photoelectric sensors to detect that the entire body-in-white has completely entered the defect detection station;

[0164] S203, The vehicle body positioning module detects the pose deviation between the current pose of the white body and the first taught pose;

[0165] S204, the vision processing module determines the target pose of the target robotic arm, controls the end effector of the target robotic arm to move to the target pose, and sends a collection command to the target acquisition device.

[0166] S205, the vision processing module performs image fusion on the received images belonging to the predetermined points, and performs defect detection on the fused images to obtain the detection results of the predetermined points;

[0167] S206, The host computer module maps at least one defect at the predetermined point to the body model of the body-in-white based on the detection results of the predetermined point.

[0168] S207, the host computer module uploads the vehicle body digital model to the car manufacturer's MES system and displays it on a large screen;

[0169] Among them, the MES (Manufacturing Execution System) is a production information management system oriented towards the factory floor level.

[0170] Finish;

[0171] As can be seen, in this embodiment of the application, defect detection of the body-in-white does not require human intervention. By controlling the target robotic arm, target acquisition equipment, and target light source device through the vision processing module in the control subsystem, defect detection of the body-in-white can be achieved, thereby improving the efficiency of defect detection of the body-in-white.

[0172] In the technical solution of this application, the operations of acquiring, storing, using, processing, transmitting, providing and disclosing multi-frame images of the workpiece are all carried out with the user's authorization.

[0173] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0174] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0175] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0176] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A detection system for a workpiece, characterized in that, include: A control subsystem, a flexible moving device, and a target acquisition device and a target light source device respectively deployed on the flexible moving device; The target light source device is equipped with a light source component; The control subsystem is used to control the flexible moving device to move to the target pose; and, after the flexible moving device moves to the target pose, to control the target acquisition device to perform image acquisition; wherein, the target pose is: a pose that enables the workpiece area of ​​the target workpiece to be within the field of view of the target acquisition device; The flexible moving device is used to move under the control of the control subsystem; The target acquisition device is used to acquire images under the control of the control subsystem and report the acquired target images to the control subsystem. The target light source device is used to control the illumination of the light source components according to a predetermined supplementary lighting control method during the image acquisition process; wherein, the supplementary lighting control method is used to provide supplementary lighting to the workpiece area in different predetermined directions; The control subsystem is also used to perform defect detection on the workpiece area based on the received target image, and obtain the defect detection result of the workpiece area.

2. The detection system according to claim 1, characterized in that, The step of controlling the illumination of the light source component according to a predetermined supplementary lighting control method includes: Based on a pre-defined combination of light source components, the lighting control of the light source components is executed; wherein, the illumination direction after different combinations of light source components are lit is different predetermined direction.

3. The detection system according to claim 1, characterized in that, The step of performing defect detection on the workpiece area based on the received target image to obtain the defect detection result of the workpiece area includes: The received target image is fused, and the defect detection of the workpiece area is performed based on the fused image to obtain the defect detection result of the workpiece area.

4. The detection system according to claim 2, characterized in that, The supplementary lighting control method is also used to ensure that: the time when the light source component combination is lit is not later than the predetermined start time, and the time when it is turned off is not earlier than the predetermined end time; Wherein, the predetermined start time is the exposure start time of a single frame image, and the corresponding predetermined end time is the exposure end time of that single frame image; or, The predetermined start time is the exposure start time of the consecutive frames, and the corresponding predetermined end time is the exposure end time of the consecutive frames.

5. The detection system according to any one of claims 1-4, characterized in that, The target acquisition device is an area array camera with a frame rate higher than a predetermined threshold; and / or, The target light source device is either a ring-shaped light or a dome-shaped light.

6. The detection system according to any one of claims 1-4, characterized in that, The defect detection results for the workpiece area include the defect type and image location information of the target defect; The control subsystem is also used for: Based on the image location information of the target defect, the mapping position of the target defect in the workpiece digital model of the target workpiece is determined, and the target defect is mapped to the workpiece digital model of the workpiece according to the mapping position and the defect type.

7. The detection system according to any one of claims 1-4, characterized in that, The control subsystem is also used to detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, and to obtain the pose detection result; Based on the pose detection results, the target pose is determined according to a predetermined analysis method. The predetermined analysis method includes: If the pose detection result indicates that there is no pose deviation, the second teaching pose in the teaching process is determined as the target pose; wherein, the second teaching pose is: a pose that includes the workpiece region within the field of view of the target acquisition device; If the pose detection result indicates that there is a pose deviation, the target pose is generated according to a predetermined generation method; wherein, the predetermined generation method is a pose generation method based on the current pose of the target workpiece or the pose deviation.

8. The detection system according to claim 7, characterized in that, The detection system further includes: a 3D camera; the current pose of the target workpiece and the first teaching pose are both poses in the coordinate system of the 3D camera, and the second teaching pose is the pose in the base coordinate system of the flexible moving device. The control subsystem detects whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, including: The 3D camera is controlled to acquire images of the target workpiece to obtain a point cloud image acquired by the 3D camera. Based on the point cloud image, analyze whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process; The step of generating the target pose according to a predetermined generation method includes: The process involves: obtaining the relative pose relationship between the first teaching pose and the specified pose corresponding to the second teaching pose; determining the pose in the coordinate system of the 3D camera that has the relative pose relationship with the current pose of the target workpiece, thus obtaining the first pose; and mapping the first pose to the base coordinate system of the flexible moving device according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device, thus obtaining the target pose; wherein the specified pose corresponding to the second teaching pose is the pose obtained by mapping the second teaching pose to the coordinate system of the 3D camera according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device. or, According to the pose deviation, the specified pose corresponding to the second teaching pose is offset to obtain the second pose; according to the mapping relationship between the coordinate system of the 3D camera and the base coordinate system of the flexible moving device, the second pose is mapped to the base coordinate system of the flexible moving device to obtain the target pose.

9. The detection system according to claim 6, characterized in that, The control subsystem determines the mapping position of the target defect in the digital model of the target workpiece based on the image location information of the target defect, including: Based on the predetermined mapping relationship between the workpiece digital model coordinate system and the target coordinate system of the target acquisition device, the image position information of the target defect is transformed by coordinate transformation to obtain the mapping position of the target defect in the workpiece digital model of the target workpiece.

10. The detection system according to any one of claims 1-4, characterized in that, The control subsystem includes: a vision processing module; The vision processing module is used to control the flexible moving device to move to the target pose, and after the flexible moving device moves to the target pose, control the target acquisition device to acquire images; and, based on the received target image, perform defect detection on the workpiece area to obtain the defect detection result of the workpiece area. The control subsystem also includes: Host computer module and / or workpiece positioning module; The host computer module is used to determine the mapping position of the target defect in the workpiece digital model of the target workpiece based on the image position information of the target defect when the defect detection result of the workpiece area includes the defect type and image position information of the target defect, and to map the target defect to the workpiece digital model of the target workpiece according to the mapping position and the defect type. The workpiece positioning module is used to detect whether there is a pose deviation between the current pose of the target workpiece and the first taught pose during the teaching process, and obtain a pose detection result; based on the pose detection result, the target pose is determined according to a predetermined analysis method.

Images