A car door quality detection system based on 3D vision fine-grained segmentation

Abstract

The application discloses the field of automobile sample quality detection, and particularly relates to a 3D vision fine-grained segmentation-based automobile door quality detection system; the detection work of the detection system comprises the following steps: S1, door handle positioning detection; S2, door to-be-inspected area extraction; S3, door surface defect detection; and S4, door surface color difference detection; the 3D machine vision automobile door quality detection system provided by the application can realize stable, continuous and reliable product detection, and can overcome the defects of easy fatigue, individual difference and poor repeatability of artificial detection, and can help automobile accessory factories to improve product quality level, improve production efficiency and reduce production cost.

Description

Technical Field

[0001] This invention relates to the field of automotive sample quality inspection, and more particularly to an automotive door quality inspection system based on 3D vision fine-grained segmentation. Background Technology

[0002] With the continuous development of machine vision technology, the automotive manufacturing industry has become one of the main application areas of machine vision. The development of intelligent industrial manufacturing has prompted many domestic and foreign automakers to focus on building intelligent automated production lines to reduce labor costs and improve production efficiency and quality. Machine vision technology, as an important component of intelligent automated production line construction, has been widely applied to all aspects of automotive manufacturing.

[0003] Automotive door painting is a crucial process in automobile production, impacting the car's appearance and the customer's visual experience. The quality of the paint directly affects the car's rust resistance and safety performance. However, defects can occur during painting, such as particles, pinholes, bubbles, and color differences on the surface. Currently, most auto parts factories inspect car doors manually using measuring tools. This method is inefficient, incurring significant labor and management costs, and it doesn't guarantee zero errors. Although machine vision-based automotive door painting is available on the market... The coating quality inspection system can identify defects such as particles, pinholes, and bubbles on the surface of automobile doors, promptly detecting defective products, improving production efficiency, and realizing intelligent production. The technology primarily involves classifying defects into categories such as particles, pinholes, bubbles, and water bubbles, all characterized by small protrusions or depressions on the paint film surface. Defect sizes are further categorized: large (>2mm), medium (1mm~2mm), small (0.5mm~1mm), and micro (<0.5mm). During inspection, the vehicle body is positioned on a conveyor belt, allowing for stationary photography. The tested surface has semi-mirror characteristics, with details exhibiting a slight orange peel texture. Figure 1 As shown, the detection system uses four high-resolution area scan cameras, mounted at the same height, forming a rectangular field of view of 1100mm × 276mm. It employs a deflection method, combined with a dedicated flat-panel display light source, to generate a set of specific images using software. These images are then displayed sequentially on the LCD screen, and the cameras take pictures in sequence. Scratches or dents reflect light at different angles in localized areas, and the deflection method can locate the corresponding area within the pattern. Finally, the defect is analyzed and its size is determined. While this method offers high accuracy in detecting surface parameters of car doors, it still relies heavily on human intervention. The four cameras make the system too expensive and structurally complex, and the detection parameters are incomplete. The level of intelligence needs improvement. Therefore, how to accurately and quickly identify defective products is a problem that urgently needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a car door quality inspection system based on 3D vision fine-grained segmentation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A vehicle door quality inspection system based on 3D vision fine-grained segmentation, the inspection operation of the system includes the following steps:

[0007] S1. Door handle positioning and detection: First, create a door handle template library for good quality doors of this model at the front end to obtain point cloud. Then, perform coarse positioning for target detection and fine positioning based on point cloud registration. Map the spatial three-dimensional coordinates of the point cloud to the planar two-dimensional coordinates to obtain the door handle area. After image point cloud recognition, comparison and judgment, use depth difference to detect defects on the surface of the door handle.

[0008] S2. Door inspection area extraction: Divide the door inspection area into three parts. Calculate the 3D point cloud of each part using the camera intrinsic parameters, the row and column coordinates of the depth map, and the depth value. Calculate the spatial three-dimensional coordinates of the point cloud. Merge the above spatial coordinates, filter out overlapping area coordinates, obtain the spatial coordinates, and calculate the point cloud P. Simultaneously, map the spatial three-dimensional coordinates of the point cloud to a two-dimensional planar coordinate to obtain the door area.

[0009] S3. Door Surface Defect Detection: The point cloud is divided into fine-grained meshes, and the curved surface is approximately decomposed into several small planes. The spatial three-dimensional coordinates and normal vectors of the point cloud are calculated. The texture map is converted to grayscale, and the spatial three-dimensional coordinates of the point cloud are mapped to the two-dimensional coordinates of the plane to obtain rows_max, cols_max, and grayscale value grays_max. The horizontal and vertical segmentation granularities are set respectively, and double looping is performed in the horizontal and vertical directions to divide the entire door area into several small regions. The spatial coordinates of the point cloud are calculated to make the small regions approximate the plane. The above point cloud is fitted to the plane to obtain the spatial normal vector. The spatial projection distance of all points to the plane is calculated. The standard deviation of each small region is calculated, and the difference between the standard deviation and the given standard value is used to determine whether it is a good or defective product.

[0010] S4. Color difference detection on the car door surface: Set the horizontal and vertical segmentation granularity respectively, and perform double-wheel circulation in the horizontal and vertical directions to divide the entire car door area into several small areas, calculate their row and column coordinates and gray values, perform plane fitting on each small area to obtain the normal vector, calculate the spatial projection distance of all points in each small area to the plane, calculate the standard deviation of each small area, and determine whether it is a good or defective product based on the difference between the standard deviation and the given standard value.

[0011] A car door quality inspection system based on 3D vision fine-grained segmentation, the system includes a robot and an image acquisition device that are communicatively connected to a motion control device. The motion control device controls the robot and the image acquisition device to perform actions. The motion control device includes an identity recognition module, an image processing module, a database module and a front-end display module.

[0012] The robot includes a robotic arm equipped with a gripping end;

[0013] The image acquisition device uses a large-format line laser for real-time image acquisition of the sample.

[0014] The identity recognition module verifies the identity of the operator through facial recognition.

[0015] The image processing module is used to identify, determine, and process images acquired and transmitted by the image acquisition device;

[0016] The database module is used to record and store the data processed by the image processing module;

[0017] The aforementioned front-end display module displays the data recorded in the database module in real time. A car door quality inspection system based on 3D vision fine-grained segmentation is in operation.

[0018] The method is characterized by the following operational steps:

[0019] A1: System Login: Operators can log in to the system after verifying their ID card through the identity verification module;

[0020] A2: Sample setup: After the operator successfully logs in to the system, they first enter the number of the car door to be tested to set up, record and read the sample, and retrieve the standard car door image from the database according to the type of car door to be monitored.

[0021] A3: System preparation: After starting the system, the robot with the image acquisition device on the gripping end is controlled by the motion control device to reach the zero position to prepare for taking pictures and capturing images.

[0022] A4: Image Acquisition: Control the robotic arm to move the image acquisition module to the car door, adjust the shooting position, and take a picture of the car door.

[0023] A5: Sample Inspection: After acquiring the car door image, the image processing module performs recognition and inspection on the image and compares it with the standard car door image to determine whether the inspected car door is qualified.

[0024] A6: Results Display: Based on the test results, the database module records and stores the data, while the front-end display module displays the test results in real time;

[0025] A7: System Reset: After the detection results are output, the robot returns to the zero position, and the system resets.

[0026] The robot includes a motion base.

[0027] The image acquisition device uses a large-format line laser imaging camera, model Gocator2670.

[0028] The output of the laser imaging camera includes a texture map and a depth map.

[0029] In step S1, after selecting the vehicle model to be inspected at the front end, the good product image of the door of that vehicle model is imported, and a door handle point cloud template and a depth map template are created and then saved locally.

[0030] In S1, the coarse localization uses the YOLOv7 target detection network.

[0031] In step S3, after performing a series of preprocessing techniques such as downsampling and smoothing filtering on the point cloud, the point cloud PC_max with the highest density under the spatial distance threshold is extracted, and the spatial three-dimensional coordinates and normal vector of the point cloud are calculated.

[0032] In S3 and S4, if the standard deviation is greater than the given standard value, the area is considered to have a defect and is regarded as a defective product; otherwise, the area is considered to be normal and is regarded as a good product.

[0033] The beneficial effects of this invention are as follows: The technical solution of this invention mainly adopts an automotive door quality inspection system that uses 3D vision imaging to acquire point cloud data of the car door. By performing sliding window and fine-grained segmentation on the point cloud data, defects such as particles, pinholes, bubbles, and color differences on the car door surface are detected, allowing for timely detection of defective products, improving production efficiency, and achieving intelligent production. The 3D machine vision automotive door quality inspection system proposed in this invention can achieve stable, continuous, and reliable product inspection, overcoming the shortcomings of manual inspection such as fatigue, individual differences, and poor repeatability. It can help auto parts factories improve product quality, increase production efficiency, and reduce production costs. More specifically:

[0034] (1) This application adopts non-contact, non-contact three-dimensional imaging detection, which can effectively avoid contact with the test piece in contact-type three-dimensional measurement;

[0035] (2) The detection speed of this application is fast, and the door appearance scanning and image processing only take about 5 seconds;

[0036] (3) The technical solution of this application can achieve a large field of view, with a scanning range of 1200*1200, which is suitable for various robot guidance needs;

[0037] (4) This application has high detection accuracy; the system imaging accuracy in the technical solution is as high as 0.05 mm, and the calculation accuracy reaches 0.1 mm. The defect detection rate is as high as 99%.

[0038] (5) The technical solution of this application is highly applicable and can be quickly and easily replicated in the fields of railways and aviation, with huge market potential. Attached Figure Description

[0039] Figure 1 This is a schematic diagram illustrating the working principle of a machine vision-based automotive door coating quality inspection system in the prior art.

[0040] Figure 2 This is a schematic diagram of the operation process of an automotive door quality inspection system based on 3D vision fine-grained segmentation proposed in this invention.

[0041] Figure 3 This is a schematic diagram of image acquisition in an embodiment of the present invention.

[0042] Figure 4 This is a working logic diagram of the detection system in an embodiment of the present invention. Detailed Implementation

[0043] The technical solution of this patent will be further described in detail below with reference to specific embodiments.

[0044] like Figure 2-4 As shown, a car door quality inspection system based on 3D vision fine-grained segmentation is disclosed. The system includes a robot and an image acquisition device that are communicatively connected to a motion control device. The motion control device controls the robot and the image acquisition device to perform actions. The motion control device includes an identity recognition module, an image processing module, a database module, and a front-end display module. The robot includes a robotic arm with a gripping end and a motion base, which can be a guide rail or wheel assembly. The robot is mounted on the motion base. Under the control of the motion control device, the motion base can drive the robot to move along a set or adjusted path. The vector and angle can be adjusted by the motion control device. Since this motion and control method is a relatively mature technology in the field, it will not be elaborated upon in this embodiment.

[0045] In this embodiment, the image acquisition device uses a large-format line laser, including but not limited to a large-format line laser imaging camera. The model of the line laser imaging camera is Gocator2670. In this embodiment, the camera line width is 1200mm, the working distance of the camera is 200mm, the Z-direction accuracy is 0.05mm, and the camera outputs a texture image and a depth image for real-time image acquisition of the sample. The output of the laser imaging camera includes a texture image and a depth image.

[0046] In this embodiment, the identity recognition module verifies the identity of the operator through facial recognition; the image processing module is used to recognize, judge, and process the images acquired and transmitted by the image acquisition device; the database module is used to record and store the data processed by the image processing module; and the front-end display module displays the data recorded in the database module in real time.

[0047] When operating this system, the operator shall follow these steps:

[0048] A1: System Login: Operators obtain system login permissions by facial recognition or card swiping. More specifically, operators can log in to the system after verifying their ID card through the identity verification module.

[0049] A2: Sample setup: After the operator successfully logs in to the system, they first enter the number of the car door to be tested to set up, record and read the sample, and retrieve the standard car door image from the database according to the type of car door to be monitored.

[0050] A3: System preparation: After starting the system, the robot with the image acquisition device on the gripping end is controlled by the motion control device to reach the zero position to prepare for taking pictures and capturing images.

[0051] A4: Image Acquisition: Control the robotic arm to move the image acquisition module to the car door, adjust the shooting position, and take a picture of the car door.

[0052] A5: Sample Inspection: After acquiring the car door image, the image processing module performs recognition and inspection on the image and compares it with the standard car door image to determine whether the inspected car door is qualified.

[0053] A6: Results Display: Based on the test results, the database module records and stores the data, while the front-end display module displays the test results in real time;

[0054] A7: System Reset: After the detection results are output, the robot returns to the zero position, and the system resets.

[0055] The automotive door quality inspection system based on 3D vision fine-grained segmentation of this invention uses a large-format line laser for image acquisition when inspecting the door. The imaging is as follows: Figure 3 As shown, P0, P1, and P2 are the detection areas, and the specific steps include:

[0056] S1. Door Handle Localization and Detection: First, create a door handle template library for good quality doors of this vehicle model on the front end. Specifically, select the vehicle model to be inspected on the front end, import the good quality door image for that model, click the "Create Door Handle" button, draw the ROI region, and the template creation algorithm will automatically search for the door handle outline and create a door handle point cloud template PC_dh_mod and a depth map template Dep_dh_mod, which are then saved locally in PLY format. Selecting the one-click template export function will automatically load the template path into the algorithm software configuration file.

[0057] Next, we perform coarse localization for object detection: due to the salient features of the car door handle, we use the YOLOv7 object detection network, which can train a model with high accuracy and recall with only a small number of samples.

[0058] The location information obtained from the coarse localization of the target detection is appropriately expanded using pixels, and the expanded area is simultaneously mapped to both the texture map and the depth map. This texture area is cropped as the second image, and the aforementioned depth area is cropped as the third image. The 3D point cloud PC_dh of the third image is calculated using intrinsic parameters, the row and column coordinates of the third image, and the depth value. The spatial 3D coordinates CorX_dh, CorY_dh, and CorZ_dh of this point cloud are then calculated. The formula for calculating the point cloud spatial coordinates is:

[0059] CorX_dh=Grayvals*(Cols-x0) / fx

[0060] CorY_dh=Grayval s*(Rows-y0) / fy

[0061] CorZ_dh=Grayvals

[0062] Where fx, fy, x0, and y0 are camera intrinsic parameters, and Rows, Cols, and Grayvals are the row and column coordinates and depth values ​​corresponding to the depth map, respectively.

[0063] Next, target detection and precise localization are performed based on point cloud registration: The point cloud template PC_dh_mod created above is loaded, and the door handle is precisely located using the ICP point cloud registration algorithm to obtain the matched point cloud PC_dh_cur. The spatial 3D coordinates CorX_dh_cur, CorY_dh_cur, and CorZ_dh_cur of this point cloud are then calculated. The spatial 3D coordinates of the point cloud PC_dh_cur are mapped to planar 2D coordinates to obtain the door handle region Dep_dh_cur. The specific formula is as follows:

[0064] Rows = ycor * fy / zcor + y0

[0065] Cols = xcor * fx / zcor + x0

[0066] Where fx, fy, x0, and y0 are camera intrinsic parameters, xcor, ycor, and zcor are the spatial XYZ coordinates, and Rows and Cols are the row and column coordinates corresponding to the depth map. Dep_dh_cur and Dep_dh_mod depth differences are used for detecting surface defects on the car door handle.

[0067] S2. Extraction of the door inspection area: (e.g., ...) Figure 3 As shown, the area to be inspected on the car door is divided into three parts: P0, P1, and P2. Each region has an overlap of 20 pixels. The 3D point clouds PC_0, PC_1, and PC_2 of P0, P1, and P2 are calculated using camera intrinsic parameters, the row and column coordinates of the depth map, and the depth value. The spatial 3D coordinates of these point clouds are then calculated: CorX_0, CorY_0, CorZ_0; CorX_1, CorY_1, CorZ_1; CorX_2, CorY_2, CorZ_2. The door handle point cloud PC_dh_cur from step one is filtered out from point cloud PC_0, resulting in point cloud PC_0 and its spatial coordinates CorX_0, CorY_0, and CorZ_0.

[0068] Depend on Figure 3 As can be seen, P0 includes a portion of the window area, so this area needs to be filtered out to avoid false alarms. The specific method is as follows: scan line by line in the row direction of the depth map, mark the endpoints at obvious depth breaks, extract the boundary line between the door and the window at P0, scan column by column in the col direction of the depth map, and filter out points in the row direction of the depth map that are smaller than the marked endpoints, finally obtaining the effective point cloud CorX_0, CorY_0, and CorZ_0 at P0.

[0069] Merge the above spatial coordinates CorX_0, CorY_0, CorZ_0; CorX_1, CorY_1, CorZ_1; CorX_2, CorY_2, CorZ_2, filter out overlapping area coordinates to obtain spatial coordinates CorX, CorY, CorZ, and calculate point cloud PC. At the same time, map the spatial three-dimensional coordinates of point cloud PC to planar two-dimensional coordinates to obtain the door region.

[0070] S3. Door Surface Defect Detection: Since the door is a three-dimensional curved surface, the point cloud PC needs to be finely meshed to approximate the curved surface into several small planes. The specific process is as follows:

[0071] Step 1: Perform a series of preprocessing techniques such as downsampling and smoothing filtering on the point cloud PC to extract the point cloud PC_max with the highest density under the spatial distance threshold, and calculate the spatial three-dimensional coordinates CorX_max, CorY_max, CorZ_max and normal vector of the point cloud.

[0072] NorX_max, NorY_max, NorZ_max.

[0073] Step 2: First, convert the TextureImage to grayscale. Map the spatial 3D coordinates of the point cloud PC_max to planar 2D coordinates to obtain rows_max, cols_max, and the grayscale value grays_max. The formulas are described above.

[0074] Step 3: Set the horizontal segmentation granularity div_x to 20 pixels, calculate the minimum and maximum values ​​of CorX_max, which are min(CorX_max) and max(CorX_max), respectively. Then, the number of horizontal segmentation steps num_x = int(ceil((max(CorX_max)-min(CorX_max))*1.0 / div_x)). Similarly, set the vertical segmentation granularity div_y to 15 pixels, calculate the minimum and maximum values ​​of CorY_max, which are min(CorY_max) and max(CorY_max), respectively. Then, the number of horizontal segmentation steps num_y = int(ceil((max(CorY_max)-min(CorY_max))*1.0 / div_y)).

[0075] Step 4: Perform a dual-cycle rotation in both the horizontal and vertical directions to divide the entire door area into several smaller regions (Region_seg), and calculate the spatial coordinates (cor_x_seg, cor_y_seg, cor_z_seg) of their point cloud (PC_seg). The smaller the Region_seg, the closer it is to a plane.

[0076] Step 5: Perform plane fitting on the point cloud PC_seg to obtain the point cloud PC_seg_plane, and obtain the spatial normal vectors A, B, C, and D. Calculate the spatial projection distance from all points within Region_seg to the plane PC_seg_plane. The formula for calculating the spatial projection distance is: Projections = cor_x_seg * A + cor_y_seg * B + cor_z_seg * C.

[0077] Step 6: Calculate the standard deviation of Region_seg. The standard deviation reflects the dispersion of a dataset and can be used as a quantitative basis for whether the region has defects. The formula is: dev_depth = sqrt(sum((Projections-D)*(Projections-D)) / |Projections|), where |Projections| represents the modulus of the point cloud, i.e., the number of points.

[0078] Finally, the system makes a judgment. If dev_depth is greater than the given standard value std_depth, the region is considered to have a depth defect and the location information is output as good. Otherwise, the region is normal.

[0079] S4. Door Surface Color Difference Detection: The detection principle for door surface color difference detection is the same as that for surface defect detection in step S3 above, except that the depth map is replaced with a texture map, and the depth value is replaced with a grayscale value. The specific process is as follows:

[0080] Step 1: Set the horizontal segmentation granularity div_x to 20 pixels, calculate the minimum and maximum values ​​of rows_max, which are min(rows_max) and max(rows_max) respectively. Then, the number of horizontal segmentation steps num_x = int(ceil((max(rows_max)-min(rows_max))*1.0 / div_x)). Similarly, set the vertical segmentation granularity div_y to 15 pixels, calculate the minimum and maximum values ​​of cols_max, which are min(cols_max) and max(cols_max) respectively. Then, the number of horizontal segmentation steps num_y = int(ceil((max(cols_max)-min(cols_max))*1.0 / div_y)).

[0081] Step 2: Perform a double-wheel loop in both the horizontal and vertical directions to divide the entire door area into several smaller regions, Region_seg, and calculate their row and column coordinates and grayscale values, namely row_seg, col_seg, and gray_seg, respectively.

[0082] Step 3: Perform plane fitting on the above Region_seg to obtain Region_seg_plane, and obtain normal vectors A, B, C, and D. Calculate the spatial projection distance from all points within Region_seg to the plane Region_seg_plane. The formula for calculating the spatial projection distance is: Projections_gray = row_seg * A + col_seg * B + gray_seg * C.

[0083] Step 4: Calculate the standard deviation of Region_seg. The standard deviation reflects the dispersion of a dataset and can be used as a quantitative basis for whether a region has defects. The formula is: dev_gray=sqrt(sum((Projections_gray-D)*(Projections_gray-D)) / |Projections_gray|), where |Projections_gray| represents the modulus of the pixels, i.e., the number of pixels.

[0084] Step 5: The system makes a judgment. If dev_gray is greater than the given standard value std_gray, it is considered that there is a color difference defect in the area and the location information is output as good. Otherwise, the area is normal.

[0085] The 3D machine vision-based automotive door quality inspection system in this application enables stable, continuous, and reliable product inspection, overcoming the drawbacks of manual inspection such as fatigue, individual differences, and poor repeatability. It helps auto parts factories improve product quality, increase production efficiency, and reduce production costs. Currently, the industrial sector primarily relies on target detection algorithms or semantic segmentation algorithms based on big data and a large number of negative samples. However, in reality, it is difficult to provide a large number of negative samples in certain scenarios. Even with data augmentation or adversarial networks to expand the negative sample count, the model's generalization ability remains poor. This patented automotive door quality inspection system eliminates the reliance on big data and negative samples. It reconstructs the 3D curved surface point cloud of the door using 3D vision and employs fine-grained mesh segmentation to approximately decompose the 3D curved surface into several small planes. By analyzing and calculating the standard deviation from the point cloud to the planes, it determines whether there are physical defects such as particles, pinholes, bubbles, and color differences on the door surface, enabling timely detection of defective products, improving production efficiency, and achieving intelligent production. Furthermore, this system can be quickly and seamlessly replicated in fields such as railways and aviation, demonstrating strong versatility.

[0086] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A vehicle door quality inspection system based on 3D vision fine-grained segmentation, characterized in that: The detection system described above includes the following steps: S1. Door handle positioning and detection: First, a door handle template library of good quality doors of a certain car model is created at the front end to obtain point cloud. Then, coarse positioning of the target is performed, and fine positioning is performed based on the point cloud registration. The spatial three-dimensional coordinates of the point cloud are mapped to the planar two-dimensional coordinates to obtain the door handle area. After selecting the car model to be inspected at the front end, the good quality door image of that car model is imported, and a door handle point cloud template and a depth map template are created and then saved locally. After image point cloud recognition, comparison and judgment, depth difference is used to detect surface defects of the door handle. S2. Door inspection area extraction: Divide the door inspection area into three parts. Calculate the 3D point cloud of each part using the camera intrinsic parameters, the row and column coordinates of the depth map, and the depth value. Calculate the spatial three-dimensional coordinates of the point cloud. Merge the spatial three-dimensional coordinates of the point cloud, filter out the coordinates of overlapping areas, obtain the spatial coordinates, and calculate the point cloud P. At the same time, map the spatial three-dimensional coordinates of the point cloud to a two-dimensional plane coordinate to obtain the door area. S3. Door Surface Defect Detection: The point cloud is segmented into a fine-grained mesh, and the curved surface is approximately decomposed into several small planes. The spatial three-dimensional coordinates and normal vector of the point cloud are calculated. The texture map is converted to grayscale, and the spatial three-dimensional coordinates of the point cloud are mapped to the two-dimensional coordinates of the plane to obtain rows_max, cols_max, and grayscale value grays_max. The horizontal and vertical segmentation granularity are set respectively, and double looping is performed in the horizontal and vertical directions to divide the entire door area into several small regions. The spatial coordinates of the point cloud are calculated to make the small regions approximate the plane. The above point cloud is fitted to the plane to obtain the spatial normal vector. The spatial projection distance of all points to the plane is calculated. The standard deviation of each small region is calculated, and the difference between the standard deviation and the given standard value is used to determine whether it is a good or defective product. S4. Color difference detection on the car door surface: Set the horizontal and vertical segmentation granularity respectively, and perform double-wheel circulation in the horizontal and vertical directions to divide the entire car door area into several small areas, calculate their row and column coordinates and gray values, perform plane fitting on each small area to obtain the normal vector, calculate the spatial projection distance of all points in each small area to the plane, calculate the standard deviation of each small area, and determine whether it is a good or defective product based on the difference between the standard deviation and the given standard value.

2. When operating the automotive door quality inspection system based on 3D vision fine-grained segmentation as described in claim 1, the system is characterized in that: The following steps are included: A1: System Login: Operators can log in to the system after verifying their ID card through the identity verification module; A2: Sample setup: After the operator successfully logs in to the system, they first enter the number of the car door to be tested to set up, record and read the sample, and retrieve the standard car door image from the database according to the type of car door to be monitored. A3: System preparation: After starting the system, the robot with the image acquisition device on the gripping end is controlled by the motion control device to reach the zero position to prepare for taking pictures and capturing images. A4: Image Acquisition: Control the robotic arm to move the image acquisition module to the car door, adjust the shooting position, and take a picture of the car door. A5: Sample Inspection: After acquiring the car door image, the image processing module performs recognition and inspection on the image and compares it with the standard car door image to determine whether the inspected car door is qualified. A6: Results Display: Based on the test results, the database module records and stores the data, while the front-end display module displays the test results in real time; A7: System Reset: After the detection results are output, the robot returns to the zero position, and the system resets.

3. The automotive door quality inspection system based on 3D vision fine-grained segmentation according to claim 2, characterized in that: The robot includes a motion base.

4. The automotive door quality inspection system based on 3D vision fine-grained segmentation according to claim 2, characterized in that: The image acquisition device uses a large-format line laser imaging camera, model Gocator2670.

5. The automotive door quality inspection system based on 3D vision fine-grained segmentation according to claim 4, characterized in that, The output of the line laser imaging camera includes a texture map and a depth map.

6. The automotive door quality inspection system based on 3D vision fine-grained segmentation according to claim 1, characterized in that, In S1, the coarse localization uses the YOLOv7 target detection network.

7. The automotive door quality inspection system based on 3D vision fine-grained segmentation according to claim 1, characterized in that, In step S3, after performing a series of downsampling and smoothing filtering preprocessing techniques on the point cloud, the point cloud PC_max with the highest density under the spatial distance threshold is extracted, and the spatial three-dimensional coordinates and normal vector of the point cloud are calculated.

8. The automotive door quality inspection system based on 3D vision fine-grained segmentation according to claim 1, characterized in that, In S3 and S4, if the standard deviation is greater than the given standard value, the area is considered to have a defect and is regarded as a defective product; otherwise, the area is considered to be normal and is regarded as a good product.

Images