A visual-based dynamic detection method and device for vehicle door rubber strips

Abstract

A kind of visual-based door rubber strip dynamic detection method and device, the method includes the following steps: image acquisition step: target image with door rubber strip as target object is obtained as input image;Image processing step: input image is processed, and binary image is obtained;Edge recognition step: binary image is Canny operator edge, and the edge in image is identified;Edge contour information output step: the rubber strip image of the edge extracted is compared with rubber strip template picture in rubber strip image with the edge point template matching mode of the target matched with rubber strip template picture is searched, and the edge contour information including possible defect information is output.Thereby, the dynamic detection of rubber strip can be realized, while not affecting the normal production line, with higher accuracy, the defect detection of target is realized.

Description

Technical Field

[0001] This invention relates to the field of vehicle inspection technology, and more specifically to a vision-based dynamic detection method and device for door seal strips. Background Technology

[0002] During vehicle production, defects in raw materials or installation errors by workers can lead to a series of quality problems, including defects in door sealing strips. Door sealing strips are products used to seal car doors, making them difficult to open and providing functions such as shock absorption, waterproofing, sound insulation, heat insulation, dust prevention, and fixation (hereinafter referred to as door strips or strips). Therefore, door sealing strips not only affect the appearance of the vehicle but also the safety of passengers. Traditionally, quality inspection was done manually, but subjective factors often resulted in low accuracy and efficiency. Machine vision inspection using photography is a stable and reliable method that avoids errors caused by subjectivity and fatigue. Therefore, it is necessary to use machine vision to inspect the quality of door sealing strips in the door production line, thereby improving the normal operation of the production line and increasing inspection efficiency.

[0003] Errors in car door rubber strips can be broadly categorized into breakage, defects, dents, and twisting. These errors are also common problems encountered during automated dispensing in industrial production. Existing detection methods first use image matching algorithms to locate marker points and obtain the specific position of the rubber strip, thus generating an image of the strip. Then, defect detection is performed on the rubber strip image. For example, in detecting whether the rubber strip is twisted, edge expansion, threshold segmentation, contour finding, and convex hull calculation are performed sequentially (see Figure 1).

[0004] While existing inspection methods can effectively identify errors, they often have stringent environmental requirements, such as needing to be in a closed environment and requiring static inspection. These requirements are often difficult to meet for high-speed and complex production lines. This is particularly true in car door production lines, where the door holder moves with the line; frequent line stops severely impact workshop efficiency. Furthermore, existing methods struggle to accurately measure smaller defects such as breaks and bulges, making it difficult to precisely detect defects in rubber strips. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide a vision-based dynamic detection method and apparatus for door seal strips. This method addresses some shortcomings of traditional methods by focusing on both system layout and image processing, particularly image processing, thereby improving the accuracy of seal strip quality detection while the production line is continuously moving.

[0006] According to one aspect of the present invention, a vision-based dynamic detection method for car door rubber strips is provided, comprising the following steps: an image acquisition step: acquiring a target image with the car door rubber strip as the target object as an input image; an image processing step: performing image processing on the input image to obtain a binarized image; an edge recognition step: recognizing edges in the image based on the binarized image; and an edge contour information output step: comparing the rubber strip image with the extracted edges with a rubber strip template image using an edge point template matching method that searches for a target matching the rubber strip template image in the rubber strip image, and outputting edge contour information including possible defect information. In the image acquisition step, a robotic arm equipped with a camera moves to the shooting point to take a picture of the moving car door rubber strip, thereby acquiring the target image. In the image processing step, the method includes a first image processing based on grayscale processing and Gaussian filtering, and a second image processing based on threshold segmentation and dilation-erosion operations.

[0007] According to another aspect of the present invention, a corresponding vision-based dynamic detection device for door rubber strips is provided, comprising: an image acquisition module for acquiring a target image with the door rubber strip as the target object as an input image; an image processing module for processing the input image to obtain a binarized image; an edge recognition module for recognizing edges in the image based on the binarized image; and an edge contour information output module for comparing the rubber strip image with the extracted edges with a rubber strip template image using an edge point template matching method that searches for a target matching the rubber strip template image in the rubber strip image, and outputting edge contour information including possible defect information.

[0008] According to another aspect of the present invention, an electronic device is provided, comprising: a processor for executing a plurality of instructions; and a memory for storing the plurality of instructions; wherein the plurality of instructions are stored in the memory and loaded by the processor to execute the method.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the above-described method.

[0010] This technical solution enables dynamic detection of adhesive strips, achieving high accuracy in defect detection without disrupting the production line. Furthermore, the improved image preprocessing method and the detection process combining caliper tools and template matching enhance the detection rate of erroneous samples. Attached Figure Description

[0011] Figure 1 A schematic diagram illustrating the detection flowchart for rubber strip twisting in the prior art is shown.

[0012] Figure 2 The diagram illustrates an application scenario for the detection device.

[0013] Figure 3 A flowchart illustrating the detection method is shown schematically.

[0014] Figure 4 The diagram schematically shows a partial structure of the car door, where the black strip-shaped area enclosed by the green lines within the frame represents the rubber strip.

[0015] Figure 5 A schematic diagram of the Gaussian filtering process is shown.

[0016] Figure 6 The diagram illustrates the effect comparison, where (A) shows before filtering and (B) shows after filtering.

[0017] Figure 7 The schematic diagram illustrates the simulated morphological operation process of Generative Adversarial Networks (GANs).

[0018] Figure 8 The shape of the adhesive strip after threshold segmentation is schematically shown.

[0019] Figure 9 The calculation process of the caliper tool is illustrated schematically.

[0020] Figure 10 The edge diagram after caliper calculation is shown schematically.

[0021] Figure 11 The template matching flowchart is shown schematically.

[0022] Figure 12 The test results for the correct sample are shown schematically.

[0023] Figure 13 The detection results for erroneous samples are shown schematically. Detailed Implementation

[0024] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The exemplary embodiments described below and illustrated in the drawings are intended to teach the principles of the invention, enabling those skilled in the art to implement and use the invention in various environments and for various applications. Therefore, the scope of protection of the present invention is defined by the appended claims, and the exemplary embodiments are not intended, and should not be considered, a limiting description of the scope of protection of the present invention. Furthermore, for ease of description, the dimensions of the various parts shown in the drawings are not necessarily drawn to actual scale. Orientation descriptions, such as the longitudinal direction corresponding to the length of the main body, and the orientations or positional relationships indicated by up, down, left, right, top, bottom, etc., are based on the orientations or positional relationships shown in the drawings and are only for the purpose of facilitating the description of the present invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Unless otherwise specifically stated, the order and numerical values ​​of the components and assembly steps described in the embodiments do not limit the scope of the present invention. Moreover, any numerical range stated herein is intended to include all sub-ranges contained therein, and a numerical range expressed as "numerical value A to numerical value B" refers to a range including endpoints numerical values ​​A and B. Those skilled in the art will understand that the terms "nth" and "Sn" in this invention are used only to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them. For example, steps two and three can be interchanged or performed in parallel.

[0025] To address the problem of dynamic detection, according to embodiments of the present invention, such as... Figure 2 As shown, a robotic arm carrying a camera and a light source is used. The robotic arm moves to the shooting point to take a picture by means of the signal transmitted by the sensor, thereby obtaining the target image.

[0026] Furthermore, the light source has a significant impact on the imaging of the target object (i.e., the target device). Therefore, this invention uses a coaxial light source for illumination. A coaxial light source can provide uniform lighting. During installation, it is necessary to ensure that the light source, camera, lens, and object under test are on the same straight line, i.e., the "coaxial" characteristic. This characteristic can effectively reduce the interference of stray light and diffuse reflection light from the object's surface on the final image, highlighting the defect information and edge contour information of the adhesive strip, making the imaging of defect images more accurate and complete.

[0027] As shown in Figure 3, after acquiring the image, image processing (grayscale conversion, dilation and erosion, etc.) is performed. After image preprocessing, a binary image is obtained. After edge-mapping the binary image using the Canny operator, a pre-designed caliper tool is used to detect defects in the target.

[0028] The caliper tool is primarily used to measure the width of objects, the position of edge features, and the position and spacing of edge pairs in an image. This caliper tool allows for fast and accurate detection and localization within an image's Region of Interest (ROI).

[0029] <Image Processing> Due to the special properties of light, some light spots and other noise are inevitably generated on the adhesive strip. If these are not processed, they will affect the measurement results and reduce the detection accuracy. The following are four image processing methods performed by this invention. Grayscale conversion and Gaussian filtering are applied in the first image processing step, while thresholding and dilation / erosion operations are applied in the second image processing step.

[0030] <Grayscale> After obtaining the target image, which is used as the input image, it is first converted to grayscale. Grayscale conversion is the process of converting a color image into a grayscale image. The purpose is to simplify the matrix, improve the processing speed, highlight the target area, and increase visual contrast.

[0031] <Gaussian Filter> Gaussian filtering is an image processing technique commonly used for denoising, smoothing, and other processes to improve image quality. Gaussian filtering is achieved using a matrix (convolution kernel) calculated from a Gaussian function.

[0032] Specifically, Gaussian filtering uses a normal distribution to calculate the transformation of each pixel in an image. The equation for the normal distribution in two-dimensional space is:

[0033] Where u represents the x-coordinate in the kernel matrix, v represents the y-coordinate in the kernel matrix, and σ is a position-dependent function.

[0034] In two-dimensional space, the contour lines of the surface generated by this formula are concentric circles that are normally distributed starting from the center. A convolution matrix composed of pixels with non-zero distributions is used to transform the original image. The value of each pixel is a weighted average of the values ​​of its surrounding pixels. In standard Gaussian filtering, σ is a fixed value, and the smoothness of the entire filter is generally moderate. However, in adhesive strip defect detection, different regions may require different degrees of smoothness. For example, cracked areas require a smaller σ to preserve edge details; background or irrelevant areas require a larger σ to reduce noise interference. Therefore, this design sets σ as a position-dependent function σ(x, y), as follows:

[0035] Where x represents the horizontal coordinate of the convolution kernel relative to the center point, and y represents the vertical coordinate of the convolution kernel relative to the center point.

[0036] This allows for adaptive adjustment based on the region's location and the image gradient. Adjust σ: Use a smaller σ for regions with large gradients (edges) to preserve the edges, and a larger σ for regions with small gradients (flat areas) to enhance the smoothing effect. The specific adjustment method is as follows: (1) Through (in, and The degree of pixel change is obtained by using the horizontal and vertical gradients of the pixel value, respectively.

[0037] (2) Adjust the magnitude of σ(x,y) according to the gradient. The formula is as follows:

[0038] in, To achieve the minimum smoothness, For maximum smoothness, The maximum gradient value is used for normalization.

[0039] As shown in the Gaussian filtering flowchart in Figure 5, the following can be obtained: Figure 6 The comparison diagrams shown in Figure 6 (A) represent before filtering and (B) represent after filtering. It can be seen that the improved Gaussian filtering process according to the present invention can effectively filter out noise in the image.

[0040] <Expansion corrosion, also known as opening and closing operations> Dilation and erosion algorithms are two types of image morphology algorithms, often used in combination. Erosion can eliminate noise and certain boundary values, reducing the overall image size; dilation increases the feature values ​​of the target, thus enlarging the overall image. Applying erosion followed by dilation, i.e., opening, can effectively eliminate noise and burrs in an image, significantly improving the accuracy of subsequent circle detection.

[0041] Traditional expansion and corrosion methods rely on fixed structural elements, and these methods have the following drawbacks: (1) Fixed structural element template: unable to adapt to the diverse defect forms of the adhesive strip (such as cracks and twists of different sizes and directions).

[0042] (2) Global operation: The entire image is processed uniformly, but it lacks dynamic adaptability to local features.

[0043] (3) Design relies on experience: the shape, size and operation sequence of structural elements need to be selected manually, and there is a lack of automatic optimization capabilities.

[0044] To address the aforementioned issues, a Generative Adversarial Network (GAN) deep learning model is introduced to simulate morphological operations. Through learning from data, it automatically designs dilation and erosion operations to adapt to different defect features and complex scenarios. The core idea is to treat dilation and erosion as image "generation" tasks, using a generator to simulate the effects of dilation / erosion, while a discriminator ensures that the generated image retains defect features. Specific steps are as follows... Figure 7 As shown, this deep learning model follows a process of "data preparation—model design—model training—model inference." In the data preparation stage, the original image of the adhesive strip and reference images processed by opening operations and manually selected form training samples. In the model design stage, the generator learns the mapping relationship between the original image and the denoised image, and the discriminator constrains the generated results to suppress noise and burrs while preserving the contour features of the adhesive strip defects. In the model training stage, the network parameters are iteratively updated based on the differences between the generated image and the reference image. After training, the image to be detected is input into the generator to obtain adaptive denoising results, providing an image basis for subsequent detection.

[0045] Threshold segmentation Threshold segmentation divides an image into foreground and background by setting a threshold. In the object of this invention, the adhesive strip is considered the foreground, and the other parts are considered the background. Therefore, a binarized image of the adhesive strip as the target can be obtained, for example... Figure 8 The shape of the adhesive strip after threshold segmentation is shown.

[0046] <Edge Detection> Edge detection is a technique used to identify edges (those with drastic changes in pixel values) in an image. After the image preprocessing steps mentioned above (Gaussian filtering, thresholding, etc.), noise in the image can be effectively removed and edge features enhanced. Based on this, the Canny edge detection algorithm is used, calculating the gradient magnitude and direction of the image using the Sobel operator. Regions with larger gradient magnitudes indicate drastic changes in pixel values ​​and may be edges. To ensure edge accuracy, non-maximum suppression, double thresholding, and edge concatenation are performed sequentially, thereby improving the accuracy of edge recognition. The results are as follows. Figure 4 As shown.

[0047] <Caliper tools, also known as caliper calculation> like Figure 9 As shown, the caliper tool integrates all the aforementioned image processing methods, including image preprocessing and image edge detection algorithms. Its workflow can be summarized as follows: For the input image, define the Region of Interest (ROI), extract one-dimensional information (i.e., one-dimensional data), perform filtering and noise reduction, opening and closing operations on this data, use the Canny edge detection method to identify edges in the image, calculate scores for these edges according to a preset method, and finally output edge contours that meet the score requirements. See [link / reference]. Figure 10The example shows the edge diagram calculated using calipers. The output contour is used as input to the template matching module for defect identification.

[0048] <Template comparison, also known as template matching> By comparing the extracted edge of the adhesive strip with the correct adhesive strip (template image), edge defect information, such as edge offset and breakage, is output, thereby achieving the purpose of defect detection. This invention employs an edge-based approach for template matching. Edge-based template matching algorithms typically include the following two main data processing steps: creating a template based on specific image features and searching for a matching target in the image using the template. See [link to relevant documentation]. Figure 11 The template matching flowchart is shown.

[0049] Through the above process, defects in the target object can be detected by comparison, and the detection results are as follows: Figure 12 , 13 As shown. Among them, Figure 12 , 13 These are the detection results for correct and incorrect samples, respectively.

[0050] <Dynamic Detection Device> Accordingly, an exemplary embodiment of the present invention provides a vision-based dynamic detection device for door sealing strips, comprising: Image acquisition module: Acquires a target image with the door rubber strip as the target object as the input image; Image processing module: Performs image processing on the input image to obtain a binarized image; Edge recognition module: Performs Canny operator edge-forming on the binarized image to identify edges in the image; Edge contour information output module: The edge contour information is compared with the edge strip image with the strip template image by searching for the edge point template matching target in the strip image. The output includes edge contour information including possible defect information.

[0051] In addition to the methods and devices described above, embodiments of this disclosure may also be an electronic device comprising: a processor for executing a plurality of instructions; and a memory for storing the plurality of instructions; wherein the plurality of instructions are stored in the memory and loaded by the processor to execute the steps of the method described above.

[0052] <Key points of the technical solution of this invention> This invention utilizes a robotic arm equipped with a camera light source to achieve dynamic detection. Furthermore, it employs existing image processing and image recognition algorithms to perform relatively accurate measurements of defects in the adhesive strip. The most crucial aspect lies in the image preprocessing, which includes filtering, opening and closing operations, and the use of calipers. It also combines various image processing algorithms to ensure the quality of the detected image and improve the accuracy of recognition.

[0053] <Electronic Devices> Accordingly, an exemplary embodiment of the present invention provides an electronic device that can be either or both of a first device and a second device, or a standalone device independent of them, which can communicate with the first and second devices to receive acquired input signals from them. The electronic device includes one or more processors and a memory. The processor can be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions. The memory can include one or more computer program products, which can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory can include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions can be stored on the computer-readable storage medium, and the processor can execute the program instructions to implement the methods of the software programs of the various embodiments of the present disclosure described above and / or other desired functions.

[0054] Computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above. The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0055] For those skilled in the art, the specific meanings of the terms used in this application will be understood according to the specific circumstances. Although the invention has been described with reference to various specific embodiments, it should be understood that modifications can be made within the spirit and scope of the described inventive concept. Therefore, it is intended that the invention be limited to the described embodiments but will have the full scope defined by the language of the appended claims.

Claims

1. A vision-based dynamic detection method for door seal strips, characterized in that, Includes the following steps: Image acquisition steps: Acquire the target image with the car door rubber strip as the target object as the input image; Image processing steps: Perform image processing on the input image to obtain a binarized image; Edge recognition steps: Identify edges in an image based on a binarized image; Edge contour information output steps: The extracted edge image of the adhesive strip is compared with the adhesive strip template image using a template matching method that searches for edge points in the adhesive strip image that match the adhesive strip template image. The result is edge contour information including possible defect information. In the image acquisition step, a robotic arm equipped with a camera moves to the image capture point to photograph the moving door seal, thereby acquiring the target image. The image processing steps include a first image processing step based on grayscale conversion and Gaussian filtering, and a second image processing step based on thresholding and dilation / erosion operations. In the image acquisition step, the data after Gaussian filtering and denoising is processed by opening and closing operations. A generative adversarial network is introduced to simulate a deep learning model for morphological operations. Dilation and erosion are regarded as image generation tasks. The generator is used to simulate the effects of dilation and erosion, and the discriminator ensures that the generated image retains the defect features.

2. The vision-based dynamic detection method for door seal strips according to claim 1, characterized in that, In the image acquisition step, the target image is converted to grayscale as the input image to obtain a grayscale image. The grayscale image is then subjected to Gaussian filtering. The normal distribution equation in two-dimensional space is: ， Where u represents the x-coordinate in the kernel matrix, v represents the y-coordinate in the kernel matrix, and σ is the position-dependent function σ(x, y): ， Where x represents the x-coordinate of the convolution kernel relative to the center point, and y represents the y-coordinate of the convolution kernel relative to the center point. The value of σ(x,y) is adjusted based on the image gradient. Smaller σ values ​​are used in regions with larger gradients to preserve edges, while larger σ values ​​are used in regions with smaller gradients to enhance the smoothing effect. ， in, To achieve the minimum smoothness, For maximum smoothness, The maximum gradient value is used for normalization. Among them, through To obtain the degree of pixel change, where, and These are the horizontal and vertical gradients of the pixel value, respectively.

3. The vision-based dynamic detection method for door seal strips according to claim 1, characterized in that, In the image processing step, threshold segmentation divides the image into foreground and background by setting a threshold. The tape strip is regarded as the foreground and other parts are regarded as the background. The image is then binarized into a binary image with the tape strip as the target. After Canny operator edge-trimming is applied to the binary image, a pre-designed caliper tool is used to detect defects in the target. This caliper tool is used to measure the width of the target object, the position of edge features, and the position and spacing of edge pairs in the image.

4. The vision-based dynamic detection method for door seal strips according to claim 3, characterized in that, The process includes the following steps: for the input image, define the ROI region, extract the one-dimensional information of the image, i.e., one-dimensional data, perform filtering and noise reduction, opening and closing operations on the data, use the Canny edge detection method to identify the edges in the image, calculate the scores of these edges according to a preset method, and finally output the edge contours that meet the score requirements. The output contours are used as the input for template matching to identify defects.

5. The vision-based dynamic detection method for door seal strips according to claim 1, characterized in that, The edge-point-based template matching method includes the following data processing steps: creating a template based on specific image features and searching for targets in the image that match the template.

6. The vision-based dynamic detection method for door seal strips according to claim 1, characterized in that, In the image acquisition step, a coaxial light source with the light source, camera lens, and target object aligned on the same straight line is used. The signal transmitted by the sensor causes the robotic arm carrying the camera to move to the shooting point and take a picture of the moving door seal.

7. A vision-based dynamic detection device for door sealing strips, characterized in that, The vision-based dynamic detection method for door seal strips according to any one of claims 1 to 6 includes: Image acquisition module: Acquires a target image with the door rubber strip as the target object as the input image; Image processing module: Performs image processing on the input image to obtain a binarized image; Edge recognition module: Recognizes edges in images based on binarized images; Edge contour information output module: The edge contour information is compared with the edge strip image with the strip template image by searching for the edge point template matching target in the strip image. The output includes edge contour information including possible defect information.

8. An electronic device, characterized in that, The electronic device includes: a processor for executing multiple instructions; and a memory for storing multiple instructions; wherein the multiple instructions are stored in the memory and loaded by the processor to execute the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program; the computer program is loaded and executed by a processor to implement the method of any one of claims 1 to 6.

Images