Vehicle frame quality detection method and system based on computer vision

Abstract

The invention discloses a frame quality detection method and system based on computer vision, and the method comprises the steps: setting two positioning points at a preset position of a standard frame, and storing a front view image of the standard frame as a reference image; two positioning points are arranged at preset positions of a to-be-detected frame, and a front view image of the to-be-detected frame is acquired through image acquisition equipment to serve as a to-be-detected image; performing semantic segmentation on the reference image and the to-be-detected image to generate a reference binary image and a to-be-detected binary image; performing exhaustion type combination transformation on a to-be-detected frame entity in the to-be-detected binary image until the two positioning points of the to-be-detected binary image coincide with the two positioning points of the reference binary image; the converted binary image to be detected is compared with the reference binary image to generate a frame error index, whether the quality of the frame to be detected is qualified or not is judged, and automatic computer vision detection on the production quality of the whole vehicle frame is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision detection, and particularly to a method and system for detecting the quality of a vehicle frame based on computer vision. Background Art

[0002] In the process of electric tricycle production, information technology and automation technology are often used to improve production efficiency and product qualification rate. In this process, it is a common technical means to use computer vision technology to detect the production quality of the whole vehicle frame, so as to verify whether the actual dimensions, shapes, position tolerances, etc. of the components meet the design specifications.

[0003] However, in the actual production line, the placement positions of the vehicle frames to be detected are usually not completely consistent, which makes it difficult for ordinary detection methods to conduct unified automatic detection of the production quality of the whole vehicle frame. In addition, the structure of the whole vehicle frame is complex, and it is difficult for ordinary image segmentation models to accurately segment the edges between the vehicle frame entity and the background in the whole vehicle frame image, resulting in insufficient accuracy of the detection method. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for detecting the quality of a vehicle frame based on computer vision, aiming to solve the problems that in the actual production line, the placement positions of the vehicle frames to be detected are usually not completely consistent, which makes it difficult for ordinary detection methods to conduct unified automatic detection of the production quality of the whole vehicle frame, and the structure of the whole vehicle frame is complex, and it is difficult for ordinary image segmentation models to accurately segment the edges between the vehicle frame entity and the background in the whole vehicle frame image, resulting in insufficient accuracy of the detection method.

[0005] In view of the above problems, the present application provides a method and system for detecting the quality of a vehicle frame based on computer vision.

[0006] In the first aspect disclosed by the present application, a method for detecting the quality of a vehicle frame based on computer vision is provided, and the method includes: Set two positioning points at preset positions on the standard vehicle frame, and store the front view image of the standard vehicle frame as a reference image; Set two positioning points at preset positions on the vehicle frame to be detected, and obtain the front view image of the vehicle frame to be detected through an image acquisition device as a to-be-detected image; Construct an HRNet image segmentation model, perform semantic segmentation on the reference image and the to-be-detected image, segment out the standard vehicle frame entity and the to-be-detected vehicle frame entity in the reference image and the to-be-detected image, and generate a reference binary image and a to-be-detected binary image; Perform an exhaustive combination transformation of scaling, rotation, and translation on the vehicle frame entity to be detected in the binary image to be detected until two positioning points on the vehicle frame entity to be detected in the binary image to be detected coincide with two positioning points on the standard vehicle frame entity in the reference binary image, generating the transformed binary image to be detected; Perform an exclusive OR operation on the transformed binary image to be detected and the reference binary image to generate a deviation binary image, calculate the ratio of non-zero pixels in the deviation binary image and the reference binary image, and generate a vehicle frame error index; Determine whether the quality of the vehicle frame to be detected is qualified according to whether the vehicle frame error index exceeds a preset threshold.

[0007] Preferably, the construction of the HRNet image segmentation model for semantic segmentation of the reference image and the image to be detected specifically includes: Input the reference image and the image to be detected into the neck network for downsampling respectively, and sequentially generate an initial reference feature map and an initial image-to-be-detected feature map; Input the initial reference feature map and the initial image-to-be-detected feature map into the first-stage network for downsampling respectively, and sequentially generate a first reference feature map and a first image-to-be-detected feature map; Input the first reference feature map and the first image-to-be-detected feature map into the second-stage network for downsampling respectively, and sequentially generate a second reference feature map and a second image-to-be-detected feature map; Input the second reference feature map and the second image-to-be-detected feature map into the third-stage network for downsampling respectively, and sequentially generate a third reference feature map and a third image-to-be-detected feature map; Input the third reference feature map and the third image-to-be-detected feature map into the fourth-stage network for downsampling respectively, and sequentially generate a fourth reference feature map and a fourth image-to-be-detected feature map; Fuse the features of the fourth reference feature map and the fourth image-to-be-detected feature map respectively, and generate the class probability of each pixel of the reference image and the image to be detected through 1×1 convolution calculation and activation function calculation, generating the semantic segmentation result.

[0008] Preferably, the resolutions of the initial reference feature map and the initial image-to-be-detected feature map are 1 / 2 of the reference image and the image to be detected, the resolutions of the first reference feature map and the first image-to-be-detected feature map are 1 / 2 of the reference image and the image to be detected, the second reference feature map and the second image-to-be-detected feature map each contain two branches, and the resolutions of the two branches are 1 / 2 and 1 / 4 of the reference image and the image to be detected respectively. The third reference feature map and the third image-to-be-detected feature map each contain three branches, and the resolutions of the three branches are 1 / 2, 1 / 4, and 1 / 8 of the reference image and the image to be detected respectively. The fourth reference feature map and the fourth image-to-be-detected feature map each contain four branches, and the resolutions of the four branches are 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the reference image and the image to be detected respectively.

[0009] Preferably, perform an exhaustive combination transformation of scaling, rotation, and translation on the frame entity to be detected in the binary image to be detected until two positioning points on the frame entity to be detected in the binary image to be detected coincide with two positioning points on the standard frame entity in the reference binary image. Specifically, it includes: Based on the binary image to be detected and the reference binary image, establish a rectangular coordinate system in the plane respectively; Obtain the coordinate positions of two positioning points on the standard frame entity in the reference binary image, and obtain the coordinate positions of two positioning points on the frame entity to be detected in the binary image to be detected; Use a homogeneous transformation matrix to perform an exhaustive combination transformation of scaling, rotation, and translation on the frame entity to be detected in the binary image to be detected. The coordinate positions of the two positioning points on the frame entity to be detected in the binary image to be detected will also undergo corresponding transformations until the transformed coordinate positions are the same as the coordinate positions of the two positioning points on the standard frame entity in the reference binary image.

[0010] In the second aspect disclosed in this application, a frame quality detection system based on computer vision is provided. The system is used for the above-mentioned frame quality detection method based on computer vision. The system includes: A reference image module, which is used to set two positioning points at preset positions on the standard frame and store the front view image of the standard frame as a reference image; An image to be detected module, which is used to set two positioning points at preset positions on the frame to be detected, and obtain the front view image of the frame to be detected through an image acquisition device as the image to be detected; An image segmentation module, which is used to construct an HRNet image segmentation model, perform semantic segmentation on the reference image and the image to be detected, segment out the standard frame entity and the frame entity to be detected in the reference image and the image to be detected, and generate a reference binary image and an image binary image to be detected; A combination transformation module, which is used to perform an exhaustive combination transformation of scaling, rotation, and translation on the frame entity to be detected in the binary image to be detected until two positioning points on the frame entity to be detected in the binary image to be detected coincide with two positioning points on the standard frame entity in the reference binary image, and generate a transformed binary image to be detected; A logical operation module, which is used to perform an exclusive OR operation on the transformed binary image to be detected and the reference binary image, generate a deviation binary image, and calculate the ratio of the non-zero pixels in the deviation binary image to the reference binary image to generate a frame error index; A determination module, which is used to determine whether the quality of the frame to be detected is qualified according to whether the frame error index exceeds a preset threshold.

[0011] The third aspect disclosed in this application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned frame quality detection method based on computer vision are implemented.

[0012] The fourth aspect disclosed in this application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned frame quality detection method based on computer vision are implemented.

[0013] The fifth aspect disclosed in this application provides a computer program product, including a computer program or instruction. When the computer program or instruction is executed by a processor, the steps of the above-mentioned frame quality detection method based on computer vision are implemented.

[0014] The beneficial effects of the present invention are as follows: (1) By adopting the method of setting positioning points and exhaustive combination transformation, the quality detection of the frame to be detected at any placement position in the production line is realized, and the production quality of the whole vehicle frame can be automatically detected by computer vision. (2) Through multiple feature extractions by the multi-stage downsampling operation of the HRNet model and retaining the high-resolution details of the image through the multi-branch structure of the feature map, the accurate semantic segmentation of the whole vehicle frame with complex structure is realized, and the accuracy of the detection method is improved. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0016] Figure 1 It is the overall flowchart of the frame quality detection method based on computer vision.

[0017] Figure 2 It is the overall structure diagram of the frame quality detection system based on computer vision. Detailed Embodiments

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] Embodiment 1: As Figure 1 shown, the embodiment of the present application provides a frame quality detection method based on computer vision, and the method includes: Step 1, set two positioning points at preset positions on the standard frame, and store the front view image of the standard frame as a reference image.

[0020] Step 2, set two positioning points at preset positions on the frame to be detected, and obtain the front view image of the frame to be detected through an image acquisition device as an image to be detected.

[0021] Step 3, construct an HRNet image segmentation model, and the HRNet image segmentation model sequentially includes a neck network, a first-stage network, a second-stage network, a third-stage network, and a fourth-stage network; Input the reference image and the image to be detected into the neck network for downsampling respectively, and generate an initial reference feature map and an initial image-to-be-detected feature map in sequence; Input the initial reference feature map and the initial image-to-be-detected feature map into the first-stage network for downsampling respectively, and generate a first reference feature map and a first image-to-be-detected feature map in sequence; Input the first reference feature map and the first image-to-be-detected feature map into the second-stage network for downsampling respectively, and generate a second reference feature map and a second image-to-be-detected feature map in sequence; Input the second reference feature map and the second image-to-be-detected feature map into the third-stage network for downsampling respectively, and generate a third reference feature map and a third image-to-be-detected feature map in sequence; Input the third reference feature map and the third image-to-be-detected feature map into the fourth-stage network for downsampling respectively, and generate a fourth reference feature map and a fourth image-to-be-detected feature map in sequence; Perform feature fusion on the fourth reference feature map and the fourth image-to-be-detected feature map respectively, and generate the class probability of each pixel of the reference image and the image to be detected through 1×1 convolution calculation and activation function calculation, perform semantic segmentation on the reference image and the image to be detected, segment out the standard frame entity and the frame entity to be detected in the reference image and the image to be detected, and generate a reference binary image and an image-to-be-detected binary image.

[0022] Among them, the resolutions of the initial reference feature map and the initial feature map to be detected are 1 / 2 of the reference image and the image to be detected. The resolutions of the first reference feature map and the first feature map to be detected are 1 / 2 of the reference image and the image to be detected. The second reference feature map and the second feature map to be detected each contain two branches, and the resolutions of the two branches are 1 / 2 and 1 / 4 of the reference image and the image to be detected respectively. The third reference feature map and the third feature map to be detected each contain three branches, and the resolutions of the three branches are 1 / 2, 1 / 4, and 1 / 8 of the reference image and the image to be detected respectively. The fourth reference feature map and the fourth feature map to be detected each contain four branches, and the resolutions of the four branches are 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the reference image and the image to be detected respectively.

[0023] Step 4: Perform an exhaustive combination transformation of scaling, rotation, and translation on the vehicle frame entity to be detected in the binary image to be detected until the two positioning points on the vehicle frame entity to be detected in the binary image to be detected coincide with the two positioning points on the standard vehicle frame entity in the reference binary image, generating the transformed binary image to be detected.

[0024] The specific steps include: Step 4.1: Respectively establish a plane rectangular coordinate system with the binary image to be detected and the reference binary image as the benchmarks. Step 4.2: Obtain the coordinate positions of the two positioning points on the standard vehicle frame entity in the reference binary image, and obtain the coordinate positions of the two positioning points on the vehicle frame entity to be detected in the binary image to be detected. Step 4.3: Perform an exhaustive combination transformation of scaling, rotation, and translation on the vehicle frame entity to be detected in the binary image to be detected using a homogeneous transformation matrix. The coordinate positions of the two positioning points on the vehicle frame entity to be detected in the binary image to be detected also undergo corresponding transformations until the transformed coordinate positions are the same as the coordinate positions of the two positioning points on the standard vehicle frame entity in the reference binary image.

[0025] Step 5: Perform an exclusive OR operation on the transformed binary image to be detected and the reference binary image to generate a deviation binary image, and calculate the ratio of the non-zero pixels in the deviation binary image and the reference binary image to generate a vehicle frame error index.

[0026] Step 6: Determine whether the quality of the vehicle frame to be detected is qualified according to whether the vehicle frame error index exceeds a preset threshold.

[0027] In addition to the above steps, it is also necessary to perform model training on the HRNet image segmentation model. The training steps include: Obtain the images of the electric tricycle frame, normalize all the images, unify the input size, and then perform image data augmentation by rotation and flipping to alleviate the problem of insufficient image data volume. Finally, perform manual annotation, convert the annotation into the form of One-Hot encoding or binary mask, establish a dataset, and divide the training set, validation set, and test set according to the ratio of 8:1:1; Use Dice Loss to optimize the overlapping area, and use weighted cross-entropy loss to alleviate the problem of frame-background pixel imbalance. At the same time, use the AdamW optimizer (learning rate 3e-4), and use the training set to train the HRNet image segmentation model. At the same time, use the validation set to verify the accuracy, and use IoU and Recall as indicators to represent the accuracy; Adopt the early stopping method. If the accuracy on the validation set does not increase for 10 consecutive epochs, stop the model training, and then use the test set to test the HRNet image segmentation model.

[0028] In summary, the method for detecting the quality of the frame based on computer vision provided by the embodiments of the present application has the following technical effects: (1) By using the method of setting positioning points and exhaustive combination transformation, the quality detection of the frame to be detected at any placement position in the production line is realized, and the production quality of the whole vehicle frame can be automatically detected by computer vision; (2) Through multiple feature extractions by the multi-stage downsampling operation of the HRNet model, and by retaining the high-resolution details of the image through the multi-branch structure of the feature map, the accurate semantic segmentation of the whole vehicle frame with complex structure is realized, and the accuracy of the detection method is improved.

[0029] Embodiment 2: Based on the same inventive concept as the method for detecting the quality of the frame based on computer vision in Embodiment 1, as Figure 2 shown, the present application provides a system for detecting the quality of the frame based on computer vision, and the system includes: A reference image module, which is used to set two positioning points at preset positions of the standard frame and store the front view image of the standard frame as a reference image; An image to be detected module, which is used to set two positioning points at preset positions of the frame to be detected, and obtain the front view image of the frame to be detected through an image acquisition device as an image to be detected; An image segmentation module, which is used to construct an HRNet image segmentation model, perform semantic segmentation on the reference image and the image to be detected, segment out the standard frame entity and the frame entity to be detected in the reference image and the image to be detected, and generate a reference binary image and an image binary image to be detected; Combination transformation module, which is used to perform exhaustive scaling, rotation, and translation combination transformation on the frame entity to be detected in the binary image to be detected until two positioning points on the frame entity to be detected in the binary image to be detected coincide with two positioning points on the standard frame entity in the reference binary image, and generate the transformed binary image to be detected; Logical operation module, which is used to perform exclusive OR operation on the transformed binary image to be detected and the reference binary image, generate a deviation binary image, calculate the ratio of the non-zero pixels in the deviation binary image and the reference binary image, and generate a frame error index; Determination module, which is used to determine whether the quality of the frame to be detected is qualified according to whether the frame error index exceeds a preset threshold.

[0030] Through the foregoing detailed description of the frame quality detection method based on computer vision in this specification, those skilled in the art can clearly know that the frame quality detection system based on computer vision in this embodiment, because it corresponds to the method disclosed in the embodiment, is described relatively simply. For related parts, refer to the description in the method part.

[0031] Embodiment 3: In Embodiment 3, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps of the above-mentioned frame quality detection method based on computer vision are implemented.

[0032] Embodiment 4: In Embodiment 4, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned frame quality detection method based on computer vision are implemented.

[0033] Embodiment 5: In Embodiment 5, a computer program product is provided, including a computer program or instruction. When the computer program or instruction is executed by a processor, the steps of the above-mentioned frame quality detection method based on computer vision are implemented.

[0034] The technical features of the above embodiments can be combined arbitrarily. For the sake of brief description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0035] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A frame quality detection method based on computer vision, characterized in that, The method includes: Set two positioning points at preset positions on the standard vehicle frame, and store the front view image of the standard vehicle frame as a reference image; Set two positioning points at preset positions on the vehicle frame to be detected, and obtain the front view image of the vehicle frame to be detected through an image acquisition device as the image to be detected; Construct an HRNet image segmentation model, perform semantic segmentation on the reference image and the image to be detected, segment the standard vehicle frame entity and the vehicle frame entity to be detected in the reference image and the image to be detected, and generate a reference binary image and an image to be detected binary image; Perform an exhaustive combination transformation of scaling, rotation, and translation on the vehicle frame entity to be detected in the image to be detected binary image until the two positioning points on the vehicle frame entity to be detected in the image to be detected binary image coincide with the two positioning points on the standard vehicle frame entity in the reference binary image, and generate a transformed image to be detected binary image; Perform an exclusive OR operation on the transformed image to be detected binary image and the reference binary image to generate a deviation binary image, calculate the ratio of the non-zero pixels in the deviation binary image and the reference binary image, and generate a vehicle frame error index; Determine whether the quality of the vehicle frame to be detected is qualified according to whether the vehicle frame error index exceeds a preset threshold.

2. The computer vision-based vehicle frame quality detection method according to claim 1, wherein, The constructing the HRNet image segmentation model and performing semantic segmentation on the reference image and the image to be detected specifically includes: Input the reference image and the image to be detected into the neck network for downsampling respectively, and sequentially generate an initial reference feature map and an initial image to be detected feature map; Input the initial reference feature map and the initial image to be detected feature map into the first stage network for downsampling respectively, and sequentially generate a first reference feature map and a first image to be detected feature map; Input the first reference feature map and the first image to be detected feature map into the second stage network for downsampling respectively, and sequentially generate a second reference feature map and a second image to be detected feature map; Input the second reference feature map and the second image to be detected feature map into the third stage network for downsampling respectively, and sequentially generate a third reference feature map and a third image to be detected feature map; Input the third reference feature map and the third image to be detected feature map into the fourth stage network for downsampling respectively, and sequentially generate a fourth reference feature map and a fourth image to be detected feature map; Perform feature fusion on the fourth reference feature map and the fourth image to be detected feature map respectively, and generate the class probability of each pixel of the reference image and the image to be detected through 1×1 convolution calculation and activation function calculation to generate a semantic segmentation result.

3. The computer vision-based vehicle frame quality detection method according to claim 2, characterized in that, The resolutions of the initial reference feature map and the initial feature map to be detected are 1 / 2 of the reference image and the image to be detected. The resolutions of the first reference feature map and the first feature map to be detected are 1 / 2 of the reference image and the image to be detected. The second reference feature map and the second feature map to be detected each contain two branches, and the resolutions of the two branches are 1 / 2 and 1 / 4 of the reference image and the image to be detected respectively. The third reference feature map and the third feature map to be detected each contain three branches, and the resolutions of the three branches are 1 / 2, 1 / 4, and 1 / 8 of the reference image and the image to be detected respectively. The fourth reference feature map and the fourth feature map to be detected each contain four branches, and the resolutions of the four branches are 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the reference image and the image to be detected respectively.

4. The computer vision-based vehicle frame quality detection method according to claim 1, characterized in that Perform an exhaustive combination transformation of scaling, rotation, and translation on the vehicle frame entity to be detected in the binary image to be detected until two positioning points on the vehicle frame entity to be detected in the binary image to be detected coincide with two positioning points on the standard vehicle frame entity in the reference binary image. Specifically, it includes: Establish a plane rectangular coordinate system respectively with the binary image to be detected and the reference binary image as the reference. Obtain the coordinate positions of two positioning points on the standard vehicle frame entity in the reference binary image, and obtain the coordinate positions of two positioning points on the vehicle frame entity to be detected in the binary image to be detected. Perform an exhaustive combination transformation of scaling, rotation, and translation on the vehicle frame entity to be detected in the binary image to be detected using a homogeneous transformation matrix. The coordinate positions of two positioning points on the vehicle frame entity to be detected in the binary image to be detected will also undergo corresponding transformations until the transformed coordinate positions are the same as the coordinate positions of two positioning points on the standard vehicle frame entity in the reference binary image.

5. A vehicle frame quality detection system based on computer vision, the system includes: A reference image module, which is used to set two positioning points at preset positions of a standard vehicle frame and store the front view image of the standard vehicle frame as a reference image. An image to be detected module, which is used to set two positioning points at preset positions of the vehicle frame to be detected and obtain the front view image of the vehicle frame to be detected through an image acquisition device as the image to be detected. An image segmentation module, which is used to construct an HRNet image segmentation model to perform semantic segmentation on the reference image and the image to be detected, segment the standard vehicle frame entity and the vehicle frame entity to be detected in the reference image and the image to be detected, and generate a reference binary image and a binary image to be detected. A combination transformation module, which is used to perform an exhaustive combination transformation of scaling, rotation, and translation on the vehicle frame entity to be detected in the binary image to be detected until two positioning points on the vehicle frame entity to be detected in the binary image to be detected coincide with two positioning points on the standard vehicle frame entity in the reference binary image, and generate a transformed binary image to be detected. A logic operation module, which is used to perform an exclusive OR operation on the transformed binary image to be detected and the reference binary image to generate a deviation binary image, calculate the ratio of non-zero pixels in the deviation binary image and the reference binary image, and generate a frame error index; A determination module, which is used to determine whether the quality of the frame to be detected is qualified according to whether the frame error index exceeds a preset threshold.

6. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the computer vision-based frame quality detection method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the computer vision-based frame quality detection method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instruction is executed by the processor, it implements the steps of the computer vision-based frame quality detection method according to any one of claims 1 to 4.

Images