Vehicle chassis defect detection method and device and electronic equipment
By preprocessing the historical defect images of the vehicle chassis and adjusting the feature point configuration parameters, more training images are generated, which solves the problem of sparse vehicle chassis defect images and improves the detection accuracy and vehicle inspection efficiency of the chassis defect recognition model.
Patent Information
- Application Number
- CN202310499267.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-05
- Publication Date
- 2024-11-08
AI Technical Summary
When there are fewer images of vehicle chassis defects, it is difficult to train an effective image recognition model, and a large number of manual inspection of chassis images are required, resulting in low vehicle inspection efficiency.
By preprocessing historical defect images, including data augmentation and data fusion, selecting closed areas surrounded by feature points for configuration parameters adjustment, generating more training images, and using the object detection model for training, improving detection accuracy.
The detection accuracy and efficiency of the chassis defect identification model are improved, the demand for manual inspection is reduced, and the efficiency of chassis defect inspection is improved.
Smart Images

Figure CN118918055A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of automobile technology, and in particular to a vehicle chassis defect detection method, device and electronic equipment. Background Art
[0002] At present, with the improvement of automobile chassis production technology, chassis defects in automobile production are becoming less and less common. When inspecting the chassis of an automobile for chassis defects, it is usually necessary to have experienced inspectors to complete the inspection. Although the current inspection process no longer requires inspectors to complete it on the production line, it can be completed by viewing the chassis image of the automobile, thereby increasing the inspection speed. However, when there are many cars to be inspected, a lot of manpower is still required to view the chassis images of the automobile one by one, and the inspection efficiency is still difficult to meet the demand. In addition, some people have proposed to use image recognition models to detect vehicle chassis defects, but there are few existing vehicle chassis defect images, and it is difficult to train an effective image recognition model. Therefore, how to obtain a sufficient number of chassis defect images to complete the training of the image recognition model and improve the inspection efficiency when there are few vehicle chassis defect images has become an urgent problem to be solved. Summary of the invention
[0003] In order to solve the above technical problems, the present disclosure provides a vehicle chassis defect detection method, device and electronic equipment.
[0004] In a first aspect, the present disclosure provides a method for acquiring training data, comprising: acquiring historical defect images of a vehicle chassis; preprocessing the historical defect images to obtain enhanced defect images and fused defect images; wherein the preprocessing includes data enhancement processing and data fusion processing; selecting a first number of first feature points in a defect image area in a target image, and selecting a second number of second feature points in an area other than the defect image area in the target image; wherein the target image includes any one of the enhanced defect image and the fused defect image; adjusting the configuration parameters of the image in a closed area enclosed by the first feature point and the second feature point to obtain a training image.
[0005] In some feasible examples, the configuration parameters of the image within the closed area surrounded by the first feature point and the second feature point are adjusted to obtain a training image, including: adjusting the configuration parameters of the image within the closed area surrounded by the first feature point and the second feature point to obtain a preprocessed image; using a first recognition model to recognize the preprocessed image, and determining that the preprocessed image with a confidence level less than or equal to a preset threshold is a training image.
[0006] In some feasible examples, the historical defect image is preprocessed to obtain an enhanced defect image, including: performing data enhancement processing on the historical defect image to obtain an enhanced defect image; wherein the data enhancement processing includes one or more of flipping, rotating, scaling, adjusting brightness, adding Gaussian noise and image mixing.
[0007] In some feasible examples, the historical defect image is preprocessed to obtain a fused defect image, including: performing data fusion processing on the defect image area in the historical defect image and the target background image to obtain a fused defect image; wherein the target background image includes a chassis image of the automobile chassis when there is no defect.
[0008] In some feasible examples, the configuration parameters include one or more of brightness, grayscale, contrast, saturation, exposure, and sharpness.
[0009] In a second aspect, the present disclosure provides a method for training a chassis defect recognition model, comprising: obtaining a training sample set and a test sample set; wherein both the training sample set and the test sample set include training images and labeling results of the training images obtained in any one of the training data acquisition methods provided in the first aspect, the labeling results include recognition results corresponding to the training images, and the recognition results include confidence, defect location and defect category; inputting the training images in the training sample set to a target detection model for training to obtain a pre-trained target detection model; inputting the training images in the test sample set to the pre-trained target detection model to obtain prediction results of the pre-trained target detection model for the training images in the test sample set; based on the prediction results and the labeling results, adjusting the network parameters of the pre-trained target detection model until the pre-trained target detection model converges, solidifying the network parameters corresponding to the convergence of the pre-trained target detection model, and obtaining a chassis defect recognition model.
[0010] In a third aspect, the present disclosure provides a vehicle chassis defect detection method, comprising: obtaining a vehicle chassis image to be identified; inputting the vehicle chassis image to be identified into a vehicle chassis defect detection model for identification to obtain an identification result; wherein the vehicle chassis defect detection model is trained by the method provided in the second aspect, and the identification result includes a confidence level, a defect location and a defect category; based on the identification result, a detection result of the vehicle chassis image to be identified is obtained.
[0011] In some feasible examples, based on the recognition result, the detection result of the chassis image of the vehicle to be identified is obtained, including: when the contents of the confidence, defect location and defect category are all empty, determining that the detection result of the chassis image of the vehicle to be identified is normal.
[0012] In some feasible examples, based on the recognition results, the detection results of the chassis image of the vehicle to be identified are obtained, including: when the contents of the confidence, defect location and defect category are all not empty, and the confidence is greater than the score threshold, it is determined that the detection result of the chassis image of the vehicle to be identified is abnormal, the abnormal location is a defect location, and the abnormal category is a defect category.
[0013] In some feasible examples, based on the recognition results, the detection results of the chassis image of the vehicle to be identified are obtained, including: when the contents of the confidence, defect location and defect category are all not empty, and the confidence is less than the score threshold, it is determined that the detection result of the chassis image of the vehicle to be identified is normal.
[0014] In some feasible examples, the defect category includes any one of oil leakage, scratches, rust, and glue overflow.
[0015] In a fourth aspect, the present disclosure provides a training data acquisition device, comprising: an acquisition unit, for acquiring historical defect images of a car chassis; a processing unit, for preprocessing the historical defect images acquired by the acquisition unit to obtain enhanced defect images and fused defect images; wherein the preprocessing includes data enhancement processing and data fusion processing; the processing unit is also used to select a first number of first feature points in a defect image area in a target image, and select a second number of second feature points in an area other than the defect image area in the target image; wherein the target image includes any one of the enhanced defect image and the fused defect image; the processing unit is also used to adjust the configuration parameters of the image in a closed area enclosed by the first feature point and the second feature point to obtain a training image.
[0016] In some feasible examples, the processing unit is specifically used to adjust the configuration parameters of the image within the closed area enclosed by the first feature point and the second feature point to obtain a preprocessed image; the processing unit is specifically used to use the first recognition model to recognize the preprocessed image, and determine that the preprocessed image with a confidence level less than or equal to a preset threshold is a training image.
[0017] In some feasible examples, the processing unit is specifically used to perform data enhancement processing on the historical defect image to obtain an enhanced defect image; wherein the data enhancement processing includes one or more of flipping, rotating, scaling, adjusting brightness, adding Gaussian noise and image mixing.
[0018] In some feasible examples, the processing unit is specifically used to perform data fusion processing on the defect image area in the historical defect image and the target background image to obtain a fused defect image; wherein the target background image includes a chassis image when the automobile chassis is free of defects.
[0019] In some feasible examples, the configuration parameters include one or more of brightness, grayscale, contrast, saturation, exposure, and sharpness.
[0020] In a fifth aspect, the present disclosure provides a training device for a chassis defect recognition model, comprising: an acquisition unit, used to acquire a training sample set and a test sample set; wherein both the training sample set and the test sample set include training images and marking results of the training images obtained in the training data acquisition method of any one of the items provided in the first aspect, the marking results include recognition results corresponding to the training images, and the recognition results include confidence, defect location and defect category; a processing unit, used to input the training images in the training sample set acquired by the acquisition unit into a target detection model for training to obtain a pre-trained target detection model; the processing unit is also used to input the training images in the test sample set acquired by the acquisition unit into the pre-trained target detection model to obtain the prediction results of the pre-trained target detection model for the training images in the test sample set; the processing unit is also used to adjust the network parameters of the pre-trained target detection model based on the prediction results and the marking results until the pre-trained target detection model converges, solidify the network parameters corresponding to the convergence of the pre-trained target detection model, and obtain the chassis defect recognition model.
[0021] In the sixth aspect, the present disclosure provides a vehicle chassis defect detection device, comprising: an acquisition unit, used to acquire a vehicle chassis image to be identified; a processing unit, used to input the vehicle chassis image to be identified acquired by the acquisition unit into a vehicle chassis defect detection model for identification, to obtain an identification result; wherein the vehicle chassis defect detection model is trained by the method provided in the second aspect, and the identification result includes a confidence level, a defect location and a defect category; the processing unit is also used to obtain a detection result of the vehicle chassis image to be identified based on the identification result.
[0022] In some feasible examples, the processing unit is specifically configured to determine that the detection result of the chassis image of the vehicle to be identified is normal when the contents of the confidence level, the defect location and the defect category are all empty.
[0023] In some feasible examples, the processing unit is specifically used to determine that the detection result of the vehicle chassis image to be identified is abnormal, the abnormal position is a defect position, and the abnormal category is a defect category when the contents of the confidence, defect location and defect category are all not empty and the confidence is greater than the score threshold.
[0024] In some feasible examples, the processing unit is specifically used to determine that the detection result of the chassis image of the vehicle to be identified is normal when the contents of the confidence, defect location and defect category are all not empty and the confidence is less than the score threshold.
[0025] In some feasible examples, the defect category includes any one of oil leakage, scratches, rust, and glue overflow.
[0026] In a seventh aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to enable the electronic device to implement the method for acquiring training data provided in the first aspect above when executing the computer program.
[0027] In an eighth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, the computing device implements the method for acquiring training data provided in the first aspect above.
[0028] In a ninth aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to enable the electronic device to implement the training method for a chassis defect recognition model provided in the second aspect when executing the computer program.
[0029] In a tenth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, the computing device implements the training method of the chassis defect recognition model provided in the second aspect above.
[0030] In an eleventh aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to enable the electronic device to implement the vehicle chassis defect detection method provided in the third aspect above when executing the computer program.
[0031] In a twelfth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computing device, the computing device implements the vehicle chassis defect detection method provided in the third aspect above.
[0032] In the thirteenth aspect, the present disclosure provides a server, including a device for acquiring training data as provided in the fourth aspect, and / or a device for training a chassis defect recognition model as provided in the fifth aspect, and / or a vehicle chassis defect detection device as provided in the sixth aspect.
[0033] In the present disclosure, the names of the above-mentioned training data acquisition device, chassis defect recognition model training device and vehicle chassis defect detection device do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they belong to the scope of the claims of the present disclosure and their equivalent technologies.
[0034] These and other aspects of the present disclosure will become more apparent from the following description.
[0035] Compared with the prior art, the technical solution provided by the present invention has the following advantages:
[0036] Since the actual defect images of automobile chassis in actual production are very rare, and the actual defect images correspond to a wide variety of defect types, and the same defect type contains multiple defects. In order to simulate the actual defect images in actual production, the method for obtaining training data provided by the embodiment of the present disclosure can achieve the expansion of training images by preprocessing the historical defect images of the automobile chassis. Furthermore, by selecting a first number of first feature points for the defect image area in the target image, and selecting a second number of second feature points for the area other than the defect image area in the target image. In this way, by connecting the first feature point and the second feature point, the image within the closed area surrounded by the first feature point and the second feature point can be obtained. Afterwards, by adjusting the configuration parameters of the image in the closed area, the training image can be further expanded.
[0037] In addition, since the training image contains more samples, the detection accuracy of the target detection model can be improved when the training image is used to train the target detection model, so that the obtained chassis defect recognition model can more accurately identify the recognition result of the chassis image of the vehicle to be identified, thereby ensuring the accuracy of the recognition result. Furthermore, when the chassis image of the vehicle to be identified is identified, the chassis image of the vehicle to be identified can be input into the vehicle chassis defect detection model for identification to obtain the recognition result. Since there is no need to manually detect the defects in the chassis image of the vehicle to be identified one by one, the inspection efficiency of the chassis defect inspection can be greatly improved, thereby solving the problem that when there are many cars to be inspected, a large amount of manpower is still required to check the chassis images of the cars one by one, resulting in low inspection efficiency of the chassis defect inspection. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0040] Figure 1 One of the flowcharts of the method for acquiring training data provided in the embodiment of the present disclosure;
[0041] Figure 2 A second flowchart of a method for acquiring training data provided in an embodiment of the present disclosure;
[0042] Figure 3 A third flowchart of a method for acquiring training data provided in an embodiment of the present disclosure;
[0043] Figure 4 A fourth flowchart of a method for acquiring training data provided in an embodiment of the present disclosure;
[0044] Figure 5 A schematic diagram of a process flow of a chassis defect recognition model training method provided in an embodiment of the present disclosure;
[0045] Figure 6 One of the flow charts of the vehicle chassis defect detection method provided in the embodiment of the present disclosure;
[0046] Figure 7 A second flow chart of a vehicle chassis defect detection method provided in an embodiment of the present disclosure;
[0047] Figure 8 A third flow chart of a vehicle chassis defect detection method provided in an embodiment of the present disclosure;
[0048] Fig. 9 A fourth flow chart of a vehicle chassis defect detection method provided in an embodiment of the present disclosure;
[0049] Fig.10 A schematic diagram of a chassis defect detection system applied to the vehicle chassis defect detection method provided in an embodiment of the present disclosure;
[0050] Fig.11 A schematic diagram of the detection process of a chassis defect detection system provided in an embodiment of the present disclosure;
[0051] Fig.12 A schematic diagram of the structure of a device for acquiring training data provided in an embodiment of the present disclosure;
[0052] Fig.13 A schematic diagram of the structure of a training device for a chassis defect recognition model provided by an embodiment of the present disclosure;
[0053] Fig.14 A schematic diagram of the structure of a vehicle chassis defect detection device provided in an embodiment of the present disclosure;
[0054] Fig.15 A schematic diagram of the structure of a server provided in an embodiment of the present disclosure;
[0055] Fig.16A schematic diagram of the structure of a computer program product provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0056] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0058] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0059] The automobile chassis in the disclosed embodiment plays an important role in supporting, connecting various components, shaping, and receiving power to enable the automobile to move.
[0060] The robot arm flange in the embodiment of the present disclosure is a disc-shaped part, which is used as a fixed connection device between the robot arm and the end effector.
[0061] The robot arm point teaching in the embodiment of the present disclosure is that the robot arm moves according to a pre-edited program. This program is generally formed by an operator recording the movement trajectory according to the teaching machine.
[0062] The mixup in the embodiment of the present disclosure is an algorithm for performing mixed-class enhancement on images, which can mix images of different classes in a linear interpolation manner to construct new training samples and expand the training data set.
[0063] The full name of R-CNN in the disclosed embodiment is Region-CNN, which is the first algorithm that successfully applies deep learning to target detection.
[0064] The SSD in the disclosed embodiment refers to Single Shot MultiBox Detector, which is a one-stage detection method that mainly directly regresses the target category and position.
[0065] CUDA in the embodiments of the present disclosure refers to Compute Unified Device Architecture, which is a computing platform launched by graphics card manufacturer NVIDIA.
[0066] OpenCV in the disclosed embodiments is a cross-platform computer vision and machine learning software library released under the Apache 2.0 license (open source), and can run on Linux, Windows, Android, and Mac OS operating systems.
[0067] YOLOX in the disclosed embodiment is an improved version of the YOLO series, which combines the advantages of the series networks, uses the Focus and CSPnet network structures for feature extraction, and performs data enhancement through Mosaic.
[0068] The MES in the embodiment of the present disclosure is a production information management system for the workshop execution layer of a manufacturing enterprise.
[0069] Exemplarily, the method for acquiring training data provided by the embodiment of the present disclosure is introduced by taking the execution subject of the method for acquiring training data provided by the embodiment of the present disclosure as a server as an example.
[0070] like Figure 1 As shown, the method for obtaining training data provided in the embodiment of the present disclosure includes the following steps S11-S14:
[0071] S11. Acquire historical defect images of the automobile chassis.
[0072] In some examples, the historical defect images may be various actual defect images related to automobile chassis involved in production lines, after-sales, etc. since production to date.
[0073] In some examples, since the actual defect images of automobile chassis in actual production are very rare, and the actual defect images correspond to a wide variety of defect types, and the same defect type contains multiple defects. In order to ensure the number of training images, the method for obtaining training data provided in the embodiment of the present disclosure expands the actual defect images in the following manner, and the specific implementation process is as follows:
[0074] 1. Through manual mapping, real defects are simulated on a normal automobile chassis or components of the automobile chassis to create defects, and the actual defect images are collected through a camera.
[0075] 2. Through data enhancement algorithms, such as illumination changes, mixup, etc., the actual defect image is processed to obtain the amplified actual defect image.
[0076] 3. Cut out various defect areas in the existing defect image, and use the data fusion algorithm to insert the defects into the background image (such as a normal car chassis, or components of the car chassis) to obtain the actual defect image after amplification.
[0077] It should be noted that the factors considered by the data fusion algorithm include but are not limited to illumination, rotation angle, scale change, etc.
[0078] 4. Select a first number of first feature points in the defect area in the actual defect image, and select a second number of second feature points in the area other than the defect image area in the target image. Adjust the configuration parameters of the image in the closed area surrounded by the first feature point and the second feature point to obtain the actual defect image after amplification.
[0079] It can be seen that the number of actual defect images can be increased by one or more of the above methods, so that the number of historical defect images is larger, and the integrity of the training images is ensured.
[0080] In some examples, the actual defect image can be obtained by image acquisition by an image acquisition device set on the robotic arm. For example, after receiving the trigger signal, the robotic arm in the initial position will start running according to the stored teaching point. When the first teaching point is reached, the image acquisition device (such as an industrial camera) fixed on the robotic arm is triggered to perform image acquisition at that point. After the image acquisition is completed, a completion signal will be sent to the robotic arm, and the robotic arm will start the image acquisition work of the next point. This process will continue until all teaching points are cruised, so that the actual chassis image of the car chassis can be obtained. Afterwards, by performing defect detection on the actual chassis image, it is determined whether there are defects in the actual chassis image. If there are defects in the actual chassis image, the actual chassis image is used as the actual defect image.
[0081] Specifically, when the image acquisition device set on the robot arm is used to acquire the actual defect image, it is necessary to teach the robot arm point position, such as: preparing the system required hardware: robot arm, image acquisition device (such as: 2D color high-definition camera (including lens)), non-standard connectors, etc., and build a chassis quality inspection system according to the needs. After that, open up the functions of robot arm fixed-point control, triggering camera photography, image storage, etc. The robot arm point position teaching is completed according to the fixed-point photography content, based on the following principles:
[0082] a. Ensure that all images collected at the teaching points can be stitched into a complete chassis without missing any position.
[0083] b. Take intensive photos of key areas, such as the front and rear of the vehicle, to ensure focus and multiple angles.
[0084] c. Ensure the workstation rhythm and control the overall system inspection time within 2-3 minutes.
[0085] In this way, the robot arm point teaching is completed.
[0086] Specifically, defect detection is performed on the actual chassis image, including:
[0087] The actual chassis image is compared with the standard chassis image to determine whether there is a difference between the actual chassis image and the standard chassis image. If there is no difference, it is determined that the actual chassis image has no defects; if there is a difference, it is determined that the actual chassis image has defects.
[0088] or,
[0089] The actual chassis image is compared with the image corresponding to the existing defective area to determine whether there is an image in the actual chassis image that matches the image corresponding to the existing defective area. If so, it is determined that the actual chassis image has defects; if not, it is determined that the actual chassis image has no defects.
[0090] S12, preprocessing the historical defect image to obtain an enhanced defect image and a fused defect image, wherein the preprocessing includes data enhancement processing and data fusion processing.
[0091] In some examples, the historical defect image may be subjected to data enhancement processing to obtain an enhanced defect image. The data enhancement processing includes one or more of flipping, rotating, scaling, adjusting brightness, adding Gaussian noise, and image mixing. Alternatively, the defect image area in the historical defect image is subjected to data fusion processing with the target background image to obtain a fused defect image. The target background image includes an image of the chassis of the automobile when the chassis is free of defects.
[0092] S13, selecting a first number of first feature points in a defect image region in a target image, and selecting a second number of second feature points in a region other than the defect image region in the target image, wherein the target image includes any one of an enhanced defect image and a fused defect image.
[0093] In some examples, the first number and the second number are both integers greater than or equal to 0.
[0094] Exemplarily, the sum of the first number and the second number is equal to 3.
[0095] S14, adjusting the configuration parameters of the image within the closed area surrounded by the first feature point and the second feature point to obtain a training image.
[0096] In some examples, the configuration parameters include one or more of brightness, grayscale, contrast, saturation, exposure, and sharpness. By adjusting the configuration parameters, more training images can be obtained to ensure the recognition accuracy of the chassis defect recognition model trained by the training images.
[0097] From the above, it can be seen that in actual production, there are relatively few actual defect images of automobile chassis, and the types of defects corresponding to the actual defect images are numerous, and the same defect type contains multiple defects. In order to simulate the actual defect images in actual production, the method for obtaining training data provided by the embodiment of the present disclosure can realize the expansion of the training images by preprocessing the historical defect images of the automobile chassis. Furthermore, by selecting a first number of first feature points for the defect image area in the target image, and selecting a second number of second feature points in the area other than the defect image area in the target image. In this way, by connecting the first feature point and the second feature point, the image within the closed area surrounded by the first feature point and the second feature point can be obtained. Afterwards, by adjusting the configuration parameters of the image in the closed area, the training image can be further expanded.
[0098] In some possible implementation examples, combined with Figure 1 ,like Figure 2 As shown, the above S12 can be specifically implemented through the following S120.
[0099] S120, performing data enhancement processing on the historical defect image to obtain an enhanced defect image, wherein the data enhancement processing includes one or more of flipping, rotating, scaling, adjusting brightness, adding Gaussian noise, and image mixing.
[0100] In some possible implementation examples, combined with Figure 1 ,like Figure 3 As shown, the above S12 can be specifically implemented through the following S121.
[0101] S121, performing data fusion processing on the defect image area in the historical defect image and the target background image to obtain a fused defect image, wherein the target background image includes a chassis image when the chassis of the automobile is free of defects.
[0102] In some possible implementation examples, combined with Figure 1 ,like Figure 4 As shown, the above S14 can be specifically implemented through the following S140 and S141.
[0103] S140 , adjusting configuration parameters of an image within a closed area enclosed by the first feature point and the second feature point to obtain a preprocessed image.
[0104] S141. Use the first recognition model to recognize the preprocessed image, and determine that the preprocessed image with a confidence level less than or equal to a preset threshold is a training image.
[0105] In some examples, the first recognition model may be an existing recognition model, such as a target detection model based on deep learning (R-CNN, SSD, etc.)
[0106] Exemplarily, taking the example of a server as the executing entity of the training method of the chassis defect recognition model provided by the embodiment of the present disclosure, the training method of the chassis defect recognition model provided by the embodiment of the present disclosure is introduced.
[0107] like Figure 5 As shown, the training method of the chassis defect recognition model provided by the embodiment of the present disclosure includes the following steps S21-S24:
[0108] S21, obtaining a training sample set and a test sample set. Both the training sample set and the test sample set include the labeling results of the training images obtained by the training data acquisition method provided in any of the above-mentioned methods, the labeling results include the recognition results corresponding to the training images, and the recognition results include the confidence, defect location and defect category.
[0109] S22: Input the training images in the training sample set into the target detection model for training to obtain a pre-trained target detection model.
[0110] S23, inputting the training images in the test sample set into the pre-trained target detection model to obtain the prediction results of the pre-trained target detection model on the training images in the test sample set.
[0111] S24. Based on the prediction results and the marking results, the network parameters of the pre-trained target detection model are adjusted until the pre-trained target detection model converges, and the corresponding network parameters when the pre-trained target detection model converges are solidified to obtain a chassis defect recognition model.
[0112] In some examples, when training an object detection model, you need to do the following:
[0113] a. Configure the training environment: Install the necessary training environment, such as CUDA, OpenCV, etc., on the training machine.
[0114] b. Training pipeline design: design network structure, training error terms, data flow and other modules.
[0115] c. Design and write training scripts.
[0116] d. Use the prepared experimental data set (such as training sample set and test sample set) and start the training script for training.
[0117] e. Use the prediction results and labeling results to calibrate the target detection model, update the parameters of the target detection model, and repeat the above steps.
[0118] f. Model solidification Based on the training error and model test results, determine whether the training model has converged. If it has converged, it means that the model has stabilized. At this time, the model parameters are fixed to obtain the chassis defect recognition model.
[0119] In some examples, the target detection model includes but is not limited to Faster RCNN, YOLOX, etc.
[0120] It can be seen that in order to improve the recognition accuracy of the chassis defect recognition model, the chassis defect recognition model training method provided by the embodiment of the present disclosure amplifies the training image so that the number of training image samples is larger. In this way, when training the target detection model, the number of training samples of the target detection model can be increased, thereby increasing the coupling of the target detection model and improving the accuracy of the target detection model. In this way, when the target detection model converges, the obtained chassis defect recognition model can more accurately identify the recognition result of the chassis image of the vehicle to be identified, thereby ensuring the accuracy of the recognition result.
[0121] Exemplarily, the vehicle chassis defect detection method provided by the embodiment of the present disclosure is introduced by taking the execution subject of the vehicle chassis defect detection method provided by the embodiment of the present disclosure as a server as an example.
[0122] like Figure 6 As shown, the vehicle chassis defect detection method provided by the embodiment of the present disclosure includes the following steps S31-S33:
[0123] S31, obtaining a chassis image of a vehicle to be identified.
[0124] S32, inputting the vehicle chassis image to be identified into the vehicle chassis defect detection model for identification, and obtaining an identification result. The vehicle chassis defect detection model is trained by the above-mentioned chassis defect recognition model training method, and the identification result includes confidence, defect location and defect category.
[0125] S33. Based on the recognition result, a detection result of the chassis image of the vehicle to be recognized is obtained.
[0126] In some possible implementation examples, combined with Figure 6 ,like Figure 7 As shown, the above S33 can be specifically implemented through the following S330.
[0127] S330: When the contents of the confidence level, defect location and defect category are all empty, it is determined that the detection result of the chassis image of the vehicle to be identified is normal.
[0128] In some possible implementation examples, combined with Figure 6 ,like Figure 8 As shown, the above S33 can be specifically implemented through the following S331.
[0129] S331. When the contents of the confidence, defect location and defect category are all not empty and the confidence is greater than the score threshold, it is determined that the detection result of the chassis image of the vehicle to be identified is that there is an abnormality, the abnormal location is a defect location, and the abnormal category is a defect category.
[0130] In some possible implementation examples, combined with Figure 6 ,like Fig. 9 As shown, the above S33 can be specifically implemented through the following S332.
[0131] S332: When the contents of the confidence level, defect location and defect category are not empty and the confidence level is less than the score threshold, it is determined that the detection result of the chassis image of the vehicle to be identified is normal.
[0132] In some feasible examples, the defect category includes any one of oil leakage, scratches, rust, and glue overflow.
[0133] In some examples, the vehicle chassis defect detection method provided by the embodiments of the present disclosure is applied to Fig.10 The chassis defect detection system shown in the figure includes: data acquisition equipment, data processing equipment, storage equipment, and result visualization equipment.
[0134] The data acquisition equipment includes a robotic arm control module, a camera imaging module, and non-standard connectors, where the non-standard connectors are used to fix the 2D color high-definition camera on the flange of the robotic arm. The purpose is to use the robotic arm to drive the camera fixed on it to image a specific chassis area, and by moving the robotic arm at a fixed point, strive to cover the entire chassis area with the overall image set, and by adjusting the camera viewing angle at a specific point, strive to obtain multi-view chassis images of the vehicle to be identified in the focus point space.
[0135] The data processing equipment must include but is not limited to having relatively powerful computing power, using the vehicle chassis defect detection method to identify the acquired vehicle chassis image to be identified, locate the defect target, and provide the identification result after logical processing.
[0136] The storage device is used to store the point images captured by the camera and also to store the defect recognition results of the images. It can be a local storage device or a cloud storage device.
[0137] The result visualization device is used to present the chassis defect detection results. It can present statistical results, such as the number of defects detected, and can also present defect visualization effects, such as the location and type of defects identified in the image. At the same time, the vehicle defect detection results can also be uploaded to the production management system (MES) used by automobile manufacturers, which helps to realize the digitalization of enterprise production.
[0138] Exemplarily, the detection process of the chassis defect detection system is as follows: Fig.11 shown.
[0139] After receiving the trigger signal, the robot arm in the initial position will start running according to the stored teaching points. When it reaches the first teaching point, it will trigger the image acquisition device (such as an industrial camera) fixed on the robot arm to acquire the image of that point. After the image acquisition is completed, a completion signal will be sent to the robot arm, and the robot arm will start the image acquisition work of the next point. This process will continue until all teaching points are cruised, so that the actual chassis image of the car chassis can be obtained. After that, by performing defect detection on the actual chassis image, it is determined whether there are defects in the actual chassis image. If there are defects in the actual chassis image, the actual chassis image is used as the actual defect image.
[0140] Specifically, when the image acquisition device set on the robot arm is used to acquire the actual defect image, it is necessary to teach the robot arm point position, such as: preparing the system required hardware: robot arm, image acquisition device (such as: 2D color high-definition camera (including lens)), non-standard connectors, etc., and build a chassis quality inspection system according to the needs. After that, open up the functions of robot arm fixed-point control, triggering camera photography, image storage, etc. The robot arm point position teaching is completed according to the fixed-point photography content, based on the following principles:
[0141] a. Ensure that all images collected at the teaching points can be stitched into a complete chassis without missing any position.
[0142] b. Take intensive photos of key areas, such as the front and rear of the vehicle, to ensure focus and multiple angles.
[0143] c. Ensure the workstation rhythm and control the overall system inspection time within 2-3 minutes.
[0144] In this way, the robot arm point teaching is completed.
[0145] Specifically, defect detection is performed on the actual chassis image, including:
[0146] The actual chassis image is compared with the standard chassis image to determine whether there is a difference between the actual chassis image and the standard chassis image. If there is no difference, it is determined that the actual chassis image has no defects; if there is a difference, it is determined that the actual chassis image has defects.
[0147] or,
[0148] The actual chassis image is compared with the image corresponding to the existing defective area to determine whether there is an image in the actual chassis image that matches the image corresponding to the existing defective area. If so, it is determined that the actual chassis image has defects; if not, it is determined that the actual chassis image has no defects.
[0149] The above mainly introduces the solution provided by the embodiment of the present disclosure from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.
[0150] The embodiments of the present disclosure may divide the functional modules of the training data acquisition device, the chassis defect recognition model training device and the vehicle chassis defect detection device respectively according to the above method examples. For example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical functional division. There may be other division methods in actual implementation.
[0151] like Fig.12 , which is a schematic diagram of the structure of a training data acquisition device 10 provided in an embodiment of the present disclosure, including an acquisition unit 101 and a processing unit 102 .
[0152] The acquisition unit 101 is used to acquire a historical defect image of a vehicle chassis; the processing unit 202 is used to preprocess the historical defect image acquired by the acquisition unit 101 to obtain an enhanced defect image and a fused defect image; wherein the preprocessing includes data enhancement processing and data fusion processing; the processing unit 202 is also used to select a first number of first feature points in a defect image area in a target image, and select a second number of second feature points in an area other than the defect image area in the target image; wherein the target image includes any one of the enhanced defect image and the fused defect image; the processing unit 202 is also used to adjust the configuration parameters of the image in a closed area surrounded by the first feature point and the second feature point to obtain a training image.
[0153] In some feasible examples, the processing unit 202 is specifically used to adjust the configuration parameters of the image within the closed area enclosed by the first feature point and the second feature point to obtain a preprocessed image; the processing unit 202 is specifically used to use the first recognition model to recognize the preprocessed image, and determine that the preprocessed image with a confidence level less than or equal to a preset threshold is a training image.
[0154] In some feasible examples, the processing unit 202 is specifically used to perform data enhancement processing on the historical defect image to obtain an enhanced defect image; wherein the data enhancement processing includes one or more of flipping, rotating, scaling, adjusting brightness, adding Gaussian noise and image mixing.
[0155] In some feasible examples, the processing unit 202 is specifically used to perform data fusion processing on the defect image area in the historical defect image and the target background image to obtain a fused defect image; wherein the target background image includes a chassis image when the automobile chassis is free of defects.
[0156] In some feasible examples, the configuration parameters include one or more of brightness, grayscale, contrast, saturation, exposure, and sharpness.
[0157] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.
[0158] Of course, the training data acquisition device 10 provided in the embodiment of the present disclosure includes but is not limited to the above modules, for example, the training data acquisition device 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the training data acquisition device 10, and may also be used to store data generated by the training data acquisition device 10 during operation, such as data in a write request, etc.
[0159] like Fig.13 , which is a schematic diagram of the structure of a chassis defect recognition model training device 20 provided in an embodiment of the present disclosure, including an acquisition unit 201 and a processing unit 202 .
[0160] The acquisition unit 201 is used to acquire a training sample set and a test sample set; wherein both the training sample set and the test sample set include training images and marking results of the training images; wherein the training images are obtained by training through any of the methods provided in the first aspect, and the marking results include recognition results corresponding to the training images, and the recognition results include confidence, defect location and defect category; the processing unit 202 is used to input the training images in the training sample set acquired by the acquisition unit 201 into the target detection model for training to obtain a pre-trained target detection model; the processing unit 202 is also used to input the training images in the test sample set acquired by the acquisition unit 201 into the pre-trained target detection model to obtain the prediction results of the pre-trained target detection model for the training images in the test sample set; the processing unit 202 is also used to adjust the network parameters of the pre-trained target detection model based on the prediction results and the marking results until the pre-trained target detection model converges, solidify the network parameters corresponding to the convergence of the pre-trained target detection model, and obtain a chassis defect recognition model.
[0161] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.
[0162] Of course, the training device 20 for chassis defect recognition model provided by the embodiment of the present disclosure includes but is not limited to the above modules, for example, the training device 20 for chassis defect recognition model may also include a storage unit 203. The storage unit 203 may be used to store the program code of the training device 20 for chassis defect recognition model, and may also be used to store data generated by the training device 20 for chassis defect recognition model during operation, such as data in a write request, etc.
[0163] like Fig.14 , which is a schematic diagram of the structure of a vehicle chassis defect detection device 30 provided in an embodiment of the present disclosure, including an acquisition unit 301 and a processing unit 302.
[0164] The acquisition unit 301 is used to acquire the vehicle chassis image to be identified; the processing unit 302 is used to input the vehicle chassis image to be identified acquired by the acquisition unit 301 into the vehicle chassis defect detection model for identification to obtain an identification result; wherein the vehicle chassis defect detection model is trained by the method provided in the second aspect, and the identification result includes confidence, defect location and defect category; the processing unit 302 is also used to obtain a detection result of the vehicle chassis image to be identified based on the identification result.
[0165] In some feasible examples, the processing unit 302 is specifically configured to determine that the detection result of the to-be-identified vehicle chassis image is normal when the contents of the confidence level, the defect location, and the defect category are all empty.
[0166] In some implementable examples, the processing unit 302 is specifically used to determine that the detection result of the vehicle chassis image to be identified is abnormal, the abnormal position is a defect position, and the abnormal category is a defect category when the contents of the confidence, defect position and defect category are all not empty and the confidence is greater than the score threshold.
[0167] In some feasible examples, the processing unit 302 is specifically configured to determine that the detection result of the chassis image of the vehicle to be identified is normal when the contents of the confidence, defect location and defect category are all not empty and the confidence is less than the score threshold.
[0168] In some feasible examples, the defect category includes any one of oil leakage, scratches, rust, and glue overflow.
[0169] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.
[0170] Of course, the vehicle chassis defect detection device 30 provided in the embodiment of the present disclosure includes but is not limited to the above modules, for example, the vehicle chassis defect detection device 30 may also include a storage unit 303. The storage unit 303 may be used to store the program code of the vehicle chassis defect detection device 30, and may also be used to store data generated by the vehicle chassis defect detection device 30 during operation, such as data in a write request, etc.
[0171] Fig.15 A schematic diagram of the structure of a server provided in an embodiment of the present disclosure is shown in FIG. Fig.15 As shown, the server may include: at least one processor 51 , a memory 52 , a communication interface 53 and a communication bus 54 .
[0172] Combine the following Fig.15 A detailed introduction to each component of the server:
[0173] The processor 51 is the control center of the electronic device 10, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).
[0174] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Fig.15 Also, as an embodiment, the electronic device may include multiple processors, such as Fig.15 51 and 55 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0175] The memory 52 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.
[0176] In a specific implementation, the memory 52 is used to store the data in the present disclosure and execute the software program in the present disclosure. The processor 51 can execute various functions of the air conditioner by running or executing the software program stored in the memory 52 and calling the data stored in the memory 52.
[0177] The communication interface 53 uses any transceiver-like device to communicate with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, the cloud, etc. The communication interface 53 may include an acquisition unit to implement the acquisition function.
[0178] The communication bus 54 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0179] As an example, combine Fig.12 The function implemented by the acquisition unit 101 in the acquisition device 10 of the training data device is the same as Fig.15 The function of the communication interface 53 in the training data acquisition device 10 is the same as that of the processing unit 102 in the training data acquisition device 10. Fig.15 The function of the processor 51 in the training data acquisition device 10 is the same as that of the storage unit 103 in the training data acquisition device 10. Fig.15 The function of the memory 52 in is the same.
[0180] As an example, combine Fig.13 The function of the acquisition unit 201 in the training device 20 of the chassis defect recognition model is similar to Fig.15 The function of the communication interface 53 in the training device 20 of the chassis defect recognition model is the same as that of the processing unit 202 in the training device 20 of the chassis defect recognition model. Fig.15 The function of the processor 51 in the training device 20 of the chassis defect recognition model is the same as that of the storage unit 203 in the training device 20 of the chassis defect recognition model. Fig.15 The function of the memory 52 in is the same.
[0181] As an example, combine Fig.14 The function of the acquisition unit 301 in the vehicle chassis defect detection device 30 is the same as Fig.15 The function of the communication interface 53 in the vehicle chassis defect detection device 30 is the same as that of the processing unit 302 in the vehicle chassis defect detection device 30. Fig.15 The function of the processor 51 in the vehicle chassis defect detection device 30 is the same as that of the storage unit 303 in the vehicle chassis defect detection device 30. Fig.15 The function of the memory 52 in is the same.
[0182] Another embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computing device, the computing device executes the method for acquiring training data, or the method for training a chassis defect recognition model, or the method for detecting vehicle chassis defects as shown in the above method embodiments.
[0183] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.
[0184] Fig.16 A conceptual partial view of a computer program product provided by an embodiment of the present disclosure is schematically shown, wherein the computer program product includes a computer program for executing a computer process on a computing device.
[0185] In one embodiment, the computer program product is provided using a signal bearing medium 410. The signal bearing medium 410 may include one or more program instructions, which when executed by one or more processors may provide the above-mentioned Figure 1 Thus, for example, reference to Figure 1 In the embodiment shown in , one or more features of S11-S14 may be undertaken by one or more instructions associated with the signal bearing medium 410. Figure 5 In the embodiment shown in , one or more features of S21-S24 may be undertaken by one or more instructions associated with the signal bearing medium 410. Figure 6 In the embodiment shown in , one or more features of S31-S33 may be undertaken by one or more instructions associated with the signal bearing medium 410. In addition, Fig.16 The program instructions in also describe example instructions.
[0186] In some examples, signal bearing medium 410 may include computer readable medium 411, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.
[0187] In some implementations, signal bearing medium 410 may include computer recordable medium 412 such as, but not limited to, memory, read / write (R / W) CD, R / W DVD, and the like.
[0188] In some implementations, signal bearing medium 410 may include communication medium 413 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).
[0189] Signal bearing medium 410 may be communicated by a wireless form of communication medium 413 (eg, wireless communication medium complying with IEEE 802.41 standard or other transmission protocol). The one or more program instructions may be, for example, computing device executable instructions or logic implementing instructions.
[0190] In some examples, such as for Fig.12 The described training data acquisition device 10 may be configured to provide various operations, functions, or actions in response to one or more program instructions in the computer-readable medium 411 , the computer-recordable medium 412 , and / or the communication medium 413 .
[0191] In some examples, such as for Fig.13 The described chassis defect recognition model training apparatus 20 may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer readable medium 411 , computer recordable medium 412 , and / or communication medium 413 .
[0192] In some examples, such as for Fig.14 The described vehicle chassis defect detection apparatus 30 may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer readable media 411 , computer recordable media 412 , and / or communication media 413 .
[0193] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0194] In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0195] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0196] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0197] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.
[0198] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for obtaining training data, characterized in that: include: Obtain historical defect images of the vehicle chassis; Preprocessing the historical defect image to obtain an enhanced defect image and a fused defect image; wherein the preprocessing includes data enhancement processing and data fusion processing; Selecting a first number of first feature points in a defect image region in a target image, and selecting a second number of second feature points in a region other than the defect image region in the target image; wherein the target image includes any one of an enhanced defect image and a fused defect image; The configuration parameters of the image within the closed area surrounded by the first feature point and the second feature point are adjusted to obtain a training image.
2. The method for obtaining training data according to claim 1, characterized in that: The step of adjusting the configuration parameters of the image within the closed area enclosed by the first feature point and the second feature point to obtain a training image includes: Adjusting configuration parameters of an image within a closed area enclosed by the first feature point and the second feature point to obtain a preprocessed image; The pre-processed image is recognized by using a first recognition model, and the pre-processed image having a confidence level less than or equal to a preset threshold is determined as a training image.
3. The acquisition method according to claim 1, characterized in that: The preprocessing of the historical defect image to obtain an enhanced defect image includes: Performing data enhancement processing on the historical defect image to obtain an enhanced defect image; wherein the data enhancement processing includes one or more of flipping, rotating, scaling, adjusting brightness, adding Gaussian noise and image mixing.
4. The acquisition method according to claim 1, characterized in that: The preprocessing of the historical defect image to obtain a fused defect image includes: The defect image area in the historical defect image is subjected to data fusion processing with the target background image to obtain a fused defect image; wherein the target background image includes a chassis image of the automobile chassis when there is no defect.
5. A method for training a chassis defect recognition model, characterized in that: include: Acquire a training sample set and a test sample set; wherein both the training sample set and the test sample set include a training image obtained in the method for acquiring training data according to any one of claims 1 to 4 and a marking result of the training image, wherein the marking result includes a recognition result corresponding to the training image, and the recognition result includes a confidence level, a defect location, and a defect category; Inputting the training images in the training sample set into the target detection model for training to obtain a pre-trained target detection model; Inputting the training images in the test sample set into the pre-trained object detection model to obtain prediction results of the pre-trained object detection model on the training images in the test sample set; Based on the prediction result and the marking result, the network parameters of the pre-trained target detection model are adjusted until the pre-trained target detection model converges, and the corresponding network parameters when the pre-trained target detection model converges are solidified to obtain a chassis defect recognition model.
6. A vehicle chassis defect detection method, characterized in that: include: Acquire a chassis image of a vehicle to be identified; Inputting the vehicle chassis image to be identified into a vehicle chassis defect detection model for identification to obtain an identification result; wherein the vehicle chassis defect detection model is obtained by training the method according to claim 6, and the identification result includes a confidence level, a defect location, and a defect category; Based on the recognition result, a detection result of the chassis image of the vehicle to be recognized is obtained.
7. The vehicle chassis defect detection method according to claim 6, characterized in that: The obtaining, based on the recognition result, a detection result of the chassis image of the vehicle to be recognized, comprises: When the contents of the confidence level, the defect position and the defect category are all empty, it is determined that the detection result of the chassis image of the vehicle to be identified is normal.
8. The vehicle chassis defect detection method according to claim 6, characterized in that: The obtaining, based on the recognition result, a detection result of the chassis image of the vehicle to be recognized, comprises: When the contents of the confidence, the defect position and the defect category are all not empty and the confidence is greater than the score threshold, it is determined that the detection result of the chassis image of the vehicle to be identified is abnormal, the abnormal position is the defect position, and the abnormal category is the defect category.
9. The vehicle chassis defect detection method according to claim 6, characterized in that: The obtaining, based on the recognition result, a detection result of the chassis image of the vehicle to be recognized, comprises: When the contents of the confidence, the defect position and the defect category are all not empty, and the confidence is less than the score threshold, it is determined that the detection result of the chassis image of the vehicle to be identified is normal.
10. A device for acquiring training data, characterized in that: include: An acquisition unit, used for acquiring a historical defect image of a vehicle chassis; A processing unit, used for preprocessing the historical defect image acquired by the acquisition unit to obtain an enhanced defect image and a fused defect image; wherein the preprocessing includes data enhancement processing and data fusion processing; The processing unit is further configured to select a first number of first feature points in a defect image region in the target image, and select a second number of second feature points in a region other than the defect image region in the target image; wherein the target image includes any one of an enhanced defect image and a fused defect image; The processing unit is further used to adjust the configuration parameters of the image within the closed area surrounded by the first feature point and the second feature point to obtain a training image.
11. A training device for a chassis defect recognition model, characterized in that: include: an acquisition unit, configured to acquire a training sample set and a test sample set; wherein both the training sample set and the test sample set include a training image obtained in the method for acquiring training data according to any one of claims 1 to 4 and a marking result of the training image, wherein the marking result includes a recognition result corresponding to the training image, and the recognition result includes a confidence level, a defect location, and a defect category; A processing unit, configured to input the training images in the training sample set acquired by the acquisition unit into a target detection model for training, so as to obtain a pre-trained target detection model; The processing unit is further configured to input the training images in the test sample set acquired by the acquisition unit into the pre-trained target detection model to obtain prediction results of the pre-trained target detection model on the training images in the test sample set; The processing unit is also used to adjust the network parameters of the pre-trained target detection model based on the prediction result and the marking result until the pre-trained target detection model converges, solidify the network parameters corresponding to the convergence of the pre-trained target detection model, and obtain a chassis defect recognition model.
12. A vehicle chassis defect detection device, characterized in that: include: An acquisition unit, used for acquiring a chassis image of a vehicle to be identified; a processing unit, configured to input the vehicle chassis image to be identified acquired by the acquisition unit into a vehicle chassis defect detection model for identification, and obtain an identification result; wherein the vehicle chassis defect detection model is obtained by training according to the method of claim 6, and the identification result includes a confidence level, a defect location, and a defect category; The processing unit is further used to obtain a detection result of the chassis image of the vehicle to be identified based on the identification result.
13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; the processor is used to, when executing the computer program, enable the electronic device to implement the method for acquiring training data described in any one of claims 1 to 4, or enable the electronic device to implement the method for training a chassis defect recognition model described in claim 5, or enable the electronic device to implement the vehicle chassis defect detection method described in any one of claims 6 to 10.
14. A computer-readable storage medium, characterized in that: include: A computer program is stored on the computer-readable storage medium. When the computer program is executed by a computing device, the computing device implements the method for acquiring training data described in any one of claims 1 to 4, or the computing device implements the method for training a chassis defect recognition model described in claim 5, or the computing device implements the vehicle chassis defect detection method described in any one of claims 6 to 10.
15. A server, comprising the device for acquiring training data as claimed in claim 10, and / or the device for training a chassis defect recognition model as claimed in claim 11, and / or the vehicle chassis defect detection device as claimed in claim 12.
Citation Information
Patent Citations
Method and device for generating binaural pin defect sample based on generative adversarial network
CN112668623A
Vehicle defect identification method and device, equipment and storage medium
CN112819055A
Data enhancement method and device, data enhancement equipment and storage medium
CN112884691A
Method for defect detection of an object
CN114331961A
Data enhancement method and device, terminal equipment and storage medium
CN114638294A
