Light defect identification model training method, light defect detection method and device

Abstract

The invention relates to a light defect recognition model training method and device and a light defect detection method and device. According to the invention, when the vehicle lamp sample image is cut, the overlapping area exists between the sample cut images, and the area of the overlapping area is set to be greater than the preset area, so that the defect area can be completely covered by at least one sample cut image regardless of the size of the defect area in the vehicle lamp sample image; therefore, the light defect identification model obtained based on sample clipping image training can identify the defect of any size in the to-be-detected vehicle lamp, and the accuracy of vehicle lamp defect detection is effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of detection technology, and in particular to a lighting defect recognition model training method, a lighting defect detection method and a lighting defect detection device. Background Art

[0002] After the vehicle completes all assembly procedures in the final assembly workshop, it is necessary to turn on all external lights for integrity inspection on the final inspection line, and promptly detect various abnormal conditions such as lights failing to light up, abnormal colors, dimming, etc.

[0003] In the prior art, an image comparison method is usually used for vehicle light detection, in which a captured vehicle light image is compared with a pre-stored normal vehicle light image. When the difference between the two images is large, it is considered that the vehicle light is abnormal.

[0004] However, the above solution cannot effectively detect defects with a smaller area, resulting in low accuracy of the detection results. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a lighting defect recognition model training method, a lighting defect detection method and a device to improve the accuracy of detection results.

[0006] In a first aspect, an embodiment of the present disclosure provides a lighting defect recognition model training method, comprising:

[0007] Obtain at least one sample image of a vehicle light;

[0008] For each headlight sample image, a cropping operation is performed on the headlight sample image to obtain a plurality of sample cropped images, wherein an area of ​​an overlapping region between two adjacent sample cropped images is greater than a preset area;

[0009] Determine a label classification result corresponding to each sample cropped image, wherein the label classification result is one of a normal headlight image and an abnormal headlight image;

[0010] The sample cropped image is used as the input of the light defect recognition model to be trained, and the label classification result corresponding to the sample cropped image is used as the output of the light defect recognition model to be trained. The light defect recognition model to be trained is trained to obtain a trained light defect recognition model.

[0011] In some embodiments, for each headlight sample image, performing a cropping operation on the headlight sample image to obtain a plurality of sample cropped images includes:

[0012] Performing a cropping operation on the headlight sample image according to a preset headlight mask to obtain a plurality of first region of interest images, wherein the direction of each first region of interest image is consistent with the direction of the headlight in the headlight sample image;

[0013] Rotating each of the first region-of-interest images to obtain a second region-of-interest image corresponding to each of the first region-of-interest images, wherein the direction of each of the second region-of-interest images is a horizontal direction;

[0014] A scaling operation is performed on each of the second region-of-interest images to obtain a plurality of sample cropped images of the same size.

[0015] In some embodiments, after determining the label classification results corresponding to each sample cropped image, the method further includes:

[0016] Data expansion is performed on the sample cropped images whose label classification results are abnormal headlight images through deep learning.

[0017] In some embodiments, acquiring at least one vehicle light sample image includes:

[0018] Obtain at least one original image of a vehicle light;

[0019] Extracting the minimum envelope rectangle of the headlight region in each headlight original image through an instance segmentation network to obtain at least one headlight envelope rectangle image, wherein the at least one headlight original image corresponds to the at least one headlight envelope rectangle image in one-to-one correspondence;

[0020] For each headlight envelope rectangular image, pixels outside the headlight area are set to zero to obtain at least one headlight sample image, and the at least one headlight envelope rectangular image corresponds to the at least one headlight sample image one by one.

[0021] In a second aspect, an embodiment of the present disclosure provides a light defect detection method, comprising:

[0022] Acquire the image of the vehicle headlight to be detected;

[0023] Performing a cropping operation on the image of the vehicle headlight to be detected to obtain a plurality of cropped images of the vehicle headlight to be detected, wherein there is an overlapping area between two adjacent cropped images of the vehicle headlight to be detected;

[0024] Input a plurality of cropped images of vehicle lights to be detected into a trained light defect recognition model, and receive the output of the trained light defect recognition model as the detection result of the vehicle lights to be detected. The light defect recognition model is obtained by the light defect recognition model training method as described in the first aspect.

[0025] In some embodiments, after inputting the plurality of cropped vehicle headlight images to be detected into a trained light defect recognition model and receiving the output of the trained light defect recognition model as the detection result of the vehicle headlight images to be detected, the method further includes:

[0026] If it is determined according to the detection result that there is a defective area in the image of the vehicle headlight to be detected, then a cropped image of the vehicle headlight to be detected corresponding to the defective area is determined;

[0027] According to the position of the defective area in the cropped image of the vehicle lamp to be detected and the position of the cropped image of the vehicle lamp to be detected in the image of the vehicle lamp to be detected, the actual fault position corresponding to the defective area is determined.

[0028] In a third aspect, an embodiment of the present disclosure provides a lighting defect model training device, the device comprising:

[0029] A first acquisition module, used to acquire at least one vehicle light sample image;

[0030] A first cropping module is used to perform a cropping operation on each headlight sample image to obtain a plurality of sample cropped images, wherein an area of ​​an overlapping region between two adjacent sample cropped images is greater than a preset area;

[0031] A first determination module is used to determine a label classification result corresponding to each sample cropped image, wherein the label classification result is one of a normal headlight image and an abnormal headlight image;

[0032] The training module is used to use the sample cropped image as the input of the light defect recognition model to be trained, and use the label classification result corresponding to the sample cropped image as the output of the light defect recognition model to be trained, so as to train the light defect recognition model to be trained and obtain a trained light defect recognition model.

[0033] In a fourth aspect, an embodiment of the present disclosure provides a light defect detection device, the device comprising:

[0034] The second acquisition module is used to acquire the image of the vehicle headlight to be detected;

[0035] A second cropping module is used to crop the image of the vehicle headlight to be detected to obtain a plurality of cropped images of the vehicle headlight to be detected, wherein there is an overlapping area between two adjacent cropped images of the vehicle headlight to be detected;

[0036] The receiving module is used to input a plurality of cropped vehicle headlight images to be detected into a trained light defect recognition model, and receive the output of the trained light defect recognition model as the detection result of the vehicle headlight images to be detected.

[0037] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including:

[0038] Memory;

[0039] Processor; and

[0040] Computer programs;

[0041] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect or the second aspect.

[0042] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described in the first aspect or the second aspect.

[0043] The light defect recognition model training method, light defect detection method and device provided by the embodiments of the present disclosure allow overlapping areas between each sample cropped image when cropping the sample image of the headlight, and set the area of ​​the overlapping area to be larger than a preset area. Regardless of the size of the defect area in the sample image of the headlight, the defect area can be completely covered by at least one sample cropped image, so that the light defect recognition model trained based on the sample cropped image can identify defects of any size in the headlight to be detected, effectively improving the accuracy of headlight defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0046] Figure 1 A flow chart of a lighting defect recognition model training method provided in an embodiment of the present disclosure;

[0047] Figure 2 A schematic diagram of a sample image of a vehicle light provided by an embodiment of the present disclosure;

[0048] Figure 3 A schematic diagram of an overlapping area provided in an embodiment of the present disclosure;

[0049] Figure 4 A schematic diagram of a plurality of first region of interest images corresponding to a sample image of a vehicle headlight provided in an embodiment of the present disclosure;

[0050] Figure 5 A schematic diagram of a plurality of second region of interest images corresponding to a sample image of a vehicle headlight provided in an embodiment of the present disclosure;

[0051] Figure 6 Schematic diagram of sample cropped image provided by the embodiment of the present disclosure

[0052] Figure 7 A schematic diagram of a sample cropped image provided in an embodiment of the present disclosure;

[0053] Figure 8 A schematic diagram of a sample cropped image provided in an embodiment of the present disclosure;

[0054] Fig. 9 A schematic diagram of a sample cropped image provided in an embodiment of the present disclosure;

[0055] Fig.10 A flow chart of a light defect detection method provided by another embodiment of the present disclosure;

[0056] Fig.11 A lighting defect recognition model training device provided in an embodiment of the present disclosure;

[0057] Fig.12 A schematic diagram of the structure of a light defect detection device provided in an embodiment of the present disclosure;

[0058] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0061] The present disclosure provides a method for detecting light defects, which will be described below in conjunction with specific embodiments.

[0062] Figure 1This is a flow chart of the lighting defect recognition model training method provided in the embodiment of the present disclosure. The method can be applied to any terminal device with data processing capabilities, such as car computers, smart phones, PDAs, tablet computers, laptops, all-in-one machines, smart driving devices, etc. It is understandable that the lighting defect recognition model training method provided in the embodiment of the present disclosure can also be applied in other scenarios.

[0063] Below Figure 1 The lighting defect recognition model training method shown in the figure is introduced, and the specific steps of the method are as follows:

[0064] S101: Acquire at least one vehicle light sample image.

[0065] The headlight sample images are photos of vehicle exterior lights collected on site, which are the data basis for building a light defect recognition model. For example, the number of headlight sample images can be 1,000.

[0066] The headlight sample image in this step may be a headlight image that has been preliminarily processed, so as to facilitate input into an image processing network or model in subsequent steps.

[0067] Specifically, at least one original image of the headlight is obtained; the minimum envelope rectangle of the headlight area in each original image of the headlight is extracted through an instance segmentation network to obtain at least one headlight envelope rectangle image, and the at least one original image of the headlight corresponds to the at least one headlight envelope rectangle image one-to-one; for each headlight envelope rectangle image, pixels outside the headlight area are set to zero to obtain at least one headlight sample image, and the at least one headlight envelope rectangle image corresponds to the at least one headlight sample image one-to-one.

[0068] Figure 2 A schematic diagram of a sample image of a vehicle light provided by an embodiment of the present disclosure. Figure 2 As shown, the image is the minimum enveloping rectangular range image of the integrated through-type headlight area.

[0069] S102 . For each headlight sample image, perform a cropping operation on the headlight sample image to obtain a plurality of sample cropped images, wherein an overlapping area between two adjacent sample cropped images is larger than a preset area.

[0070] The preset area may be the area of ​​the largest defect region in the sample image of the vehicle headlight.

[0071] In some embodiments, the width of the overlapping area between two adjacent sample cropped images can be determined according to the width of the overlapping area between the two adjacent sample cropped images, so that the width of the overlapping area between the two adjacent sample cropped images is greater than the width of the largest defect area in the sample image of the vehicle lamp. Specifically, the width of the defect area refers to the maximum length of the defect area in the image in the horizontal direction.

[0072] In some embodiments, the sample images of the headlights can be cropped based on the attention mechanism to obtain multiple sample cropped images. The basic idea of ​​the attention mechanism is to let the system learn to ignore irrelevant information and focus on key information. Figure 2 In the obtained headlight sample image, the key information for the headlight detection task should be the headlight area. Therefore, in fact, this step performs a cropping operation based on the attention mechanism on the headlight sample image, that is, the headlight area in the headlight sample image is cropped to obtain multiple sample cropped images, each of which corresponds to an AFROI (Attention-Focused ROI).

[0073] To ensure that the defective area in the headlight sample image can be completely included in at least one sample cropped image, an overlapping area is set between two adjacent sample cropped images, and the width of the overlapping area is greater than the width of the largest defect in the cropped headlight sample image. Figure 3 A schematic diagram of an overlapping area provided in an embodiment of the present disclosure. Figure 3 As shown, rectangular frame 31 , rectangular frame 32 , rectangular frame 33 , and rectangular frame 34 represent two adjacent sample cropped images, respectively. There is an overlapping area between rectangular frame 31 and rectangular frame 32 , and there is an overlapping area between rectangular frame 33 and rectangular frame 34 .

[0074] In some embodiments, the area or width of the overlapping region may also be set based on the maximum value statistically obtained from the uneven defects encountered in actual production, or flexibly based on actual needs, and the embodiments of the present disclosure are not limited to this.

[0075] S103: Determine a label classification result corresponding to each sample cropped image, wherein the label classification result is one of a normal headlight image and an abnormal headlight image.

[0076] After cropping the headlight sample image to obtain multiple sample cropped images, it is necessary to determine whether there is a defective area in each sample cropped image, that is, to determine whether there is a defect in the actual headlight range corresponding to the sample image. Based on this, all sample cropped images are divided into normal headlight images or abnormal headlight images, and the images are assigned corresponding class labels.

[0077] S104, using the sample cropped image as the input of the light defect recognition model to be trained, using the label classification result corresponding to the sample cropped image as the output of the light defect recognition model to be trained, training the light defect recognition model to be trained, and obtaining a trained light defect recognition model.

[0078] After the sample cropped images are classified, a training data set of the light defect recognition model is obtained, which can be used to train the light defect recognition model to be trained. Specifically, the sample cropped images are used as the input of the light defect recognition model to be trained, and the label classification results corresponding to the sample cropped images are used as the output of the light defect recognition model to be trained. The light defect recognition model to be trained is trained to obtain a trained light defect recognition model.

[0079] Among them, the lighting defect recognition model can be a deep learning classification task network (including but not limited to ResNet series, MobileNet series, ShuffleNet series, SENet series, Transformer series). The lighting defect recognition model to be trained is built based on the deep learning classification task network, and is trained using a training data set. The trained classification network structure and the weight parameters of each layer therein are saved, and a trained lighting defect recognition model is obtained.

[0080] In some embodiments, the types of vehicle headlight defects can also be classified, and corresponding sample data sets can be established for different defects to train the light defect recognition model, so that the trained light defect recognition model can output whether there is a defective area in the vehicle headlight image to be detected, and can also output the defect type of the vehicle headlight image to be detected. The embodiments of the present disclosure are not limited to this.

[0081] The disclosed embodiment acquires at least one sample image of a headlight; for each sample image of the headlight, performs a cropping operation on the sample image of the headlight to obtain a plurality of sample cropped images, wherein the area of ​​the overlapping region between two adjacent sample cropped images is greater than a preset area; determines a label classification result corresponding to each sample cropped image, wherein the label classification result is one of a normal headlight image and an abnormal headlight image; uses the sample cropped image as an input of a light defect recognition model to be trained, uses the label classification result corresponding to the sample cropped image as an output of the light defect recognition model to be trained, and trains the light defect recognition model to be trained to obtain a trained light defect recognition model; when cropping the sample image of the headlight, an overlapping region is provided between each sample cropped image, and the area of ​​the overlapping region is set to be greater than a preset area; regardless of the size of the defect region in the sample image of the headlight, the defect region can be completely covered by at least one sample cropped image, so that the light defect recognition model trained based on the sample cropped image can recognize defects of any size in the headlight to be detected, thereby effectively improving the accuracy of headlight defect detection.

[0082] On the basis of the above embodiment, for each headlight sample image, a cropping operation is performed on the headlight sample image to obtain multiple sample cropped images, including: performing an attention-based cropping operation on the headlight sample image according to a preset headlight mask to obtain multiple first region of interest images, wherein the direction of each first region of interest image is consistent with the direction of the headlights in the headlight sample image; rotating each first region of interest image to obtain a second region of interest image corresponding to each first region of interest image, wherein the direction of each second region of interest image is horizontal; scaling each second region of interest image to obtain multiple sample cropped images of the same size.

[0083] The mask is used to control the image processing area by blocking the processed image (all or part) through a selected image, graphic or object. In the embodiment of the present disclosure, the preset headlight mask is used to determine the headlight area in the headlight sample image. The headlight area in the headlight sample image can be determined by multiplying the preset headlight mask with the headlight sample image. The headlight area is further cropped based on the attention mechanism to obtain multiple first region of interest images. The direction of each first region of interest image is consistent with the direction of the headlight in the headlight sample image, that is, the segmented ROI (Rotated-Attention-Focused ROI, RAFROI) of the headlight area under the attention mechanism is obtained. Figure 4 Schematic diagram of multiple first region of interest images corresponding to the vehicle light sample image provided in the embodiment of the present disclosure. Figure 4As shown, each rectangular frame corresponds to a first region of interest image, and the directions thereof are consistent with the direction of the vehicle lights. There is also an overlapping area between two adjacent first region of interest images.

[0084] Each of the first region of interest images is rotated to obtain a second region of interest image corresponding to each first region of interest image, that is, a segmented ROI (Aligned-Attention-Focused ROI, AAFROI) under an aligned attention focused mechanism is obtained after rotation. Figure 5 Schematic diagram of multiple second region of interest images corresponding to the vehicle light sample image provided in the embodiment of the present disclosure. Figure 5 As shown, each rectangular frame corresponds to a second region of interest image, and the directions thereof are all horizontal.

[0085] A scaling operation is performed on each of the second region-of-interest images to obtain a plurality of sample cropped images of the same size. Figure 6-Figure 9 This is a schematic diagram of a sample cropped image provided by an embodiment of the present disclosure. Figure 6-Figure 9 As shown, after the scaling operation, the multiple second region of interest images are processed into multiple sample cropped images of the same size. In the embodiment of the present disclosure, a square sample cropped image is taken as an example. The processed images are convenient for inputting into the lighting defect recognition model to train the model. Figure 6 , Figure 7 is a normal sample cropped image, and its corresponding label classification result is a normal car light image, while Figure 8 , Fig. 9 The image is cropped for the abnormal sample, and its corresponding label classification result is an abnormal headlight image, indicating that the corresponding headlight area has color abnormality.

[0086] In some embodiments, data expansion can also be performed on the sample cropped images whose label classification results are abnormal headlight images through deep learning. For example, a deep learning method based on CycleGan is used to perform data expansion on the sample cropped images whose label classification results are abnormal headlight images, and style conversion and other operations are performed using the sample cropped images whose label classification results are abnormal headlight images actually obtained, so as to obtain more expanded images whose label classification results are abnormal headlight images, thereby increasing the amount of data for training the light defect recognition model at a lower cost, and improving the effect and efficiency of the light defect recognition model training.

[0087] Fig.10 This is a flow chart of a method for detecting light defects provided by another embodiment of the present disclosure. Fig.10 As shown, the method includes the following steps:

[0088] S1001. Acquire an image of a vehicle headlight to be detected.

[0089] Specifically, an original image to be detected corresponding to the headlight to be detected is obtained; and a headlight region in the original image to be detected is extracted through an instance segmentation network to obtain the headlight image to be detected.

[0090] S1002: Crop the image of the vehicle headlight to be detected to obtain a plurality of cropped images of the vehicle headlight to be detected, wherein there is an overlapping area between two adjacent cropped images of the vehicle headlight to be detected.

[0091] Referring to the above embodiment in which the clipping operation is performed on the vehicle light sample image, the vehicle light image to be detected is clipped in the same manner as the vehicle light sample image.

[0092] Specifically, the area of ​​the overlapping region between each cropped headlight image to be detected can be determined according to the area of ​​the overlapping region between two adjacent sample cropped images. For example, the overlapping region between each cropped headlight image to be detected is smaller than a preset area.

[0093] S1003, inputting a plurality of cropped vehicle headlight images to be detected into the trained light defect recognition model, and receiving an output of the trained light defect recognition model as a detection result of the vehicle headlight images to be detected.

[0094] The lighting defect recognition model is obtained by the lighting defect recognition model training method as described in the above embodiment.

[0095] S1004: If it is determined according to the detection result that there is a defective area in the image of the vehicle headlight to be detected, determine a cropped image of the vehicle headlight to be detected corresponding to the defective area.

[0096] According to the output of the light defect recognition model, it can be determined whether there is a defective area in the headlight image to be detected, that is, whether the headlight to be detected has a defect. Specifically, a classification model can be established to classify the cropped headlight images to be detected corresponding to the headlight image to be detected. As long as one cropped headlight image to be detected is classified as an abnormal image, it is considered that the side light to be detected has a defect.

[0097] S1005: Determine an actual fault position corresponding to the defective area according to the position of the defective area in the cropped image of the vehicle headlight to be detected and the position of the cropped image of the vehicle headlight to be detected in the image of the vehicle headlight to be detected.

[0098] Since a single light defect recognition model can be trained for one type of vehicle light, the position of each cropped vehicle light image to be detected in the vehicle light image to be detected is relatively fixed.

[0099] On this basis, first determine whether the defective area is in the overlapping area of ​​the cropped image of the headlight to be detected and the adjacent cropped image of the headlight to be detected. If so, determine the actual fault position corresponding to the defective area based on the positions of the two adjacent cropped images of the headlight to be detected in the image of the headlight to be detected; if not, determine the actual fault position corresponding to the defective area based on the position of the cropped image of the headlight to be detected corresponding to the defective area in the image of the headlight to be detected.

[0100] The disclosed embodiment can determine the location of the actual fault in the headlight by the position of the cropped headlight image to be detected according to the position of the fault area in the headlight image to be detected, so as to quickly locate the fault position and further improve the efficiency of headlight fault identification.

[0101] Fig.11 The lighting defect recognition model training device provided in the embodiment of the present disclosure. The lighting defect recognition model training device may be a terminal device with data processing capability as described in the above embodiment, or the lighting defect detection device may be a component or assembly in the terminal device with data processing capability. The lighting defect recognition model training device provided in the embodiment of the present disclosure may execute the processing flow provided in the lighting defect recognition model training method embodiment, such as Fig.11 As shown, the lighting defect recognition model training device 110 includes: a first acquisition module 111, a first cropping module 112, a first determination module 113, and a training module 114; the first acquisition module 111 is used to acquire at least one vehicle light sample image; the first cropping module 112 is used to crop each vehicle light sample image to obtain a plurality of sample cropped images, wherein the area of ​​the overlapping region between two adjacent sample cropped images is greater than a preset area; the determination module 113 is used to determine the label classification result corresponding to each sample cropped image, wherein the label classification result is one of a normal vehicle light image and an abnormal vehicle light image; the training module 114 is used to use the sample cropped image as the input of the lighting defect recognition model to be trained, and use the label classification result corresponding to the sample cropped image as the output of the lighting defect recognition model to be trained, and train the lighting defect recognition model to be trained to obtain a trained lighting defect recognition model.

[0102] Optionally, the first cropping module 112 includes a cropping unit 1121, a rotation unit 1122, and a scaling unit 1123; the cropping unit 1121 is used to perform a cropping operation on the headlight sample image according to a preset headlight mask to obtain multiple first area of ​​interest images, wherein the direction of each first area of ​​interest image is consistent with the direction of the headlights in the headlight sample image; the rotation unit 1122 is used to rotate each first area of ​​interest image to obtain a second area of ​​interest image corresponding to each first area of ​​interest image, wherein the direction of each second area of ​​interest image is horizontal; the scaling unit 1123 is used to scale each second area of ​​interest image to obtain multiple sample cropped images of the same size.

[0103] Optionally, the lighting defect recognition model training device 110 further includes an expansion module 115 for performing data expansion on the sample cropped images whose label classification results are abnormal vehicle light images through deep learning.

[0104] Optionally, the first acquisition module 111 includes a first acquisition unit 1111, a first extraction unit 1112, and a processing unit 1113; the first acquisition unit 1111 is used to acquire at least one original headlight image; the first extraction unit 1112 is used to extract the minimum envelope rectangle of the headlight area in each original headlight image through an instance segmentation network to obtain at least one headlight envelope rectangle image, and the at least one headlight original image corresponds one-to-one to the at least one headlight envelope rectangle image; the processing unit 1113 is used to set the pixels outside the headlight area to zero for each headlight envelope rectangle image to obtain at least one headlight sample image, and the at least one headlight envelope rectangle image corresponds one-to-one to the at least one headlight sample image.

[0105] Fig.11 The lighting defect recognition model training device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0106] Fig.12 The structure diagram of the light defect detection device provided in the embodiment of the present disclosure. The light defect detection device may be a terminal device with data processing capability as described in the above embodiment, or the light defect detection device may be a component or assembly in the terminal device with data processing capability. The light defect detection device provided in the embodiment of the present disclosure may execute the processing flow provided in the light defect detection method embodiment, such as Fig.12As shown, the light defect detection device 120 includes a second acquisition module 121, a second cropping module 122, and a receiving module 123; the second acquisition module 121 is used to acquire the image of the headlight to be detected; the second cropping module 122 is used to crop the image of the headlight to be detected to obtain multiple cropped images of the headlight to be detected, wherein there is an overlapping area between two adjacent cropped images of the headlight to be detected; the receiving module 123 is used to input the multiple cropped images of the headlight to be detected into a trained light defect recognition model, and receive the output of the trained light defect recognition model as the detection result of the image of the headlight to be detected.

[0107] Optionally, the light defect detection device 120 also includes a second determination module 124 and a third determination module 125; the second determination module 124 is used to determine the cropped image of the headlight to be detected corresponding to the defective area if it is determined according to the detection result that there is a defective area in the headlight image to be detected; the third determination module 125 is used to determine the actual fault position corresponding to the defective area according to the position of the defective area in the cropped image of the headlight to be detected and the position of the cropped image of the headlight to be detected in the headlight image to be detected.

[0108] Fig.12 The lighting defect detection device of the illustrated embodiment can be used to implement the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effect are similar and will not be described in detail here.

[0109] In addition, an embodiment of the present disclosure also provides a vehicle, which includes a lighting defect recognition model training device and / or a lighting defect detection device as described in the above embodiments.

[0110] Fig.13 The electronic device provided in the embodiment of the present disclosure can be a terminal device with data processing capability as described in the above embodiment. The electronic device provided in the embodiment of the present disclosure can execute the processing flow provided in the light defect recognition model training method or the light defect detection method embodiment, such as Fig.13 As shown, the electronic device 130 includes: a memory 131, a processor 132, a computer program and a communication interface 133; wherein the computer program is stored in the memory 131 and is configured so that the processor 132 executes the lighting defect recognition model training method or the lighting defect detection method as described above.

[0111] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the lighting defect recognition model training method or the lighting defect detection method described in the above embodiment.

[0112] In addition, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the lighting defect recognition model training method or the lighting defect detection method as described above is implemented.

[0113] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0116] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A lighting defect recognition model training method, characterized in that: The method comprises: Obtain at least one sample image of a vehicle light; For each headlight sample image, a cropping operation is performed on the headlight sample image to obtain a plurality of sample cropped images, wherein an area of ​​an overlapping region between two adjacent sample cropped images is greater than a preset area; Determine a label classification result corresponding to each sample cropped image, wherein the label classification result is one of a normal headlight image and an abnormal headlight image; The sample cropped image is used as the input of the light defect recognition model to be trained, and the label classification result corresponding to the sample cropped image is used as the output of the light defect recognition model to be trained. The light defect recognition model to be trained is trained to obtain a trained light defect recognition model.

2. The method according to claim 1, characterized in that: The step of performing a cropping operation on each headlight sample image to obtain a plurality of sample cropped images includes: Performing a cropping operation on the headlight sample image according to a preset headlight mask to obtain a plurality of first region of interest images, wherein the direction of each first region of interest image is consistent with the direction of the headlight in the headlight sample image; Rotating each of the first region-of-interest images to obtain a second region-of-interest image corresponding to each of the first region-of-interest images, wherein the direction of each of the second region-of-interest images is a horizontal direction; A scaling operation is performed on each of the second region-of-interest images to obtain a plurality of sample cropped images of the same size.

3. The method according to claim 1, characterized in that After determining the label classification results corresponding to each sample cropped image, the method further includes: Data expansion is performed on the sample cropped images whose label classification results are abnormal headlight images through deep learning.

4. The method according to claim 1, characterized in that: The obtaining of at least one vehicle light sample image comprises: Obtain at least one original image of a vehicle light; Extracting the minimum envelope rectangle of the headlight region in each headlight original image through an instance segmentation network to obtain at least one headlight envelope rectangle image, wherein the at least one headlight original image corresponds to the at least one headlight envelope rectangle image in a one-to-one manner; For each headlight envelope rectangular image, pixels outside the headlight area are set to zero to obtain at least one headlight sample image, and the at least one headlight envelope rectangular image corresponds to the at least one headlight sample image one by one.

5. A method for detecting light defects, characterized in that: The method comprises: Acquire the image of the vehicle headlight to be detected; Performing a cropping operation on the image of the vehicle headlight to be detected to obtain a plurality of cropped images of the vehicle headlight to be detected, wherein there is an overlapping area between two adjacent cropped images of the vehicle headlight to be detected; Input a plurality of cropped images of vehicle lights to be detected into a trained light defect recognition model, and receive the output of the trained light defect recognition model as the detection result of the vehicle lights to be detected. The light defect recognition model is obtained by the light defect recognition model training method as described in claims 1-4.

6. The method according to claim 5, characterized in that After inputting the plurality of cropped vehicle headlight images to be detected into the trained light defect recognition model and receiving the output of the trained light defect recognition model as the detection result of the vehicle headlight images to be detected, the method further includes: If it is determined according to the detection result that there is a defective area in the image of the vehicle headlight to be detected, then a cropped image of the vehicle headlight to be detected corresponding to the defective area is determined; According to the position of the defective area in the cropped image of the vehicle lamp to be detected and the position of the cropped image of the vehicle lamp to be detected in the image of the vehicle lamp to be detected, the actual fault position corresponding to the defective area is determined.

7. A lighting defect model training device, characterized in that: The device comprises: A first acquisition module, used to acquire at least one vehicle light sample image; A first cropping module is used to perform a cropping operation on each headlight sample image to obtain a plurality of sample cropped images, wherein an area of ​​an overlapping region between two adjacent sample cropped images is greater than a preset area; A first determination module is used to determine a label classification result corresponding to each sample cropped image, wherein the label classification result is one of a normal headlight image and an abnormal headlight image; The training module is used to use the sample cropped image as the input of the light defect recognition model to be trained, and use the label classification result corresponding to the sample cropped image as the output of the light defect recognition model to be trained, so as to train the light defect recognition model to be trained and obtain a trained light defect recognition model.

8. A lighting defect detection device, characterized in that: The device comprises: The second acquisition module is used to acquire the image of the vehicle headlight to be detected; A second cropping module is used to crop the image of the vehicle headlight to be detected to obtain a plurality of cropped images of the vehicle headlight to be detected, wherein there is an overlapping area between two adjacent cropped images of the vehicle headlight to be detected; The receiving module is used to input a plurality of cropped vehicle headlight images to be detected into a trained light defect recognition model, and receive the output of the trained light defect recognition model as the detection result of the vehicle headlight images to be detected.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images