Vehicle lamp bead quality detection method of vehicle, electronic equipment and storage medium
By extracting the binarization and RGB feature data in the car light image, and using preset model processing, the coordinate information and quality information of the lamp beads are obtained, combined with confidence screening, the problem of inaccurate detection of car light quality is solved and the accuracy of detection is improved.
Patent Information
- Application Number
- CN202411903411.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-23
- Publication Date
- 2025-04-15
AI Technical Summary
In the prior art, the accuracy of the quality detection of the car light beads is not high, resulting in inaccurate detection of the car light quality.
By obtaining the image data of the car lights, extracting the binary feature data and RGB feature data, and inputting them into the preset model for processing, we obtain the coordinate information and the light bead quality information corresponding to the coordinate box of the light bead, as well as the confidence of the coordinate information, and filtering out the accurate light bead related information based on the confidence.
Improve the accuracy of lamp bead quality inspection and ensure the reliability of car light quality inspection.
Smart Images
Figure CN119832319A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of vehicle lamp quality detection, and in particular to a vehicle lamp quality detection method, electronic equipment and storage medium. Background Art
[0002] The light emitted by the vehicle headlights can ensure a good vision when driving at night. The headlights are an important device to ensure safe driving of the vehicle. The headlights are generally composed of many lamp beads, and multiple lamp beads emit light at the same time to ensure the brightness of the headlights.
[0003] In order to ensure the lighting intensity of the headlights, the headlights will be inspected for quality. If the number of damaged and non-luminous lamp beads is greater than a certain percentage, the headlights are considered to be of unqualified quality. In the related art, during the quality inspection of headlights, the accuracy of the lamp bead quality inspection is not high, resulting in low accuracy of headlight quality inspection. Summary of the invention
[0004] In view of the above problems, the present application provides a method, electronic device and storage medium for detecting the quality of vehicle headlight beads, which obtains more accurate lamp bead-related information based on binary feature data and RGB feature data, and filters the lamp bead-related information according to the confidence level to make it more accurate, thereby improving the accuracy of lamp bead quality detection.
[0005] The first aspect of the present application provides a method for detecting the quality of lamp beads of a vehicle headlight, comprising: obtaining image data of the headlight; wherein the headlight comprises a plurality of lamp beads; obtaining binary feature data and RGB feature data of the image data, and inputting the binary feature data and the RGB feature data into a preset model for processing to obtain first initial data, second initial data and third initial data; processing the first initial data, the second initial data and the third initial data by an encoding module and a decoding module of the preset model to obtain coordinate information corresponding to a coordinate frame of the lamp beads, lamp bead quality information and confidence corresponding to the coordinate information; obtaining coordinate information and lamp bead quality information higher than a preset confidence as target coordinate information and target lamp bead quality information, and obtaining the lamp bead quality of the headlight based on the target coordinate information and the target lamp bead quality information.
[0006] In some specific embodiments, the steps of inputting the binary feature data and the RGB feature data into a preset model for processing to obtain first initial data, second initial data, and third initial data include: inputting the binary feature data into a first multi-layer convolution layer for feature extraction to output first convolution data, and inputting the RGB feature data into a second multi-layer convolution layer for feature extraction to output second convolution data; fusing the first convolution data with the second convolution data to obtain first fused data, and inputting the first fused feature data into a backbone network to output first initial data, second initial data, and third initial data.
[0007] In some specific embodiments, the first initial data, the second initial data, and the third initial data are processed by the encoding module and the decoding module of the preset model to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information, and the confidence corresponding to the coordinate information, including: performing feature extraction on the first initial data, the second initial data, and the third initial data by the first feature extraction module and the second feature extraction module of the encoding module of the preset model to obtain initial extracted data; performing feature extraction on the initial extracted data by the decoding module of the preset model to obtain target extracted data, and processing the target extracted data by the fully connected layer of the decoding module to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information, and the confidence corresponding to the coordinate information.
[0008] In some specific embodiments, the step of performing feature extraction on the first initial data, the second initial data and the third initial data through the first feature extraction module and the second feature extraction module of the encoding module of the preset model to obtain initial extracted data includes: performing feature extraction on the first initial data through the first feature extraction module in the encoding module of the preset model to obtain first extracted data; performing feature extraction on the first extracted data, the second initial data and the third initial data through the second feature extraction module of the encoding module to obtain second extracted data, third extracted data and fourth extracted data as initial extracted data.
[0009] In some specific embodiments, the step of performing feature extraction on initial extracted data through a decoding module of a preset model to obtain target extracted data includes: performing feature extraction on second extracted data, third extracted data and fourth extracted data through a preset three-layer convolutional layer of the decoding module of the preset model to obtain fifth extracted data, sixth extracted data and seventh extracted data; fusing the fifth extracted data, the sixth extracted data and the seventh extracted data to form second fused data, performing feature extraction on the second fused data through a preset attention module in the decoding module to obtain eighth extracted data, and using the eighth extracted data as target extracted data.
[0010] In some specific embodiments, the step of performing feature extraction on initial extracted data through a decoding module of a preset model to obtain target extracted data includes: performing feature extraction on the second extracted data, the third extracted data and the fourth extracted data through a preset three-layer convolutional layer of the decoding module of the preset model to obtain fifth extracted data, sixth extracted data and seventh extracted data; fusing the fifth extracted data, the sixth extracted data and the seventh extracted data to form second fused data, performing feature extraction on the second fused data through a preset attention module in the decoding module to obtain eighth extracted data; performing feature deletion processing on the eighth extracted data through a preset processing layer in the decoding module to obtain ninth extracted data, and using the ninth extracted data as target extracted data.
[0011] In some specific embodiments, the step of processing the target extraction data through the fully connected layer of the decoding module to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information, and the confidence corresponding to the coordinate information includes: processing the target extraction data through the first fully connected layer in the decoding module to obtain the coordinate information of the coordinate frame of the lamp bead; processing the target extraction data through the second fully connected layer in the decoding module to obtain the lamp bead quality information corresponding to the coordinate frame of the lamp bead and the confidence corresponding to the coordinate information.
[0012] In some specific embodiments, the method also includes: obtaining a connected area of the corresponding image based on the image data; determining a connected area that meets preset requirements based on the shape, size and brightness of the pixels in the connected area, and the connected area that meets the preset requirements will be used as a target connected area; the step of obtaining the lamp bead quality of the car light based on the target coordinate frame information and the target lamp bead quality information includes: obtaining the lamp bead quality of the car light based on the target coordinate frame information and the corresponding target lamp bead quality information that the coordinate frame is located in the target connected area.
[0013] A second aspect of the present application provides an electronic device, comprising: a processor; a memory for storing a computer program, which implements any of the above-mentioned methods for detecting the quality of vehicle lamp beads when the computer program is executed by the processor.
[0014] A third aspect of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, a method for detecting the quality of a lamp bead of a vehicle headlight as described above is implemented.
[0015] The present application has at least the following beneficial technical effects: Based on the vehicle headlight lamp bead quality detection method, electronic device and storage medium provided by the present application, the method includes: obtaining image data of the headlight; wherein the headlight includes a plurality of lamp beads; obtaining binary feature data and RGB feature data of the image data, inputting the binary feature data and RGB feature data into a preset model for processing to obtain first initial data, second initial data and third initial data; processing the first initial data, second initial data and third initial data through the encoding module and decoding module of the preset model to obtain the coordinate information corresponding to the coordinate frame of the lamp bead and the lamp bead quality information and the confidence corresponding to the coordinate information; obtaining the coordinate information and lamp bead quality information higher than the preset confidence as the target coordinate information and the target lamp bead quality information, and obtaining the lamp bead quality of the headlight based on the target coordinate information and the target lamp bead quality information. Therefore, relatively accurate lamp bead related information is obtained based on the binary feature data and the RGB feature data, and the lamp bead related information is screened according to the confidence to make it more accurate, thereby improving the accuracy of lamp bead quality detection.
[0016] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:
[0018] Figure 1 It is a flow chart of an embodiment of a method for detecting the quality of lamp beads of a vehicle lamp provided by the present application;
[0019] Figure 2 is a vehicle light image acquired by an image acquisition device;
[0020] Figure 3 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided by the present application;
[0021] Figure 4 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided by the present application;
[0022] Figure 5 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided by the present application;
[0023] Figure 6It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided by the present application;
[0024] Figure 7 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided by the present application;
[0025] Figure 8 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided by the present application;
[0026] Fig. 9 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided by the present application;
[0027] Fig.10 It is a schematic diagram of the structural framework of an embodiment of the electronic device provided by the present application;
[0028] Fig.11 It is a schematic diagram of the structural framework of an embodiment of the computer-readable storage medium provided by the present application. DETAILED DESCRIPTION
[0029] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.
[0030] If there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited to "first" and "second" may explicitly or implicitly include at least one of the features. In addition, if the meaning of "and / or" appearing in the full text is to include three parallel schemes, taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0031] A first aspect of the present application provides a method for testing the quality of lamp beads of a vehicle headlight. This method can perform quality testing on the lamp beads of the vehicle headlights, but is certainly not limited to the testing of the headlights. Figure 1 This is a flow chart of an embodiment of the method for detecting the quality of lamp beads of a vehicle lamp provided by the present application, combined with Figure 1, the method comprises the following steps:
[0032] S101: Acquire image data of a vehicle lamp; wherein the vehicle lamp includes a plurality of lamp beads.
[0033] In order to obtain image data of the headlights, an image acquisition device may be pre-installed on the vehicle production line. The image acquisition device actively triggers shooting when a vehicle passes by to obtain the headlight image, and then processes the headlight image to obtain the image data of the headlight.
[0034] Figure 2 It is a car light image captured by an image acquisition device.
[0035] Combination Figure 2 A plurality of small lamp beads are arranged in a headlight, so that the light beam of the headlight is formed by the light emitted by the plurality of small lamp beads together. The headlight lamp bead quality detection method of the present application is used to perform quality detection on all the lamp beads of the headlight, and then the quality detection result of the headlight is obtained by combining the quality detection results of all the lamp beads.
[0036] S102: Obtaining binary feature data and RGB feature data of the image data, and inputting the binary feature data and the RGB feature data into a preset model for processing to obtain first initial data, second initial data, and third initial data.
[0037] As for the method of obtaining binary feature data, the image data can be read through opencv and grayscaled, and then the original features are extracted using the binarization function and normalized to 0 or 1, thereby obtaining the binary feature data.
[0038] The binary feature data finally obtained is an array data with four data dimensions, and the data format is [B, C, H, W], and the four data dimensions are B, C, H, W. At this time, B is the batch size (representing how much data is sent to the model at a time in the subsequent steps), C is the number of channels, H and W are the height and width of the headlight image after resizing, and the number of channels of the binary feature data C = 1.
[0039] As for the method of obtaining RGB feature data, you can use the cv2.imread function to read the image data, and use the cv2.cvtColor function to convert the channel from BGR to RGB to obtain the original RGB value, and then normalize it to normalize the original RGB value to obtain the normalized data.
[0040] For example, the original RGB value is [230, 100, 20]. After normalization, it is divided by 255 to get [0.0784, 0.0824, 0.0980]. Then, the average value [0.485, 0.456, 0.406] is subtracted in RGB order, and then divided by the standard deviation [0.229, 0.224, 0.225] to get [-1.7754, -1.6681, -1.3687] as the normalized data, and finally get the RGB feature data. The data format of RGB feature data is also [B, C, H, W], and the number of channels C = 3.
[0041] Among them, the preset model is pre-set and trained. After obtaining the binary feature data and the RGB feature data, the two can be input into the preset model for further processing. Of course, this embodiment only shows the input of the binary feature data and the RGB feature data into the preset model. The data required by other models can also be input into this model. Because other input data does not involve the invention point of this application, it will not be introduced too much.
[0042] The preset model performs data processing based on the input binary feature data and RGB feature data, specifically feature extraction processing, and finally obtains first initial data, second initial data, and third initial data. At this time, the first initial data, the second initial data, and the third initial data are also array data with four data dimensions.
[0043] S103: Processing the first initial data, the second initial data and the third initial data through the encoding module and the decoding module of the preset model to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information and the confidence corresponding to the coordinate information.
[0044] The preset model is pre-set with an encoding module and a decoding module. After obtaining the first initial data, the second initial data and the third initial data, they are further processed by the encoding module and the decoding module. The specific processing method includes feature extraction. Among them, the encoding module can process the first initial data, the second initial data and the third initial data to obtain intermediate data, and then process the intermediate data through the decoding module to obtain final data, and then obtain the required coordinate information, quality information and confidence through the final data.
[0045] It should be understood that one lamp bead corresponds to a lamp bead coordinate frame, and the lamp bead is located in the corresponding coordinate frame. The specific position of the corresponding lamp bead can be determined based on the coordinate information of the lamp bead's coordinate frame. The quality information corresponding to the coordinate frame is the quality information of the lamp bead corresponding to the coordinate frame. The quality information may include two modes: normal light emission and abnormal light emission. Among them, for the quality information of the lamp bead, after determining the coordinate frame, the brightness information in the coordinate frame can be used to determine whether the light emission is normal or abnormal. The confidence of the coordinate information reflects the accuracy of the coordinate frame for the lamp bead and the accuracy of the coordinate information of the coordinate frame. The higher the confidence, the higher the accuracy.
[0046] S104: Acquire coordinate information and lamp bead quality information with a higher reliability than a preset value as target coordinate information and target lamp bead quality information, and acquire the lamp bead quality of the vehicle lamp based on the target coordinate information and the target lamp bead quality information.
[0047] The setting of the preset confidence can be based on actual needs and can be changed according to the application situation. When the confidence obtained in the above steps is higher than the preset confidence, it means that the coordinate frame is more likely to be the coordinate frame of the lamp bead and the coordinate information of the coordinate frame is more likely to be accurate. At this time, the coordinate information and the lamp bead quality information are used as the target coordinate information and the target lamp bead quality information. The target coordinate information and the target lamp bead quality information are information that can be used in subsequent processes.
[0048] Among them, one target coordinate information corresponds to one target lamp bead quality information. The target lamp bead quality information corresponding to the target coordinate information reflects the quality information of the lamp bead of the car light. Through the target lamp bead quality information, the damage ratio of the lamp bead can be obtained, and then the quality information of the car light can be obtained. For example, one hundred target coordinate information is obtained, and one hundred target coordinate information corresponds to one hundred target lamp bead quality information. Among them, ninety of the one hundred target lamp bead quality information are normal, then the normal luminescence rate of the lamp bead of the car light is 90%. If the car light with a luminescence rate of the lamp bead lower than 95% is regarded as an abnormal car light, then the car light quality detection information of abnormal quality is obtained.
[0049] In summary, based on the binary feature data and RGB feature data, more accurate lamp bead related information can be obtained, and the lamp bead related information can be screened according to the confidence level to make the final target coordinate information and target lamp bead quality information more accurate, thereby improving the accuracy of lamp bead quality detection and improving the accuracy of vehicle light quality detection.
[0050] Figure 3 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided in the present application.
[0051] Combination Figure 3In some specific embodiments, the step of inputting the binary feature data and the RGB feature data into a preset model for processing to obtain the first initial data, the second initial data and the third initial data includes:
[0052] S201: Input the binary feature data into a first multi-layer convolution layer for feature extraction to output first convolution data, and input the RGB feature data into a second multi-layer convolution layer for feature extraction to output second convolution data.
[0053] The first multi-layer convolution layer is preset in the preset model, and the main function of the first multi-layer convolution layer is to perform feature extraction and then form the first convolution data. Among them, the first multi-layer convolution layer is used to extract feature information of brightness difference to better judge the quality of the lamp beads.
[0054] In combination with the above content, the data format of the binary feature data is [B, 1, H, W], and the data format of the first convolution data obtained after the first multi-layer convolution layer is [B, 64, dim0, dim1]. At this time, the number of channels of the first convolution data becomes 64. In some application scenarios, the first multi-layer convolution layer can be conv_bin. Since most detection models do not have such a convolution layer, conv_bin can be added in the process of forming a preset model.
[0055] The second convolution layer is also preset in the preset model. The main function of the second multi-layer convolution layer is to extract features and then form the second convolution data. The second convolution layer is mainly used to extract feature information such as edge information of the image. Combined with the above content, the data format of RGB feature data is [B, 3, H, W], and the data format of the second convolution data obtained by processing the second multi-layer convolution layer is [B, 64, dim0, dim1]. In some application scenarios, the second multi-layer convolution layer can be conv_rgb, of course, it is not limited to this.
[0056] S202: Fusing the first convolution data with the second convolution data to obtain first fused data, and inputting the first fused feature data into the backbone network to output first initial data, second initial data, and third initial data.
[0057] After obtaining the first convolution data and the second convolution data, the first convolution data and the second convolution data are further fused to obtain the first fused data. In some application scenarios, the first convolution data and the second convolution data can be fused in dimension 1, and the data format of the obtained first fused data is [B, 128, dim0, dim1].
[0058] Among them, the backbone network can use a residual network, and the BasicBlock block in the preset model will add an SE layer, so that the preset model pays more attention to the importance of channels. It should be understood that when the first fusion data is input into such a backbone network, the accuracy of the data output of the model will be increased.
[0059] The data formats of the output first initial data, second initial data and third initial data are different, namely, fea1=[B, 256, dim2, dim3], fea2=[B, 512, dim2 / 2, dim3 / 2], fea3=[B, 1024, dim2 / 4, dim3 / 4]. It should be understood that the first initial data, the second initial data and the third initial data include basic edge and texture information, as well as more complex local feature information, shape and pattern information, and can also extract global feature information of the image, which represents the overall information of the entire image.
[0060] Figure 4 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided in the present application.
[0061] Combination Figure 4 In some specific embodiments, the first initial data, the second initial data and the third initial data are processed by the encoding module and the decoding module of the preset model to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information and the confidence corresponding to the coordinate information, and the above-mentioned step S103, including:
[0062] S301: extracting features from the first initial data, the second initial data and the third initial data by using the first feature extraction module and the second feature extraction module of the encoding module of the preset model to obtain initial extracted data.
[0063] The preset model is provided with an encoding module and a decoding module. In this embodiment, the first feature extraction module and the second feature extraction module of the encoding module are used to extract features of the first initial data, the second initial data and the third initial data, thereby obtaining initial extracted data.
[0064] Before extracting features from the first initial data, the second initial data, and the third initial data, the three types of data can be unified into 256 dimensions. At this time, fea1 = [B, 256, dim2 / 4, dim3 / 4], fea2 = [B, 256, dim2, dim3], and fea3 = [B, 256, dim2 / 2, dim3 / 2].
[0065] This embodiment uses two feature extraction modules to perform feature extraction on the first initial data, the second initial data, and the third initial data, which can obtain a better feature extraction effect, thereby making the initial extracted data more in line with actual needs.
[0066] S302: Perform feature extraction on the initial extracted data through the decoding module of the preset model to obtain target extracted data, and process the target extracted data through the fully connected layer of the decoding module to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information, and the confidence level corresponding to the coordinate information.
[0067] After the initial extraction data is obtained through the encoding module, the initial extraction data is further feature extracted through the decoding module to obtain the target extraction data. The target extraction data is the data that meets the requirements and is used to obtain the coordinate information, lamp bead quality information and confidence after processing.
[0068] In this embodiment, the final target extraction data is obtained by extracting features layer by layer, so that the target extraction data is more in line with actual needs, and the coordinate information, lamp bead quality information and confidence level finally obtained are more accurate and in line with actual needs.
[0069] Figure 5 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided in the present application.
[0070] Combination Figure 5 In some specific embodiments, the step of extracting features from the first initial data, the second initial data, and the third initial data by the first feature extraction module and the second feature extraction module of the encoding module of the preset model to obtain initial extracted data, that is, the above-mentioned step S301, includes:
[0071] S401: extracting features from first initial data using a first feature extraction module in an encoding module of a preset model to obtain first extracted data.
[0072] Combined with the above, after the first initial data fea1 is input into the first feature extraction module, high-level features with richer semantic concepts will be extracted, which can capture the connection between conceptual entities in the image, which is helpful for detecting and identifying objects in the image, and finally the first feature data fea1 is obtained. ’ In some application scenarios, the first feature extraction module may be an attention intra-scale feature interaction module.
[0073] S402: Perform feature extraction on the first extracted data, the second initial data and the third initial data by a second feature extraction module of the encoding module to obtain second extracted data, third extracted data and fourth extracted data as initial extracted data.
[0074] Combined with the above content, the first extracted data fea1 ’ The second initial data fea2 and the third initial data fea3 are input into the second feature extraction module for feature extraction to obtain the second extracted data, the third extracted data and the fourth extracted data. At this time, the second extracted data, the third extracted data and the fourth extracted data are fea11 = [B, 256, dim2 / 4, dim3 / 4], fea22 = [B, 256, dim2, dim3], and fea33 = [B, 256, dim2 / 2, dim3 / 2].
[0075] In some application scenarios, the second feature extraction module may be a CNN cross-scale feature fusion module, but this is not a limitation.
[0076] Figure 6 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided in the present application.
[0077] Combination Figure 6 As well as the related contents of the above embodiments, in some specific embodiments, the step of extracting features from the initial extracted data by a decoding module of a preset model to obtain target extracted data includes:
[0078] S501: Perform feature extraction on the second extracted data, the third extracted data and the fourth extracted data through the preset three-layer convolutional layer of the decoding module of the preset model to obtain the fifth extracted data, the sixth extracted data and the seventh extracted data.
[0079] In the above embodiment, the second extracted data, the third extracted data and the fourth extracted data are obtained. This embodiment further performs feature extraction of the data through a specific layer in the encoding module.
[0080] Among them, the preset three-layer convolutional layer is preset in the preset model. Combined with the above content, after the second extracted data, the third extracted data and the fourth extracted data are subjected to feature extraction through the preset three-layer convolutional layer, the fifth extracted data, the sixth extracted data and the seventh extracted data obtained are [B, dim2 / 4xd im3 / 4, 256], [B, dim2 xd im3, 256], [B, dim2 / 2x d im3 / 2, 256] respectively.
[0081] S502: The fifth extracted data, the sixth extracted data and the seventh extracted data are fused to form second fused data, and the eighth extracted data is obtained by performing feature extraction on the second fused data through a preset attention module in the decoding module, so as to use the eighth extracted data as the target extracted data.
[0082] Specifically, the fifth extracted data, the sixth extracted data and the seventh extracted data can be fused in dimension 1 in concat, and finally a feature fea=[B, dim2 xd im3+d im2 / 2x dim3 / 2+d im2 / 4x dim3 / , 256] is obtained as the second fused data.
[0083] After the second fused data is input into the preset attention module, the preset attention module will perform feature extraction on the second fused data, mainly extracting global attention information, and then obtaining the eighth extracted data. In this embodiment, the eighth extracted data is directly used as the target extracted data, which is not limited to this in other embodiments.
[0084] Figure 7 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided in the present application.
[0085] Combination Figure 7 , after the above step S501, comprising:
[0086] S601: The fifth extracted data, the sixth extracted data and the seventh extracted data are fused to form second fused data, and the eighth extracted data is obtained by performing feature extraction on the second fused data through a preset attention module in the decoding module.
[0087] For the related contents of forming the second fused data by fusing the fifth extracted data, the sixth extracted data and the seventh extracted data, and further extracting features from the second fused data to obtain the eighth extracted data, please refer to the above embodiments and will not be described in detail.
[0088] Different from the above embodiment, this embodiment does not directly use the eighth extracted data as target extracted data, but further processes the eighth extracted data in the subsequent step S602 to obtain target extracted data.
[0089] S602: Perform feature deletion processing on the eighth extracted data through a preset processing layer in the decoding module to obtain ninth extracted data, and use the ninth extracted data as target extracted data.
[0090] In some application scenarios, the preset processing layer may be a DropBlock2D layer, which is mainly used to delete some features to improve the robustness of the model and the accuracy of occlusion information. After the preset processing layer performs feature deletion, the ninth extracted data is finally obtained, and the ninth extracted data is then used as the target extracted data.
[0091] Figure 8 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided in the present application.
[0092] Combination Figure 8 As well as the related contents of the above embodiments, in some specific embodiments, the step of processing the target extraction data through the fully connected layer of the decoding module to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information and the confidence corresponding to the coordinate information includes:
[0093] S701: Process the target extraction data through the first fully connected layer in the decoding module to obtain the coordinate information of the coordinate frame of the lamp bead.
[0094] In this embodiment, the fully connected layer of the encoding module includes a first fully connected layer and a second fully connected layer, and the two fully connected layers are used to output different information. At this time, the first fully connected layer is used to output the coordinate information of the coordinate frame based on the target extraction data.
[0095] S702: Process the target extraction data through the second fully connected layer in the decoding module to obtain the lamp bead quality information corresponding to the coordinate frame of the lamp bead and the confidence level corresponding to the coordinate information.
[0096] Different from the output information of the first fully connected layer, the second fully connected layer is used to output the lamp bead quality information of the coordinate frame of the lamp bead and the confidence corresponding to the coordinate information.
[0097] Fig. 9 It is a flow chart of another embodiment of the vehicle lamp bead quality detection method provided in the present application.
[0098] Combination Fig. 9 In some specific embodiments, the method further comprises the following steps:
[0099] S801: Obtain a connected area of a corresponding image according to image data.
[0100] Combination Figure 2 , there are some bright areas in the headlight image, these bright areas may have lamp beads, which are connected areas. In a headlight image, there may be multiple bright areas, some of which are where the lamp beads are located, while some are not where the lamp beads are located. For example, Figure 2 The first connected area represented by the dotted frame S1 is the connected area where the lamp beads are located, while the second connected area represented by the dotted frame S2 is not the connected area where the lamp beads are located.
[0101] According to the above content, there may be multiple connected areas obtained according to the image data, but some connected areas are not the connected areas where the lamp beads are located.
[0102] S802: Determine a connected area that meets preset requirements according to the shape, size and brightness of the pixels of the connected area, and use the connected area that meets the preset requirements as a target connected area.
[0103] After the connected area is obtained, it is further determined whether the connected area meets the preset requirements based on the shape, size and brightness of the pixels.
[0104] In some specific embodiments, a preset size, a preset shape, and a preset ratio may be set. Only when the shape of the connected area meets the preset shape, the size meets the preset size, and the proportion of pixels with brightness greater than a certain brightness exceeds a preset ratio, can the connected area be regarded as a connected area that meets the preset requirements.
[0105] Based on the determination of the above target connected area, the coordinate frame is located within the target connected area, which means that the coordinate frame is an accurate coordinate frame, and the coordinate frame information and lamp bead quality information corresponding to this coordinate frame are usable information.
[0106] Based on this, the step of obtaining the lamp bead quality of the vehicle lamp based on the target coordinate frame information and the target lamp bead quality information includes: obtaining the lamp bead quality of the vehicle lamp based on the target coordinate frame information and the corresponding target lamp bead quality information of the coordinate frame being located in the target connected area. That is, after determining the target coordinate frame information and the target lamp bead quality information, further screening will be performed, and only when the coordinate frame is located in the target connected area, the corresponding target coordinate frame information and the target lamp bead quality information are available.
[0107] In addition, for the training of the preset model, it is necessary to input a label file including the coordinate information of the coordinate frame and the quality information of the lamp beads corresponding to the coordinate frame. When obtaining the coordinate information of the coordinate frame, the coordinate information of part of the coordinate frame can be obtained by manual annotation. The target detection model is further trained with this part of the coordinate information, and then the coordinate information of other coordinate frames is acquired through the trained target detection model. For the quality information of the lamp beads, two categories such as [OK, NG] can be included, corresponding to the two types of good and bad lamp beads respectively. At this time, a one-to-one correspondence can be established between the coordinate information of the coordinate frame and the quality information of the lamp beads.
[0108] After obtaining the original label file, the label file needs to be converted into a format that can be input into the model training. The original image size is Ho x Wo, and the coordinate frame information needs to be converted. The conversion process is as follows: for example, the original coordinate frame information is (x0, y0, x1, y1), which is converted to (x0, y0, y1-y0, x1-x0), and then to (x0, y0, h, w). Finally, the coordinate information is divided by the H and W of the original image, and the final result is (x0 / Wo, y0 / Ho, h / Ho, w / Wo), which is written here as (x0_r, y0_r, h_r, w_r). Next is the mapping of the target frame category. Category OK is mapped to 0 and category NG is mapped to 1.
[0109] During the training process of the preset model, the first fully connected layer and the second fully connected layer of the preset model will respectively collect coordinate information and quality information categories. At this time, the label file is input into the model to calculate the loss and update the gradient. At this time, the result of the first fully connected layer is used to calculate the iou loss with the coordinate information in the label file, which is box_loss. The result of the second fully connected layer is used to calculate the cross entropy loss with the quality information category in the label file, which is cls_loss. The final loss loss = boX_Oss + CIs_lOsS. In this way, the gradient is continuously iterated and updated to achieve the desired effect, so that the preset model can obtain better training results.
[0110] A second aspect of the present application provides an electronic device, comprising: a processor; a memory for storing a computer program, which, when executed by the processor, implements the vehicle lamp quality detection method in any of the above-mentioned embodiments.
[0111] Fig.10 It is a schematic diagram of the structural framework of an embodiment of the electronic device 500 provided in the present application.
[0112] In some specific embodiments, the electronic device 500 includes a central processing unit (CPU) 501 and a read-only memory (ROM) 502, wherein the CPU 501 is a processor and the ROM 502 is a memory. The CPU 501 can perform various appropriate actions and processes according to the program stored in the ROM 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method in the above embodiment. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, the ROM 502 and the RAM 503 are connected to each other via the bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.
[0113] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.
[0114] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the system of the present application are executed.
[0115] A third aspect of the present application provides a computer-readable storage medium 40, Fig.11 It is a schematic diagram of the structural framework of an embodiment of the computer-readable storage medium 40 provided in the present application.
[0116] The computer readable storage medium 40 stores a computer program 41 , and when the computer program 41 is executed by the processor, the method for detecting the quality of lamp beads of a vehicle lamp as described in any of the above embodiments is implemented.
[0117] It should be noted that the computer-readable medium 40 shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0118] In summary, based on the vehicle headlight lamp bead quality detection method, electronic device and storage medium provided by the present application, the method includes: obtaining image data of the headlight; wherein the headlight includes a plurality of lamp beads; obtaining binary feature data and RGB feature data of the image data, inputting the binary feature data and RGB feature data into a preset model for processing to obtain first initial data, second initial data and third initial data; processing the first initial data, second initial data and third initial data through the encoding module and decoding module of the preset model to obtain the coordinate information corresponding to the coordinate frame of the lamp bead and the lamp bead quality information and the confidence corresponding to the coordinate information; obtaining the coordinate information and lamp bead quality information higher than the preset confidence as the target coordinate information and target lamp bead quality information, and obtaining the lamp bead quality of the headlight based on the target coordinate information and the target lamp bead quality information. Therefore, relatively accurate lamp bead related information is obtained based on the binary feature data and RGB feature data, and the lamp bead related information is screened according to the confidence to make it more accurate, thereby improving the accuracy of lamp bead quality detection.
[0119] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.
Claims
1. A method for detecting the quality of lamp beads of a vehicle, characterized in that: include: Acquire image data of a vehicle lamp; wherein the vehicle lamp comprises a plurality of lamp beads; Obtaining binary feature data and RGB feature data of the image data, and inputting the binary feature data and the RGB feature data into a preset model for processing to obtain first initial data, second initial data, and third initial data; The first initial data, the second initial data and the third initial data are processed by the encoding module and the decoding module of the preset model to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information and the confidence corresponding to the coordinate information; The coordinate information and the lamp bead quality information having a reliability higher than a preset reliability are obtained as target coordinate information and target lamp bead quality information, and the lamp bead quality of the vehicle lamp is obtained based on the target coordinate information and the target lamp bead quality information.
2. The method for detecting the quality of lamp beads of a vehicle lamp according to claim 1, characterized in that: The step of inputting the binary feature data and the RGB feature data into a preset model for processing to obtain first initial data, second initial data and third initial data comprises: Input the binary feature data into a first multi-layer convolution layer for feature extraction to output first convolution data, and input the RGB feature data into a second multi-layer convolution layer for feature extraction to output second convolution data; The first convolution data and the second convolution data are fused to obtain first fused data, and the first fused feature data is input into a backbone network to output first initial data, second initial data and third initial data.
3. The method for detecting the quality of lamp beads of a vehicle lamp according to claim 1, characterized in that: The step of processing the first initial data, the second initial data and the third initial data by the encoding module and the decoding module of the preset model to obtain the coordinate information corresponding to the coordinate frame of the lamp bead and the lamp bead quality information and the confidence corresponding to the coordinate information includes: Performing feature extraction on the first initial data, the second initial data, and the third initial data by using a first feature extraction module and a second feature extraction module of the encoding module of the preset model to obtain initial extracted data; The initial extracted data is subjected to feature extraction by the decoding module of the preset model to obtain target extracted data, and the target extracted data is processed by the fully connected layer of the decoding module to obtain the coordinate information corresponding to the coordinate frame of the lamp bead, the lamp bead quality information and the confidence level corresponding to the coordinate information.
4. The method for detecting the quality of lamp beads of a vehicle lamp according to claim 3, characterized in that: The step of extracting features from the first initial data, the second initial data, and the third initial data by using the first feature extraction module and the second feature extraction module of the encoding module of the preset model to obtain initial extracted data includes: Performing feature extraction on the first initial data by a first feature extraction module in the encoding module of the preset model to obtain first extracted data; The first extracted data, the second initial data and the third initial data are subjected to feature extraction by the second feature extraction module of the encoding module to obtain second extracted data, third extracted data and fourth extracted data as initial extracted data.
5. The method for detecting the quality of lamp beads of a vehicle lamp according to claim 4, characterized in that: The step of extracting features from the initial extracted data by using the decoding module of the preset model to obtain target extracted data comprises: Performing feature extraction on the second extracted data, the third extracted data, and the fourth extracted data through the preset three-layer convolutional layer of the decoding module of the preset model to obtain fifth extracted data, sixth extracted data, and seventh extracted data; The fifth extracted data, the sixth extracted data and the seventh extracted data are fused to form second fused data, and the second fused data is subjected to feature extraction by a preset attention module in the decoding module to obtain eighth extracted data, and the eighth extracted data is used as target extracted data.
6. The method for detecting the quality of lamp beads of a vehicle lamp according to claim 4, characterized in that: The step of extracting features from the initial extracted data by using the decoding module of the preset model to obtain target extracted data comprises: Performing feature extraction on the second extracted data, the third extracted data, and the fourth extracted data through the preset three-layer convolutional layer of the decoding module of the preset model to obtain fifth extracted data, sixth extracted data, and seventh extracted data; The fifth extracted data, the sixth extracted data and the seventh extracted data are fused to form second fused data, and a preset attention module in the decoding module is used to perform feature extraction on the second fused data to obtain eighth extracted data; The eighth extracted data is subjected to feature deletion processing by a preset processing layer in the decoding module to obtain ninth extracted data, and the ninth extracted data is used as target extracted data.
7. The method for detecting the quality of lamp beads of a vehicle lamp according to claim 3, characterized in that: The step of extracting data from the target through the fully connected layer of the decoding module to obtain coordinate information corresponding to the coordinate frame of the lamp bead and the quality information of the lamp bead and the confidence corresponding to the coordinate information includes: Processing the target extraction data through the first fully connected layer in the decoding module to obtain the coordinate information of the coordinate frame of the lamp bead; The target extraction data is processed through the second fully connected layer in the decoding module to obtain the lamp bead quality information corresponding to the coordinate frame of the lamp bead and the confidence corresponding to the coordinate information.
8. The method for detecting the quality of lamp beads of a vehicle lamp according to claim 3, characterized in that: The method further comprises: Acquire a connected area of a corresponding image according to the image data; Determine the connected area that meets the preset requirements according to the shape, size and brightness of the pixels of the connected area, and the connected area that meets the preset requirements will be used as the target connected area; The step of obtaining the lamp bead quality of the vehicle lamp based on the target coordinate frame information and the target lamp bead quality information comprises: The lamp bead quality of the vehicle lamp is obtained based on the target coordinate frame information that the coordinate frame is within the target connected area and the corresponding target lamp bead quality information.
9. An electronic device, characterized in that: include: processor; A memory for storing a computer program, wherein when the computer program is executed by the processor, the method for detecting the quality of lamp beads of a vehicle lamp according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for detecting the quality of lamp beads of a vehicle lamp as described in any one of claims 1 to 8 is implemented.