Image processing method for improving light thorn detection precision and storage medium

Abstract

The invention relates to the technical field of image processing, and provides an image processing method for improving light thorn detection precision and a storage medium, and aims at performing image enhancement on a light thorn region, on one hand, model training is performed through coverage service scene collection, and the recognition accuracy of a deep learning model on an image is effectively improved; on the other hand, through division, labeling and identification definition of a target area and a domain area, a corresponding light thorn image can be effectively screened out, and then a basis is provided for subsequent light thorn identification; and on the other hand, image enhancement is executed, a target region is highlighted, a neighborhood region is suppressed, and the probability of occurrence of false detection is effectively reduced by suppressing the overall brightness of the image and highlighting the characteristics of a light thorn region.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and a storage medium for improving light thorn detection accuracy. Background Art

[0002] During driving, the cleanliness of the CMS (Camera Monitor System) camera is crucial to driving safety. Especially in bad weather or adverse environments, dirt, such as oil stains and water droplets, easily accumulates on the camera surface. These stains not only affect the clarity of the image, but may also cause obvious light thorns at night. Light thorns will seriously interfere with the driver's vision and increase the risk of traffic accidents.

[0003] To solve this problem, the light sting recognition function based on deep learning algorithms came into being. This technology can intelligently identify light stings caused by dirty lenses by analyzing the images captured by the camera. The deep learning model has learned how to distinguish between normal light and abnormal light stings through a large amount of data training, and can accurately judge even under complex lighting conditions.

[0004] When the system detects light stinging, it will immediately alert the driver that the CMS camera may be dirty and advise the driver to wipe the lens in time. This function greatly improves driving safety because it can actively remind the driver of potential problems when the driver is not easily aware of it, avoiding dangers caused by obstructed vision.

[0005] In the existing technical solutions, there are the following scenario limitations:

[0006] 1. In the vehicle environment, due to cost constraints, CMS cameras usually use relatively low-priced chips, which are far inferior to high-end devices in image processing capabilities. Therefore, when capturing and transmitting images, the original image quality may already have noise and blur problems. In this case, image enhancement is required to improve image quality before sending the image to the deep learning network for processing.

[0007] 2. When the vehicle is moving or even at high speed, the images collected by the on-board CMS camera are often affected by a variety of factors, resulting in a decrease in image quality. Among them, afterimage and light dispersion are two significant problems, which seriously affect the clarity and detail of the image, and may interfere with the deep learning network's accurate analysis and understanding of the image content.

[0008] 3. In the field of deep learning, processing large amounts of data to obtain prior knowledge is the basis of model training. However, if there is too much dirty data in the data set, this will not only affect the model's understanding of the distribution of real data, but also lead to an increase in the cumulative error in the learning process. The cumulative effect of this error will significantly reduce the accuracy and reliability of the model, causing it to perform poorly in practical applications.

[0009] 4. Traditional image algorithms often face the challenge of insufficient robustness when processing images. In particular, the performance of traditional algorithms is often unsatisfactory under complex environmental conditions such as lighting changes, rainy days, and foggy days. Light changes can significantly affect the contrast and brightness of images, causing algorithms designed based on fixed thresholds or specific lighting conditions to fail.

[0010] In order to improve the robustness of recognition and address the problems existing in the current on-board chip optical spike recognition algorithm, a deep learning solution is adopted for processing. However, data is crucial for deep learning algorithms. Before processing, the image needs to be enhanced to improve the recognition accuracy of the model. The image enhancement algorithm needs to deal with the light scattering and noise problems in the data. Summary of the invention

[0011] The present invention provides an image processing method and a storage medium for improving the accuracy of light thorn detection, which solves the technical problem that the existing image processing scheme cannot effectively filter light scatter and noise, and thus cannot effectively detect light thorns, resulting in low image recognition accuracy.

[0012] In order to solve the above technical problems, the present invention provides an image processing method for improving the accuracy of light thorn detection, comprising:

[0013] Collect data based on different business scenarios and obtain corresponding scene images;

[0014] Performing generalization screening on the scene image to obtain a representative image;

[0015] Marking regions of interest for the light spikes in the representative image to determine a first region and a second region;

[0016] Identify the first area and the second area, determine whether they are corresponding target areas and neighborhood areas, and if so, proceed to the next step of performing image enhancement;

[0017] Image enhancement is performed to highlight the target area and suppress the neighborhood area to obtain a target image.

[0018] This basic solution performs image enhancement on the light thorn area in a targeted manner. On the one hand, it conducts model training through comprehensive business scenario collection to effectively improve the recognition accuracy of the deep learning model for images. On the other hand, through the division, labeling, and identification definition of target areas and field areas, it can effectively screen out the corresponding light thorn images, thereby providing a basis for subsequent light thorn recognition. On the other hand, it performs image enhancement to highlight the target area and suppress the neighboring area. By suppressing the overall brightness of the image and highlighting the characteristics of the light thorn area, the probability of false detection is effectively reduced.

[0019] In a further embodiment, marking the region of interest on the representative image to determine the first region and the second region includes:

[0020] Identify the light thorn area in the representative image, mark the region of interest, and obtain a first region;

[0021] Based on the first area, the areas on both sides of the light thorn area are marked with equal amplitude to obtain a second area.

[0022] This solution sets the first area and the second area for equal-amplitude marking, with consistent collection standards, to improve the accuracy of comparative identification; directly marking the areas on both sides of the light-thorn area with equal amplitude can more intuitively determine whether it is a light-thorn area.

[0023] In a further embodiment, identifying the first area and the second area and determining whether they are corresponding target areas and neighboring areas includes:

[0024] Calculate a first gradient of the first region based on a gradient calculation formula;

[0025] Calculating a second gradient of the second region based on a gradient calculation formula;

[0026] The first gradient and the second gradient are compared to determine whether the gradient difference is greater than a preset threshold, and if so, the first area and the second area are determined to be the corresponding target area and the neighborhood area.

[0027] In a further embodiment, the gradient calculation formula is as follows:

[0028]

[0029] Where d(x,y) is the gradient, (x i ,y i ) is the pixel value of the ith pixel, (x,y) is the coordinate of the ith pixel.

[0030] This scheme uses gradient contrast for regional comparison and recognition based on the difference in pixel values ​​between the light thorn area and the adjacent areas. It can intuitively and quickly determine whether the collected representative image contains light thorns, thereby providing a stable basis for light thorn recognition.

[0031] In a further embodiment, the scene image is subjected to a generalization screening to obtain a representative image, comprising:

[0032] Calculate the MD5 value of each scene image according to the hash function, use it as the name of the image sample and store it in a preset folder;

[0033] Calculate the mode M of the last digit of the MD5 value of all image samples in the preset folder o , for each image sample n in the preset folder i The MD5 value of M o Take the remainder, and when the remainder is 0, select the image sample;

[0034] Calculate the gradient of the image sample that jumps out based on a gradient calculation formula;

[0035] A similarity comparison is performed based on the gradient, and a representative image is extracted from image samples with similar similarity as a representative image.

[0036] Before performing image recognition, this solution pre-screens image samples with similarities based on hash functions and gradient calculations, and then extracts a representative image as a representative image. While ensuring image scene coverage, it also reduces the load on system operations and improves efficiency.

[0037] In a further embodiment, performing image enhancement to highlight the target area and suppress the neighborhood area to obtain a target image comprises:

[0038] Predefine at least two pixel ranges and corresponding stage enhancement functions;

[0039] Obtaining a pixel value of each pixel in the representative image;

[0040] Determine the pixel range of the corresponding pixel point according to the pixel value, and then determine the corresponding stage enhancement function;

[0041] Calculate the pixel update value of the pixel point according to the stage enhancement function and the pixel value;

[0042] Image enhancement is performed based on the pixel update value to highlight the target area and suppress the neighborhood area to obtain a target image.

[0043] In a further embodiment, the stage enhancement function is as follows:

[0044]

[0045] Among them, x is the pixel value of the pixel point, f(x) is the pixel update value, 255 is the highest bit of the image pixel value, 127 is the median of the image, α is the smoothing coefficient, and e is a natural number.

[0046] This solution is based on the pixel difference between the light thorn scene and the surrounding scene itself, and further sets a stage enhancement function to perform pixel enhancement with different variation amplitudes on pixel points in different pixel ranges, so that the image contrast is more prominent, and the light thorn area can be further highlighted, thereby providing a basis for vehicle driving recognition.

[0047] In a further implementation, the business scenario includes one or more of an indoor scenario, an outdoor scenario, a weather scenario, a driving scenario, and a road condition scenario.

[0048] This solution collects images from a variety of scenarios and conducts training and recognition, which can effectively improve the application recognition of the recognition algorithm model corresponding to the actual scenario, thereby providing a good user experience.

[0049] The present invention also provides a storage medium on which a computer program is stored, and the computer program is used to implement the above-mentioned image processing method for improving the accuracy of optical thorn detection. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM). BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a workflow diagram of an image processing method for improving light thorn detection accuracy provided by an embodiment of the present invention;

[0051] Figure 2 is a marked schematic diagram of a first area provided in an embodiment of the present invention;

[0052] Figure 3 is a marked schematic diagram of the second area provided in an embodiment of the present invention;

[0053] Figure 4 is a schematic diagram of gradient comparison between the first region and the second region provided by an embodiment of the present invention;

[0054] Figure 5 is a function curve diagram of a stage enhancement function provided by an embodiment of the present invention;

[0055] Figure 6 It is the image comparison before and after the function processing provided by the embodiment of the present invention;

[0056] Figure 7This is the experimental comparison result of different algorithm models provided in the embodiment of the present invention before and after image processing using this embodiment. DETAILED DESCRIPTION

[0057] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and cannot be understood as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0058] An embodiment of the present invention provides an image processing method for improving the accuracy of light thorn detection, such as Figure 1 As shown, in this embodiment, it includes S1 to S5:

[0059] S1. Collect data based on different business scenarios and obtain corresponding scene images;

[0060] In this embodiment, the business scenario includes one or more of an indoor scenario, an outdoor scenario, a weather scenario, a driving scenario, and a road condition scenario. Figure 2 Among them, weather scenes include rainy days, foggy days and sunny days; road conditions include wide sections, crowded sections, etc.; through the combination of different conditions, we try to collect scene distribution and representative data sets.

[0061] This embodiment collects images in various scenarios and performs training and recognition, thereby effectively improving the application recognition of the recognition algorithm model corresponding to the actual scenario, thereby providing a good user experience.

[0062] S2, performing generalization screening on the scene image to obtain a representative image, including steps S21 to S24:

[0063] S21, calculating the MD5 value of each scene image according to the hash function, taking the MD5 value as the name of the image sample and storing it in a preset folder;

[0064] Specifically, the MD5 value of each image sample is calculated by using the hash function N*512+R with 512 bits as a group.

[0065] S22, calculate the mode M of the last digit of the MD5 value of all image samples in the preset folder o , for each image sample n in the preset folder i The MD5 value of M o Take the remainder, and when the remainder is 0, select the image sample;

[0066] S23, calculating the gradient of the image sample that is jumped out based on a gradient calculation formula;

[0067] In this embodiment, the gradient calculation formula is as follows:

[0068]

[0069] Where d(x,y) is the gradient, (x i ,y i ) is the pixel value of the ith pixel, (x,y) is the coordinate of the ith pixel.

[0070] This embodiment uses gradient contrast to perform regional comparison and identification based on the pixel value difference between the light thorn area and the adjacent area, which can intuitively and quickly determine whether the collected representative image contains light thorns, thereby providing a stable basis for light thorn identification.

[0071] S24, performing similarity comparison based on the gradient, and extracting a representative image from the image samples with similar similarity as a representative image.

[0072] Among them, chemical screening refers to selecting according to the characteristics of the data, and selecting images that meet the expected characteristics. For example, the closer the image gradient / pixel value mean in the data is to 0, the darker the collected image is, and the lower the probability of light thorns. Moreover, the closer the gradient mean values ​​of two images are, the higher the similarity is, and there is no need for repeated collection.

[0073] Before performing image recognition, this embodiment pre-screens image samples with similarities based on hash functions and gradient calculations, and then extracts a representative image as a representative image. While ensuring the image scene coverage, it also reduces the load on system operations and improves efficiency.

[0074] S3, marking the region of interest for the light spikes in the representative image to determine the first region and the second region, including:

[0075] Identify the light thorn area in the representative image, see Figure 2 , use the marking tool to mark the region of interest, calibrate the position and size of the light thorn, and obtain the first region;

[0076] Based on the first area, the areas on both sides of the light thorn area are marked with equal amplitude to obtain a second area.

[0077] Among them, equal-width annotation specifically refers to performing ROI operations according to the size of equal width and height. Figure 3 The middle rectangular frame is the first area, and the two side rectangular frames are the second area.

[0078] In this embodiment, the first area and the second area are set to be marked with equal amplitude, and the collection standard is consistent, which improves the accuracy of comparison and identification; the equal amplitude marking is directly performed on the areas on both sides of the light thorn area, which can more intuitively judge whether it is a light thorn area.

[0079] S4, identifying the first area and the second area, determining whether they are corresponding target areas and neighborhood areas, and if so, proceeding to the next step of performing image enhancement;

[0080] In this embodiment, identifying the first area and the second area and determining whether they are corresponding target areas and neighboring areas includes:

[0081] Calculate a first gradient of the first region based on a gradient calculation formula;

[0082] Calculating a second gradient of the second region based on a gradient calculation formula;

[0083] The first gradient and the second gradient are compared to determine whether the gradient difference is greater than a preset threshold, and if so, the first area and the second area are determined to be the corresponding target area and the neighborhood area.

[0084] See also Figure 4 , the four groups of histograms from left to right are the gradient comparisons of the pixel channels R, G, B and the pixel mean, and each group of histograms contains the gradient values ​​of the target area, the neighborhood area on the left, and the neighborhood area on the right from left to right. Among them, the Mean in the rectangular box is the mean of the pixel channels R, G, and B. From the gradient map, it can be seen that compared with the light thorn area Bbox (i.e., the target area) of the labeled box, the adjacent left and right areas Bbox-Left and Bbox-Right (i.e., the neighborhood area on the left and the neighborhood area on the right) have lower gradients, and the image needs to be further enhanced to highlight the light thorn area and suppress the left and right neighboring areas.

[0085] S5, performing image enhancement, highlighting the target area and suppressing the neighborhood area, to obtain a target image, including:

[0086] Predefine at least two pixel ranges and corresponding stage enhancement functions;

[0087] Obtaining a pixel value of each pixel in the representative image;

[0088] Determine the pixel range of the corresponding pixel point according to the pixel value, and then determine the corresponding stage enhancement function;

[0089] Calculate the pixel update value of the pixel point according to the stage enhancement function and the pixel value;

[0090] Image enhancement is performed based on the pixel update value to highlight the target area and suppress the neighborhood area to obtain a target image.

[0091] In this embodiment, the stage enhancement function is as follows:

[0092]

[0093] Among them, x is the pixel value of the pixel point, f(x) is the pixel update value, 255 is the highest bit of the image pixel value, 127 is the image median, α is the smoothing coefficient; e is a natural number; the function image is as follows Figure 5 shown.

[0094] Among them, when the pixel value x of the pixel point is in [0,150], the enhancement function curve is steeper, and the change is larger; when the pixel value x of the pixel point is in [150,200], the enhancement function curve is slightly slower, and the change amplitude becomes smaller; when the pixel value x of the pixel point is in [200,255], the enhancement function curve tends to be flat, and the change amplitude is basically small.

[0095] This embodiment is based on the pixel difference between the light thorn scene and the surrounding scene itself, and further sets a stage enhancement function to perform pixel enhancement with different variation amplitudes on pixel points in different pixel ranges, so that the image contrast is more prominent, and the light thorn area can be further highlighted, thereby providing a basis for vehicle driving recognition.

[0096] See also Figure 6 The upper and lower images are the image comparisons before and after the function processing, respectively. The data processed by the image processing algorithm of this embodiment and the unprocessed data are trained and tested for accuracy on a common target detection model, respectively. The contrast between the light spikes and the background after the processing of this embodiment is more significant.

[0097] See also Figure 7 , four groups of algorithm models Mobilenet-SSD, YOLIv5-s, YOLIv8-s, and YOLIv10-s are compared, where the left bar graph of each group represents the unprocessed original data, and the right bar graph represents the processed data. It can be seen from the figure that the use of processed data (right bar graph) is relative to the unprocessed data (left bar graph) in test accuracy. In some scenes with obvious astigmatism, the image processing method proposed in this embodiment can suppress the overall brightness of the image, highlight the characteristics of the light thorn area, and reduce the occurrence of false detection.

[0098] The embodiment of the present invention performs image enhancement on the light thorn area in a targeted manner. On the one hand, model training is performed through comprehensive business scene collection, thereby effectively improving the recognition accuracy of the deep learning model for the image; on the other hand, the corresponding light thorn image can be effectively screened out through the division, labeling, and identification definition of the target area and the field area, thereby providing a basis for subsequent light thorn recognition; on the other hand, image enhancement is performed to highlight the target area and suppress the neighboring area. By suppressing the overall brightness of the image, the characteristics of the light thorn area are highlighted, thereby effectively reducing the probability of false detection.

[0099] Example 2

[0100] The embodiment of the present invention further provides a storage medium on which a computer program is stored, and the computer program is used to implement an image processing method for improving the accuracy of optical thorn detection as described in the above embodiment 1. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0101] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An image processing method for improving the accuracy of light thorn detection, characterized in that: include: Collect data based on different business scenarios and obtain corresponding scene images; Performing generalization screening on the scene image to obtain a representative image; Marking regions of interest for the light spikes in the representative image to determine a first region and a second region; Identify the first area and the second area, determine whether they are corresponding target areas and neighborhood areas, and if so, proceed to the next step of performing image enhancement; Image enhancement is performed to highlight the target area and suppress the neighborhood area to obtain a target image.

2. The image processing method for improving the accuracy of optical thorn detection according to claim 1, characterized in that: Marking the region of interest on the representative image to determine the first region and the second region includes: Identify the light thorn area in the representative image, mark the region of interest, and obtain a first region; Based on the first area, the areas on both sides of the light thorn area are marked with equal amplitude to obtain a second area.

3. The image processing method for improving the accuracy of optical thorn detection according to claim 2, characterized in that: Identifying the first area and the second area, and determining whether they are corresponding target areas and neighboring areas, includes: Calculate a first gradient of the first region based on a gradient calculation formula; Calculating a second gradient of the second region based on a gradient calculation formula; The first gradient and the second gradient are compared to determine whether the gradient difference is greater than a preset threshold, and if so, the first area and the second area are determined to be the corresponding target area and the neighborhood area.

4. The image processing method for improving the accuracy of optical thorn detection according to claim 3, characterized in that: The gradient calculation formula is as follows: Where d(x,y) is the gradient, (x i ,y i ) is the pixel value of the ith pixel, (x,y) is the coordinate of the ith pixel.

5. The image processing method for improving the accuracy of optical thorn detection according to claim 4, characterized in that: The scene image is screened for generalization to obtain a representative image, including: Calculate the MD5 value of each scene image according to the hash function, use it as the name of the image sample and store it in a preset folder; Calculate the mode M of the last digit of the MD5 value of all image samples in the preset folder o , for each image sample n in the preset folder i The MD5 value of M o Take the remainder, and when the remainder is 0, select the image sample; Calculate the gradient of the image sample that jumps out based on a gradient calculation formula; A similarity comparison is performed based on the gradient, and a representative image is extracted from image samples with similar similarity as a representative image.

6. The image processing method for improving the accuracy of optical thorn detection according to claim 1, characterized in that: Performing image enhancement to highlight the target area and suppress the neighborhood area to obtain a target image includes: Predefine at least two pixel ranges and corresponding stage enhancement functions; Obtaining a pixel value of each pixel in the representative image; Determine the pixel range of the corresponding pixel point according to the pixel value, and then determine the corresponding stage enhancement function; Calculate the pixel update value of the pixel point according to the stage enhancement function and the pixel value; Image enhancement is performed based on the pixel update value to highlight the target area and suppress the neighborhood area to obtain a target image.

7. The image processing method for improving the accuracy of optical thorn detection according to claim 6, characterized in that: The stage enhancement function is as follows: Among them, x is the pixel value of the pixel point, f(x) is the pixel update value, 255 is the highest bit of the image pixel value, 127 is the median of the image, α is the smoothing coefficient, and e is a natural number.

8. The image processing method for improving the accuracy of optical thorn detection according to claim 1, characterized in that: The business scenario includes one or more of an indoor scenario, an outdoor scenario, a weather scenario, a driving scenario, and a road condition scenario.

9. A storage medium having a computer program stored thereon, characterized in that: The computer program is used to implement an image processing method for improving the accuracy of optical thorn detection as described in any one of claims 1 to 8.

Images