Highway construction image defogging method, device and equipment and storage medium

Abstract

The invention discloses a highway construction image defogging method, device and equipment and a storage medium, and relates to the technical field of image defogging, and the method comprises the following steps: collecting a fogless image and a plurality of foggy images under the same construction link, selecting a foggy image with the highest coincidence degree with the fogless image from the plurality of foggy images as a first image, and taking the foggy image as a second image; the other foggy images are second images; performing feature extraction on the first image to obtain a first feature map, and performing feature extraction on the plurality of second images to obtain a plurality of second feature maps; obtaining a plurality of similarities between the first feature map and the plurality of second feature maps, and weighting the plurality of second feature maps according to the plurality of similarities; and obtaining a defogging mapping relationship through the fogless image and the first feature map, and inputting the plurality of weighted second feature maps into the defogging mapping relationship to obtain a corresponding defogged image. According to the invention, a plurality of foggy images under the simultaneous construction link can be processed in batches, so that the image processing time is greatly saved.

Description

Technical Field

[0001] The present invention relates to the technical field of image defogging, and in particular to a method, device, equipment and storage medium for defogging images for highway construction. Background Art

[0002] Highways refer to roads built according to national technical standards that connect cities, urban and rural areas, villages, and industrial and mining bases. They are roads approved by the highway authorities and include expressways, first-class highways, second-class highways, third-class highways, and fourth-class highways. At present, with the rapid development of domestic transportation, my country's expressways are still in a period of rapid development. During the construction of expressways, imaging equipment with cameras is usually used to monitor and record the specific progress of highway construction.

[0003] Since highway construction is often carried out in an open-air environment, it is greatly affected by the weather. Bad weather conditions often have a certain impact on the collected images or videos. In foggy environments, due to the influence of atmospheric scattering, the visibility of the target scene is reduced. The image information obtained by outdoor image acquisition equipment often has poor visual effects and serious lack of clarity, resulting in the inability to accurately obtain the detailed conditions of the construction process, resulting in the loss of records of a certain construction link. Therefore, defogging images collected in foggy days has important practical significance.

[0004] In recent years, domestic and foreign research institutions and scholars have paid more and more attention to the problem of defogging images in foggy weather, and have proposed or optimized many image defogging algorithms. For example, defogging algorithms based on image enhancement and defogging algorithms based on deep learning. Defogging algorithms based on image enhancement include Retinex algorithm, histogram equalization algorithm, partial differential equation algorithm and wavelet transform algorithm, etc., which mainly enhance the image's contrast, brightness and other features to make the image look clearer. Defogging algorithms based on deep learning use deep learning models such as convolutional neural networks (CNN) to establish a mapping relationship between foggy images and fog-free images.

[0005] The existing defogging algorithms have a very obvious defogging effect when applied to the images to be defogged. However, the defogging algorithms based on image enhancement usually process a single image during the defogging process. When highway construction is carried out in areas prone to fog, foggy weather accounts for a large proportion, and more construction images are taken on foggy days. Therefore, for a large number of images to be defogged, the defogging algorithms based on image enhancement need to spend a lot of time to process multiple images, and their computational efficiency is low and the time cost is high. The defogging algorithms based on deep learning usually need to obtain a large number of foggy images and corresponding fog-free images as training sets to train the model. However, in specific construction scenarios, the environment is changeable and it is difficult to obtain a large number of foggy images and corresponding fog-free images. Summary of the invention

[0006] The present invention provides a method, device, equipment and storage medium for defogging highway construction images, which solves the problem that the existing defogging algorithm based on image enhancement has a large time cost in processing construction images and the training set of the defogging algorithm based on deep learning is difficult to obtain when processing construction images.

[0007] In a first aspect, the present invention provides a method for defogging a highway construction image, comprising the following steps:

[0008] Collect a fog-free image and multiple foggy images in the same construction link, and select a foggy image with the highest overlap with the fog-free image from the multiple foggy images as the first image, and the remaining foggy images as the second images;

[0009] Performing feature extraction on the first image to obtain a first feature map for characterizing the fog state, and performing feature extraction on the plurality of second images to obtain a plurality of second feature maps;

[0010] Obtaining multiple similarities between the first feature map and the multiple second feature maps, and weighting the multiple second feature maps according to the multiple similarities;

[0011] A defogging mapping relationship is obtained through the haze-free image and the first feature map, and a plurality of weighted second feature maps are input into the defogging mapping relationship to obtain a corresponding defogging image.

[0012] Preferably, the step of extracting features from the first image to obtain a first feature map for characterizing the fog state comprises the following steps:

[0013] Obtain pixel values ​​of each pixel in the first image on three channels, namely, red, green and blue, respectively, and extract the minimum value of the pixel values ​​of each pixel on the three channels;

[0014] Combining the minimum values ​​of all pixels to obtain a first grayscale image having the same size as the first image;

[0015] Minimum filtering is performed on the first grayscale image to obtain a first dark channel image, where the first dark channel image is a first feature map.

[0016] Preferably, the obtaining of multiple similarities between the first feature map and the multiple second feature maps, and weighting the multiple second feature maps according to the multiple similarities, comprises the following steps:

[0017] Extracting a first texture feature of the first feature map, and extracting a plurality of second texture features of a plurality of second feature maps;

[0018] Obtaining similarity values ​​between the first texture feature and the plurality of second texture features through a texture feature matching algorithm, and arranging the plurality of similarity values ​​in descending order;

[0019] The second feature maps corresponding to the plurality of second texture features are assigned corresponding weights in the order of arrangement to obtain a weighted second feature map, as shown below:

[0020]

[0021] Where F is the weighted second feature map, b i is the i-th weighting coefficient, x i is the i-th second feature map, and n is the number of second feature maps.

[0022] Preferably, the obtaining of the defogging mapping relationship through the haze-free image and the first feature map comprises the following steps:

[0023] Taking the pixel with the highest brightness in the first dark channel image as the first atmospheric light intensity;

[0024] Averaging a plurality of transmittances corresponding to a plurality of pixel points of the first dark channel feature map to obtain a first transmittance;

[0025] Based on the image defogging algorithm, a mapping relationship between the first atmospheric light intensity and the light intensity of the fog-free image and a mapping relationship between the first transmittance and the transmittance of the fog-free image are established.

[0026] Preferably, the step of inputting the plurality of weighted second feature maps into the defogging mapping relationship to obtain the corresponding defogging image comprises the following steps:

[0027] Acquire a plurality of second atmospheric light intensities and second transmittances corresponding to a plurality of weighted second feature maps;

[0028] The second atmospheric light intensity of each weighted second feature map is input into the light intensity mapping relationship, and the second transmittance is input into the transmission mapping relationship to obtain the corresponding defogging image.

[0029] Preferably, the step of selecting a foggy image having the highest overlap with the fog-free image from among the multiple foggy images as the first image comprises the following steps:

[0030] Adjust multiple foggy images and fog-free images to the same size;

[0031] Converting the adjusted multiple foggy images and fog-free images into grayscale images to obtain multiple foggy grayscale images and fog-free grayscale images;

[0032] Obtaining a plurality of first hash values ​​of a plurality of foggy grayscale images and a second hash value of a non-foggy grayscale image, and obtaining a plurality of Hamming distances between the plurality of first hash values ​​and the second hash values;

[0033] Multiple similarities are obtained through multiple Hamming distances, and the foggy image with the highest similarity is used as the first image.

[0034] In a second aspect, the present invention provides a highway construction image defogging device, comprising:

[0035] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned highway construction image defogging method when executing the program.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned highway construction image defogging method is implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention provides a method for defogging highway construction images. First, a fog-free image and multiple foggy images with similar backgrounds within a certain period of time under the same construction link are collected. Since the backgrounds of the multiple images of the present invention are similar, it is convenient to find the association between the multiple images in the subsequent scheme. At the same time, a foggy image with the highest overlap with the fog-free image is selected from the multiple foggy images as the first image, and the remaining foggy images are the second images. There is no need to obtain a large number of foggy images and corresponding fog-free images for training. Then, the second feature map of the second image is weighted by the first feature map of the first image, and the association between the multiple second feature maps and the first feature map is obtained. Finally, the defogging mapping relationship between the fog-free image and the first feature map is obtained by the existing algorithm, and the multiple weighted second feature maps are input into the defogging mapping relationship to obtain the corresponding defogging image. The present invention can batch process multiple foggy images under the same construction link, thereby greatly saving the processing time of the image, and at the same time, there is no need to obtain a large number of foggy images and corresponding fog-free images. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of a highway construction image defogging method according to the present invention;

[0041] Figure 2The figure is a flowchart of a highway construction image defogging method according to the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Since most links of highway construction require a lot of time to process, and the construction process of each link is the same, all the work is repetitive, so the difference in the background of the images sampled in a certain period of time in a single link is not very large. For these foggy images with small background differences, the correlation between them can be found and batch processed based on this correlation, thereby greatly saving the image processing time. Figure 1 and Figure 2 The present invention provides a method for defogging a highway construction image, which specifically comprises the following steps:

[0044] S1: Collect a fog-free image and multiple foggy images in the same construction link, and select a foggy image with the highest background overlap with the background in the fog-free image from the multiple foggy images as the first image, and the remaining foggy images as the second images.

[0045] Highway construction is a complex and systematic process. Each link requires strict control and fine construction to ensure the quality and safety of the highway. It mainly includes roadbed construction, pavement base construction and pavement surface construction. Among them, roadbed construction includes:

[0046] Clearing and digging: Clear debris, plants, etc. on the roadbed and dig the roadbed to form an appropriate height and shape.

[0047] Drainage system construction: Set up drainage facilities such as drainage ditches and drainage pipes to ensure smooth drainage of the roadbed during the construction process and in future use.

[0048] Roadbed compaction: Use compaction machinery to compact the roadbed to ensure its stability and durability. If necessary, carry out soil reinforcement treatment, such as soil replacement, drainage and reinforcement.

[0049] Pavement base construction includes:

[0050] Selection and laying of base materials: Select base materials such as gravel, crushed stone or asphalt concrete according to design requirements, and lay and compact them.

[0051] Quality control and testing: Strictly control and test the quality of base construction to ensure that it meets the design requirements.

[0052] Pavement surface construction includes:

[0053] Surface material selection and paving: Select surface materials, such as asphalt concrete, cement concrete, etc., and pave them. During the paving process, ensure the uniformity and density of the materials.

[0054] Paving and compaction: Use a paver to evenly spread the surface material on the base layer, and use a roller to compact it to ensure the flatness and density of the road surface.

[0055] Surface treatment: Carry out surface treatment on the road surface as needed, such as spraying emulsified asphalt, laying anti-skid materials, etc.

[0056] Existing defogging algorithms based on deep learning usually need to obtain a large number of foggy images and corresponding fog-free images as training sets to train the model. However, in specific construction scenarios, the environment is changeable and it is difficult to obtain a large number of foggy images and corresponding fog-free images. To address this problem, the present invention first collects a fog-free image and multiple foggy images under the same construction link, and then selects a foggy image with the highest overlap with the fog-free image from the multiple foggy images as the first image, and the remaining foggy images are used as the second images.

[0057] In this embodiment, the present invention uses the first image as a reference to perform defogging on multiple second images. Therefore, the present invention selects a foggy image with the highest overlap with the fog-free image as the first image, and ensures that the background of the first image and the fog-free image are the same to the greatest extent, thereby improving the defogging effect of the multiple second images.

[0058] When selecting a foggy image with the highest overlap with the fog-free image from multiple foggy images as the first image, firstly, multiple foggy images and fog-free images need to be resized and adjusted to the same size to eliminate the influence of size differences. Then the processed images are grayed and converted into grayscale images to reduce the influence of color on similarity calculation. The pixel mean of each grayscale image is calculated for subsequent comparison. By comparing the difference between the grayscale value of the pixel and the average value, the hash value of the image is generated, and the similarity of the images is evaluated by comparing the Hamming distance of these hash values. The foggy image with the highest similarity is taken as the first image.

[0059] S2: Extract features from the first image to obtain a first feature map for characterizing the fog state.

[0060] In this embodiment, the first feature map is a dark channel image corresponding to the parameters required by the defogging algorithm based on image enhancement, and feature extraction of the first image includes the following steps:

[0061] Step 1: Obtain pixel values ​​of each pixel in the first image on three channels of red (R), green (G) and blue (B), respectively, and extract the minimum value of the pixel values ​​of each pixel on the three channels.

[0062] Step 2: Combine the minimum values ​​of all pixels to obtain a first grayscale image of the same size as the first image.

[0063] Step 3: Perform minimum filtering on the first grayscale image to obtain the first dark channel image. The filtering window size can be selected as needed, for example, the commonly used window size is 15x15. In each window, find the minimum value of all pixel values ​​and assign the minimum value to the pixel at the center of the window.

[0064] Through these three steps, a dark channel image can be obtained. The areas with lower grayscale values ​​usually correspond to the fog or shadow parts in the original image.

[0065] S3: Extract features from the multiple second images to obtain multiple second feature maps.

[0066] The second feature map is a dark channel image corresponding to the parameters required by the defogging algorithm based on image enhancement, and its extraction method is the same as that of the first image.

[0067] S4: Obtain multiple similarities between the first feature map and the multiple second feature maps, and weight the multiple second feature maps according to the multiple similarities.

[0068] Step S4 includes:

[0069] The first step: extracting a first texture feature of a first feature map, and extracting a plurality of second texture features of a plurality of second feature maps.

[0070] Image texture features are a kind of visual features that reflect homogeneous phenomena in images. They reflect the surface structural organization and arrangement properties of objects with slow or periodic changes. Texture has three major characteristics: a certain local sequence is repeated continuously, non-random arrangement, and the texture area is roughly uniform. The present embodiment first obtains the grayscale co-occurrence matrix of the first feature map and normalizes it, and then obtains the energy of the grayscale co-occurrence matrix, that is, the sum of the squares of the values ​​of each element, to obtain the texture features of the image. The energy reflects the uniformity of the grayscale distribution of the image and the coarseness of the texture. A large energy value indicates that the current texture is a relatively stable texture with regular changes.

[0071] Step 2: Obtain similarity values ​​between the first texture feature and the plurality of second texture features through a texture feature matching algorithm, and arrange the plurality of similarity values ​​in descending order.

[0072] The present invention uses the cosine similarity between two texture feature vectors to measure their similarity. The value range of cosine similarity is [-1, 1], and the larger the value, the more similar the two vectors are. The specific calculation formula is as follows:

[0073]

[0074] In the formula, x i and i They represent the i-th component of the two texture feature vectors, n is the dimension of the vector, and θ is the angle between the two vectors.

[0075] Step 3: assign corresponding weights to the second feature maps corresponding to the multiple second texture features in the order of arrangement, and the sum of the multiple weight values ​​is 1, so as to obtain the weighted second feature map, as shown below:

[0076]

[0077] Where F is the weighted second feature map, b i is the i-th weighting coefficient, x i is the i-th second feature map, and n is the number of second feature maps.

[0078] S5: Obtain a defogging mapping relationship through the haze-free image and the first feature map.

[0079] The pixel with the highest brightness in the first dark channel image is taken as the first atmospheric light intensity. The multiple transmittances corresponding to the multiple pixel points of the first dark channel feature map are averaged to obtain the first transmittance. Based on the image defogging algorithm, the mapping relationship between the first atmospheric light intensity and the light intensity of the fog-free image and the mapping relationship between the first transmittance and the transmittance of the fog-free image are established.

[0080] S6: Input the multiple weighted second feature maps into the defogging mapping relationship to obtain a corresponding defogging image.

[0081] Obtain multiple corresponding second atmospheric light intensities and second transmittances in multiple weighted second feature maps. Input the second atmospheric light intensity of each weighted second feature map into the intensity mapping relationship, and input the second transmittance into the transmission mapping relationship to obtain the corresponding defogging image.

[0082] Based on the same concept, the present invention also provides a highway construction image defogging device, which includes a collection module, an extraction module, a weighting module and a defogging module.

[0083] The acquisition module is used to acquire a fog-free image and multiple foggy images in the same construction link, and select a foggy image with the highest overlap with the fog-free image from the multiple foggy images as the first image, and the remaining foggy images as the second images;

[0084] The extraction module is used to perform feature extraction on the first image to obtain a first feature map for characterizing the fog state, and to perform feature extraction on the plurality of second images to obtain a plurality of second feature maps;

[0085] The weighting module is used to obtain multiple similarities between the first feature map and the multiple second feature maps, and weight the multiple second feature maps according to the multiple similarities;

[0086] The defogging module is used to obtain a defogging mapping relationship through a haze-free image and a first feature map, and input a plurality of weighted second feature maps into the defogging mapping relationship to obtain a corresponding defogging image.

[0087] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the above-mentioned highway construction image defogging method is implemented when the processor executes the program.

[0088] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned highway construction image defogging method is implemented.

[0089] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0090] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A highway construction image defogging method, characterized in that: The following steps are involved: Collect a fog-free image and multiple foggy images in the same construction link, and select a foggy image with the highest overlap with the fog-free image from the multiple foggy images as the first image, and the remaining foggy images as the second images; Performing feature extraction on the first image to obtain a first feature map for characterizing the fog state, and performing feature extraction on the plurality of second images to obtain a plurality of second feature maps; Obtaining multiple similarities between the first feature map and the multiple second feature maps, and weighting the multiple second feature maps according to the multiple similarities; A defogging mapping relationship is obtained through the haze-free image and the first feature map, and a plurality of weighted second feature maps are input into the defogging mapping relationship to obtain a corresponding defogging image.

2. A highway construction image defogging method as claimed in claim 1, characterized in that: The step of extracting features from the first image to obtain a first feature map for characterizing the fog state comprises the following steps: Obtain pixel values ​​of each pixel in the first image on three channels, namely, red, green and blue, respectively, and extract the minimum value of the pixel values ​​of each pixel on the three channels; Combining the minimum values ​​of all pixels to obtain a first grayscale image having the same size as the first image; Minimum filtering is performed on the first grayscale image to obtain a first dark channel image, where the first dark channel image is a first feature map.

3. A highway construction image defogging method as claimed in claim 1, characterized in that: The method of obtaining multiple similarities between the first feature map and the multiple second feature maps, and weighting the multiple second feature maps according to the multiple similarities, comprises the following steps: Extracting a first texture feature of the first feature map, and extracting a plurality of second texture features of a plurality of second feature maps; Obtaining similarity values ​​between the first texture feature and the plurality of second texture features through a texture feature matching algorithm, and arranging the plurality of similarity values ​​in descending order; The second feature maps corresponding to the plurality of second texture features are assigned corresponding weights in the order of arrangement to obtain a weighted second feature map, as shown below: Where F is the weighted second feature map, b i is the i-th weighting coefficient, x i is the i-th second feature map, and n is the number of second feature maps.

4. A highway construction image defogging method as claimed in claim 2, characterized in that: The step of obtaining a defogging mapping relationship through a fog-free image and a first feature map comprises the following steps: Taking the pixel with the highest brightness in the first dark channel image as the first atmospheric light intensity; Averaging a plurality of transmittances corresponding to a plurality of pixel points of the first dark channel feature map to obtain a first transmittance; Based on the image defogging algorithm, a mapping relationship between the first atmospheric light intensity and the light intensity of the fog-free image and a mapping relationship between the first transmittance and the transmittance of the fog-free image are established.

5. A highway construction image defogging method as claimed in claim 4, characterized in that: The step of inputting a plurality of weighted second feature maps into a defogging mapping relationship to obtain a corresponding defogging image comprises the following steps: Obtaining a plurality of second atmospheric light intensities and second transmittances corresponding to a plurality of weighted second feature maps; The second atmospheric light intensity of each weighted second feature map is input into the light intensity mapping relationship, and the second transmittance is input into the transmission mapping relationship to obtain the corresponding defogging image.

6. A highway construction image defogging method as claimed in claim 1, characterized in that: The step of selecting a foggy image with the highest overlap with the non-fog image from among the multiple foggy images as the first image comprises the following steps: Adjust multiple foggy images and fog-free images to the same size; Converting the adjusted multiple foggy images and fog-free images into grayscale images to obtain multiple foggy grayscale images and fog-free grayscale images; Obtaining a plurality of first hash values ​​of a plurality of foggy grayscale images and a second hash value of a non-foggy grayscale image, and obtaining a plurality of Hamming distances between the plurality of first hash values ​​and the second hash values; Multiple similarities are obtained through multiple Hamming distances, and the foggy image with the highest similarity is used as the first image.

7. A highway construction image defogging device, characterized in that: include: An acquisition module is used to acquire a fog-free image and multiple foggy images in the same construction link, and select a foggy image with the highest overlap with the fog-free image from the multiple foggy images as the first image, and the remaining foggy images as the second images; An extraction module, used to perform feature extraction on the first image to obtain a first feature map for characterizing the fog state, and to perform feature extraction on the plurality of second images to obtain a plurality of second feature maps; A weighting module, used to obtain multiple similarities between the first feature map and the multiple second feature maps, and weight the multiple second feature maps according to the multiple similarities; The defogging module is used to obtain a defogging mapping relationship through a haze-free image and a first feature map, and input a plurality of weighted second feature maps into the defogging mapping relationship to obtain a corresponding defogging image.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for defogging highway construction images as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the highway construction image defogging method described in any one of claims 1 to 6 is implemented.

Images