Surveillance video image enhancement method for traffic construction road obstacle detection

Abstract

The present invention discloses a monitoring video image enhancement method for detecting obstacles on traffic construction roads, which relates to the technical field of image enhancement. First, each frame image in the traffic monitoring video is obtained, and the image is subjected to illumination correction to obtain the current illumination-corrected image; by performing differential analysis on each pixel point in the current illumination-corrected image, a region mask is generated, and the region mask is used to identify abnormal areas in the current illumination-corrected image, including shadow areas and strong light reflection areas; then, the shadow areas and strong light reflection areas are restored to obtain a processed image, and then edge detection is performed on the processed image to form two boundary pixel sets, and a matching relationship is established based on the two boundary pixel sets, and the established matching relationship is verified to generate verification result information, and the verification result information is used to determine whether the established matching relationship is correct, thereby ultimately realizing the identification of the broken part of the obstacle.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to a monitoring video image enhancement method for detecting obstacles on traffic construction roads. Background Art

[0002] In the field of computer vision and image processing, with the rapid development of intelligent transportation systems, the importance of road obstacle detection during construction is becoming increasingly prominent as a key component in ensuring road construction safety and smooth traffic flow. Traffic surveillance videos can visually display the dynamic conditions of construction areas. However, due to the complex and ever-changing construction environment, surveillance video images are often affected by factors such as uneven lighting, shadows, and strong light reflections, resulting in blurred obstacle features and information loss. This makes it difficult for traditional obstacle detection methods to accurately identify and locate obstacles, hindering road construction safety supervision and traffic diversion efficiency.

[0003] Currently, illumination issues severely hamper the accuracy and reliability of obstacle detection during analysis of road construction surveillance video. For example, in road construction scenarios, large shadows often appear in surveillance video images due to obstructions from construction equipment and buildings. This makes key features of obstacles, such as warning signs and the outlines and colors of construction vehicles, difficult to discern, leading to missed detections. Furthermore, in conditions such as direct sunlight or reflections from warning signs, strong light reflections can cause areas of the image to become overly bright, obscuring details such as the surface texture of reflective cones and warning signs. This obscured detail can lead to inaccurate obstacle detection, misjudgments, and serious safety hazards. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention provides a monitoring video image enhancement method for detecting obstacles on traffic construction roads, which solves the problems in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A monitoring video image enhancement method for detecting road obstacles during traffic construction comprises the following steps:

[0006] S1: Obtain each frame of the traffic monitoring video and perform illumination correction on the image to obtain the current illumination-corrected image;

[0007] S2: Generate a region mask by performing differential analysis on each pixel in the current illumination-corrected image. The region mask is used to identify abnormal regions in the current illumination-corrected image, including shadow regions and strong light reflection regions.

[0008] S3: Restoring the shadow area and the strong light reflection area to obtain a processed image;

[0009] S4: Perform edge detection on the processed image to form two boundary pixel sets, establish a matching relationship based on the two boundary pixel sets, and verify the established matching relationship to generate verification result information. The verification result information includes a verification pass signal and a verification fail signal to determine whether the established matching relationship is correct; after traversal, extract the boundary pixel point pairs corresponding to the verification pass signal in the verification result information, and use the position of the boundary pixel point pair corresponding to the verification pass signal as the broken part of the obstacle.

[0010] Preferably, a starting frame position is set in the traffic monitoring video to read each frame of the image one by one, and the read image is converted from the RGB color space to the HSV color space, and the brightness channel is extracted separately. For each pixel in the image, its pixel value in the brightness channel is recorded as V (x, y) to obtain brightness information;

[0011] According to the brightness information, the pixel values ​​at each position in the image are averaged to calculate the global average brightness value in the image, specifically: ,in, is the global average brightness value, M and N are the width and height of the image respectively, and x and y are the points with width and height as horizontal and vertical coordinates respectively.

[0012] Preferably, the brightness channel is processed using homomorphic filtering technology in combination with the global average brightness value, and the image frequency components are adjusted using a homomorphic filtering function during the homomorphic filtering process, specifically including:

[0013] When using the homomorphic filter function to adjust the image frequency components, the high-frequency gain and low-frequency gain are dynamically adjusted in combination with the global average brightness value, and the dynamic adjustment coefficient is set as follows: , where w is the dynamic adjustment coefficient, is the maximum pixel value in the brightness channel;

[0014] According to the dynamic adjustment coefficient, the high-frequency gain and low-frequency gain are dynamically adjusted, specifically: , ,in, and are the high frequency gain and low frequency gain before adjustment respectively. is the gain adjustment amplitude coefficient, which is used to control the degree of adjustment;

[0015] Substitute the adjusted high-frequency gain and low-frequency gain into the homomorphic filter function before adjustment to perform illumination correction, including:

[0016] The adjusted homomorphic filtering function is expressed as follows:

[0017] ,in, is the adjusted homomorphic filter function, is the adjusted high frequency gain, is the adjusted low-frequency gain, c is the sharpening coefficient, which determines the sharpness of the homomorphic filter function. is the cutoff frequency, D(u, v) is the distance from the midpoint (u, v) in the frequency domain to the origin of the frequency plane, and e is the Euler number;

[0018] Perform Fourier transform on the pixel values ​​at each position in the brightness channel, and process the Fourier transformed results in the frequency domain through the adjusted homomorphic filter function to obtain the filtered frequency domain brightness information, specifically: ,in, is the result of Fourier transform and homomorphic filtering, is the result of Fourier transform on the pixel values ​​at each position in the brightness channel;

[0019] Then perform inverse Fourier transform on the filtered frequency domain brightness information to obtain the corrected brightness channel;

[0020] The corrected brightness channel is recombined with the hue and saturation channels of the original image, and then converted back to the RGB color space to obtain the image corrected for the current lighting.

[0021] Preferably, a region mask is generated by performing a differential analysis on each pixel in the current illumination-corrected image, including:

[0022] For each pixel in the current illumination-corrected image, extract the values ​​of its red, green, and blue channels to obtain the RGB value of each pixel. Based on the RGB values ​​of each pixel, calculate the normalized RGB color vector of each pixel. The normalization method is to divide the value of each channel by the sum of the three channel values.

[0023] Traverse all pixels in the illumination-corrected image and obtain the normalized RGB color vector of each pixel;

[0024] Multiple sets of historical illumination-corrected images within a historical period are obtained, and the normalized RGB color vectors of all pixels in all the historical illumination-corrected images are used as samples. The distribution of the samples in the color space is statistically analyzed, and the frequency of occurrence of different color vectors is recorded using a histogram to construct a color distribution model. Based on the color distribution model, the pixels within the normal illumination area are determined.

[0025] Preferably, the normalized RGB color vector mean and covariance matrix of all pixels in the normal illumination area are calculated;

[0026] Combined with the pixels in the current illumination-corrected image, calculate the Mahalanobis distance between the pixels in the current illumination-corrected image and all pixels in the normal illumination area;

[0027] If the Mahalanobis distance exceeds the preset distance threshold, the corresponding pixel point is regarded as the shadow area, otherwise it is not regarded as the shadow area.

[0028] Preferably, using the gradient information of the image after current illumination correction to detect the strong light reflection area includes:

[0029] Use the Sobel operator to calculate the gradient information of the image in the x direction and the y direction after the current illumination correction. The gradient information includes the gradient of the image in the x direction and the gradient in the y direction after the current illumination correction.

[0030] Based on the gradient information, the gradient amplitude is calculated. The gradient amplitude is used to represent the gradient intensity of the pixel point in the image after the current illumination correction.

[0031] A gradient amplitude threshold is set in advance, and all pixels in the current illumination-corrected image are traversed. If the gradient amplitude of a pixel exceeds the gradient amplitude threshold, the corresponding pixel is marked as a strong light reflection area. Otherwise, no corresponding marking is performed.

[0032] Combine the shadow area and the bright reflection area to generate an area mask.

[0033] Preferably, a plurality of groups of historical illumination-corrected images containing shadow areas are selected from a plurality of groups of historical illumination-corrected images within a historical period as a training set, and their corresponding reference images under normal illumination are obtained. A set of image pairs is generated by establishing a one-to-one correspondence between the historical illumination-corrected images containing shadow areas and their corresponding reference images under normal illumination;

[0034] After unifying and normalizing the size of each image pair in the image pair set, the image is input into a generative adversarial network. The generator generates an image with no shadow areas corrected for historical illumination, which is labeled as the generated image. The discriminator then analyzes the difference in feature distribution between the generated image and the corresponding reference image under normal illumination to output the probability that the generated image is the corresponding reference image under normal illumination. Based on the probability that the generated image is the corresponding reference image under normal illumination, the generator gradually optimizes its own parameters through adversarial training to produce a trained generative adversarial network.

[0035] The illumination-corrected image containing the shadow area is input into the trained generative adversarial network as the image to be detected to generate a shadow-restored image.

[0036] Based on the shadow restoration image, a nonlinear mapping function is applied to the illumination-corrected image containing strong light reflection areas to reduce the brightness of highlight pixels. The highlight-suppressed image is divided into several groups of local regions, and the histogram of each local region is calculated and histogram equalization is performed.

[0037] The processing results of each local area are stitched together to obtain a processed image, and the processing results of each local area represent an image after histogram equalization.

[0038] Preferably, the Canny algorithm is used to implement edge detection of the processed image to obtain the edges of the shadow area and the strong light reflection area;

[0039] Extracting boundary pixel points from the edges of the shadow area and the strong light reflection area to form two boundary pixel sets, corresponding to the shadow area boundary pixel set and the strong light reflection area boundary pixel set respectively;

[0040] Extract the feature vectors of all boundary pixel points in the two boundary pixel sets. The feature vectors include color features and gradient features. The color features include RGB values ​​and HSV values.

[0041] Based on the feature vector, the feature similarity of each boundary pixel point in the shadow area boundary pixel set and each boundary pixel point in the strong light reflection area boundary pixel set is analyzed, and the feature similarity is obtained by calculating the cosine similarity. If the feature similarity exceeds the preset similarity threshold, it is initially indicated that the corresponding two boundary pixels belong to the broken part of the same obstacle; otherwise, they do not belong to the broken part of the same obstacle.

[0042] Traverse all boundary pixels in the shadow area boundary pixel set, find the boundary pixel in the strong light reflection area boundary pixel set whose feature similarity exceeds the similarity threshold for each boundary pixel, and establish a matching relationship;

[0043] The established matching relationship is verified to generate verification result information.

[0044] Preferably, the established matching relationship is verified to generate verification result information, including:

[0045] According to the established matching relationship, the mutually matching boundary pixel pairs are determined;

[0046] For each pair of matched boundary pixels, the neighborhood of the set range is expanded around the pixel itself to obtain the neighborhood pixel set of the corresponding boundary pixel point in the shadow area and the neighborhood pixel set of the corresponding boundary pixel point in the strong light reflection area respectively;

[0047] The cosine similarity calculation method is used again to calculate the feature similarity between the neighborhood pixel set of the corresponding boundary pixel point in the shadow area and the neighborhood pixel set of the corresponding boundary pixel point in the strong light reflection area, which is recorded as the feature similarity between boundary pixels. If the feature similarity between boundary pixels exceeds the similarity threshold, it indicates that there is similarity between the corresponding two boundary pixels.

[0048] In the neighborhood pixel sets of the corresponding boundary pixels in the shadow area and the neighborhood pixel sets of the corresponding boundary pixels in the strong light reflection area, the number of boundary pixel pairs whose feature similarity between boundary pixels exceeds the similarity threshold is counted. If the number of boundary pixel pairs whose feature similarity between boundary pixels exceeds the similarity threshold exceeds If the neighborhood is within the set range, a verification pass signal is generated; otherwise, a verification fail signal is generated;

[0049] Combine the verification pass signal and the verification fail signal to generate verification result information;

[0050] According to the established matching relationship, all mutually matching boundary pixel pairs are traversed to obtain boundary pixel pairs that have been verified by the neighborhood, and the boundary pixel pairs corresponding to the verification result information as the verification qualified signal are extracted from them. After statistics, the obstacle break part is generated.

[0051] The present invention provides a monitoring video image enhancement method for detecting obstacles on traffic construction roads, which has the following beneficial effects:

[0052] (1) By converting the image from RGB color space to HSV color space and extracting the brightness channel separately, and combining it with homomorphic filtering technology for illumination correction, the image quality problem caused by uneven illumination can be effectively improved. For example, in scenes with large differences in illumination, the details of overly dark or overly bright areas can be clearly presented. Differential analysis of pixel points is performed to generate regional masks to accurately identify abnormal areas, laying the foundation for subsequent restoration processing, so that obstacles in the image can be better presented under various illumination conditions.

[0053] (2) A generative adversarial network is used to restore shadow areas. Through an adversarial training mechanism, the generator generates images close to those under real normal lighting. For obstacles such as construction signs and equipment that are obscured by shadows, their shapes and details can be effectively restored. For areas with strong light reflections, a nonlinear mapping function is used to suppress highlights, and local histogram equalization is combined to enhance details. This can solve the problem of object detail loss caused by strong light reflections, making the image information more complete and accurate, and facilitating subsequent obstacle analysis.

[0054] (3) By extracting the features of boundary pixels and calculating the similarity, a preliminary matching relationship is established, and then the matching relationship is verified in the neighborhood. Starting from the overall feature similarity of the boundary pixel neighborhood, misjudgment caused by local interference or noise is eliminated. In complex traffic construction scenes, different parts of the same obstacle that are divided by shadows and strong light reflections can be accurately associated, and the broken parts of the obstacle can be determined. This provides accurate obstacle information for traffic construction management, helps to scientifically plan construction areas, formulate traffic diversion plans, and ensure construction safety and smooth traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the process of the present invention;

[0056] Figure 2 This is the logic diagram of the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] Example 1, please refer to Figure 1 and Figure 2 The present invention provides a monitoring video image enhancement method for detecting obstacles on traffic construction roads, comprising the following steps:

[0059] S1: Obtain each frame of the traffic monitoring video and perform illumination correction on the image to obtain the current illumination-corrected image;

[0060] S2: Generate a region mask by performing differential analysis on each pixel in the current illumination-corrected image. The region mask is used to identify abnormal regions in the current illumination-corrected image, including shadow regions and strong light reflection regions.

[0061] S3: Restoring the shadow area and the strong light reflection area to obtain a processed image;

[0062] S4: Perform edge detection on the processed image to form two boundary pixel sets, establish a matching relationship based on the two boundary pixel sets, and verify the established matching relationship to generate verification result information. The verification result information includes a verification pass signal and a verification fail signal to determine whether the established matching relationship is correct; after traversal, extract the boundary pixel point pairs corresponding to the verification pass signal in the verification result information, and use the position of the boundary pixel point pair corresponding to the verification pass signal as the broken part of the obstacle.

[0063] In this embodiment, step S1: The image in the traffic monitoring video may be affected by various lighting conditions, such as changes in natural light at different times, artificial lighting in the construction area, etc. These uneven lighting conditions may cause some areas in the image to be too bright or too dark, affecting subsequent analysis and processing. Through lighting correction, the overall brightness and contrast of the image can be made more uniform, providing a more stable foundation for subsequent processing. Taking the actual traffic construction scene as an example, in the evening, due to insufficient light, the image in the monitoring video may have large areas of shadows, making obstacles on the road, such as construction signs and roadblocks, difficult to clearly identify. After lighting correction, the shadow area in the image is significantly improved, and the outline and details of the obstacle become clearer, which helps to improve the accuracy of obstacle detection in subsequent steps.

[0064] Step S2: In the image after illumination correction, pixels in different areas have different characteristics. By analyzing these pixels and comparing them with the surrounding pixels, areas with characteristics that are significantly different from normal areas can be identified. These areas may interfere with the accurate detection of obstacles and therefore require special processing. For example, in a construction area, there may be some metal objects such as construction equipment, which will reflect sunlight or light, forming strong light reflection areas. These areas appear as extremely bright areas in the image and will obscure the details of surrounding objects. By generating a region mask, these strong light reflection areas, as well as other possible shadow areas, can be accurately identified. This allows subsequent processing to operate on these abnormal areas in a targeted manner to avoid interference with obstacle detection.

[0065] Step S3: The presence of shadow areas and areas of strong light reflection can affect image quality and information integrity, hindering obstacle detection. Restoring these areas allows obstacles in the image to appear complete and detailed, thereby improving obstacle detection accuracy. For example, suppose a construction sign partially obscured by a shadow is identified in step S2. In step S3, shadow restoration can be used to restore the sign information in the shadowed area, making the shape and text of the sign more clearly visible. If part of the sign is located in the shadowed area, this method can also be used to identify its location.

[0066] Similarly, for areas with strong light reflections, after processing, the details of obstacles that were originally obscured by the reflected light can also be revealed, which helps to more accurately determine the type and location of the obstacles.

[0067] Step S4: After the previous steps, the obstacles in the image have been well restored and enhanced. Edge detection can extract edge information of objects in the image, forming a set of boundary pixels. Since obstacles may be segmented into different parts in different lighting areas (such as shadows and strong light reflection areas), by establishing and verifying matching relationships between these boundary pixel sets, it is possible to determine which parts belong to the same obstacle, thereby identifying the broken parts of the obstacle. For example, in a traffic construction scene, there may be multiple obstacles, and due to lighting and occlusion, these obstacles appear as discontinuous parts in the image. Through edge detection and the establishment and verification of matching relationships, these seemingly broken parts can be correctly associated to form a complete obstacle outline. This is crucial for accurately determining the shape, size, and location of the obstacle, helping traffic management departments to take timely measures to ensure the safety and smooth progress of road construction.

[0068] In summary, this method starts with image illumination correction, gradually identifies and processes abnormal areas, and finally verifies through edge detection and matching. It effectively solves the problem of obstacle detection in traffic construction road monitoring videos, improves image quality and the accuracy of obstacle detection, and has important practical application value.

[0069] Example 2, please refer to Figure 1 Specifically, the starting frame position is set in the traffic monitoring video to read each frame of the image one by one, and the read image is converted from the RGB color space to the HSV color space, and the brightness channel, that is, the V channel, is extracted separately for further processing or enhancement. For each pixel point (x, y) in the image, its pixel value in the brightness channel is recorded as V(x, y) to obtain the brightness information;

[0070] According to the brightness information, the pixel values ​​at each position in the image are averaged to calculate the global average brightness value in the image, specifically: ,in, is the global average brightness value, M and N are the width and height of the image respectively, and x and y are the points with width and height as horizontal and vertical coordinates respectively.

[0071] The global average brightness value represents the overall brightness of the image and is an important basis for determining whether the image lighting is uniform and the degree of lighting deviation. By calculating the global average brightness value, we can gain a quantitative understanding of the overall lighting conditions of the image and provide a benchmark for subsequent correction operations.

[0072] Combined with the global average brightness value, homomorphic filtering technology is used to process the brightness channel. During the homomorphic filtering process, the homomorphic filtering function is used to adjust the image frequency components. Specifically, the following steps are performed:

[0073] When using the homomorphic filter function to adjust the image frequency components, the high-frequency gain and low-frequency gain are dynamically adjusted in combination with the global average brightness value, and the dynamic adjustment coefficient is set as follows: , where w is the dynamic adjustment coefficient, is the maximum pixel value in the brightness channel;

[0074] According to the dynamic adjustment coefficient, the high-frequency gain and low-frequency gain are dynamically adjusted, specifically: , ,in, and are the high frequency gain and low frequency gain before adjustment respectively. is the gain adjustment amplitude coefficient, which is used to control the degree of adjustment;

[0075] Substitute the adjusted high-frequency gain and low-frequency gain into the homomorphic filter function before adjustment to perform illumination correction, including:

[0076] The adjusted homomorphic filtering function is expressed as follows: ,in, is the adjusted homomorphic filter function, is the adjusted high frequency gain, is the adjusted low-frequency gain, c is the sharpening coefficient, which determines the sharpness of the homomorphic filter function. is the cutoff frequency (controls the boundary between high and low frequencies), D(u, v) is the distance from the midpoint (u, v) in the frequency domain to the origin of the frequency plane, and e is the Euler number, which is approximately 2.71828;

[0077] The calculation formula for the distance D(u, v) from the midpoint (u, v) in the frequency domain to the origin of the frequency plane is: , where u and v are coordinate variables in the frequency domain, used to represent frequency components at different locations in the frequency domain. u corresponds to the horizontal frequency coordinate, and v corresponds to the vertical frequency coordinate.

[0078] In a two-dimensional plane coordinate system, the distance from a point (x, y) to the origin ((0, 0)) can be calculated using the Pythagorean theorem, and the formula is d= In the frequency domain, the origin of the frequency plane is usually located at ( , ), this is because in the frequency domain representation after discrete Fourier transform, the origin is located at half the width and height of the image, so the distance from a point ((u, v) in the frequency domain to the origin ( , ), the coordinate difference is substituted into the Pythagorean theorem formula, which is mainly used to measure the distance between different positions in the frequency domain and the origin. This plays an important role in distinguishing high-frequency and low-frequency components in operations involving frequency processing such as homomorphic filtering.

[0079] Perform Fourier transform on the pixel values ​​at each position in the brightness channel, and process the Fourier transformed results in the frequency domain through the adjusted homomorphic filter function to obtain the filtered frequency domain brightness information, specifically: ,in, The result after Fourier transformation is processed by homomorphic filtering, that is, in the frequency domain, the result after Fourier transformation is processed by the adjusted homomorphic filtering function to obtain the filtered frequency domain brightness information; It is the result of Fourier transform of the pixel values ​​at each position in the brightness channel, that is to say: it is the frequency domain representation obtained after Fourier transform of the brightness channel V(x, y) (which describes brightness information in the spatial domain, x, y are spatial coordinates). Specifically, Fourier transform can convert the image from the spatial domain to the frequency domain, and the frequency components of the image can be analyzed in the frequency domain. For example, low frequency represents the general outline of the image, and high frequency represents the details of the image.

[0080] The homomorphic filter function refers to a frequency domain filter control function used to control the effects of low-frequency (light) suppression and high-frequency (detail) enhancement. It is the only mathematical model with a regulatory role in the homomorphic filtering algorithm. In short, it is a regulator or weight distributor in the homomorphic filtering process, controlling the degree of retention or suppression of different frequency components.

[0081] After the above processing, when the overall brightness of the image is low (that is, the global average brightness value is small and the dynamic adjustment coefficient is small), Increase and enhance the high-frequency components of the image and highlight the image details; at the same time Reduce, appropriately reduce the overall brightness of the image to avoid excessive enhancement of bright areas. On the contrary, when the overall brightness of the image is high (the global average brightness value is large, the dynamic adjustment coefficient is large), Reduce to prevent excessive enhancement of details leading to noise amplification; Increase to ensure that the overall brightness of the image is not too dark.

[0082] By incorporating the global average brightness value into the homomorphic filtering gain adjustment process, illumination correction can be adaptively adjusted based on the overall brightness of the image, further improving the effect of illumination correction and providing a higher-quality image foundation for subsequent steps such as shadow and strong light reflection area detection and feature enhancement.

[0083] The meaning of the formula is: the adjusted homomorphic filtering function and frequency domain brightness information Element-by-element multiplication is performed to achieve filtering operations on frequency domain brightness information.

[0084] Correcting illumination non-uniformity through homomorphic filtering can enhance the overall contrast of the image, making the details of different areas in the image clearer, avoiding misjudgment of obstacle features due to illumination differences, and providing a more accurate basic image for subsequent detection of shadows and strong light reflection areas.

[0085] Then the filtered frequency domain brightness information, that is Perform inverse Fourier transform to obtain the corrected brightness channel;

[0086] The corrected brightness channel is recombined with the hue and saturation channels of the original image, and then converted back to the RGB color space to obtain the image after the current illumination correction. The original image here refers to the corresponding image in the traffic monitoring video obtained at the beginning of S1;

[0087] The hue channel is the H channel in the HSV color space, and the saturation channel is the S channel in the HSV color space;

[0088] In this embodiment, the image is converted from the RGB color space to the HSV color space because the HSV color space better separates color information into brightness, hue, and saturation. Extracting the brightness channel (V channel) alone allows for focused processing of image illumination information, avoiding interference from color information. This makes subsequent corrections for uneven illumination more accurate and efficient. For example, in actual traffic monitoring scenarios, early morning or evening light often exhibits a specific hue, such as warm tones, and exhibits uneven illumination intensity. Directly addressing illumination in the RGB color space would also alter color information, resulting in image color distortion. However, after converting to the HSV color space, the V channel is extracted for processing. For example, in an evening surveillance image, one side of the road is shaded by trees, while the other side is brighter due to streetlights. By extracting the V channel, only the brightness can be adjusted, without affecting the original colors of the road or vehicles. The resulting image is uniformly illuminated and has realistic colors, providing an excellent foundation for subsequent obstacle detection.

[0089] Calculating the image's global average brightness value provides an indication of the overall illumination level. This value serves as an important reference for subsequent homomorphic filtering, helping the algorithm determine whether the image is brighter or darker overall. This supports dynamic adjustment of filtering parameters, allowing homomorphic filtering to better reflect the image's actual illumination conditions and achieve adaptive illumination correction. For example, in traffic surveillance videos on cloudy days, the image is generally darker, resulting in a lower calculated global average brightness value. Based on this, the algorithm can adjust parameters during subsequent homomorphic filtering to increase the image brightness, avoiding under- or overcorrection caused by fixed parameters. This effectively improves the overall visual quality of the image and makes the surveillance footage clearer and more discernible.

[0090] Homomorphic filtering technology, based on the image's illumination-reflectance model, processes the image's low-frequency components (corresponding to illumination, reflecting overall brightness and background) separately from its high-frequency components (corresponding to reflection, reflecting image details and edges). By setting dynamic adjustment coefficients, the high-frequency and low-frequency gains are dynamically adjusted based on the global average brightness value and the maximum pixel value in the brightness channel.

[0091] Low-frequency gain adjustment changes the overall brightness of the image, while high-frequency gain adjustment enhances image detail and contrast. The adjusted gain is fed into a homomorphic filtering function, which filters the luminance channel in the frequency domain. Finally, an inverse Fourier transform is used to convert the processed frequency domain information back to the spatial domain, achieving correction of the luminance channel. For example, in traffic surveillance videos, some areas are often shadowed by buildings, while others are overly bright due to direct sunlight. For example, on a road section with an overpass, the area beneath the bridge is heavily shadowed, making it difficult to see road and vehicle details, while the top of the overpass is strongly reflective due to sunlight, causing detail to be lost. By using homomorphic filtering technology combined with dynamic gain adjustment, adjusting the low-frequency gain can boost the brightness of shadowed areas, allowing objects such as vehicles and pedestrians to appear clearly beneath the bridge. Adjusting the high-frequency gain enhances details in overly bright areas, such as the texture and signage on the overpass roof. Ultimately, the brightness and contrast of the entire image are optimized, enhancing image quality by enhancing detail in both shadowed and brightly lit areas.

[0092] The corrected brightness channel is recombined with the hue channel (H channel) and saturation channel (S channel) of the original image in order to restore the complete color information of the image.

[0093] The conversion back to RGB color space is done because it is a common image display and storage format, facilitating subsequent image display, transmission, and integration with other image processing algorithms. Combining the adjusted brightness channel with the unchanged hue and saturation channels ensures uniform image illumination while maintaining the original color style and color characteristics of objects. For example, traffic signs and vehicle colors on the road can be accurately restored. After conversion back to RGB color space, they can be directly displayed in the monitoring system or used in RGB-based obstacle recognition algorithms, ensuring the consistency and practicality of the entire traffic monitoring video image enhancement process.

[0094] In summary, this method separates the illumination information from the conversion of the image color space, calculates the global average brightness value to provide a basis for subsequent processing, then uses homomorphic filtering technology combined with dynamic parameter adjustment to achieve accurate illumination correction, and finally recombines the channels to restore the image color and convert it back to a common format.

[0095] Example 3, please refer to Figure 1Specifically, by performing differential analysis on each pixel in the current illumination-corrected image, a region mask is generated, including:

[0096] For each pixel in the current illumination-corrected image, extract the values ​​of its red, green, and blue channels to obtain the RGB value of each pixel. Based on the RGB value of each pixel, calculate the normalized RGB color vector of each pixel. The normalization method is to divide the value of each channel by the sum of the three channel values. The calculation formula is:

[0097] ,

[0098] in, It is the normalized RGB color vector, 、 and It is the original value of the red (Red), green (Green), and blue (Blue) channels of the corresponding pixel in the RGB color space after illumination correction, representing the intensity value of the corresponding color channel of the corresponding pixel. 、 and The red, green, and blue channel values ​​of the corresponding pixel in the RGB color space after normalization. Normalization is to divide the original value of each channel by the sum of the three channel values ​​(if the sum is not 0). The purpose is to eliminate the influence of factors such as lighting on the color value, making the color representation more consistent and comparable.

[0099] A normalized RGB color vector is a vector obtained by normalizing the RGB color values ​​of each pixel. In the RGB color space, each pixel's color information is represented by the values ​​of the three color channels: red (R), green (G), and blue (B). Normalization involves dividing the values ​​of each of these three channels by the sum of their values, so that the vector's modulus is 1. The resulting vector is a normalized RGB color vector. A normalized RGB color vector reflects the relative proportions of pixel colors, rather than relying on their absolute brightness. It eliminates the effects of light intensity on color representation, ensuring that objects of the same color have more similar color vector representations under different lighting conditions, facilitating subsequent analysis and comparison of color features.

[0100] The red channel is also the R channel, the green channel is also the G channel, and the blue channel is also the B channel;

[0101] Traverse all pixels in the illumination-corrected image and obtain the normalized RGB color vector of each pixel;

[0102] Acquire multiple sets of historical illumination-corrected images within a historical period, and use the normalized RGB color vectors of all pixels in all historical illumination-corrected images as samples. Statistically analyze the distribution of these samples in the color space, and use a histogram to record the frequency of occurrence of different color vectors to construct a color distribution model.

[0103] The color distribution model is used to describe the distribution probability of various colors in the image in the normalized RGB color space;

[0104] According to the color distribution model, the pixels in the normal illumination area are determined as follows:

[0105] By analyzing the color distribution model, we can determine the color range of the normal lighting area. Generally speaking, the color distribution in the normal lighting area is relatively concentrated and has a higher frequency of occurrence. The frequency of occurrence of different color vectors is compared with the preset frequency threshold to obtain the normal lighting area. If the frequency of occurrence of different color vectors exceeds the preset frequency threshold, the corresponding pixel point is considered to be in the normal lighting area, otherwise it is considered to be an abnormal lighting area.

[0106] Calculate the normalized RGB color vector mean and covariance matrix of all pixels in the normal illumination area;

[0107] Combined with the pixels in the current light-corrected image, the Mahalanobis distance between the pixels in the current light-corrected image and all pixels in the normal light area is calculated. The Mahalanobis distance is used to measure the difference between the pixel color and the color of the normal light area, thereby detecting the shadow area.

[0108] If the Mahalanobis distance exceeds the preset distance threshold, the corresponding pixel point is regarded as the shadow area, otherwise it is not regarded as the shadow area.

[0109] In this embodiment, the RGB value of each pixel is extracted from the illumination-corrected image. Then, through normalization, each channel value is divided by the sum of the three channel values ​​to obtain a normalized RGB color vector. This is used to eliminate the impact of illumination changes on the color value. This makes the color representation of the same object under different illumination conditions more consistent and comparable, laying the foundation for subsequent accurate analysis of pixel color features.

[0110] Multiple sets of illumination-corrected images from a historical period are collected. The normalized RGB color vectors of all pixels in these images are used as samples. Their distribution in color space is statistically analyzed. The frequency of occurrence of different color vectors is recorded using a histogram to construct a color distribution model. This step, based on historical data, depicts the distribution probability of image colors in the normalized RGB color space under normal illumination, providing a reference standard for determining whether pixels in the current image are in normal illumination. The frequency of occurrence of different color vectors in the color distribution model is compared with a preset frequency threshold. Pixels with frequencies exceeding the threshold are considered to be in normal illumination areas; otherwise, they are considered to be in abnormal illumination areas.

[0111] First, the normalized RGB color vector mean and covariance matrix of all pixels in the normally illuminated area are calculated. Then, combined with the current image pixel, the Mahalanobis distance between the current pixel and the pixels in the normally illuminated area is calculated. The Mahalanobis distance takes into account the covariance relationship of the data and can more accurately measure the degree of difference between a pixel and the normally illuminated area. If the Mahalanobis distance exceeds a preset distance threshold, the corresponding pixel is classified as a shadow area. Otherwise, it is not considered a shadow area, thus accurately identifying shadow areas in the image. This method can accurately identify these abnormal areas.

[0112] In summary, this method builds a color distribution model based on historical data and is adaptable to a variety of complex lighting environments. Whether under cloudy or sunny skies, or even in nighttime artificial lighting environments, the accumulation of historical data and model adjustments can accurately identify abnormal areas. For example, in nighttime construction scenes, lighting is uneven, with some areas too bright and others too dark. Based on the color distribution model constructed from previous nighttime construction image data, this method can accurately determine which areas are abnormally illuminated due to lighting, providing effective support for subsequent image enhancement and obstacle detection. It has strong environmental adaptability and practicality.

[0113] Example 4, please refer to Figure 1 Specifically: Use the gradient information of the image after current illumination correction to detect the strong light reflection area, including:

[0114] The Sobel operator is used to calculate the gradient information of the image in the x direction and the y direction after the current illumination correction. The gradient information includes the gradient of the image in the x direction and the gradient in the y direction after the current illumination correction. Specifically:

[0115] ;

[0116] ;

[0117] in, and are the gradient of the image in the x direction and the gradient in the y direction after the current illumination correction, is the convolution operation, Represents the pixel value of the image at the coordinate ((x, y)) after illumination correction. The image here is obtained after the previous illumination correction processing step and is used for subsequent operations such as strong light reflection area detection. It is the basic data for the entire calculation; Is the convolution kernel used by the Sobel operator when calculating the x-direction gradient of an image. In the convolution operation, it is Perform convolution operation to extract edge and gradient information of the image in the x direction by weighted calculation of image pixel values; It is the convolution kernel used by the Sobel operator to calculate the y-direction gradient of the image. Performing convolution operations can extract the edge and gradient information of the image in the y direction.

[0118] Calculate the gradient magnitude: ,in, is the gradient amplitude;

[0119] Based on the gradient information, the gradient amplitude is calculated. The gradient amplitude is used to represent the gradient intensity of the pixel in the image after the current illumination correction. The strong light reflection area usually has a higher gradient amplitude change.

[0120] A gradient amplitude threshold is set in advance, and all pixels in the current illumination-corrected image are traversed. If the gradient amplitude of a pixel exceeds the gradient amplitude threshold, the corresponding pixel is marked as a strong light reflection area. Otherwise, no corresponding marking is performed.

[0121] Combine the shadow area and the bright reflection area to generate an area mask.

[0122] A region mask is a binary image that identifies which pixels in the image belong to shadows and which pixels belong to brightly reflected areas. In the mask image, pixels belonging to the corresponding region are assigned a value of 1, while pixels not belonging to the corresponding region are assigned a value of 0. The main purpose of obtaining a region mask is to enable different enhancement strategies for different regions during subsequent image enhancement.

[0123] Accurately detecting shadows and strong light reflection areas enables special enhancement and optimization of these special areas in subsequent processing, preventing these areas from interfering with obstacle detection and improving detection accuracy.

[0124] In this embodiment, the Sobel operator is a commonly used edge detection operator. By performing a convolution operation with the image, the gradient information of the image in the x and y directions is obtained respectively. Strong light reflection areas will produce obvious brightness mutations in the image, and this mutation will be reflected in the gradient information. By calculating the gradients in the x and y directions, the changing trends of the pixels in the image can be captured from different dimensions, providing basic data for subsequent analysis. For example, in a traffic construction scene, when light shines on the metal shell of a construction vehicle, reflective warning signs, and other objects, strong light reflections will be generated, resulting in a significant difference in brightness between the reflected area and the surrounding area. After using the Sobel operator to calculate the gradient information, the edges of these strong light reflection areas will appear as higher gradient values ​​in the gradient image, allowing them to be effectively captured. Compared with other simple brightness judgment methods, the Sobel operator is more sensitive to strong light reflection areas from the perspective of gradient changes and can more accurately locate the boundaries of the reflection area.

[0125] Obtaining only gradient information in the x and y directions makes it difficult to intuitively determine the gradient strength of a pixel. By calculating the gradient magnitude, the gradient information in both directions is combined to produce a numerical value that quantifies the gradient strength of the pixel. Highly reflective areas typically have larger gradient magnitudes due to the dramatic brightness variations. Therefore, the gradient magnitude serves as an important indicator for identifying high-light-reflective areas. For example, under strong sunlight, the reflected light from the reflective cones on the road causes the brightness of that area in the image to change dramatically. After calculating the gradient magnitude, pixels at the edges of the reflective cones and areas with significant reflections will have higher gradient magnitudes, while pixels in the surrounding normal areas will have relatively lower gradient magnitudes. This allows the gradient magnitude to distinguish high-light-reflective areas from other areas, avoiding false or missed detections caused by inaccurate gradient information in a single direction, and improving the accuracy and reliability of high-light-reflective area detection.

[0126] Shadows and strong light reflections are both considered abnormal regions in an image, and their presence can affect accurate obstacle detection. Combining these two regions to generate a unified region mask fully identifies these abnormal regions. This region mask can serve as a basis for subsequent image restoration, enhancement, and other processing, enabling targeted processing operations on these abnormal regions, improving processing efficiency and effectiveness. For example, in a traffic construction monitoring image with both shadows and strong light reflections, the previous steps have detected the shadows and strong light reflections. By combining these regions to generate a region mask, subsequent image restoration can be performed based on the region mask to process only the shadows and strong light reflections, avoiding unnecessary impact on normal areas. For obstacles covered by shadows or where details have been lost due to strong light reflections, the region mask can be used to accurately identify the locations that need repair, effectively restoring complete obstacle information, improving obstacle detection accuracy and ensuring traffic construction safety.

[0127] Example 5, please refer to Figure 1 Specifically, in traffic construction road monitoring scenarios, when obstacles are in both shadows and strong light reflection areas, their shape and color features are prone to interruption or distortion, making it difficult for the detection algorithm to obtain complete information. Therefore, it is necessary to consider and analyze the situation where obstacles are in both shadows and strong light reflection areas at the same time;

[0128] A plurality of sets of historical illumination-corrected images within a historical period are selected as training sets, and corresponding reference images under normal illumination are obtained (which can be taken by artificial fill light or selected from an existing image library). A one-to-one correspondence is established between the historical illumination-corrected images containing shadow areas and their corresponding reference images under normal illumination to generate a set of image pairs; the set of image pairs includes a plurality of image pairs with a one-to-one correspondence.

[0129] After unifying and normalizing the size of each image pair in the image pair set, the image pairs are input into the generative adversarial network;

[0130] Among them, normalizing each image pair in the image pair set (scaling the pixel values ​​from [0, 255] to the range expected by the model) can stabilize the training process and avoid gradient explosion or disappearance. The unified size is because the generative adversarial network usually requires a fixed input dimension, and the jump connections of architectures such as U-Net also rely on size matching.

[0131] The generator generates an image with historical illumination correction in the shadow-free area, which is marked as the generated image. The discriminator then analyzes the difference in feature distribution between the generated image and the corresponding reference image under normal illumination to output the probability that the generated image is the reference image under normal illumination, indicating the likelihood that the input image is a real image and the probability that the corresponding reference image under normal illumination is the real image. Based on the probability that the generated image is the reference image under normal illumination, the generator gradually optimizes its own parameters through adversarial training to produce a trained generative adversarial network.

[0132] Wherein, gradually optimizing its own parameters includes but is not limited to weight parameters of the neural network layer and bias parameters of the neural network layer;

[0133] Specifically, the generator uses a U-Net architecture, consisting of a contraction path (downsampling) and an expansion path (upsampling). In the contraction path, convolutional and pooling layers continuously extract semantic features from the image, gradually reducing the image resolution and increasing the number of channels. In the expansion path, deconvolution layers and skip connections are used to fuse the high-resolution features extracted from the contraction path with the upsampled features to generate a detailed shadow-restored image. For example, in the contraction path, a 3×3 convolution kernel is first used for convolution, followed by 2×2 max pooling for downsampling. In the expansion path, 2×2 deconvolution is used for upsampling and then concatenated with the feature map from the corresponding contraction path.

[0134] The discriminator extracts image features through convolution operations, and finally uses the Sigmoid activation function to output the probability that the generated image is a real image.

[0135] Use the Adam optimizer to train the generator and discriminator alternately, and set an appropriate learning rate, for example, the learning rate is 0.0002.

[0136] During the training process, the parameters of the generator and discriminator are continuously adjusted so that the shadow-restored image generated by the generator can deceive the discriminator while being consistent in content with the real normal lighting image.

[0137] After the training is completed, the image to be processed containing the shadow area is input into the generator to obtain the image after shadow restoration.

[0138] The illumination-corrected image containing the shadow area is input into the trained generative adversarial network as the image to be detected to generate a shadow-restored image.

[0139] On the basis of the shadow restoration image, a nonlinear mapping function is applied to the illumination-corrected image containing the strong light reflection area to reduce the brightness value of the highlight pixels. Specifically: ,in, is the brightness value of the image at position (x, y) after illumination correction, is the adjusted brightness value, 255 represents the maximum brightness value in an 8-bit image;

[0140] In an 8-bit grayscale image or a single channel of a color image, the pixel brightness value ranges from 0 to 255, where 0 represents the darkest and 255 represents the brightest. 255 is used as the denominator here to convert the original brightness value, that is, the brightness value of the image at the (x, y) position after illumination correction, to Zoom in and out to adjust the brightness of highlight pixels.

[0141] Through the above method, the brightness of the highlight area is compressed and the reflection intensity is reduced.

[0142] The image after highlight suppression is divided into several groups of local regions, the histogram of each local region is calculated and histogram equalization is performed; the image after highlight suppression refers to the result of applying a nonlinear mapping function to the image after illumination correction containing strong light reflection areas;

[0143] For each local area (for example, a 32×32 pixel block), count the number of times each gray level appears in the area. You can use an array (for example, an array of length 256, corresponding to the 256 gray levels of an 8-bit image) to record the number of times each gray level appears. Iterate over all pixels in the local area, and for each pixel's gray value, add 1 to the corresponding array element. In this way, you get the histogram of the local area.

[0144] Histogram equalization is achieved by calculating the cumulative distribution function and performing grayscale mapping based on the cumulative distribution function. The local area histogram counts the number of occurrences of each grayscale, while the cumulative distribution function calculates the cumulative probability of pixels with a grayscale less than or equal to a certain value based on the histogram. The cumulative distribution function can be used to map the original grayscale to a new grayscale to achieve histogram equalization, thereby enhancing the details and contrast of the image.

[0145] The processing results of each local area are stitched together to obtain a processed image, and the processing results of each local area represent an image after histogram equalization.

[0146] In this embodiment, images containing shadow areas are selected from multiple sets of historical illumination-corrected images within a historical period as a training set, and corresponding reference images under normal illumination are obtained to construct a set of image pairs. The purpose of this is that the Generative Adversarial Network (GAN) requires a large number of samples with corresponding relationships for learning. By matching shadow images with normal illumination images one-to-one, the model can learn the feature conversion relationship between shadow areas and normal areas. For example, in a traffic construction road monitoring, historical images contain images of construction areas with shadows at different times and in different weather conditions. These images are combined with corresponding images under normal illumination taken with artificial fill light to form image pairs, providing a rich set of learning samples for subsequent model training. The model can learn the characteristic performance of obstacles such as construction signs and roadblocks under normal illumination under different shadow conditions, thus laying the foundation for subsequent shadow recovery.

[0147] The processed image pairs are fed into a generative adversarial network. The generator generates an image without shadows, while the discriminator analyzes the feature distribution differences between the generated image and the real image (a reference image under normal lighting) and outputs a probability that the generated image is the real image. The generator uses this probability for adversarial training, optimizing its parameters to make the generated image closer to the real image. This gradually trains the generator to achieve high-performance generative adversarial networks. This process leverages the game mechanism between the generator and discriminator in a GAN, improving the generator's image generation capabilities through continuous adversarial learning.

[0148] Taking the training process as an example, as training progresses, the shadow-restored image generated by the generator gradually becomes more similar to the real image, changing from initial blur and distortion. In practical applications, when fed a current surveillance image containing shadows, the trained generative adversarial network can accurately restore the shadowed areas, clearly rendering obscured construction equipment and warning signs, and avoiding missed detection or misidentification of obstacles caused by shadows. The illumination-corrected image of the current shadowed area is fed into the trained generative adversarial network to generate a shadow-restored image, eliminating the shadow's influence on the image information. Furthermore, a nonlinear mapping function is used to reduce the brightness of highlight pixels in images containing strong light reflections. Histogram equalization is then performed by dividing the local regions to enhance image detail. This processing order is based on the different effects of shadows and strong light reflections on the image. Restoring shadows first provides a more complete image foundation, while processing strong light reflections later further optimizes image quality.

[0149] The final processed image is obtained by stitching together the local region images processed with histogram equalization. Since local region processing is intended to enhance image detail, the stitching operation integrates these local enhancements to form a complete, high-quality image. For example, in a surveillance image containing multiple construction obstacles and a complex background, image detail in different regions is enhanced after local histogram equalization. Through stitching, the entire image is richer in detail and has more balanced contrast. Whether it's a small warning sign in the distance or large construction machinery nearby, they can all be clearly presented, providing strong visual support for traffic construction safety supervision.

[0150] Example 6, please refer to Figure 1 ,Specifically: Use the Canny algorithm to detect the edge of the ,processed image to obtain the edges of the shadow area and the ,strong light reflection area;

[0151] Extracting boundary pixel points from the edges of the shadow area and the strong light reflection area to form two boundary pixel sets, corresponding to the shadow area boundary pixel set and the strong light reflection area boundary pixel set respectively;

[0152] Extract the feature vectors of all boundary pixels in the two boundary pixel sets. The feature vectors include color features and gradient features. Color features include RGB values ​​and HSV values. Gradient features refer to the gradient information and gradient intensity in the x and y directions.

[0153] Based on the feature vector, the feature similarity of each boundary pixel point in the shadow area boundary pixel set and each boundary pixel point in the strong light reflection area boundary pixel set is analyzed, and the feature similarity is obtained by calculating the cosine similarity. If the feature similarity exceeds the preset similarity threshold, it is initially indicated that the corresponding two boundary pixels belong to the broken part of the same obstacle; otherwise, they do not belong to the broken part of the same obstacle.

[0154] The closer the cosine similarity value is to 1, the higher the feature similarity between the two boundary pixels is;

[0155] Traverse all boundary pixels in the shadow area boundary pixel set, find the boundary pixel in the strong light reflection area boundary pixel set whose feature similarity exceeds the similarity threshold for each boundary pixel, and establish a matching relationship;

[0156] Through these matching relationships, the boundary pixels in the shadow area and the strong light reflection area can be associated, thereby identifying the broken parts of the same obstacle in different areas;

[0157] The established matching relationship is verified to generate verification result information.

[0158] In this embodiment, although the processed image improves the lighting problem, it is still necessary to locate the boundaries of the shadow and strong light reflection areas. The Canny algorithm can effectively extract edge information with obvious grayscale changes in the image through Gaussian filtering noise reduction, calculation of gradient amplitude and direction, non-maximum suppression, and hysteresis threshold processing, providing a clear area range for subsequent pixel point analysis. For example, at a construction site, part of a large construction machinery is in the shadow, and part is directly illuminated by sunlight to form strong light reflections. The Canny algorithm can accurately outline the edges of the machinery in different lighting areas, avoiding blurred or missing edges caused by light interference, laying the foundation for accurately identifying broken parts of obstacles.

[0159] The edge detected by edge detection is composed of a series of pixel points. By extracting and classifying these pixel points into sets of shadow areas and strong light reflection areas, the boundary data of different areas can be clearly divided, so that subsequent processing can focus on key pixel points, improving computational efficiency and accuracy. Assuming that there are multiple obstacles at the construction site, after first obtaining the edges of each obstacle in different areas, this step separates the edge pixels of each area into independent sets. For example, for cone-shaped traffic signs and warning fences, boundary pixel sets of their respective shadow and strong light reflection areas are formed to avoid confusion between pixels of different obstacles and provide clear data for subsequent accurate matching.

[0160] A single feature cannot fully describe the characteristics of a pixel, so combining color and gradient features can enhance differentiation. RGB values ​​directly reflect the original color intensity, while HSV values ​​describe color from the perspectives of hue, saturation, and brightness, which is more consistent with human visual perception. Gradient features reflect the trend of grayscale changes around pixels. The combination of the two can accurately depict the characteristics of pixels in an image. For example, in a yellow construction warning sign, its RGB value may change due to reduced brightness in shadowed areas, but the hue in the HSV value still maintains its yellow characteristics. Combined with gradient features (such as the regularity of the gradient direction and intensity at the edge of the warning sign), it can be effectively distinguished from surrounding pixels and its features can be extracted even under complex lighting conditions.

[0161] Cosine similarity measures similarity by calculating the cosine value of the angle between the feature vectors of two regions. It does not depend on the length of the vector and can effectively compare the feature distribution of pixels in different regions and screen out pixel pairs that may belong to the same obstacle. Through comprehensive traversal and matching, pixels with similar features in different regions are associated one by one, and the pixel correspondence of the same obstacle in different lighting areas is established, thereby connecting the seemingly broken parts and restoring the complete form of the obstacle. In complex scenes with multiple obstacles, such as images of construction vehicles, roadblocks and warning signs, this step can accurately match the broken parts of each obstacle in the shadow and strong light reflection areas. For example, the part of the construction vehicle in the shadow is matched with the part under strong light reflection, avoiding incorrect association with pixels of other obstacles and clearly presenting the complete outline of each obstacle.

[0162] In actual scenarios, relying on feature similarity may result in false matches, which require further confirmation through verification steps. Adjacent pixel associations, such as whether the matching relationships between adjacent pixel points are consistent, can be combined to ensure the accuracy of the matching results. Suppose two pixels are matched due to accidental similarity, but verification reveals that the matching relationships between their adjacent pixel points are chaotic, and the resulting shape does not conform to any known obstacle shape. In this case, the match can be determined to be a false match and eliminated. After determining whether the established matching relationship is correct, the final generated verification result information can further reflect the actual broken part of the obstacle, providing reliable data support for traffic construction safety monitoring, such as helping workers accurately determine the location and status of obstacles and take timely protective measures.

[0163] In summary, the present invention achieves high-precision identification of broken parts of road obstacles during traffic construction in complex lighting environments through an interlocking processing flow, effectively improving the reliability and practicality of monitoring video image analysis, and is of great significance for ensuring construction safety and optimizing traffic management.

[0164] Example 7, please refer to Figure 1 Specifically: verify the established matching relationship to generate verification result information, including:

[0165] According to the established matching relationship, the mutually matching boundary pixel pairs are determined;

[0166] For each pair of matched boundary pixels, the neighborhood of a set range (such as a 3×3 or 5×5 window) is expanded around the pixel itself. In this way, each boundary pixel corresponds to a neighborhood set containing multiple pixels, so as to obtain the neighborhood pixel set of the corresponding boundary pixel point in the shadow area and the neighborhood pixel set of the corresponding boundary pixel point in the strong light reflection area respectively;

[0167] The cosine similarity calculation method is used again to calculate the feature similarity between the neighborhood pixel set of the corresponding boundary pixel point in the shadow area and the neighborhood pixel set of the corresponding boundary pixel point in the strong light reflection area, which is recorded as the feature similarity between boundary pixels. If the feature similarity between boundary pixels exceeds the similarity threshold, it indicates that there is similarity between the corresponding two boundary pixels.

[0168] In the neighborhood pixel sets of the corresponding boundary pixels in the shadow area and the neighborhood pixel sets of the corresponding boundary pixels in the strong light reflection area, the number of boundary pixel pairs whose feature similarity between boundary pixels exceeds the similarity threshold is counted. If the number of boundary pixel pairs whose feature similarity between boundary pixels exceeds the similarity threshold exceeds If the neighborhood of the set range is within the specified range, a verification pass signal is generated, indicating that the corresponding two boundary pixels belong to the broken part of the same obstacle. Otherwise, a verification failure signal is generated, indicating that they do not belong to the broken part of the same obstacle.

[0169] For example, using cosine similarity, the value range is between [-1, 1]. When the threshold is set to 0.8, if the cosine similarity of two pixels is greater than 0.8, it means that the similarity of the two pixels exceeds the threshold, indicating that the angle between the feature vectors of the two pixels is small and the directions in the feature space are similar. To some extent, it reflects that the local features of the image represented by the two pixels have a high consistency, which may correspond to similar structures or contents in the image.

[0170] Combine the verification pass signal and the verification fail signal to generate verification result information;

[0171] Based on the established matching relationship, all mutually matching boundary pixel pairs are traversed to obtain boundary pixel pairs that have undergone neighborhood verification, and boundary pixel pairs corresponding to verification-qualified signals are extracted from them. After statistical analysis, the obstacle break portion is generated. Specifically, the obstacle break portion is obtained by statistically analyzing multiple boundary pixel pairs corresponding to verification-qualified signals.

[0172] In this embodiment, based on the established matching relationships, matching boundary pixel pairs are screened to provide specific targets for subsequent neighborhood analysis. This step forms the foundation for subsequent operations and clarifies the set of pixels requiring further verification. For each matched boundary pixel pair, a set neighborhood (e.g., a 3×3 or 5×5 window) is expanded around the corresponding boundary pixel to obtain the neighboring pixel sets within the shadow and strong light reflection areas.

[0173] By expanding the analysis scope and incorporating local information surrounding the pixel, more accurate judgments about the relationship between pixels are provided. Cosine similarity is again employed to calculate the feature similarity between two sets of neighboring pixels. Cosine similarity measures the degree of directional similarity between two vectors. By calculating the cosine similarity of the feature vectors of a set of neighboring pixels, similarity is determined between corresponding boundary pixels. If the similarity exceeds a threshold, it indicates that the two boundary pixels have a high degree of similarity in their neighborhood features. The number of boundary pixel pairs in the set of neighboring pixels whose feature similarity exceeds the threshold is counted. If this number exceeds a set range, such as a certain percentage of the total number of neighboring pixels, a pass signal is generated, confirming that the two boundary pixels belong to the same obstacle fracture. Otherwise, a fail signal is generated. This quantitative judgment method comprehensively determines whether boundary pixels belong to the same obstacle based on the similarity of a large number of neighboring pixels, avoiding misjudgments due to the similarity of individual pixels.

[0174] The verification result information is combined with the pass and fail signals. All matching boundary pixel pairs are then traversed to extract the boundary pixel pairs corresponding to the pass signals. These pairs are then counted to ultimately identify the broken sections of the obstacle. By verifying and screening all matching relationships, the identification of broken sections of complete obstacles is achieved, moving from pixel matching relationships to complete obstacle identification. For example, in traffic construction road monitoring scenarios, obstacles are often segmented into discontinuous sections due to shadows and strong light reflections. Relying solely on single pixel matching relationships can easily lead to misidentification. This verification method analyzes the overall feature similarity of the boundary pixel neighborhood and combines the similarity of multiple pixels to make a judgment. This effectively eliminates false matches caused by local interference or noise, further improving the accuracy of identifying broken sections of the same obstacle. For example, under complex lighting conditions, a construction fence partially obscured by shadows and strong light reflections may appear as multiple, seemingly independent regions in the image. This verification method accurately associates these regions and identifies them as different parts of the same fence.

[0175] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A surveillance video image enhancement method for detecting obstacles on traffic construction roads, characterized by: The following steps are included: S1: Obtain each frame of the traffic monitoring video and perform illumination correction on the image to obtain the current illumination-corrected image; S2: Generate a region mask by performing differential analysis on each pixel in the current illumination-corrected image. The region mask is used to identify abnormal regions in the current illumination-corrected image, including shadow regions and strong light reflection regions. S3: Restoring the shadow area and the strong light reflection area to obtain a processed image; S4: performing edge detection on the processed image to form two boundary pixel sets, establishing a matching relationship based on the two boundary pixel sets, and verifying the established matching relationship to generate verification result information. The verification result information includes a verification pass signal and a verification fail signal, which are used to determine whether the established matching relationship is correct; After traversal, the boundary pixel pairs corresponding to the verification result signals are extracted, and the positions of the boundary pixel pairs corresponding to the verification signal are taken as the broken parts of the obstacle.

2. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 1, characterized in that: The starting frame position is set in the traffic monitoring video, and each frame image is read one by one. The read image is converted from the RGB color space to the HSV color space, and the brightness channel is extracted separately. For each pixel in the image, its pixel value in the brightness channel is recorded as V(x, y) to obtain the brightness information; According to the brightness information, the pixel values ​​at each position in the image are averaged to calculate the global average brightness value in the image, specifically: ,in, is the global average brightness value, M and N are the width and height of the image respectively, and x and y are the points with width and height as horizontal and vertical coordinates respectively.

3. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 2, characterized in that: Combined with the global average brightness value, homomorphic filtering technology is used to process the brightness channel. During the homomorphic filtering process, the homomorphic filtering function is used to adjust the image frequency components. Specifically, the following steps are performed: When using the homomorphic filter function to adjust the image frequency components, the high-frequency gain and low-frequency gain are dynamically adjusted in combination with the global average brightness value, and the dynamic adjustment coefficient is set as follows: , where w is the dynamic adjustment coefficient, is the maximum pixel value in the brightness channel; According to the dynamic adjustment coefficient, the high-frequency gain and low-frequency gain are dynamically adjusted, specifically: , ,in, and are the high frequency gain and low frequency gain before adjustment respectively. is the gain adjustment amplitude coefficient, which is used to control the degree of adjustment; Substitute the adjusted high-frequency gain and low-frequency gain into the homomorphic filter function before adjustment to perform illumination correction, including: The adjusted homomorphic filtering function is expressed as follows: ,in, is the adjusted homomorphic filter function, is the adjusted high frequency gain, is the adjusted low-frequency gain, c is the sharpening coefficient, which determines the sharpness of the homomorphic filter function. is the cutoff frequency, D(u, v) is the distance from the midpoint (u, v) in the frequency domain to the origin of the frequency plane, and e is the Euler number; Perform Fourier transform on the pixel values ​​at each position in the brightness channel, and process the Fourier transformed results in the frequency domain through the adjusted homomorphic filter function to obtain the filtered frequency domain brightness information, specifically: ,in, is the result of Fourier transform and homomorphic filtering, is the result of Fourier transform on the pixel values ​​at each position in the brightness channel; Then perform inverse Fourier transform on the filtered frequency domain brightness information to obtain the corrected brightness channel; The corrected brightness channel is recombined with the hue and saturation channels of the original image, and then converted back to the RGB color space to obtain the image corrected for the current lighting.

4. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 3, characterized in that: Generate a region mask by performing differential analysis on each pixel in the current illumination-corrected image, including: For each pixel in the current illumination-corrected image, extract the values ​​of its red, green, and blue channels to obtain the RGB value of each pixel. Based on the RGB values ​​of each pixel, calculate the normalized RGB color vector of each pixel. The normalization method is to divide the value of each channel by the sum of the three channel values. Traverse all pixels in the illumination-corrected image and obtain the normalized RGB color vector of each pixel; Multiple sets of historical illumination-corrected images within a historical period are obtained, and the normalized RGB color vectors of all pixels in all the historical illumination-corrected images are used as samples. The distribution of the samples in the color space is statistically analyzed, and the frequency of occurrence of different color vectors is recorded using a histogram to construct a color distribution model. Based on the color distribution model, the pixels within the normal illumination area are determined.

5. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 4, characterized in that: Calculate the normalized RGB color vector mean and covariance matrix of all pixels in the normal illumination area; Combined with the pixels in the current illumination-corrected image, calculate the Mahalanobis distance between the pixels in the current illumination-corrected image and all pixels in the normal illumination area; If the Mahalanobis distance exceeds the preset distance threshold, the corresponding pixel point is regarded as the shadow area, otherwise it is not regarded as the shadow area.

6. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 5, characterized in that: Use the gradient information of the image after current illumination correction to detect strong light reflection areas, including: Use the Sobel operator to calculate the gradient information of the image in the x direction and the y direction after the current illumination correction. The gradient information includes the gradient of the image in the x direction and the gradient in the y direction after the current illumination correction. Based on the gradient information, the gradient amplitude is calculated. The gradient amplitude is used to represent the gradient intensity of the pixel point in the image after the current illumination correction. A gradient amplitude threshold is set in advance, and all pixels in the current illumination-corrected image are traversed. If the gradient amplitude of a pixel exceeds the gradient amplitude threshold, the corresponding pixel is marked as a strong light reflection area. Otherwise, no corresponding marking is performed. Combine the shadow area and the bright reflection area to generate an area mask.

7. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 6, characterized in that: A number of sets of historical illumination-corrected images containing shadow areas are selected from multiple sets of historical illumination-corrected images within a historical period as training sets, and their corresponding reference images under normal illumination are obtained. A set of image pairs is generated by establishing a one-to-one correspondence between the historical illumination-corrected images containing shadow areas and their corresponding reference images under normal illumination. After unifying and normalizing the size of each image pair in the image pair set, the image is input into a generative adversarial network. The generator generates an image with no shadow areas corrected for historical illumination, which is labeled as the generated image. The discriminator then analyzes the difference in feature distribution between the generated image and the corresponding reference image under normal illumination to output the probability that the generated image is the corresponding reference image under normal illumination. Based on the probability that the generated image is the corresponding reference image under normal illumination, the generator gradually optimizes its own parameters through adversarial training to produce a trained generative adversarial network. The illumination-corrected image containing the shadow area is input into the trained generative adversarial network as the image to be detected to generate a shadow-restored image. Based on the shadow restoration image, a nonlinear mapping function is applied to the illumination-corrected image containing strong light reflection areas to reduce the brightness of highlight pixels. The highlight-suppressed image is divided into several groups of local regions, and the histogram of each local region is calculated and histogram equalization is performed. The processing results of each local area are stitched together to obtain a processed image, and the processing results of each local area represent an image after histogram equalization.

8. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 7, characterized in that: Use the Canny algorithm to detect the edges of the processed image to obtain the edges of the shadow area and the strong light reflection area; Extracting boundary pixel points from the edges of the shadow area and the strong light reflection area to form two boundary pixel sets, corresponding to the shadow area boundary pixel set and the strong light reflection area boundary pixel set respectively; Extract the feature vectors of all boundary pixel points in the two boundary pixel sets. The feature vectors include color features and gradient features. The color features include RGB values ​​and HSV values. Based on the feature vector, the feature similarity of each boundary pixel point in the shadow area boundary pixel set and each boundary pixel point in the strong light reflection area boundary pixel set is analyzed, and the feature similarity is obtained by calculating the cosine similarity. If the feature similarity exceeds the preset similarity threshold, it is initially indicated that the corresponding two boundary pixels belong to the broken part of the same obstacle; otherwise, they do not belong to the broken part of the same obstacle; Traverse all boundary pixels in the shadow area boundary pixel set, find the boundary pixel in the strong light reflection area boundary pixel set whose feature similarity exceeds the similarity threshold for each boundary pixel, and establish a matching relationship; The established matching relationship is verified to generate verification result information.

9. The monitoring video image enhancement method for detecting road obstacles during traffic construction according to claim 8, characterized in that: Verify the established matching relationship to generate verification result information, including: According to the established matching relationship, the mutually matching boundary pixel pairs are determined; For each pair of matched boundary pixels, the neighborhood of the set range is expanded around the pixel itself to obtain the neighborhood pixel set of the corresponding boundary pixel point in the shadow area and the neighborhood pixel set of the corresponding boundary pixel point in the strong light reflection area respectively; The cosine similarity calculation method is used again to calculate the feature similarity between the neighborhood pixel set of the corresponding boundary pixel point in the shadow area and the neighborhood pixel set of the corresponding boundary pixel point in the strong light reflection area, which is recorded as the feature similarity between boundary pixels. If the feature similarity between boundary pixels exceeds the similarity threshold, it indicates that there is similarity between the corresponding two boundary pixels. In the neighborhood pixel sets of the corresponding boundary pixels in the shadow area and the neighborhood pixel sets of the corresponding boundary pixels in the strong light reflection area, the number of boundary pixel pairs whose feature similarity between boundary pixels exceeds the similarity threshold is counted. If the number of boundary pixel pairs whose feature similarity between boundary pixels exceeds the similarity threshold exceeds If the neighborhood is within the set range, a verification pass signal is generated; otherwise, a verification fail signal is generated; Combine the verification pass signal and the verification fail signal to generate verification result information; According to the established matching relationship, all mutually matching boundary pixel pairs are traversed to obtain boundary pixel pairs that have been verified by the neighborhood, and the boundary pixel pairs corresponding to the verification result information as the verification qualified signal are extracted from them. After statistics, the obstacle break part is generated.

Images