Night road surface recognition method, system and equipment based on illumination condition improvement

Abstract

The invention discloses a night road surface recognition method based on illumination condition improvement, and relates to the technical field of intelligent perception. The method comprises the following steps: receiving real-time image data, and recovering an image with low illumination and non-uniform illumination distribution at night by using an illumination enhancement algorithm; building a horizon line detection network, inputting the improved image into the horizon line detection network to separate environment information and road surface information, and obtaining image data without the environment information; dividing a road surface area in the image into image blocks according to a driving lane of the vehicle, and focusing image features on an actual driving area; and building a classification network, carrying out feature extraction and classification on the image blocks, and carrying out decision-level fusion on prediction results of the image blocks to obtain a final road surface recognition result. According to the method, the image quality is recovered by using the illumination enhancement algorithm, the night road surface recognition precision is improved, and the variance can be reduced and the perception robustness can be improved by performing decision-level fusion through the prediction results of multiple regions in the image.

Description

Technical Field

[0001] The present invention relates to the field of intelligent perception technology, and specifically to a method, system and device for nighttime road surface recognition based on improved lighting conditions. Background Art

[0002] The accuracy of road surface recognition and detection has gradually become an important factor affecting driving safety.

[0003] At present, various sensors are deployed on smart vehicles, among which cameras are widely used due to their low price and ease of use. However, most of the existing road detection technologies are based on daytime lighting conditions, and have poor robustness when dealing with complex and severe weather conditions (such as rainy days, snowy days and nights). Although there are usually car lights for illumination in night driving scenes, in the absence of ambient light, the road area in the night image is usually too dim, which can easily lead to misidentification. To this end, we propose a night road recognition method, system and device based on improved lighting conditions. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system and device for nighttime road surface recognition based on improved lighting conditions, which can improve nighttime images with low brightness and insufficient contrast, eliminate interference from environmental information, and effectively realize the recognition of common road conditions under complex lighting conditions.

[0005] According to a first aspect of the present invention, in order to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for nighttime road surface recognition based on improved lighting conditions, comprising the following steps:

[0006] Receive real-time image data and use the light enhancement algorithm to restore night images with low lighting conditions and uneven light distribution to obtain improved clear night images;

[0007] Build a horizon detection network, input the improved nighttime image into the horizon detection network for horizon segmentation, separate environmental information and road surface information, and obtain image data without environmental information;

[0008] Divide the image blocks according to the road surface area in the image data after the vehicle and the ego vehicle drive lane are separated, so that the image features are focused on the actual driving road surface;

[0009] Build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road recognition result.

[0010] Furthermore, the illumination enhancement algorithm is used to restore the night images with low lighting conditions and uneven illumination distribution, and obtain improved clear night images, as follows:

[0011] (1) Process the input image with a logarithmic scaling function to estimate the transformation achieved by the retina of the human visual system:

[0012]

[0013] Where I0 is the input RGB image, and I1 is the image obtained by the logarithmic scaling function;

[0014] (2) Processing image I0 through an exponential function to adjust the local contrast and reduce the high brightness part of the input image:

[0015] I2=1-exp(-I0) (2)

[0016] Where I2 is the image obtained by the exponential function;

[0017] (3) The parameter λ is introduced into the LIP model, and the generated I1 and I2 images are combined using the adjusted LIP model. The LIP model used is as follows:

[0018]

[0019] Where λ is a scalar that controls the enhancement process, and λ satisfies 2≤λ≤7. Through equations (1) to (3), the output image I3 has the characteristics of both image I1 and image I2.

[0020] (4) The overall brightness of the image I3 is enhanced by using the modified cumulative distribution function of the hyperbolic secant distribution, so that the potential details in the image are correctly displayed. The modified cumulative distribution function of the hyperbolic secant distribution is expressed as follows:

[0021] I4=erf(λ+arctan(exp(I3))-0.(4)

[0022] Where I4 is the output image obtained by the cumulative distribution function equation of the modified hyperbolic secant distribution, erf is the error function used to increase the brightness of the dark area;

[0023] (5) Apply a normalization function to linearly scale the pixel values. The normalization function is expressed as follows:

[0024]

[0025] Where I5 ​​is the final output result of the illumination enhancement algorithm.

[0026] Furthermore, a horizon detection network is built, and the improved night image is input into the horizon detection network for horizon segmentation, separating environmental information and road information, and obtaining image data without environmental information, as follows:

[0027] (1) The horizon detection network uses ResNet-34 as the backbone network and adds two lightweight attention mechanisms, CBAM and ECA, to the horizon detection network structure;

[0028] (2) The image is horizontally divided into different small areas, and the lightweight attention mechanism CBAM and ECA are used to enhance the detection network feature extraction and separate the environmental information and road surface information.

[0029] Furthermore, the lightweight attention mechanism CBAM includes two independent sub-modules: a channel attention module CAM and a spatial attention module SAM, which are used to perform channel and spatial attention respectively, and change the two sub-modules from a serial structure to a parallel structure, which is specifically expressed as:

[0030]

[0031] M CBAM (F) = σ(M c (F)+M s (F)) (8)

[0032] Where M c is the channel attention feature, M s is the spatial attention feature, F is the input image feature, and is the weight of MLP, C is the number of channels of the input image, r is the scaling factor, and f 7×7 represents the convolution operation with a 7×7 convolution kernel size, σ represents the Sigmoid function, and M CBAM is the final parallel CBAM attention feature. Further, the lightweight attention mechanism ECA is expressed as:

[0033]

[0034] In the formula, F is the input image feature, f k×k represents a one-dimensional convolutional layer with a convolution kernel size of k, C is the number of channels of the input feature, Υ and b are hyperparameters, and the absolute value is taken and rounded down to the nearest odd number to ensure that the size of the convolution kernel is an odd number, M ECA is the final ECA attention feature.

[0035] Furthermore, the image blocks are divided according to the road surface area in the image data after the vehicle and the self-vehicle driving lane are separated, so that the image features are focused on the actual driving road surface, as follows:

[0036] (1) Based on the images segmented by the horizon detection network, small-resolution image blocks are cropped from the main area of ​​the vehicle's driving lane. According to the position of the horizon and the shape of the road surface, four image blocks of size [250 × 200] are intercepted from each image, focusing on the area where the wheels are driving, while ensuring that the image has a sufficient receptive field;

[0037] (2) When the driving lane area is too small to crop an image block of a specified size, the cropping size can be adjusted.

[0038] Furthermore, a classification network is built, and the classification network is used to extract and classify the image blocks, and the prediction results of each image block are fused at the decision level to obtain the final road surface recognition result, as follows:

[0039] (1) Build a classification network. The classification network uses the EfficientNetV2 model. A parallel CBAM module is added to the classification network to improve the classification accuracy. The four image blocks are input into the classification network respectively to obtain the prediction probabilities of the four image blocks: P c , P r , P l , P u ;

[0040] (2) Soft voting probability fusion:

[0041] The soft voting method in ensemble learning is used to perform weighted averaging of the prediction probabilities obtained from all image patches, expressed as:

[0042]

[0043] Where P c , P r , P l , P u denote the predicted probabilities obtained from the center image block, the right image block, the left image block, and the upper image block, respectively. n denotes the total number of image blocks. P dry , P wet , P snow They represent the final prediction probabilities of dry asphalt road, wet asphalt road, and snowy or icy road, respectively. The category with the highest probability is selected as the final prediction result.

[0044] According to a second aspect of the present invention, the present invention provides a nighttime road surface recognition system based on improved lighting conditions, which is used to implement the above-mentioned nighttime road surface recognition method based on improved lighting conditions, comprising:

[0045] The illumination enhancement module is used to receive real-time image data and use the illumination enhancement algorithm to restore night images with low lighting conditions and uneven illumination distribution to obtain improved clear night images;

[0046] A separation module is used to build a horizon detection network, input the improved night image into the horizon detection network to perform horizon segmentation, separate environmental information and road surface information, and obtain image data without environmental information;

[0047] A focusing module, used to divide the image blocks according to the road surface area in the image data after the vehicle and the self-driving lane are separated, so that the image features are focused on the actual driving road surface;

[0048] The fusion output module is used to build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road recognition result.

[0049] According to a third aspect of the present invention, the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the above-mentioned method for night road surface recognition based on improved lighting conditions is adopted.

[0050] According to a fourth aspect of the present invention, the present invention provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the above-mentioned method for nighttime road surface recognition based on improved lighting conditions.

[0051] The present invention has at least the following beneficial effects:

[0052] 1. High recognition accuracy: The present invention utilizes a light enhancement algorithm to enhance low- and medium-intensity pixels in the image, while avoiding an extreme increase in high-intensity pixels, restoring image quality and thereby improving nighttime road recognition accuracy.

[0053] 2. High robustness: The present invention integrates multiple results and fuses the prediction results of multiple regions in the image at the decision level, which can reduce variance and improve the robustness of perception.

[0054] 3. High engineering application value: The present invention expands road surface recognition from single daytime lighting conditions to complex nighttime lighting conditions, which is more in line with actual driving scenarios and has higher engineering application value.

[0055] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the process of the method of the present invention;

[0057] Figure 2 A schematic diagram of the structure of the method of the present invention

[0058] Figure 3 is a schematic diagram of the structure of a horizon detection network in an embodiment of the present invention;

[0059] Figure 4 Schematic diagram of the parallel CBAM attention mechanism structure in the horizon detection network according to an embodiment of the present invention;

[0060] Figure 5 This is a schematic diagram of the structure of the ECA attention mechanism in the horizon detection network according to an embodiment of the present invention;

[0061] Figure 6 A schematic diagram of regional image block extraction in an embodiment of the present invention;

[0062] Figure 7 Schematic diagram of the structure of the image block input classification network in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0064] Embodiment 1:

[0065] See also Figure 1-Figure 7 The present invention provides a technical solution: a nighttime road surface recognition method based on improved lighting conditions, comprising the following steps:

[0066] S1. Receive real-time image data and use the light enhancement algorithm to restore the night image under low lighting conditions and uneven light distribution to obtain an improved clear night image, as follows:

[0067] (S11) Processing the input image with a logarithmic scaling function for estimating the transformation implemented by the retina of the human visual system:

[0068]

[0069] Where I0 is the input RGB image, and I1 is the image obtained by the logarithmic scaling function;

[0070] (S12) The image I0 is processed by an exponential function to adjust the local contrast and reduce the high brightness part of the input image:

[0071] I2=1-exp(-I0) (2)

[0072] Where I2 is the image obtained by the exponential function;

[0073] (S13) Introducing parameter λ into the LIP model, the generated I1 and I2 images are combined using the adjusted LIP model. The LIP model used is as follows:

[0074]

[0075] Where λ is a scalar that controls the enhancement process, and λ satisfies 2≤λ≤7. Through equations (1) to (3), the output image I3 has the characteristics of both image I1 and image I2.

[0076] (S14) The overall brightness of the image I3 is enhanced by using the modified cumulative distribution function of the hyperbolic secant distribution, so as to correctly display the potential details in the image. The modified cumulative distribution function of the hyperbolic secant distribution is expressed as follows:

[0077] I4=erf(λ+arctan(exp(I3))-0.(4)

[0078] Where I4 is the output image obtained by the cumulative distribution function equation of the modified hyperbolic secant distribution, erf is the error function used to increase the brightness of the dark area;

[0079] (S15) Apply a normalization function to linearly scale the pixel value, the normalization function is expressed as follows:

[0080]

[0081] Where I5 ​​is the final output result of the illumination enhancement algorithm;

[0082] It should be noted that the real-time image data is obtained by shooting with the vehicle-mounted camera;

[0083] S2. Build a horizon detection network, input the improved night image into the horizon detection network for horizon segmentation, separate environmental information and road information, and obtain image data without environmental information, as follows:

[0084] (S21) Figure 3 As shown in the figure, the horizon detection network uses ResNet-34 as the backbone network, and adds two lightweight attention mechanisms in parallel, CBAM and ECA, to the horizon detection network structure;

[0085] (S22) The image is horizontally divided into different small areas, and a lightweight attention mechanism is used to perform parallel CBAM and ECA to enhance the detection network feature extraction and separate the environmental information and road surface information;

[0086] Furthermore, the parallel CBAM contains two independent sub-modules: the channel attention module (CAM) and the spatial attention module (SAM), which are used to perform channel and spatial attention respectively. Here, the two sub-modules are changed from serial structure to parallel structure, combined with Figure 4 , expressed as:

[0087]

[0088] M CBAM (F) = σ(M c (F)+M s (F)) (8)

[0089] Where M c is the channel attention feature, M s is the spatial attention feature, F is the input image feature, and is the weight of MLP, C is the number of channels of the input image, r is the scaling factor, and f 7×7 represents the convolution operation with a 7×7 convolution kernel size, σ represents the Sigmoid function, and M CBAM is the final parallel CBAM attention feature;

[0090] like Figure 5 As shown, ECA is expressed as:

[0091]

[0092] In the formula, F is the input image feature, f k×k represents a one-dimensional convolutional layer with a convolution kernel size of k, C is the number of channels of the input feature, Υ and b are hyperparameters, and the absolute value is taken and rounded down to the nearest odd number to ensure that the size of the convolution kernel is an odd number, M ECA is the final ECA attention feature;

[0093] S3. Divide the image blocks according to the road surface area in the image data after the vehicle and the self-driving lane are separated, so that the image features are focused on the actual driving road surface, such as Figure 6 As shown, the details are as follows:

[0094] (S31) based on the image segmented by the horizon detection network, a small-resolution image block is cropped on the main area of ​​the lane where the vehicle is traveling, and four image blocks of size [250×200] are intercepted from each image according to the position of the horizon and the shape of the road surface area, focusing on the area where the wheels are traveling, while ensuring that the image has a sufficient receptive field;

[0095] (S32) When the driving lane area is too small to crop an image block of a specified size, the cropping size may be adjusted;

[0096] S4. Build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road recognition result, such as Figure 7 As shown, the details are as follows:

[0097] (S41) Build a classification network. The classification network uses the EfficientNetV2 model. A parallel CBAM module is added to the classification network to improve the classification accuracy. The four image blocks are input into the classification network respectively to obtain the prediction probabilities of the four image blocks: P c , P r , P l , P u ;

[0098] (S42) Soft voting probability fusion:

[0099] The soft voting method in ensemble learning is used to perform weighted averaging of the prediction probabilities obtained from all image patches, expressed as:

[0100]

[0101] Where P c , P r , P l , P u Respectively represent the predicted probabilities obtained from the center image block, the right image block, the left image block, and the upper image block, n represents the total number of image blocks,

[0102] P dry , P wet , P snow They represent the final prediction probabilities of dry asphalt road, wet asphalt road, and snowy or icy road, respectively. The category with the highest probability is selected as the final prediction result.

[0103] In summary, this embodiment is based on the convolutional neural networks ResNet-34 and EfficientNetV2, uses night-time visible light images as network input, and uses a light enhancement algorithm to restore acceptable image quality from night-time images, thereby improving the accuracy of night-time recognition. At the same time, it takes environmental information interference into consideration, extracts and focuses on road surface information, and fuses the prediction results of multiple image areas to reduce prediction variance and improve the robustness of perception. It has broad application prospects in the field of intelligent connected vehicle perception.

[0104] Embodiment 2:

[0105] This embodiment provides a nighttime road surface recognition system based on improved lighting conditions, which is used to implement a nighttime road surface recognition method based on improved lighting conditions described in Embodiment 1, including:

[0106] The illumination enhancement module is used to receive real-time image data and use the illumination enhancement algorithm to restore night images with low lighting conditions and uneven illumination distribution to obtain improved clear night images;

[0107] A separation module is used to build a horizon detection network, input the improved night image into the horizon detection network to perform horizon segmentation, separate environmental information and road surface information, and obtain image data without environmental information;

[0108] A focusing module, used to divide the image blocks according to the road surface area in the image data after the vehicle and the self-driving lane are separated, so that the image features are focused on the actual driving road surface;

[0109] The fusion output module is used to build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road recognition result.

[0110] Specifically, the above-mentioned illumination enhancement module, separation module, focusing module and fusion output module can be embedded in a computer processing system. The computer calls the above-mentioned modules to complete the task of accurately identifying the night road surface according to the above-mentioned method for night road surface recognition based on improved lighting conditions; the above-mentioned illumination enhancement module, separation module, focusing module and fusion output module can perform operations according to the specific steps given in the above-mentioned method for night road surface recognition based on improved lighting conditions.

[0111] It should be noted that it should be understood that the division of the various modules of the above system is only the division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated, and these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the light enhancement module can be a separately established processing element, or it can be integrated in a chip of the above-mentioned device. In addition, it can also be stored in the memory of the above-mentioned device in the form of program code, and called and executed by a processing element of the above-mentioned device. The functions of the above signal processing module, and the implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or software instructions.

[0112] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0113] Embodiment three:

[0114] The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the above-mentioned method for nighttime road surface recognition based on improved lighting conditions is adopted.

[0115] It should be noted that the terminal device can be a computer device such as a desktop computer, a laptop computer or a cloud server, and the terminal device includes but is not limited to a processor and a memory. For example, the terminal device can also include input and output devices, network access devices and buses.

[0116] Furthermore, the processor may adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. may also be adopted. The general-purpose processor may adopt a microprocessor or any conventional processor, etc., and the present application does not impose any restrictions on this.

[0117] Embodiment 4:

[0118] The present invention provides a storage medium containing computer executable instructions, and the computer executable instructions are used to execute the above-mentioned nighttime road surface recognition method based on improved lighting conditions when executed by a computer processor.

[0119] Among them, the computer program can be stored in a computer-readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or certain middleware, etc. The computer-readable medium includes any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer-readable medium includes but is not limited to the above-mentioned components.

[0120] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0121] For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. When an element is referred to as being "assembled on", "installed on", "fixed on" or "set on" another element, it can be directly on the other element or there can also be a centered element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there can be a centered element at the same time. The terms "vertical", "horizontal", "up", "down", "left", "right" and similar expressions used herein are only for illustrative purposes and are not intended to be the only implementation method.

[0122] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0123] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

Claims

1. A nighttime road surface recognition method based on improved lighting conditions, characterized in that: The following steps are involved: Receive real-time image data and use the light enhancement algorithm to restore night images with low lighting conditions and uneven light distribution to obtain improved clear night images; Build a horizon detection network, input the improved nighttime image into the horizon detection network for horizon segmentation, separate environmental information and road surface information, and obtain image data without environmental information; Divide the image blocks according to the road surface area in the image data after the vehicle and the ego vehicle drive lane are separated, so that the image features are focused on the actual driving road surface; Build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road recognition result.

2. The method for nighttime road surface recognition based on improved lighting conditions according to claim 1, characterized in that: The illumination enhancement algorithm is used to restore night images with low lighting conditions and uneven illumination distribution to obtain improved clear night images, as follows: (1) Process the input image with a logarithmic scaling function to estimate the transformation achieved by the retina of the human visual system: Where I0 is the input RGB image, and I1 is the image obtained by the logarithmic scaling function; (2) Processing image I0 through an exponential function to adjust the local contrast and reduce the high brightness part of the input image: I2=1-exp(-I0) (2) Where I2 is the image obtained by the exponential function; (3) The parameter λ is introduced into the LIP model, and the generated I1 and I2 images are combined using the adjusted LIP model. The LIP model used is as follows: Where λ is a scalar that controls the enhancement process, and λ satisfies 2≤λ≤7. Through equations (1) to (3), the output image I3 has the characteristics of both image I1 and image I2. (4) The overall brightness of the image I3 is enhanced by using the modified hyperbolic secant distribution cumulative distribution function, so that the potential details in the image are correctly displayed. The modified hyperbolic secant distribution cumulative distribution function is expressed as follows: I4=erf(λ+arctan(exp(I3))-0.(4) Where I4 is the output image obtained by the cumulative distribution function equation of the modified hyperbolic secant distribution, erf is the error function used to increase the brightness of the dark area; (5) Apply a normalization function to linearly scale the pixel values. The normalization function is expressed as follows: Where I5 ​​is the final output result of the illumination enhancement algorithm.

3. The nighttime road surface recognition method based on improved lighting conditions according to claim 2, characterized in that: Build a horizon detection network, input the improved night image into the horizon detection network for horizon segmentation, separate environmental information and road information, and obtain image data without environmental information, as follows: (1) The horizon detection network uses ResNet-34 as the backbone network, and adds two lightweight attention mechanisms, CBAM and ECA, to the horizon detection network structure in parallel; (2) The image is horizontally divided into different small areas, and a lightweight attention mechanism is used to perform CBAM and ECA in parallel to enhance the detection network feature extraction and separate environmental information and road surface information.

4. The method for nighttime road surface recognition based on improved lighting conditions according to claim 3, characterized in that: The lightweight attention mechanism parallel CBAM includes two independent sub-modules: channel attention module CAM and spatial attention module SAM, which are used to perform channel and spatial attention respectively, and change the two sub-modules from serial structure to parallel structure, which is specifically expressed as: M CBAM (F)= σ(M c (F)+ M s (F)) (8) Where M c is the channel attention feature, M s is the spatial attention feature, F is the input image feature, and is the weight of MLP, C is the number of channels of the input image, r is the scaling factor, and f 7×7 represents the convolution operation with a 7×7 convolution kernel size, σ represents the Sigmoid function, and M CBAM is the final parallel CBAM attention feature.

5. The method for nighttime road surface recognition based on improved lighting conditions according to claim 3, characterized in that: The lightweight attention mechanism ECA is expressed as: In the formula, F is the input image feature, f k×k represents a one-dimensional convolutional layer with a convolution kernel size of k, C is the number of channels of the input feature, Υ and b are hyperparameters, and the absolute value is taken and rounded down to the nearest odd number to ensure that the size of the convolution kernel is an odd number, M ECA is the final ECA attention feature.

6. The method for nighttime road surface recognition based on improved lighting conditions according to claim 5, characterized in that: The image blocks are divided according to the road surface area in the image data after the vehicle and the ego vehicle driving lane are separated, so that the image features are focused on the actual driving road surface, as follows: (1) Based on the images segmented by the horizon detection network, small-resolution image blocks are cropped from the main area of ​​the vehicle's driving lane. According to the position of the horizon and the shape of the road surface, four image blocks of size [250 × 200] are intercepted from each image, focusing on the area where the wheels are driving, while ensuring that the image has a sufficient receptive field; (2) When the driving lane area is too small to crop an image block of a specified size, the cropping size can be adjusted.

7. The method for nighttime road surface recognition based on improved lighting conditions according to claim 6, characterized in that: Build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road recognition result, as follows: (1) Build a classification network. The classification network uses the EfficientNetV2 model. A parallel CBAM module is added to the classification network to improve the classification accuracy. The four image blocks are input into the classification network respectively to obtain the prediction probabilities of the four image blocks: P c , P r , P l , P u ; (2) Soft voting probability fusion: The soft voting method in ensemble learning is used to perform weighted averaging of the prediction probabilities obtained from all image patches, expressed as: Where P c , P r , P l , P u denote the predicted probabilities obtained from the center image block, the right image block, the left image block, and the upper image block, respectively. n denotes the total number of image blocks. P dry , P wet , P snow They represent the final prediction probabilities of dry asphalt road, wet asphalt road, and snowy or icy road, respectively. The category with the highest probability is selected as the final prediction result.

8. A nighttime road surface recognition system based on improved lighting conditions, used to implement a nighttime road surface recognition method based on improved lighting conditions as claimed in any one of claims 1 to 7, characterized in that: include: The illumination enhancement module is used to receive real-time image data and use the illumination enhancement algorithm to restore night images with low lighting conditions and uneven illumination distribution to obtain improved clear night images; A separation module is used to build a horizon detection network, input the improved night image into the horizon detection network to perform horizon segmentation, separate environmental information and road surface information, and obtain image data without environmental information; A focusing module, used to divide the image blocks according to the road surface area in the image data after the vehicle and the self-vehicle driving lane are separated, so that the image features are focused on the actual driving road surface; The fusion output module is used to build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road recognition result.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, a nighttime road surface recognition method based on improved lighting conditions as described in any one of claims 1 to 7 is adopted.

10. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute a nighttime road surface recognition method based on improved lighting conditions as described in any one of claims 1 to 7.

Images