A night road surface recognition method, system and device based on improved illumination conditions

Abstract

The application discloses a night road surface recognition method based on improved illumination conditions, and relates to the technical field of intelligent sensing.The application comprises the following steps: receiving real-time image data, using an illumination enhancement algorithm to recover images with low night lighting and uneven illumination distribution;building a horizon detection network, inputting the improved images into the horizon detection network to separate environmental information and road surface information, and obtaining image data without environmental information; dividing the road surface area in the image into image blocks according to the driving lane of the vehicle, so that the image features are focused on the actual driving area; building a classification network to extract features and classify the image blocks, and performing decision-level fusion on the prediction results of the image blocks to obtain the final road surface recognition result.The application uses the illumination enhancement algorithm to recover the image quality, improves the night road surface recognition accuracy, and performs decision-level fusion on the prediction results of multiple areas in the image, so that the variance is reduced and the robustness of perception is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent sensing technology, in particular to a night road surface recognition method, system and device based on improved illumination conditions. BACKGROUND

[0002] The accuracy of road surface recognition and detection has gradually become an important factor affecting driving safety.

[0003] At present, various sensors are deployed on intelligent vehicles, among which cameras are widely used due to their low price and ease of use. However, most existing road detection technologies are based on daytime lighting conditions, and have poor robustness when dealing with complex and harsh weather conditions such as rainy days, snowy days and nights. Although there is usually car light for illumination in the night driving scene, the road surface area in the night image is usually too dark in the absence of environmental light, which can easily cause misrecognition. Therefore, we propose a night road surface recognition method, system and device based on improved illumination conditions. SUMMARY

[0004] The purpose of the present application is to provide a night road surface recognition method, system and device based on improved illumination conditions, which can improve night images with low brightness and insufficient contrast, eliminate the interference of environmental information, and effectively realize the recognition of common road surface states under complex lighting conditions.

[0005] According to the first aspect of the present application, in order to achieve the above-mentioned purpose, the present application provides the following technical scheme: a night road surface recognition method based on improved illumination conditions, comprising the following steps:

[0006] Receiving real-time image data, using an illumination enhancement algorithm to restore night images under low lighting conditions and uneven illumination distribution, obtaining improved clear night images;

[0007] Building a horizon detection network, inputting the improved night images into the horizon detection network for horizon segmentation, separating environmental information and road surface information, and obtaining image data without environmental information;

[0008] According to the vehicle's driving lane, the road surface area in the separated image data is divided into image blocks, so that the image features are focused on the actual driving road surface;

[0009] Building a classification network, using the classification network to extract and classify image blocks, and performing decision-level fusion on the prediction results of each image block to obtain the final road surface recognition result.

[0010] Further, using the illumination enhancement algorithm, the night images under low lighting conditions and uneven illumination distribution are restored to obtain improved clear night images, which are as follows:

[0011] (1) Processing the input image with a logarithmic scaling function for estimating the conversion realized by the retina of the human visual system:

[0012]

[0013] where I0 is the input RGB image, and I1 is the image obtained by the logarithmic scaling function;

[0014] (2) Adjusting the local contrast and weakening the highlight part of the input image by processing the image I0 with an exponential function:

[0015] I2 = 1 - exp(-I0) (2)

[0016] where I2 is the image obtained by the exponential function;

[0017] (3) Introducing a parameter λ in the LIP model, and using the adjusted LIP model to combine the generated I1 and I2 images, the LIP model used is as follows:

[0018]

[0019] where λ is a scalar that controls the enhancement process, and λ satisfies 2≤λ≤7, through equations (1) to (3), the output image I3 has the characteristics of both image I1 and image I2;

[0020] (4) Enhancing the overall brightness of the image I3 using the cumulative distribution function of the modified hyperbolic secant distribution, which is used to correctly display the potential details in the image, and the cumulative distribution function of the modified hyperbolic secant distribution is represented as follows:

[0021] I4 = erf(λ + arctan(exp(I3)) - 0. (4)

[0022] where I4 is the output image obtained by the cumulative distribution function equation of the modified hyperbolic secant distribution, and erf is the error function used to increase the brightness of the dark area;

[0023] (5) Applying a normalization function to linearly scale the pixel values, and the normalization function is represented as follows:

[0024]

[0025] where I5 is the final output result of the illumination enhancement algorithm.

[0026] Further, the improved night image is input into the horizon detection network for horizon segmentation, separation of environmental information and road surface information, and obtaining image data without environmental information, as follows:

[0027] (1) Horizon detection network uses ResNet-34 as the backbone network, and adds two kinds of lightweight attention mechanisms CBAM and ECA in the horizon detection network structure;

[0028] (2) The picture is horizontally divided into different small areas, and the lightweight attention mechanisms CBAM and ECA are used to enhance the feature extraction of the detection network, separate the environmental information and the road surface information.

[0029] Further, the lightweight attention mechanism CBAM includes two independent sub-modules: a channel attention module CAM and a spatial attention module SAM, which are used to perform channel and spatial attention respectively, and the two sub-modules are changed from a serial structure to a parallel structure, which is specifically represented as:

[0030]

[0031] M CBAM (F)=σ(M c (F)+M s (F)) (8)

[0032] In the formula, M c is the channel attention feature, M s is the spatial attention feature, F is the input image feature, and are the weights of the MLP, C is the channel number of the input image, r is the scaling factor, f 7×7 represents the convolution operation with a 7x7 convolution kernel size, σ represents the Sigmoid function, M CBAM is the final parallel CBAM attention feature. Further, the lightweight attention mechanism ECA is represented as:

[0033]

[0034] In the formula, F is the input image feature, f k×k represents a one-dimensional convolution layer with a convolution kernel size of k, C is the channel number of the input feature, Y and b are hyperparameters, and taking the absolute value and rounding down to the nearest odd number is to ensure that the size of the convolution kernel is odd, M ECA is the final ECA attention feature.

[0035] Further, according to the vehicle self-vehicle driving lane, the image blocks in the separated image data are divided, so that the image features focus on the actual driving road surface, which is specifically as follows:

[0036] (1) According to the horizon detection network, the image after segmentation is cropped to obtain small resolution image blocks in the main area of the driving lane of the ego vehicle, four image blocks with a size of [250x200] are cut from each image according to the position of the horizon and the shape of the road surface area, focusing on the area where the wheels drive, while ensuring that the image has sufficient receptive field;

[0037] (2) When the driving lane area is too small to crop the specified size image block, the size of the cropped image can be adjusted.

[0038] Further, a classification network is built to extract features and classify the image blocks, and the prediction results of each image block are fused at the decision level to obtain the final road surface recognition result, as follows:

[0039] (1) Build a classification network, the classification network uses an EfficientNetV2 model, and a parallel CBAM module is added to the classification network to improve classification accuracy. Four image blocks are input into the classification network to obtain four image block prediction probabilities: P c , P r , P l , and P u .

[0040] (2) Soft voting probability fusion:

[0041] The soft voting method in ensemble learning is used to weight and average the prediction probabilities obtained from all image blocks, which is expressed as:

[0042]

[0043] In the formula, P c , P r , P l , and P u represent the prediction probabilities obtained from the center image block, the right image block, the left image block, and the upper image block, respectively, n represents the total number of image blocks, P dry , P wet , and P snow represent the final prediction probabilities of dry asphalt road surface, wet and slippery asphalt road surface, and snow or icy road surface, respectively, and the class with the highest probability is selected as the final prediction result.

[0044] According to the second aspect of the present application, the present application provides a night road surface recognition system based on improved illumination conditions, which is used to implement the night road surface recognition method based on improved illumination conditions as described above, comprising:

[0045] The illumination enhancement module is used for receiving real-time image data, recovering night images under low illumination conditions and uneven illumination distribution by using an illumination enhancement algorithm, and obtaining improved clear night images.

[0046] The separation module is used for building a horizon detection network, inputting the improved night images into the horizon detection network for horizon segmentation, separating environmental information and road surface information, and obtaining image data without environmental information.

[0047] The focusing module is used for dividing the road surface area in the separated image data into image blocks according to the driving lane of the vehicle, and focusing image features on the actual driving road surface.

[0048] The fusion output module is used for building a classification network, extracting and classifying image blocks by using the classification network, and obtaining a final road surface recognition result by decision-level fusion of the prediction results of the image blocks.

[0049] According to a third aspect of the present application, the present application provides a terminal device comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the above-mentioned night road surface recognition method based on improved illumination conditions is used.

[0050] According to a fourth aspect of the present application, the present application provides a storage medium containing computer executable instructions for executing the above-mentioned night road surface recognition method based on improved illumination conditions when executed by a computer processor.

[0051] The present application has at least the following advantages:

[0052] 1. High recognition accuracy: The present application uses an illumination enhancement algorithm to enhance low and medium intensity pixels in the image, while avoiding excessive increase of high intensity pixels, thereby improving the quality of the image and the accuracy of night road surface recognition.

[0053] 2. High robustness: The present application integrates multiple results and performs decision-level fusion of the prediction results of multiple regions in the image, thereby reducing variance and improving the robustness of perception.

[0054] 3. High engineering application value: The present application extends road surface recognition from single daytime illumination conditions to complex night illumination conditions, which is more consistent with actual driving scenarios and has higher engineering application value.

[0055] Of course, any product implementing the present application does not necessarily need to achieve all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 Flowchart of the method of the present application

[0057] Figure 2 Structural schematic diagram of the method of the present application

[0058] Figure 3 Structural schematic diagram of the horizon detection network in the embodiment of the present application

[0059] Figure 4 Structural schematic diagram of the parallel CBAM attention mechanism in the horizon detection network in the embodiment of the present application

[0060] Figure 5 Structural schematic diagram of the ECA attention mechanism in the horizon detection network in the embodiment of the present application

[0061] Figure 6 Structural schematic diagram of the region image block extraction in the embodiment of the present application

[0062] Figure 7 Structural schematic diagram of the image block input classification network in the embodiment of the present application DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present disclosure.

[0064] Embodiment one

[0065] Please refer to Figures 1-7 The present application provides a technical solution: a night road recognition method based on improved illumination conditions, comprising the following steps:

[0066] S1. Receive real-time image data, use an illumination enhancement algorithm to restore night images under low lighting conditions and uneven illumination distribution, and obtain improved clear night images, as follows:

[0067] (S11) Process the input image with a logarithmic scaling function, which is used to estimate the conversion realized by the retina of the human visual system:

[0068]

[0069] In the formula, I0 is the input RGB image, and I1 is the image obtained by the logarithmic scaling function;

[0070] (S12) Adjusting local contrast and weakening highlight parts of the input image by processing the image I0 through an exponential function:

[0071] I2 = 1 - exp(-I0) (2)

[0072] In the formula, I2 is the image obtained by the exponential function;

[0073] (S13) Introducing a parameter λ in the LIP model, combining the generated I1 and I2 images using the adjusted LIP model, and using the LIP model as follows:

[0074]

[0075] In the formula, λ is a scalar that controls the enhancement process, and λ satisfies 2≤λ≤7, and through the formula (1) to formula (3), the output image I3 has the characteristics of both image I1 and image I2;

[0076] (S14) Enhancing the overall brightness of the image I3 using the cumulative distribution function of the modified hyperbolic secant distribution, which is used to correctly display the potential details in the image, and the cumulative distribution function of the modified hyperbolic secant distribution is represented as follows:

[0077] I4 = erf(λ + arctan(exp(I3)) - 0. (4)

[0078] In the formula, I4 is the output image obtained by the cumulative distribution function equation of the modified hyperbolic secant distribution, and erf is the error function used to increase the brightness of the dark area;

[0079] (S15) Applying a normalization function to linearly scale the pixel value, and the normalization function is represented as follows:

[0080]

[0081] In the formula, I5 is the final output result of the illumination enhancement algorithm;

[0082] It should be noted that the real-time image data is obtained by shooting through a vehicle-mounted camera;

[0083] S2. Building a horizon detection network, inputting the improved night image into the horizon detection network for horizon segmentation, separating environmental information and road surface information, and obtaining image data without environmental information, as follows:

[0084] (S21) As shown in Figure 3 , the horizon detection network uses ResNet-34 as the backbone network, and adds two kinds of lightweight attention mechanisms in parallel CBAM and ECA in the horizon detection network structure;

[0085] (S22) The picture is horizontally divided into different small areas, and the parallel CBAM and ECA enhanced detection network feature extraction is adopted using a lightweight attention mechanism to separate environmental information and road surface information;

[0086] Further, the parallel CBAM contains two independent sub-modules: a channel attention module (CAM) and a spatial attention module (SAM) for performing channel and spatial attention, respectively. Here, the two sub-modules are changed from a serial structure to a parallel structure, combined with Figure 4 , which is expressed as:

[0087]

[0088] M CBAM (F)=σ(M c (F)+M s (F)) (8)

[0089] In the formula, M c is the channel attention feature, M s is the spatial attention feature, F is the input image feature, and are the weights of the MLP, C is the number of channels of the input image, r is the scaling factor, f 7×7 represents a convolution operation with a 7×7 convolution kernel size, σ represents the Sigmoid function, M CBAM is the final parallel CBAM attention feature;

[0090] As shown in Figure 5 , ECA is expressed as:

[0091]

[0092] In the formula, F is the input image feature, f k×k represents a one-dimensional convolution layer with a convolution kernel size of k, C is the number of channels of the input feature, Y and b are hyperparameters, and taking the absolute value and rounding down to the nearest odd number is to ensure that the size of the convolution kernel is odd, M ECA is the final ECA attention feature;

[0093] S3. According to the vehicle self-driving lane, the road surface area in the separated image data is divided into image blocks, and the image features are focused on the actual driving road surface, as shown in Figure 6 , and the specific process is as follows:

[0094] (S31) According to the image segmented by the horizon detection network, a small resolution image block is cropped on the main area of the driving lane of the ego vehicle, four image blocks with a size of [250x200] are cut from each image according to the position of the horizon and the shape of the road surface area, focusing on the area where the wheels drive, while ensuring that the image has enough receptive field;

[0095] (S32) When the driving lane area is too small to crop a specified size image block, the size of the cropped image can be adjusted;

[0096] S4. Build a classification network, use the classification network to extract features and classify the image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road surface recognition result, as shown in Figure 7 , specifically as follows:

[0097] (S41) Build a classification network, the classification network uses an EfficientNetV2 model, and a parallel CBAM module is added to the classification network to improve classification accuracy. Four image blocks are input into the classification network to obtain four image block prediction probabilities: P c , P r , P l , and P u .

[0098] (S42) Soft voting probability fusion:

[0099] The soft voting method in ensemble learning is used to weight and average the prediction probabilities obtained from all image blocks, which can be expressed as:

[0100]

[0101] In the formula, P c , P r , P l , and P u represent the prediction probabilities obtained from the center image block, the right image block, the left image block, and the upper image block, respectively, n represents the total number of image blocks,

[0102] P dry , P wet , and P snow represent the final prediction probabilities of dry asphalt pavement, wet and slippery asphalt pavement, and snow or icy pavement, respectively, and the class with the highest probability is selected as the final prediction result.

[0103] In summary, the embodiment is based on convolutional neural network ResNet-34 and EfficientNetV2, uses night visible light images as network input; uses illumination enhancement algorithm to restore acceptable image quality from night images, improves the accuracy of night recognition; at the same time, considering the environmental information interference, extracting and focusing on the road surface information, and fusing the prediction results of multiple image regions, reducing the prediction variance, improving the robustness of perception, and having wide application prospect in intelligent networked vehicle perception field.

[0104] Embodiment two:

[0105] The embodiment provides a night road surface recognition system based on improved illumination conditions, which is used to realize the night road surface recognition method based on improved illumination conditions described in embodiment one, comprising:

[0106] The illumination enhancement module is used to receive real-time image data, use an illumination enhancement algorithm to restore night images under low illumination conditions and uneven illumination distribution at night, and obtain improved clear night images;

[0107] The separation module is used to build a horizon detection network, input the improved night images into the horizon detection network for horizon segmentation, separate the environmental information and the road surface information, and obtain image data without environmental information;

[0108] The focusing module is used to divide the road surface area in the separated image data into image blocks according to the driving lane of the ego vehicle, so that the image features are focused on the actual driving road surface;

[0109] The fusion output module is used to build a classification network, use the classification network to extract and classify the image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road surface recognition result

[0110] Specifically, the above-mentioned illumination enhancement module, separation module, focusing module and fusion output module can be embedded into a computer processing system. The computer calls each module to complete the task of accurately recognizing the night road surface according to the above-mentioned night road surface recognition method based on improved illumination conditions. The above-mentioned illumination enhancement module, separation module, focusing module and fusion output module can perform operations according to the specific steps given by the above-mentioned night road surface recognition method based on improved illumination conditions.

[0111] It should be understood that the division of the above system modules is only a logical function division, and all or part of the modules can be integrated into a physical entity or physically separated in actual implementation, and the modules can be all implemented in the form of software calling by a processing element, all implemented in the form of hardware, or part of the modules are implemented in the form of software calling by a processing element and part of the modules are implemented in the form of hardware. For example, the illumination enhancement module can be a separate processing element, or can be integrated in a chip of the above device, and in addition, the illumination enhancement module can be stored in the form of program code in the memory of the above device and called and executed by a processing element of the above device to implement the function of the above signal processing module, and the implementation of other modules is similar. In addition, all or part of the modules can be integrated together or independently implemented, and the processing element described herein can be an integrated circuit having a signal processing capability, and in the implementation process, each step of the above method or each module can be completed by an integrated logic circuit of hardware in the processing element or an instruction in the form of software.

[0112] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of program code calling by a processing element, the processing element can be a general-purpose processor such as a central processing unit (CPU) or other processor capable of calling program code. For another example, the modules can be integrated together to implement in the form of a system on a chip (SOC).

[0113] Embodiment three:

[0114] The application provides a terminal device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores a computer program capable of running on the processor. When the processor loads and executes the computer program, the above-mentioned night road surface identification method based on illumination condition improvement is adopted.

[0115] It should be noted that the terminal device can adopt a computer device such as a desktop computer, a notebook computer or a cloud server, and the terminal device includes but is not limited to a processor and a memory, for example, the terminal device can also include an input / output device, a network access device and a bus, etc.

[0116] Further, the processor can adopt a central processing unit (CPU), of course, according to the actual use case, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. can also be adopted, the general-purpose processor can adopt a microprocessor or any conventional processor, etc. The present application does not make any limitation in this regard.

[0117] Embodiment four:

[0118] The application provides a storage medium containing computer executable instructions, which are used for executing the above-mentioned night road surface identification method based on illumination condition improvement when executed by a computer processor.

[0119] Wherein, the computer program can be stored in the computer readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or some middleware form, etc., the computer readable medium includes any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code, it should be noted that the computer readable medium includes but is not limited to the above components.

[0120] It should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device.

[0121] Those of ordinary skill in the art, with the benefit of this description, can appreciate the disclosure in conjunction with its specific features, embodiments, and examples. The above-described features and specific embodiments are to be considered illustrative of specific ways to make and use the disclosure, and such to provide a concrete basis to enabling the patent. Various modifications, both as to the specific nature of the features and embodiments and methods of their implementation, can be apparent from the description. The scope of the disclosure is defined by the appended claims and their equivalents.

[0122] While the embodiments of the present disclosure have been illustrated and described, it will be understood by those skilled in the art that various changes, modifications, alternatives, and variations can be made therein without departing from the spirit and scope of the present disclosure, which is defined by the appended claims and their equivalents.

[0123] In the description of the specification, reference to "one embodiment", "an example", "a specific example", and so on, means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the disclosure. The appearances of the above-described terms in various places in the specification are not necessarily referring to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A nighttime road surface recognition method based on improved lighting conditions, characterized in that: The following steps are involved: Receive real-time image data and use light enhancement algorithms to restore night images in low-light conditions and with uneven light distribution, obtaining improved, clear night images. Build a horizon detection network and input the improved nighttime image into the horizon detection network for horizon segmentation, separating environmental information from road surface information to obtain image data without environmental information. Divide the image blocks according to the road surface area in the image data after the vehicle and the ego vehicle are separated by lanes, so that the image features are focused on the actual driving road surface; Build a classification network, use it to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road surface recognition result; The illumination enhancement algorithm is used to restore night images with low lighting conditions and uneven illumination distribution, and obtain improved clear night images, as follows: (1) Process the input image with a logarithmic scaling function to estimate the transformation implemented by the retina of the human visual system: Where, is the input RGB image, The image obtained by the logarithmic scaling function; (2) Processing images through exponential functions To adjust the local contrast and reduce the bright highlights of the input image: Where, The image obtained for the exponential function; (3) Introducing parameters into the LIP model , using the adjusted LIP model to generate and For image combination, the LIP model used is as follows: Where, To control the scalar of the enhancement process, satisfy , through equations (1) to (3), the output image Make it both an image and images characteristics; (4) Enhance the image using the cumulative distribution function of the modified hyperbolic secant distribution The overall brightness is used to correctly display the potential details in the image. The cumulative distribution function of the modified hyperbolic secant distribution is expressed as follows: Where, It is the output image obtained by the cumulative distribution function equation of the modified hyperbolic secant distribution, is the error function, used to increase the brightness of the dark area; (5) Apply the normalization function to linearly scale the pixel values. The normalization function is expressed as follows: Where, It is the final output result of the illumination enhancement algorithm.

2. The method for nighttime road surface recognition based on improved lighting conditions according to claim 1, characterized in that: Build a horizon detection network and input the improved nighttime image into the horizon detection network for horizon segmentation, separating environmental information and road surface information to obtain image data without environmental information. The details are as follows: (1) The horizon detection network uses ResNet-34 as the backbone network, and adds two lightweight attention mechanisms, CBAM and ECA, to the horizon detection network structure in parallel; (2) The image is horizontally divided into different small areas, and a lightweight attention mechanism is used to perform parallel CBAM and ECA to enhance the detection network feature extraction and separate environmental information and road surface information.

3. The method for nighttime road surface recognition based on improved lighting conditions according to claim 2, characterized in that: The lightweight attention mechanism parallel CBAM consists of two independent sub-modules: the channel attention module CAM and the spatial attention module SAM, which are used to perform channel and spatial attention respectively. The two sub-modules are changed from a serial structure to a parallel structure, which is specifically expressed as follows: Where, is the channel attention feature, is the spatial attention feature, is the input image feature, and is the weight of MLP, is the number of channels of the input image, is the scaling factor, Indicates the use of Convolution operation with kernel size, represents the Sigmoid function, is the final parallel CBAM attention feature.

4. The method for nighttime road surface recognition based on improved lighting conditions according to claim 3, characterized in that: The lightweight attention mechanism ECA is expressed as: Where, is the input image feature, Represents a convolution kernel with a size of One-dimensional convolutional layer, is the number of channels of the input feature, and Is a hyperparameter. Taking the absolute value and rounding it down to the nearest odd number is to ensure that the size of the convolution kernel is an odd number. is the final ECA attention feature.

5. The method for nighttime road surface recognition based on improved lighting conditions according to claim 4, characterized in that: The image blocks are divided according to the road surface area in the image data after the vehicle and the ego vehicle are separated into lanes, so that the image features are focused on the actual road surface, as follows: (1) Based on the image segmented by the horizon detection network, a small-resolution image patch is cropped from the main area of ​​the vehicle's driving lane. Based on the position of the horizon and the shape of the road area, four image patches of size [250 × 200] are intercepted from each image, focusing on the area where the wheels are driving, while ensuring that the image has a sufficient receptive field; (2) When the driving lane area is too small to crop an image block of the specified size, the cropping size can be adjusted.

6. The method for nighttime road surface recognition based on improved lighting conditions according to claim 5, characterized in that: Build a classification network, use it to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road surface recognition result, as follows: (1) Build a classification network. The classification network uses the EfficientNetV2 model. A parallel CBAM module is added to the classification network to improve classification accuracy. The four image blocks are input into the classification network respectively to obtain the prediction probabilities of the four image blocks: 、 、 、 ; (2) Soft voting probability fusion: The soft voting method in ensemble learning is used to perform weighted averaging of the prediction probabilities obtained from all image blocks, which can be expressed as: Where, 、 、 、 denote the predicted probabilities obtained from the center image block, the right image block, the left image block, and the upper image block, respectively. represents the total number of image blocks, 、 、 They represent the final predicted probabilities of dry asphalt road, wet asphalt road, and snowy or icy road, respectively. The category with the highest probability is selected as the final prediction result.

7. A nighttime road surface recognition system based on improved lighting conditions, used to implement the nighttime road surface recognition method based on improved lighting conditions according to any one of claims 1 to 6, characterized in that: include: The illumination enhancement module is used to receive real-time image data and use the illumination enhancement algorithm to restore night images with low lighting conditions and uneven illumination distribution to obtain improved clear night images; The separation module is used to build a horizon detection network, input the improved night image into the horizon detection network to perform horizon segmentation, separate environmental information and road surface information, and obtain image data without environmental information; A focusing module is used to divide the image blocks according to the road surface area in the image data after the vehicle and the ego vehicle are separated by the lane, so that the image features are focused on the actual driving road surface; The fusion output module is used to build a classification network, use the classification network to extract features and classify image blocks, and perform decision-level fusion on the prediction results of each image block to obtain the final road surface recognition result.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, the method for nighttime road surface recognition based on improved lighting conditions according to any one of claims 1 to 6 is adopted.

9. A storage medium containing computer-executable instructions, characterized in that: When executed by a computer processor, the computer executable instructions are used to perform a nighttime road surface recognition method based on improved lighting conditions according to any one of claims 1 to 6.

Images