A lane line recognition method under strong light and weak light background based on YOLO v9

Abstract

The application relates to the technical field of image recognition, and discloses a lane line recognition method in a strong light and weak light background based on YOLO v9. In the automatic driving process of a vehicle, first to-be-recognized image data and illumination intensity data at the same moment are used to perform scene judgment on the first to-be-recognized image data, strong light suppression processing is performed on the first to-be-recognized image data belonging to a strong light scene, weak light enhancement processing is performed on the first to-be-recognized image data belonging to a weak light scene, and second to-be-recognized image data is obtained; according to the illumination intensity data, a pre-trained lane line recognition model is used to perform lane line recognition on the second to-be-recognized image data, and a lane line recognition result is obtained; the lane line recognition model is constructed by using an improved YOLO v9 model, and the neck network of the improved YOLO v9 model is integrated with a channel attention mechanism and a spatial attention mechanism. The method disclosed by the application effectively improves the accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a lane line recognition method in strong light and weak light backgrounds based on YOLO v9. BACKGROUND

[0002] Lane line recognition is one of the core functions in automatic driving and advanced driving assistance systems (ADAS), and the accuracy of the recognition result is directly related to the safety, reliability of driving and stability of the automatic driving system. In the actual driving environment, lane line detection faces complex and variable lighting challenges. When the vehicle is in a strong light environment, it is easy to cause image overexposure and lane line detail loss. When the vehicle is in a weak light environment, the image signal-to-noise ratio obtained by the camera is low, and the lane line has poor contrast with the background. The complex lighting conditions result in poor quality of the collected image data to be recognized, and the existing preprocessing methods have obvious shortcomings. In the existing preprocessing methods, whether it is a strong light condition or a weak light condition, it relies on the fixed image enhancement strategy of histogram equalization processing, resulting in poor preprocessing effect of the image data to be recognized, which affects the accuracy of lane line recognition.

[0003] In addition, the lane line recognition model based on the YOLO v9 model has certain advantages in improving the accuracy of lane line recognition, but in the face of the structure characteristics of long and continuous lane lines, the existing YOLO v9 model cannot fully utilize multi-scale features, and it is difficult to capture global structure information and local detail information of lane lines at the same time, which significantly reduces the recognition accuracy of the model under complex lighting conditions.

[0004] Therefore, how to improve the accuracy of lane line recognition has become a technical problem to be solved by those skilled in the art. SUMMARY

[0005] The present application provides a lane line recognition method, system, device and medium in strong light and weak light backgrounds based on YOLO v9 to solve the technical problem of how to improve the accuracy of lane line recognition, and achieve the effect of improving the accuracy of lane line recognition.

[0006] In a first aspect, the present application provides a lane line recognition method in strong light and weak light backgrounds based on YOLO v9, which comprises:

[0007] In the process of vehicle automatic driving, first image data to be recognized and light intensity data at the same time are collected, and scene judgment is performed on the first image data to be recognized according to the light intensity data, to obtain the scene to which the first image data to be recognized belongs, the scene at least including a strong light scene and a weak light scene;

[0008] According to the scene to which the first to-be-recognized image data belongs, a corresponding preprocessing method is determined to preprocess the first to-be-recognized image data, to obtain second to-be-recognized image data, wherein the preprocessing is configured to perform strong light suppression processing on the first to-be-recognized image data belonging to the strong light scene, and to perform weak light enhancement processing on the first to-be-recognized image data belonging to the weak light scene;

[0009] According to the illumination intensity data, a pre-trained lane line recognition model is used to perform lane line recognition on the second to-be-recognized image data, to obtain a lane line recognition result of the first to-be-recognized image data, wherein the lane line recognition model is constructed by using an improved YOLO v9 model, a neck network of the improved YOLO v9 model is configured to perform channel attention mechanism processing and spatial attention mechanism processing on combined feature data composed of illumination feature data and image feature data, respectively, to obtain channel weights and spatial weights, respectively, and the illumination feature data and the image feature data are weighted and fused based on the channel weights and the spatial weights, to obtain a fusion feature map, the fusion feature map is used to input a detection head, the illumination feature data is extracted from the illumination intensity data, and the image feature data is extracted from the second to-be-recognized image data.

[0010] Preferably, the scene to which the first to-be-recognized image data belongs is determined according to the illumination intensity data, and the scene to which the first to-be-recognized image data belongs is determined according to the illumination intensity data.

[0011] According to the illumination intensity data, an illumination intensity calibration threshold is set, wherein the illumination intensity calibration threshold includes a highest illumination intensity calibration threshold and a lowest illumination intensity calibration threshold.

[0012] The first to-be-recognized image data is subjected to brightness statistical analysis processing, to obtain a brightness mean value and a brightness variance.

[0013] If the brightness mean value is greater than or equal to the highest illumination intensity calibration threshold, and the brightness variance is greater than or equal to a pre-set brightness variance calibration threshold, the scene to which the first to-be-recognized image data belongs is determined as a strong light scene.

[0014] If the brightness mean value is less than the lowest illumination intensity calibration threshold, and the brightness variance is less than the brightness variance calibration threshold, the scene to which the first to-be-recognized image data belongs is determined as a weak light scene.

[0015] Preferably, the corresponding preprocessing method is determined according to the scene to which the first to-be-recognized image data belongs, to preprocess the first to-be-recognized image data, to obtain second to-be-recognized image data, and the corresponding preprocessing method includes:

[0016] If the first to be identified image data belongs to the strong light scene, a modified limited contrast adaptive histogram equalization algorithm is used to perform strong light suppression processing on the first to be identified image data, to obtain strong light suppressed image data;

[0017] If the first to be identified image data belongs to the weak light scene, a modified Retinex network model is used to perform weak light enhancement processing on the first to be identified image data, to obtain weak light enhanced image data.

[0018] The strong light suppressed image data or the weak light enhanced image data is determined as second to be identified image data.

[0019] Preferably, the strong light suppression processing on the first to be identified image data by the modified limited contrast adaptive histogram equalization algorithm to obtain strong light suppressed image data comprises:

[0020] According to the brightness variance of the first to be identified image data, adaptive region division is performed on the first to be identified image data, to obtain a plurality of sub-image data;

[0021] According to the brightness mean value of the first to be identified image data, an adaptive local contrast limit threshold is set, and each sub-image data is subjected to histogram equalization processing according to the adaptive local contrast limit threshold, to obtain strong light suppressed image data.

[0022] Preferably, the setting formula of the adaptive local contrast limit threshold is:

[0023] Clip Limit = 2.0 + 0.1 x (I avg -128)

[0024] Wherein, Clip Limit represents the adaptive local contrast limit threshold, I avg represents the brightness mean value of the first to be identified image data.

[0025] Preferably, the weak light enhancement processing on the first to be identified image data by the modified Retinex network model to obtain weak light enhanced image data comprises:

[0026] The first to be identified image data is decomposed to obtain an illumination layer and a reflection layer;

[0027] The reflection layer is subjected to adaptive gamma correction and fused with the illumination layer, to obtain weak light enhanced image data.

[0028] Preferably, the combined feature data consisting of the illumination feature data and the image feature data is respectively subjected to channel attention mechanism processing and spatial attention mechanism processing, respectively corresponding to obtain channel weights and spatial weights, comprising:

[0029] The illumination feature data is mapped to a high-dimensional vector space to obtain illumination high-dimensional feature data;

[0030] The spatial alignment algorithm is adopted to perform spatial alignment on the illumination high-dimensional feature data and the image feature data to obtain combined feature data;

[0031] The combined feature data is subjected to channel attention mechanism processing to obtain channel weights;

[0032] The combined feature data is subjected to spatial attention mechanism processing to obtain spatial weights.

[0033] In a second aspect, the present application also provides a lane line recognition system under strong light and weak light background based on YOLO v9, which realizes the lane line recognition method under strong light and weak light background based on YOLO v9 described above, and the system comprises a scene confirmation unit, a preprocessing unit and a lane line recognition unit.

[0034] The scene confirmation unit is used to collect first to-be-recognized image data and illumination intensity data at the same time in the process of automatic driving of a vehicle, and according to the illumination intensity data, to perform scene judgment on the first to-be-recognized image data to obtain the scene to which the first to-be-recognized image data belongs, wherein the scene at least includes a strong light scene and a weak light scene.

[0035] The preprocessing unit is used to determine a corresponding preprocessing method according to the scene to which it belongs, so as to preprocess the first to-be-recognized image data to obtain second to-be-recognized image data, wherein the preprocessing is set to perform strong light suppression processing on the first to-be-recognized image data belonging to the strong light scene, and to perform weak light enhancement processing on the first to-be-recognized image data belonging to the weak light scene.

[0036] The lane line identification unit is configured for identifying lane lines of the second to-be-identified image data according to the light intensity data by using a pre-trained lane line identification model, to obtain a lane line identification result of the first to-be-identified image data, wherein the lane line identification model is constructed by using an improved YOLO v9 model, the neck network of the improved YOLO v9 model is configured to perform channel attention mechanism processing and spatial attention mechanism processing on combined feature data composed of light feature data and image feature data, respectively, to obtain channel weights and spatial weights, respectively, and perform weighted fusion on the light feature data and the image feature data based on the channel weights and the spatial weights, to obtain a fusion feature map, the fusion feature map is used for inputting a detection head, the light feature data is extracted from the light intensity data, and the image feature data is extracted from the second to-be-identified image data.

[0037] In a third aspect, the present application further provides a computer device, which comprises a memory, a processor and a transceiver connected through a bus; the memory is used for storing a set of computer program instructions and data, and transmitting the stored data to the processor; the processor executes the program instructions stored in the memory to perform the lane line identification method in strong light and weak light backgrounds based on YOLOv9.

[0038] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and when the computer program is executed, the lane line identification method in strong light and weak light backgrounds based on YOLO v9 is realized.

[0039] The lane line identification method in strong light and weak light backgrounds based on YOLO v9 provided by the present application has the following beneficial effects compared with the prior art:

[0040] The lane line identification method in strong light and weak light backgrounds based on YOLO v9 disclosed by the present application can optimize the image quality under various complex light conditions by judging the scene to which the first to-be-identified image data belongs, performing strong light suppression processing on the first to-be-identified image data if it belongs to a strong light scene, and performing weak light enhancement processing on the first to-be-identified image data if it belongs to a weak light scene, thereby improving the accuracy of lane line identification; the channel attention mechanism and the spatial attention mechanism jointly act on the processing process of the feature map, so that the lane line identification model can automatically select the optimal image feature data according to the light feature data under multiple scales, appropriately enhance the image feature data under different light conditions, realize dynamic light perception adjustment, effectively fuse and optimize the image features from different scales under multiple scales, and effectively improve the accuracy of target detection. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a YOLO v9-based strong light and weak light background lane line recognition method step schematic diagram provided by one preferred embodiment of the present application;

[0042] Figure 2 is a structure schematic diagram of a YOLO v9-based strong light and weak light background lane line recognition system provided by one preferred embodiment of the present application;

[0043] Figure 3 is an internal structure diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The embodiments of the present application will be described in detail below with reference to the drawings. The embodiments are given only for illustrative purposes and cannot be understood as limiting the present application. The accompanying drawings are used for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. In the description of the present application, the terms "first", "second", "third", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features limited by "first", "second", "third", etc. can explicitly or implicitly include one or more features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0045] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or the communication inside two elements. The terms "vertical", "horizontal", "left", "right", "up", "down" and similar expressions used herein are only for illustrative purposes, and cannot be understood as indicating or implying that the devices or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. The term "and / or" used herein includes any and all combinations of one or more related listed items. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art.

[0046] In the description of the present application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present application have the same meaning as commonly understood by those skilled in the art. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. For those skilled in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0047] Please refer to Figure 1 In an embodiment of the present application, a YOLO v9-based lane line recognition method in strong light and weak light background is provided, which comprises:

[0048] S1, in the process of automatic driving of the vehicle, the first to be identified image data and the light intensity data at the same time are collected, and the scene judgment is made on the first to be identified image data according to the light intensity data, so as to obtain the scene to which the first to be identified image data belongs, which at least includes strong light scene and weak light scene; in the preferred embodiment of the present application, the first to be identified image data and the light intensity data of the environment where the vehicle is located are collected at the same time, and the first to be identified image data needs to be further preprocessed before it can be used for lane line recognition. In the preferred embodiment of the present application, the scene to which the first to be identified image data belongs needs to be judged before the image is preprocessed, and the corresponding preprocessing method is selected according to the scene judgment result. Among them, the scene at least includes strong light scene, weak light scene and balanced light scene, and different preprocessing methods need to be taken for different scenes.

[0049] In the preferred embodiment of the present application, the specific implementation of the scene judgment on the first to be identified image data according to the light intensity data is that the light intensity calibration threshold is set according to the light intensity data, the light intensity calibration threshold includes the highest light intensity calibration threshold and the lowest light intensity calibration threshold, in the preferred embodiment of the present application, the highest light intensity calibration threshold is:

[0050] T high = 0.7L sensor

[0051] Among them, T high represents the highest light intensity calibration threshold, L sensor represents the light intensity.

[0052] The lowest light intensity calibration threshold is:

[0053] T low = 0.3L sensor

[0054] Among them, T low represents the lowest light intensity calibration threshold.

[0055] The first to be identified image data is subjected to brightness statistical analysis processing to obtain the brightness mean and brightness variance of the first to be identified image data. Further, the brightness variance threshold is set to 50, and the first to be identified image data is subjected to scene judgment according to the light intensity threshold and the brightness variance threshold. If the brightness mean of the first to be identified image data is greater than or equal to the highest light intensity threshold, and the brightness variance is greater than or equal to the pre-set brightness variance threshold, the scene to which the first to be identified image data belongs is determined as a strong light scene. If the brightness mean of the first to be identified image data is less than the lowest light intensity threshold, and the brightness variance is less than the brightness variance threshold, the scene to which the first to be identified image data belongs is determined as a weak light scene.

[0056] For the first to be identified image data that neither belongs to the strong light scene nor belongs to the weak light scene, the scene to which the first to be identified image data belongs is determined as a balanced light scene. Correspondingly, the first to be identified image data belonging to the balanced light scene does not need to be pre-processed and can be directly used for lane line identification.

[0057] In the preferred embodiments of the present application, the first to be identified image data is subjected to dynamic scene division based on the light intensity data of the environment in which the vehicle is located, to obtain an accurate scene division result. The first to be identified image data can be further subjected to a targeted pre-processing method according to the scene division result, to improve the pre-processing effect and obtain high-quality to-be-identified image data, thereby effectively improving the accuracy of subsequent lane line identification.

[0058] S2, a corresponding pre-processing method is determined according to the scene to which the first to be identified image data belongs, to pre-process the first to be identified image data and obtain second to be identified image data. The pre-processing is set to perform strong light suppression processing on the first to be identified image data belonging to the strong light scene, and perform weak light enhancement processing on the first to be identified image data belonging to the weak light scene. In the preferred embodiments of the present application, a targeted pre-processing method is selected for different scenes. For the first to be identified image data belonging to the strong light scene, an improved limited contrast self-adaptive histogram equalization algorithm is used for strong light suppression processing, to obtain strong light suppressed image data. For the first to be identified image data belonging to the weak light scene, an improved Retinex network model is used for weak light enhancement processing, to obtain weak light enhanced image data.

[0059] The limited contrast adaptive histogram equalization algorithm is a technique for image enhancement. The traditional histogram equalization algorithm is a global histogram equalization adjustment of the entire image, which enhances the contrast by mapping the image gray value distribution to a more uniform range. However, this method may over amplify the noise in the image and has limited effect on local contrast enhancement. The limited contrast adaptive histogram equalization algorithm divides the image into many small blocks and performs histogram equalization on each small block to better adapt to the local contrast changes of the image, avoiding the problems of over-enhancement or detail loss caused by global equalization. At the same time, in order to prevent noise amplification caused by excessive contrast enhancement, a contrast limit threshold is set in advance. When the frequency of a certain gray level exceeds the contrast limit threshold, it is clipped, and then the excess part is evenly distributed to other gray levels before histogram equalization operation.

[0060] In the preferred embodiment of the present application, a dynamic block strategy is proposed. First, the global variance of the brightness channel of the first to-be-identified image data is calculated to obtain the brightness variance. Further, the block size is determined, and the block size is used as the window size, and the sliding is performed with a step size of 1 / 2 of the block size to generate a plurality of overlapping sub-image data. The edge area that cannot be completely covered is filled with mirror image to ensure that all pixels are covered. The brightness variance of the first to-be-identified image data is used to adaptively divide the first to-be-identified image data to obtain a plurality of sub-image data, and the division of the high brightness area is more fine, avoiding the edge artifacts caused by fixed block. 2 The block size is dynamically adjusted, and the block size is used as the window size, and the sliding is performed with a step size of 1 / 2 of the block size to generate a plurality of overlapping sub-image data. The edge area that cannot be completely covered is filled with mirror image to ensure that all pixels are covered. The brightness variance of the first to-be-identified image data is used to adaptively divide the first to-be-identified image data to obtain a plurality of sub-image data, and the division of the high brightness area is more fine, avoiding the edge artifacts caused by fixed block.

[0061] Further, according to the set adaptive local contrast limit threshold, histogram equalization processing is performed on each sub-image data. In the preferred embodiment of the present application, the setting formula of the adaptive local contrast limit threshold is:

[0062] Clip Limit=2.0+0.1×(I avg -128)

[0063] Wherein, Clip Limit represents the adaptive local contrast limit threshold, I avg represents the brightness mean value of the first to-be-identified image data.

[0064] The sub-image data with a brightness value greater than the adaptive local contrast limit threshold is subjected to histogram equalization processing, and the brightness of the sub-image data after histogram equalization is represented as:

[0065] I output (i,j)=min(I(i,j),220+0.5×(I(i,j)-220))

[0066] wherein, I output (i,j) represents the luminance data of the histogram equalized sub-image data, I(i,j) represents the original luminance data of the sub-image data, and min represents taking the minimum value.

[0067] After obtaining the equalized sub-image data, the inter-block difference is eliminated by bilinear interpolation to obtain the strong light suppressed image data.

[0068] The improved limited contrast adaptive histogram equalization algorithm of the application performs fine processing on the image according to the illumination intensity of different regions, effectively suppresses the overexposed regions in the image while retaining the lane line texture, thereby reducing the visual interference caused by strong light and ensuring that the key structures such as lane lines are clear and visible.

[0069] The Retinex network model is a model for image enhancement and color constancy processing, which is based on the characteristics of human visual system for color perception, and believes that the color of an object is mainly determined by its reflection characteristics, rather than the color and intensity of the illuminating light. The Retinex network model decomposes the formation process of an image into a reflection component and an illumination component, that is, an image can be represented as the product of a reflection component and an illumination component. The reflection component reflects the inherent properties of the object surface and is related to the color and texture of the object, and the illumination component represents the lighting conditions in the scene, which may cause problems such as uneven illumination and color deviation in the image.

[0070] In the preferred embodiment of the application, a depth separable convolution is used to extract the illumination feature and the reflection feature of the first to-be-identified image data, and then a 1x1 convolution is used to integrate the channel information to obtain an illumination layer and a reflection layer, and the number of channels is compressed to 128. The reflection layer is subjected to adaptive gamma value correction, and the adaptive gamma value is corrected by the following formula:

[0071] R enhanced =R 1 / γ

[0072] γ=1+0.5(1-I avg )

[0073] wherein, R enhanced represents the enhanced reflection layer data after gamma correction, R represents the original reflection layer data, and γ represents the adaptive gamma value for controlling the correction intensity.

[0074] In the preferred embodiment of the application, the γ parameter is automatically adjusted according to the average luminance of the first to-be-identified image data, so as to adaptively adjust the gamma value of the reflection layer and optimize the contrast of the reflection layer.

[0075] The reflection layer and the illumination layer after adaptive gamma correction are multiplied pixel by pixel to obtain the weak light enhanced image data.

[0076] The improved Retinex network model can enhance the reflection component, effectively constrain the smoothness of the illumination layer, thereby improving the brightness and contrast of the image, avoiding overfitting of noise, and reducing the computational overhead.

[0077] Further, the image data after strong light suppression or the image data after weak light enhancement is determined as the second to-be-identified image data. If the first to-be-identified image data belongs to a strong light scene, the image data after strong light suppression is determined as the second to-be-identified image data. If the first to-be-identified image data belongs to a weak light scene, the image data after weak light enhancement is determined as the second to-be-identified image data. If the first to-be-identified image data belongs to a balanced illumination scene, the first to-be-identified image data can be directly determined as the second to-be-identified image data.

[0078] In the preferred embodiment of the present application, by judging the scene to which the first to-be-identified image data belongs and selecting the corresponding image preprocessing method according to the scene judgment result, the image quality can be optimized in various complex illumination conditions, and the accuracy of lane line recognition is improved.

[0079] S3, according to the illumination intensity data, using a pre-trained lane line recognition model to perform lane line recognition on the second to-be-identified image data, to obtain a lane line recognition result of the first to-be-identified image data, the lane line recognition model is constructed by using an improved YOLO v9 model, the neck network of the improved YOLO v9 model is set to respectively perform channel attention mechanism processing and spatial attention mechanism processing on the combined feature data composed of illumination feature data and image feature data, to respectively obtain channel weight and spatial weight, and based on the channel weight and the spatial weight, the illumination feature data and the image feature data are weighted and fused to obtain a fused feature map, the fused feature map is used to input a detection head, the illumination feature data is extracted from the illumination intensity data, and the image feature data is extracted from the second to-be-identified image data; in the preferred embodiment of the present application, the YOLO v9 model is selected to construct the lane line recognition model, the YOLO v9 model includes a backbone network, a neck network and a detection head, the backbone network is used to extract features of different scales from the input illumination intensity data and second to-be-identified image data to obtain feature map data of different scales. The neck network further processes and fuses the feature map data output by the backbone network, and through a series of convolution, up-sampling and down-sampling operations, the feature map data of different scales are fused to provide more rich and representative feature information for the detection head. The detection head classifies and identifies targets based on the fused feature information of the neck network, and finally outputs a lane line detection result.

[0080] In the preferred embodiment of the present application, in order to introduce the perception of illumination intensity data, the neck network of the YOLO v9 model is improved, and a channel attention mechanism and a spatial attention mechanism are introduced in the neck network to dynamically adjust the weight of the illumination-sensitive region in the feature map under different illumination conditions, thereby improving the adaptability of the lane line recognition model to illumination changes and the accuracy of lane line recognition.

[0081] After training the improved YOLO v9 model, a lane line recognition model is obtained. Before inputting the illumination intensity data and the second to-be-recognized image data into the lane line recognition model in the actual vehicle automatic driving process, a learnable fully connected layer is needed to map the illumination feature data to a high-dimensional vector space to obtain illumination high-dimensional feature data, and the dimension is set to be consistent with the number of feature channels of the backbone network, so as to convert the illumination intensity data into high-level feature representation that can be combined with the second to-be-recognized image data, thereby injecting illumination-related information into the second to-be-recognized image data. Further, a spatial broadcasting algorithm is used to expand the illumination high-dimensional feature data into a tensor with the same size as the second to-be-recognized image data, and the illumination high-dimensional feature data is aligned with the second to-be-recognized image data in space to ensure that the illumination features of each pixel point can be effectively combined with the image features at the corresponding position to obtain combined feature data.

[0082] Further, the combined feature data is respectively processed by a channel attention mechanism and a spatial attention mechanism, and channel weights and spatial weights are respectively obtained. In the channel attention mechanism processing process, the spatial aligned combined feature data is sequentially processed by channel-by-channel multiplication processing, global average pooling processing, fully connected layer weight compression processing and function activation processing to obtain channel weights. The channel attention mechanism calculates the response intensity of each channel of the first feature map after channel-by-channel multiplication processing to evaluate which channels are most important for lane line detection under the current illumination condition, and the introduction of illumination features can automatically adjust the weights of each channel, so that the lane line recognition model pays more attention to the feature channels that are more affected by illumination, thereby improving the accuracy of lane line recognition.

[0083] In the spatial attention mechanism processing process, the combined feature data after spatial alignment is sequentially subjected to channel concatenation processing, 3x3 convolution processing and function activation processing to obtain spatial weights. In the spatial attention mechanism processing process, the illumination high-dimensional feature data and the image feature data are finely adjusted in the spatial dimension. By calculating the response importance of each position in the second feature map after channel concatenation processing, the spatial attention mechanism can adjust the weights of different regions of the second to-be-identified image according to the local illumination features, especially in regions with large changes in illumination, such as shadow, overexposure or reflection regions, so that the lane line recognition model can more intelligently pay attention to and enhance the features of the illumination-sensitive regions in the second to-be-identified image, and ensure that the details of these regions will not be lost due to changes in illumination.

[0084] Finally, based on the channel weights and the spatial weights, the illumination feature data and the image feature data are weightedly fused to obtain a fused feature map. The channel weights are taken as the weights of the first feature map after the channel-by-channel multiplication processing, and the spatial weights are taken as the weights of the second feature map after the channel concatenation processing, and then the channel-by-channel addition is performed to weightly fuse the illumination feature data and the image feature data. The fused feature map is input into the detection head to perform lane line recognition and obtain a lane line recognition result.

[0085] In the preferred embodiments of the present application, the channel attention mechanism and the spatial attention mechanism jointly act on the processing process of the feature map, so that the lane line recognition model can automatically select the optimal image feature data according to the illumination feature data in multiple scales, and appropriately enhance the image feature data under different illumination conditions, realize dynamic illumination perception adjustment, so that the improved neck network can effectively fuse and optimize the image features from different scales in multiple scales, provide more accurate and robust feature representation for subsequent detection tasks, and effectively improve the accuracy of target detection.

[0086] In the preferred embodiment of the present application, during automatic driving of the vehicle, first to-be-recognized image data and illumination intensity data at the same time are collected, scene judgment is performed on the first to-be-recognized image data according to the illumination intensity data, a scene to which the first to-be-recognized image data belongs is obtained, and the scene at least includes a strong light scene and a weak light scene; a corresponding preprocessing method is determined according to the scene to which the first to-be-recognized image data belongs, so as to preprocess the first to-be-recognized image data, and second to-be-recognized image data is obtained, the preprocessing is set to be strong light suppression processing on the first to-be-recognized image data belonging to the strong light scene, and weak light enhancement processing on the first to-be-recognized image data belonging to the weak light scene; a lane line recognition model pre-trained is used to perform lane line recognition on the second to-be-recognized image data according to the illumination intensity data, and a lane line recognition result of the first to-be-recognized image data is obtained, the lane line recognition model is constructed by using an improved YOLO v9 model, a neck network of the improved YOLO v9 model is set to respectively perform channel attention mechanism processing and spatial attention mechanism processing on combined feature data composed of illumination feature data and image feature data, respectively to obtain channel weight and spatial weight, and the illumination feature data and the image feature data are weighted and fused based on the channel weight and the spatial weight, to obtain a fusion feature map, the fusion feature map is used to input a detection head, the illumination feature data is extracted from the illumination intensity data, and the image feature data is extracted from the second to-be-recognized image data. The lane line recognition method under strong light and weak light background based on YOLO v9 disclosed in the present application can optimize the image quality under various complex illumination conditions and improve the accuracy of lane line recognition by judging the scene to which the first to-be-recognized image data belongs, performing strong light suppression processing on the first to-be-recognized image data belonging to the strong light scene, and performing weak light enhancement processing on the first to-be-recognized image data belonging to the weak light scene; the channel attention mechanism and the spatial attention mechanism jointly act on the processing process of the feature map, so that the lane line recognition model can automatically select the optimal image feature data according to the illumination feature data under multiple scales, appropriately enhance the image feature data under different illumination conditions, realize dynamic illumination perception adjustment, effectively fuse and optimize the image features from different scales under multiple scales, and effectively improve the accuracy of target detection.

[0087] Correspondingly, as shown in Figure 2 According to the lane line recognition method under strong light and weak light background based on YOLO v9, the embodiment of the present application also provides a lane line recognition system under strong light and weak light background based on YOLO v9, which realizes the lane line recognition method under strong light and weak light background based on YOLO v9 disclosed in the embodiment of the present application, and the system includes a scene confirmation unit, a preprocessing unit and a lane line recognition unit.

[0088] The scene confirmation unit is configured to collect first to-be-identified image data and illumination intensity data at the same moment during automatic driving of the vehicle, and perform scene judgment on the first to-be-identified image data according to the illumination intensity data to obtain a scene to which the first to-be-identified image data belongs, wherein the scene at least includes a strong light scene and a weak light scene.

[0089] The preprocessing unit is configured to determine a corresponding preprocessing method according to the scene to which the first to-be-identified image data belongs, to preprocess the first to-be-identified image data to obtain second to-be-identified image data, wherein the preprocessing is configured to perform strong light suppression processing on the first to-be-identified image data belonging to the strong light scene, and perform weak light enhancement processing on the first to-be-identified image data belonging to the weak light scene.

[0090] The lane line identification unit is configured to perform lane line identification on the second to-be-identified image data according to the illumination intensity data, by using a pre-trained lane line identification model to obtain a lane line identification result of the first to-be-identified image data, wherein the lane line identification model is constructed by using an improved YOLO v9 model, a neck network of the improved YOLO v9 model is configured to perform channel attention mechanism processing and spatial attention mechanism processing on combined feature data composed of illumination feature data and image feature data respectively, to obtain channel weight and spatial weight respectively, and perform weighted fusion on the illumination feature data and the image feature data based on the channel weight and the spatial weight to obtain a fusion feature map, wherein the fusion feature map is used for inputting a detection head, the illumination feature data is extracted from the illumination intensity data, and the image feature data is extracted from the second to-be-identified image data.

[0091] The specific limitations of the lane line identification system under strong light and weak light backgrounds based on YOLO v9 can refer to the limitations of the lane line identification method under strong light and weak light backgrounds based on YOLO v9 described above, which will not be repeated here. Those skilled in the art can realize that the various modules and steps described in combination with the embodiments disclosed in the present application can be realized by hardware, software or a combination of both. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0092] As Figure 3As shown, the computer device provided by the embodiment of the present application comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the steps in the above-mentioned YOLO v9-based lane line identification method embodiment in strong light and weak light backgrounds, for example Figure 1 S1-S3 in the above-mentioned embodiment.

[0093] Those skilled in the art can understand that the above-mentioned Figure 3 is only an example of the computer device and does not constitute a limitation on the computer device, which can include more or fewer components than the diagram, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0094] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and connects all parts of the computer device through various interfaces and lines.

[0095] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer device by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0096] Wherein, the module integrated by the computer device, if realized in the form of software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0097] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. The program can include the processes of each method embodiment when executed.

[0098] Correspondingly, the embodiment of the present application provides a computer readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer readable storage medium is located to execute the steps in the above-mentioned embodiment of a lane line identification method in strong light and weak light background based on YOLO v9, such as the steps S1-S3 in the above-mentioned embodiment. Figure 1

[0099] ​​In summary, the embodiment of the present application provides a lane line recognition method, system, device and medium based on YOLO v9 in strong light and weak light background, which solves the technical problem of how to improve the accuracy of lane line recognition. The method comprises: in the process of automatic driving of a vehicle, collecting first to-be-recognized image data and illumination intensity data at the same time, and performing scene judgment on the first to-be-recognized image data according to the illumination intensity data to obtain the scene to which the first to-be-recognized image data belongs, the scene at least including a strong light scene and a weak light scene; determining a corresponding preprocessing method according to the scene to which the first to-be-recognized image data belongs to preprocess the first to-be-recognized image data to obtain second to-be-recognized image data, the preprocessing being set to perform strong light suppression processing on the first to-be-recognized image data belonging to the strong light scene and weak light enhancement processing on the first to-be-recognized image data belonging to the weak light scene; performing lane line recognition on the second to-be-recognized image data by using a pre-trained lane line recognition model according to the illumination intensity data to obtain a lane line recognition result of the first to-be-recognized image data, the lane line recognition model being constructed by using an improved YOLO v9 model, the neck network of the improved YOLO v9 model being set to perform channel attention mechanism processing and spatial attention mechanism processing on combined feature data composed of illumination feature data and image feature data respectively to correspondingly obtain channel weights and spatial weights, and the illumination feature data and the image feature data are weighted and fused based on the channel weights and the spatial weights to obtain a fusion feature map, the fusion feature map being used for inputting a detection head, the illumination feature data being extracted from the illumination intensity data, and the image feature data being extracted from the second to-be-recognized image data. The lane line recognition method based on YOLO v9 in strong light and weak light background disclosed in the present application can optimize the image quality under various complex illumination conditions and improve the accuracy of lane line recognition by judging the scene to which the first to-be-recognized image data belongs, performing strong light suppression processing on the first to-be-recognized image data if it belongs to a strong light scene, and performing weak light enhancement processing on the first to-be-recognized image data if it belongs to a weak light scene; the channel attention mechanism and the spatial attention mechanism jointly act on the processing process of the feature map, so that the lane line recognition model can automatically select the optimal image feature data according to the illumination feature data under multiple scales, appropriately enhance the image feature data under different illumination conditions, realize dynamic illumination perception adjustment, effectively fuse and optimize the image features from different scales under multiple scales, and effectively improve the accuracy of target detection.

[0100] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0101] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the technical principles of this application, and these improvements and substitutions should also be considered within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the scope of the claims.

Claims

1. A YOLO v9-based lane line recognition method in strong light and weak light backgrounds, characterized by, The method comprises: In the automatic driving process of the vehicle, first to-be-recognized image data and illumination intensity data at the same time are collected, and scene judgment is performed on the first to-be-recognized image data according to the illumination intensity data, so that the scene to which the first to-be-recognized image data belongs is obtained, wherein the scene at least comprises a strong light scene and a weak light scene; According to the scene to which the first to-be-recognized image data belongs, a corresponding preprocessing method is determined to preprocess the first to-be-recognized image data, so that second to-be-recognized image data is obtained, wherein the preprocessing is set to perform strong light suppression processing on the first to-be-recognized image data belonging to the strong light scene, and perform weak light enhancement processing on the first to-be-recognized image data belonging to the weak light scene; According to the illumination intensity data, a pre-trained lane line recognition model is used to perform lane line recognition on the second to-be-recognized image data, so that a lane line recognition result of the first to-be-recognized image data is obtained, wherein the lane line recognition model is constructed by using an improved YOLO v9 model, a neck network of the improved YOLO v9 model is set to perform channel attention mechanism processing and spatial attention mechanism processing on combined feature data composed of illumination feature data and image feature data respectively, so that channel weight and spatial weight are obtained respectively, and the illumination feature data and the image feature data are weighted and fused based on the channel weight and the spatial weight, so that a fusion feature map is obtained, the fusion feature map is used to input a detection head, the illumination feature data is extracted from the illumination intensity data, and the image feature data is extracted from the second to-be-recognized image data.

2. The YOLO v9-based lane line recognition method in strong light and weak light backgrounds according to claim 1, characterized in that, The scene judgment performed on the first to-be-recognized image data according to the illumination intensity data comprises: An illumination intensity calibration threshold is set according to the illumination intensity data, wherein the illumination intensity calibration threshold comprises a highest illumination intensity calibration threshold and a lowest illumination intensity calibration threshold; Brightness statistical analysis processing is performed on the first to-be-recognized image data, so that a brightness mean value and a brightness variance are obtained; If the brightness mean value is greater than or equal to the highest illumination intensity calibration threshold, and the brightness variance is greater than or equal to a pre-set brightness variance calibration threshold, then the scene to which the first to-be-recognized image data belongs is determined as a strong light scene; If the brightness mean value is less than the lowest illumination intensity calibration threshold, and the brightness variance is less than the brightness variance calibration threshold, then the scene to which the first to-be-recognized image data belongs is determined as a weak light scene.

3. The YOLO v9-based lane line recognition method in strong light and weak light backgrounds of claim 2, wherein, The determination of the corresponding preprocessing method according to the scene to which the first to-be-recognized image data belongs comprises: If the first to-be-recognized image data belongs to the strong light scene, then an improved limited contrast self-adaptive histogram equalization algorithm is used to perform strong light suppression processing on the first to-be-recognized image data, so that strong light suppression image data is obtained; If the first to-be-recognized image data belongs to the weak light scene, then an improved Retinex network model is used to perform weak light enhancement processing on the first to-be-recognized image data, so that weak light enhancement image data is obtained; The strong light suppressed image data or the weak light enhanced image data is determined as second to-be-recognized image data.

4. The YOLO v9-based lane line recognition method in strong light and weak light backgrounds of claim 3, wherein, The improved limited contrast self-adaptive histogram equalization algorithm is used to perform strong light suppression processing on the first to-be-recognized image data, to obtain strong light suppressed image data, including: According to the brightness variance of the first to-be-recognized image data, the first to-be-recognized image data is adaptively divided into regions, to obtain a plurality of sub-image data; According to the brightness mean value of the first to-be-recognized image data, an adaptive local contrast limit threshold is set, and each sub-image data is subjected to histogram equalization processing according to the adaptive local contrast limit threshold, to obtain strong light suppressed image data.

5. The YOLO v9-based lane line recognition method in strong light and weak light backgrounds of claim 4, wherein, The setting formula of the adaptive local contrast limit threshold is: Clip Limit = 2.0 + 0.1 x (I avg -128) wherein Clip Limit represents an adaptive local contrast limit threshold, I avg represents the average brightness of the first to-be-identified image data.

6. The YOLO v9-based lane line recognition method in strong light and weak light backgrounds of claim 3, wherein, The improved Retinex network model is used to perform weak light enhancement processing on the first to-be-recognized image data, to obtain weak light enhanced image data, including: The first to-be-recognized image data is decomposed to obtain an illumination layer and a reflection layer; After adaptive gamma correction of the reflection layer, the reflection layer is fused with the illumination layer to obtain weak light enhanced image data.

7. The YOLO v9-based lane line recognition method in strong light and weak light backgrounds of claim 1, wherein, The combined feature data composed of illumination feature data and image feature data is subjected to channel attention mechanism processing and spatial attention mechanism processing respectively, to obtain channel weight and spatial weight respectively, including: The illumination feature data is mapped to a high-dimensional vector space to obtain illumination high-dimensional feature data; A spatial broadcasting algorithm is used to perform spatial alignment on the illumination high-dimensional feature data and the image feature data to obtain combined feature data; The combined feature data is subjected to channel attention mechanism processing to obtain channel weight; The combined feature data is subjected to spatial attention mechanism processing to obtain spatial weight.

8. A YOLO v9-based lane line recognition system in strong light and weak light backgrounds, used to implement the YOLO v9-based lane line recognition method in strong light and weak light backgrounds of any one of claims 1-7, characterized in that, The system comprises a scene confirmation unit, a preprocessing unit and a lane line recognition unit. The scene confirmation unit is configured to, in the process of automatic driving of a vehicle, collect first to-be-recognized image data and illumination intensity data at the same time, and perform scene judgment on the first to-be-recognized image data according to the illumination intensity data, to obtain a scene to which the first to-be-recognized image data belongs, the scene at least including a strong light scene and a weak light scene. The preprocessing unit is configured to determine a corresponding preprocessing method according to the scene to which the first to-be-recognized image data belongs, to preprocess the first to-be-recognized image data, to obtain second to-be-recognized image data, the preprocessing being set to perform strong light suppression processing on the first to-be-recognized image data belonging to the strong light scene, and to perform weak light enhancement processing on the first to-be-recognized image data belonging to the weak light scene. The lane line identification unit is configured to perform lane line identification on the second to-be-identified image data according to the light intensity data by using a pre-trained lane line identification model to obtain a lane line identification result of the first to-be-identified image data, wherein the lane line identification model is constructed by using an improved YOLO v9 model, and a neck network of the improved YOLO v9 model is configured to perform channel attention mechanism processing and spatial attention mechanism processing on combined feature data composed of light feature data and image feature data respectively to obtain channel weight and spatial weight respectively, and perform weighted fusion on the light feature data and the image feature data based on the channel weight and the spatial weight to obtain a fusion feature map, the fusion feature map is used for inputting a detection head, the light feature data is extracted from the light intensity data, and the image feature data is extracted from the second to-be-identified image data.

9. A computer device, comprising: The computer device comprises a memory, a processor and a transceiver which are connected through a bus; the memory is used for storing a set of computer program instructions and data, and transmitting the stored data to the processor; the processor executes the program instructions stored in the memory to execute the lane line identification method in strong light and weak light backgrounds based on YOLO v9 according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, when the computer program is executed, a lane line identification method in strong light and weak light backgrounds based on YOLO v9 according to any one of claims 1 to 7 is realized.

Images