Vehicle-mounted road surface divider detection method and system

Abstract

The application relates to the technical field of computer vision and automatic driving, and discloses a vehicle-mounted road surface separator detection method and system, which comprises the following steps: generating an initial light source direction vector based on vehicle-mounted sensor data at a current time, calibrating the initial light source direction vector by using current vehicle attitude data to obtain a calibrated light source direction vector; calculating an actual height value of a road surface separator according to a shadow geometric feature formed by the road surface separator and extracted from the current image frame collected by a vehicle-mounted camera and the calibrated light source direction vector; and determining the type of the road surface separator by combining the actual height value with a shadow shape feature formed by the road surface separator in the current image frame. The application can solve the problems of low detection accuracy of a vehicle-mounted road surface separator and difficulty in distinguishing similar separators in a weather change and light dynamic change scene, reduces detection cost, and improves detection accuracy and stability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and automatic driving, and particularly relates to a vehicle-mounted road surface separator detection method and system. BACKGROUND

[0002] At present, in the field of automatic driving, it is crucial to accurately measure the height of road surface separators. Traditional road edge detection methods mostly rely on laser radar or stereo vision. However, these technologies are high in cost and are easily affected by weather factors, for example, rain and fog weather will interfere with the normal work of laser radar and reduce the detection accuracy. Although monocular vision methods are low in cost, it is difficult to distinguish between separators with similar heights but different functions, such as low road shoulders and wheel marks, which are easily confused. In addition, in the vehicle-mounted dynamic scene, due to the constant change of the sun angle and the frequent switching of light and dark in the tunnel, the traditional static light assumption is no longer applicable, making it difficult for methods based on this to stably and accurately detect vehicle-mounted road surface separators.

[0003] Therefore, there is an urgent need to provide a technical solution to solve the above problems. SUMMARY

[0004] To solve the above technical problems, the present application provides a vehicle-mounted road surface separator detection method and system.

[0005] In a first aspect, the present application provides a vehicle-mounted road surface separator detection method, and the technical scheme of the method is as follows:

[0006] An initial light source direction vector is dynamically generated based on vehicle-mounted sensor data at the current time, and the initial light source direction vector is calibrated using current vehicle attitude data to obtain a calibrated light source direction vector; the initial light source direction vector is a natural light initial direction vector or a man-made light initial direction vector; the vehicle-mounted sensor data at the current time includes current geographic position data, a current timestamp, a current image frame and current vehicle attitude data;

[0007] According to the calibrated light source direction vector and the shadow geometric features formed by the road surface separators extracted from the current image frame collected by the vehicle-mounted camera, the actual height value of the road surface separators is calculated; the shadow geometric features include an original shadow length value, an original shadow angle value and a shadow depth value;

[0008] The type of the road surface separator is determined by combining the actual height value with the shadow shape features formed by the road surface separators in the current image frame.

[0009] The vehicle-mounted road surface separator detection method of the present application has the following beneficial effects:

[0010] The method can solve the problems of low detection accuracy of vehicle-mounted road dividers, difficulty in distinguishing similar dividers, and the like in weather changes and dynamic light change scenes, reduces detection cost, and improves detection accuracy and stability.

[0011] Based on the above scheme, the vehicle-mounted road dividers detection method of the application can be further improved as follows.

[0012] In an optional manner, the step of dynamically generating the initial light source direction based on the current vehicle-mounted sensor data comprises:

[0013] According to the brightness distribution and the high-light region spatial distribution characteristics in the current image frame, the dominance of the natural light source and the artificial light source is distinguished, wherein the brightness distribution is global average brightness, the high-light region spatial distribution characteristics include high-light region proportion and spatial dispersion, and the expression for distinguishing the dominance of the natural light source and the artificial light source is:

[0014] ;

[0015] wherein, is the global average brightness, is the high-light region proportion, is the spatial dispersion; denotes the image diagonal length, , denotes the image width, denotes the image height;

[0016] When the natural light source is dominant, based on the current geographic location data and the current time stamp, the solar azimuth angle is calculated by using an astronomical algorithm to generate the initial direction vector of the natural light in the world coordinate system;

[0017] When the artificial light source is dominant, based on the high-light region spatial distribution characteristics, the light direction is fitted by using the Hough transform to generate the initial direction vector of the artificial light in the vehicle-mounted camera coordinate system.

[0018] The above optional manner has the beneficial effects that by distinguishing the dominance of the natural light and the artificial light, the initial light source direction vector is generated by using different data, so that the detection can be dynamically adjusted according to different light conditions, the adaptability and accuracy of the method are enhanced, and the detection effect in a complex light environment is further improved.

[0019] In an optional manner, the step of calibrating the initial light source direction vector by using the current vehicle attitude data to obtain the calibrated light source direction vector comprises:

[0020] When the initial light source direction vector is the natural light initial direction vector, a conversion from a world coordinate system to a vehicle camera coordinate system and a posture compensation operation are performed on the natural light initial direction vector to obtain a calibrated light source direction vector;

[0021] When the initial light source direction vector is the artificial light initial direction vector, a posture compensation operation is performed on the artificial light initial direction vector to obtain a calibrated light source direction vector.

[0022] The rotation matrix constructed based on the current vehicle posture data implements the posture compensation operation.

[0023] The above-mentioned optional mode has the beneficial effect that: through coordinate conversion and posture compensation on the initial light source direction vector, the light source direction is accurately calibrated, ensuring that the height of the partition can still be accurately calculated when the vehicle posture changes, improving the reliability and stability of the detection, and making the result more accurate.

[0024] In an optional mode, the shadow geometric features include: an original shadow length value, an original shadow angle value, and a shadow depth value; and the step of calculating the actual height value of the road partition based on the calibrated light source direction vector and the shadow geometric features of the road partition formed in the current image frame collected by the vehicle camera includes:

[0025] An angle between the calibrated light source direction vector and the road normal vector is determined as a light source incidence angle value.

[0026] The original shadow length value is corrected based on the shadow depth value to obtain a target shadow length value, and the original shadow angle value is corrected based on the intrinsic parameters of the vehicle camera to obtain a target shadow angle value.

[0027] The actual height value is calculated based on the target shadow length value, the target shadow angle value, and the light source incidence angle value.

[0028] The above-mentioned optional mode has the beneficial effect that: by introducing the shadow depth value to correct the original shadow length value, the actual height value is calculated by comprehensively considering multiple factors such as the light source incidence angle, making the height measurement more accurate, and improving the accuracy and reliability of the height detection of the road partition.

[0029] In an optional mode, the shadow morphological features include: a shadow aspect ratio, a shadow edge sharpness value, and shadow continuity; and the step of determining the type of the road partition based on the actual height value and the shadow morphological features of the road partition formed in the current image frame includes:

[0030] A to-be-measured feature vector is constructed based on the shadow aspect ratio, the shadow edge sharpness value, the shadow continuity, and the actual height value.

[0031] The feature vector to be tested is input into the trained classification decision model to obtain the predicted type probability distribution, and the type mapped by the maximum probability value in the predicted type probability distribution is determined as the type of road divider.

[0032] The beneficial effects of the above-mentioned optional methods are as follows: by constructing the feature vector to be tested and inputting it into the classification decision model to determine the type of separator, similar separators can be effectively distinguished, the accuracy and efficiency of classification are improved, and more reliable road information is provided for autonomous driving.

[0033] Secondly, the present invention provides a vehicle-mounted road surface divider detection system, the technical solution of which is as follows:

[0034] It includes: a processing module, a calculation module, and a detection module;

[0035] The processing module is used for: the initial light source direction vector being either the initial direction vector of natural light or the initial direction vector of artificial light; and the vehicle sensor data at the current moment including: current geographical location data, current timestamp, current image frame, and current vehicle attitude data.

[0036] The calculation module is used to: calculate the actual height of the road surface dividers based on the calibration light source direction vector and the shadow geometric features formed by the road surface dividers extracted from the current image frame captured by the vehicle-mounted camera; the shadow geometric features include: the original shadow length value, the original shadow angle value, and the shadow depth value;

[0037] The detection module is used to determine the type of road divider by combining the actual height value with the shadow morphology features formed by the road divider in the current image frame.

[0038] The beneficial effects of the vehicle-mounted road surface divider detection system of the present invention are as follows:

[0039] The system of this invention can solve the problems of low detection accuracy of vehicle road surface dividers and difficulty in distinguishing similar dividers in scenarios of weather changes and dynamic changes in lighting, thereby reducing detection costs and improving detection accuracy and stability.

[0040] Based on the above solution, the vehicle-mounted road surface divider detection system of the present invention can be further improved as follows.

[0041] In one alternative approach, the processing module is specifically used for:

[0042] Based on the brightness distribution and highlight region spatial distribution characteristics in the current image frame, the dominance of natural light sources and artificial light sources is distinguished; whereby the brightness distribution is defined as the global average brightness; and the highlight region spatial distribution characteristics include the highlight region proportion and spatial dispersion; the expression for distinguishing the dominance of natural light sources and artificial light sources is:

[0043] ;

[0044] in, The global average brightness The percentage of the highlight area. Spatial dispersion; Indicates the length of the image diagonal. , Indicates the image width. Indicates the image height;

[0045] When natural light is dominant, the solar azimuth angle is calculated using astronomical algorithms based on the current geographical location data and the current timestamp, in order to generate the initial direction vector of natural light in the world coordinate system;

[0046] When artificial light sources are dominant, the initial direction vector of artificial light in the vehicle camera coordinate system is generated by fitting the light direction through Hough transform based on the spatial distribution characteristics of the highlight area.

[0047] The beneficial effects of the above-mentioned alternative methods are as follows: by distinguishing between the dominance of natural light and artificial light, and using different data to generate the initial light source direction vector, the detection can be dynamically adjusted according to different lighting conditions, which enhances the adaptability and accuracy of the method and further improves the detection effect in complex lighting environments.

[0048] In one alternative approach, the processing module is specifically used for:

[0049] When the initial light source direction vector is the same as the initial natural light direction vector, a transformation from the world coordinate system to the vehicle camera coordinate system and an attitude compensation operation are performed on the initial natural light direction vector to obtain the calibrated light source direction vector.

[0050] When the initial light source direction vector is the same as the initial artificial light direction vector, an attitude compensation operation is performed on the initial artificial light direction vector to obtain the calibrated light source direction vector.

[0051] The rotation matrix, constructed based on the current vehicle attitude data, enables attitude compensation.

[0052] The advantages of the above-mentioned optional methods are as follows: by performing coordinate transformation and attitude compensation on the initial light source direction vector, the light source direction is accurately calibrated, ensuring that the height of the partition can still be accurately calculated when the vehicle attitude changes, thus improving the reliability and stability of the detection and making the results more accurate.

[0053] Thirdly, the technical solution of an electronic device according to the present invention is as follows:

[0054] It includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps of the vehicle-mounted road surface divider detection method of the present invention.

[0055] Fourthly, the technical solution of a computer-readable storage medium provided by the present invention is as follows:

[0056] The computer-readable storage medium stores instructions that, when read, cause the computer-readable storage medium to perform the steps of the vehicle-mounted road surface divider detection method of the present invention.

[0057] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0058] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0059] Figure 1 This is a schematic flowchart of an embodiment of the vehicle-mounted road surface divider detection method of the present invention;

[0060] Figure 2 This is a schematic diagram of an embodiment of the vehicle-mounted road surface divider detection system of the present invention;

[0061] Figure 3 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0062] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0063] Figure 1 This diagram illustrates a flowchart of a first embodiment of a vehicle-mounted road surface divider detection method provided by the present invention. This method can be executed by an electronic device such as a terminal device or a server. The terminal device can be any fixed or mobile terminal, such as a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, vehicle-mounted device, or wearable device. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the vehicle-mounted road surface divider detection method by having its processor call computer-readable instructions stored in its memory.Figure 1 As shown, it includes the following steps:

[0064] S1. Dynamically generate an initial light source direction vector based on the vehicle sensor data at the current moment, and calibrate the initial light source direction vector using the current vehicle attitude data to obtain a calibrated light source direction vector.

[0065] The vehicle-mounted sensor data includes, but is not limited to: geographic location data, timestamps, image frames, and vehicle attitude data. Geographic location data refers to the vehicle's latitude and longitude coordinates provided by the Global Positioning System (GPS); timestamps refer to precise time information in Coordinated Universal Time (UTC) format; and image frames refer to single-frame road images captured by the vehicle's onboard cameras. Vehicle attitude data refers to the vehicle's three-axis Euler angles (pitch, roll, and yaw) output by the Inertial Measurement Unit (IMU). Pitch angle refers to the vehicle's forward and backward tilt angle; roll angle refers to the vehicle's left and right tilt angle; and yaw angle refers to the vehicle's horizontal steering angle.

[0066] The initial light source direction vector refers to the uncalibrated three-dimensional vector representing the incident direction of the ambient light source, and is divided into two categories: ① Initial direction vector of natural light: the unit vector of the incident direction of sunlight in the world coordinate system; ② Initial direction vector of artificial light: the unit vector of the direction of the artificial light source in the vehicle camera coordinate system. The calibrated light source direction vector refers to the three-dimensional vector of the light source direction after vehicle attitude compensation. Specifically: ① For natural light, after transformation from the world coordinate system to the vehicle camera coordinate system, attitude deviation is eliminated by a rotation matrix; ② For artificial light, attitude deviation is eliminated directly in the vehicle camera coordinate system by a rotation matrix.

[0067] S2. Calculate the actual height of the road dividers based on the calibration light source direction vector and the geometric features of the shadows formed by the road dividers extracted from the current image frame captured by the vehicle camera.

[0068] The vehicle-mounted camera is a monocular vision sensor installed by default at the front of the vehicle (this can be adjusted according to actual conditions). Its acquisition parameters are: frame rate ≥ 30fps, field of view ≥ 120°; built-in calibration parameters are: focal length and optical center coordinates. The current image frame refers to a single road image captured by the vehicle-mounted camera at the current moment. Road surface dividers refer to the physical structures on the road surface used to demarcate traffic areas, and their types include but are not limited to: ① Shoulders: hardened curb structures (height 5-15cm); ② Fences: vertical isolation facilities (height 50-100cm); ③ Wheel tracks: tire marks (depth 0.5-2cm).

[0069] Shadow geometric features refer to the shadow spatial attribute parameters extracted from the current image frame. Shadow geometric features include: original shadow length value, original shadow angle value, and shadow depth value;

[0070] The actual height value refers to the physical dimension of the road divider in the vertical direction.

[0071] S3. Combine the actual height value with the shadow morphology features formed by the road surface dividers in the current image frame to determine the type of road surface dividers.

[0072] Among them, shadow morphology features refer to parameters that describe the visual characteristics of shadow areas.

[0073] The technical solution of this embodiment can solve the problems of low detection accuracy of vehicle road surface dividers and difficulty in distinguishing similar dividers under scenarios of weather changes and dynamic changes in lighting, thereby reducing detection costs and improving detection accuracy and stability.

[0074] Second embodiment:

[0075] Based on the first embodiment, the step of dynamically generating the initial light source direction based on the vehicle-mounted sensor data at the current moment includes:

[0076] Based on the brightness distribution and spatial distribution characteristics of highlight areas in the current image frame, the dominance of natural light sources and artificial light sources can be distinguished.

[0077] Specifically, brightness distribution refers to the global average brightness. The spatial distribution characteristics of highlight regions include: the proportion of highlight regions and spatial dispersion. Specifically: ① Calculate the global average brightness of the current image frame. The formula is: ; , representing the total number of pixels in the image; Represents pixels The grayscale value ranges from 0 to 255. ② Calculate the proportion of the highlight area in the current image frame. The formula is: ; This represents the total number of highlight pixels; the default highlight pixel threshold is 220. ③ Calculate the spatial dispersion of the current image frame. The formula is: ; Indicates the first The centroid coordinates of a highly connected region; This represents the centroid coordinates of the highlight region in the entire image. It should be noted that this is used when calculating the spatial dispersion of the current image frame. Previously, the specular connected components were detected in the current image frame, and the results were obtained. 4. A highly connected region (8-neighbor connectivity). The dominant expression distinguishing natural light sources from artificial light sources is: ;in, , which represents the length of the image diagonal.

[0078] For example, if the current image frame has 1280×720 pixels, then In a sunny setting: ;at this time: , , The dominant light source is determined to be natural light. In tunnel scenarios: ;at this time: , , The study determined that artificial light sources were the dominant source.

[0079] When natural light is dominant, the solar azimuth angle is calculated using astronomical algorithms based on the current geographical location data and the current timestamp, in order to generate the initial direction vector of natural light in the world coordinate system.

[0080] Specifically, when natural light is dominant, based on the latitude and longitude information in the current geographical location data and the Coordinated Universal Time in the current timestamp, astronomical algorithms are used to calculate the azimuth and altitude angles of the sun. Based on the calculated azimuth and altitude angles of the sun, an initial direction vector of natural light pointing towards the sun is generated in the world coordinate system. The components of the initial direction vector of natural light are determined by a trigonometric function combination of the azimuth and altitude angles of the sun.

[0081] When artificial light sources are dominant, the initial direction vector of artificial light in the vehicle camera coordinate system is generated by fitting the light direction through Hough transform based on the spatial distribution characteristics of the highlight area.

[0082] Specifically, when artificial light source is dominant, based on the spatial distribution characteristics of the highlight region in the current image frame, significant highlight pixels are first screened by gradient magnitude threshold, then Hough transform is applied to detect the dominant line direction in the highlight region, the farthest highlight point is selected along the detected dominant line direction, and combined with the focal length and optical center coordinate parameters of the vehicle camera, an initial direction vector of artificial light from the optical center of the camera to the farthest highlight point is generated in the coordinate system of the vehicle camera, and the initial direction vector of artificial light is normalized to a unit vector.

[0083] The technical solution of the second embodiment distinguishes between the dominance of natural light and artificial light, and uses different data to generate an initial light source direction vector, enabling the detection to be dynamically adjusted according to different lighting conditions, thereby enhancing the adaptability and accuracy of the method and further improving the detection effect in complex lighting environments.

[0084] Third embodiment:

[0085] Based on the second embodiment, the step of calibrating the initial light source direction vector using the current vehicle attitude data to obtain the calibrated light source direction vector includes:

[0086] When the initial light source direction vector is the same as the initial natural light direction vector, a transformation from the world coordinate system to the vehicle camera coordinate system and an attitude compensation operation are performed on the initial natural light direction vector to obtain the calibrated light source direction vector.

[0087] Specifically, when the initial light source direction vector is the natural light initial direction vector, a transformation from the world coordinate system to the vehicle camera coordinate system and attitude compensation operation are performed: First, the natural light initial direction vector in the world coordinate system is transformed to the vehicle camera reference coordinate system through a fixed rotation matrix. This fixed rotation matrix is ​​determined by the installation angle of the vehicle camera relative to the world coordinate system. Then, an attitude compensation operation is performed on the transformed vector using a rotation matrix constructed using the current vehicle attitude data to eliminate the observation deviation caused by the vehicle pitch angle, roll angle and yaw angle, and output a calibration light source direction vector.

[0088] When the initial light source direction vector is the same as the initial artificial light direction vector, an attitude compensation operation is performed on the initial artificial light direction vector to obtain the calibrated light source direction vector.

[0089] Specifically, when the initial light source direction vector is the initial artificial light direction vector, an attitude compensation operation is performed: the transpose of the rotation matrix constructed from the current vehicle attitude data is directly used to rotate the initial artificial light direction vector in the vehicle camera coordinate system in reverse, thereby offsetting the vector offset caused by the change in vehicle attitude and outputting a calibrated light source direction vector.

[0090] The rotation matrix, constructed based on the current vehicle attitude data, enables attitude compensation.

[0091] Specifically, the rotation matrix is: ; This represents the roll angle rotation matrix. ; This represents the pitch angle rotation matrix. ; This represents the yaw angle rotation matrix. ; Indicates pitch angle, Indicates the roll angle. This represents the yaw angle. It should be noted that the rotation matrix... The attitude compensation sequence is: yaw angle; pitch angle; roll angle.

[0092] The technical solution of the third embodiment accurately calibrates the light source direction by performing coordinate transformation and attitude compensation on the initial light source direction vector, ensuring that the height of the separator can still be accurately calculated when the vehicle attitude changes, thereby improving the reliability and stability of the detection and making the results more accurate.

[0093] Fourth embodiment:

[0094] Based on any of the first to third embodiments, the step of calculating the actual height value of the road surface divider according to the calibration light source direction vector and the shadow geometric features formed by the road surface divider extracted from the current image frame captured by the vehicle-mounted camera includes:

[0095] The angle between the calibration light source direction vector and the road surface normal vector is determined as the light source incident angle value.

[0096] The original shadow length value is corrected based on the shadow depth value to obtain the target shadow length value, and the original shadow angle value is corrected based on the intrinsic parameters of the vehicle camera to obtain the target shadow angle value.

[0097] The actual height value is calculated based on the target shadow length, the target shadow angle, and the incident angle of the light source.

[0098] Specifically, the angle between the calibration light source direction vector and the road surface normal vector is determined as the light source incident angle value, which is obtained by calculating the inverse cosine of the dot product of the calibration light source direction vector and the road surface normal vector. Based on the shadow depth value, the original shadow length value is subjected to perspective correction, and the image pixel distance is converted into physical space length to obtain the target shadow length value. At the same time, based on the focal length and optical center coordinate parameters of the vehicle camera, the original shadow angle value is subjected to optical center offset correction to obtain the target shadow angle value. Based on the target shadow length value, the target shadow angle value, and the light source incident angle value, the actual height value of the road surface divider is calculated using the geometric projection formula.

[0099] Among them, the incident angle value of the light source ; Indicates the calibration light source direction vector; , represents the road surface normal vector; This indicates the incident angle of the light source, ranging from 0° to 90°.

[0100] The correction formula for the target shadow length value is as follows: ; This represents the original shadow length value. This represents the shadow depth value (the vertical distance from the camera's optical center to the road surface). Indicates the focal length of the vehicle camera. This indicates the length of the target shadow.

[0101] The formula for correcting the target shadow angle is as follows: ; Indicates the original shadow angle value; , representing the vertical offset between the optical center and the shadow endpoint. Represents the ordinate of the optical center. This represents the ordinate of the shadow endpoint (the ordinate of the farthest endpoint of the shadow area in the image's two-dimensional coordinate system). This represents the target shadow angle value.

[0102] The expression for the geometric projection formula is as follows: ; This represents the target shadow length value. Indicates the incident angle value of the light source. Indicates the target shadow angle value. This indicates the actual height value.

[0103] The technical solution of the fourth embodiment corrects the original shadow length value by introducing a shadow depth value and calculates the actual height value by comprehensively considering multiple factors such as the incident angle of the light source, making the height measurement more accurate and improving the accuracy and reliability of detecting the height of road dividers.

[0104] In one alternative approach, shadow morphological features include: shadow aspect ratio, shadow edge sharpness value, and shadow continuity; the step of determining the type of road divider by combining the actual height value with the shadow morphological features formed by road dividers in the current image frame includes:

[0105] The feature vector to be tested is constructed based on the aspect ratio of the shadow, the sharpness value of the shadow edge, the continuity of the shadow, and the actual height value.

[0106] Among them, based on the aspect ratio of the shadow ( ), Shadow edge sharpness value ( ), shadow continuity ( ) and actual height value ( Construct a four-dimensional feature vector to be tested. ; = ;

[0107] ; =Percentage of the longest continuous segment Indicates the total number of pixels at the shadow boundary. Represents pixels The gradient vector.

[0108] The feature vector to be tested is input into the trained classification decision model to obtain the predicted type probability distribution, and the type mapped by the maximum probability value in the predicted type probability distribution is determined as the type of road divider.

[0109] The pre-trained classification decision model defaults to a lightweight convolutional neural network (Lightweight CNN). A Lightweight CNN consists of: an input layer (feature vector to be tested), a fully connected layer (128 nodes), ReLU activation, a fully connected layer (64 nodes), ReLU activation, and an output layer (Softmax). The output is a seven-dimensional probability distribution. ; ; Mapping to type: 1 shoulder, 2 fence, 3 tire tracks, 4 speed bump, 5 construction cone, 6 ditch cover, 7 cat's eye reflector.

[0110] It should be noted that: ① The training dataset for Lightweight CNN must contain fully labeled road surface divider samples, covering seven types: road shoulders, fences, tire tracks, speed bumps, construction cones, drainage ditch covers, and cat eye reflectors. For each type, at least 1000 samples should be collected under various lighting conditions, including sunny days, rain, fog, and nighttime. Each sample should provide a four-dimensional feature vector as input, consisting of the shadow aspect ratio, edge sharpness value, shadow continuity, and actual height value, and should be labeled with the true type label. Before training, the feature vectors are standardized by dividing the aspect ratio by 10 to scale to the [0,1] interval, dividing the edge sharpness value by 100 to scale to the [0,1] interval, and mapping the height value to the [0,1] interval, ensuring that all features are of the same magnitude. ② Lightweight CNN employs a four-layer fully connected architecture. The input layer receives a four-dimensional feature vector and connects to a 128-node hidden layer with ReLU activation. This function sets all negative values ​​to zero and retains positive values ​​to introduce non-linear computational capabilities. The output of the hidden layer connects to a 64-node second hidden layer with ReLU activation, and finally to a 7-node output layer with Softmax activation, transforming the output into a probability distribution of seven class separators. During training, cross-entropy loss is used to quantify the difference between the predicted probability distribution and the true labels. The gradient of the loss function with respect to the network parameters is calculated using the backpropagation algorithm. The Adam optimizer is used to update the weight matrix and bias vector with a learning rate of 0.001. In each iteration, 32 samples are randomly selected from the training set to form a mini-batch for gradient calculation. A training cycle ends after all samples have been traversed. ③ An early stopping strategy is implemented during training to monitor the validation set loss in real time. If the validation loss does not decrease for five consecutive training cycles, training is automatically terminated and the optimal parameters are saved. At the same time, a class weighting mechanism is used to improve the recognition ability of small sample types. Low-frequency types such as cat eye reflectors are given a tenfold loss weight to balance the type distribution. The final trained model needs to be validated on an independent test set to ensure that the overall classification accuracy is higher than 93% and the accuracy in rain and fog scenarios is not lower than 90%. After meeting the requirements of real-time computing in vehicles, it is deployed to an embedded platform.

[0111] The technical solution of the fifth embodiment determines the type of separator by constructing the feature vector to be tested and inputting it into the classification decision model. It can effectively distinguish similar separators, improve the accuracy and efficiency of classification, and provide more reliable road information for autonomous driving.

[0112] Figure 2 A schematic diagram of an embodiment of a vehicle-mounted road surface divider detection system 200 provided by the present invention is shown. Figure 2 As shown, the system 200 includes: a processing module 210, a calculation module 220, and a detection module 230;

[0113] The processing module 210 is used to: dynamically generate an initial light source direction vector based on the vehicle sensor data at the current moment, and calibrate the initial light source direction vector using the current vehicle attitude data to obtain a calibrated light source direction vector; the initial light source direction vector is: the initial direction vector of natural light or the initial direction vector of artificial light; the vehicle sensor data at the current moment includes: current geographical location data, current timestamp, current image frame and current vehicle attitude data;

[0114] The calculation module 220 is used to: calculate the actual height value of the road surface divider based on the calibration light source direction vector and the shadow geometric features formed by the road surface divider extracted from the current image frame captured by the vehicle-mounted camera; the shadow geometric features include: the original shadow length value, the original shadow angle value, and the shadow depth value;

[0115] The detection module 230 is used to: determine the type of road divider by combining the actual height value with the shadow morphology features formed by the road divider in the current image frame.

[0116] In one alternative approach, the processing module is specifically used for:

[0117] Based on the brightness distribution and highlight region spatial distribution characteristics in the current image frame, the dominance of natural light sources and artificial light sources is distinguished; whereby the brightness distribution is defined as the global average brightness; and the highlight region spatial distribution characteristics include the highlight region proportion and spatial dispersion; the expression for distinguishing the dominance of natural light sources and artificial light sources is:

[0118] ;

[0119] in, The global average brightness The percentage of the highlight area. Spatial dispersion; Indicates the length of the image diagonal. , Indicates the image width. Indicates the image height;

[0120] When natural light is dominant, the solar azimuth angle is calculated using astronomical algorithms based on the current geographical location data and the current timestamp, in order to generate the initial direction vector of natural light in the world coordinate system;

[0121] When artificial light sources are dominant, the initial direction vector of artificial light in the vehicle camera coordinate system is generated by fitting the light direction through Hough transform based on the spatial distribution characteristics of the highlight area.

[0122] In one alternative approach, the processing module is specifically used for:

[0123] When the initial light source direction vector is the same as the initial natural light direction vector, a transformation from the world coordinate system to the vehicle camera coordinate system and an attitude compensation operation are performed on the initial natural light direction vector to obtain the calibrated light source direction vector.

[0124] When the initial light source direction vector is the same as the initial artificial light direction vector, an attitude compensation operation is performed on the initial artificial light direction vector to obtain the calibrated light source direction vector.

[0125] The rotation matrix, constructed based on the current vehicle attitude data, enables attitude compensation.

[0126] In an alternative embodiment, the computing module 220 is specifically used for:

[0127] The angle between the calibration light source direction vector and the road surface normal vector is determined as the light source incident angle value;

[0128] The original shadow length value is corrected based on the shadow depth value to obtain the target shadow length value, and the original shadow angle value is corrected based on the intrinsic parameters of the vehicle camera to obtain the target shadow angle value.

[0129] The actual height value is calculated based on the target shadow length, the target shadow angle, and the incident angle of the light source.

[0130] In one alternative approach, shadow morphological features include: shadow aspect ratio, shadow edge sharpness value, and shadow continuity; the detection module 230 is specifically used for:

[0131] Based on the aspect ratio of the shadow, the sharpness of the shadow edge, the continuity of the shadow, and the actual height, a feature vector to be tested is constructed.

[0132] The feature vector to be tested is input into the trained classification decision model to obtain the predicted type probability distribution, and the type mapped by the maximum probability value in the predicted type probability distribution is determined as the type of road divider.

[0133] It should be noted that the beneficial effects of the vehicle-mounted road surface divider detection system 200 provided in the above embodiments are the same as those of the vehicle-mounted road surface divider detection method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0134] The vehicle-mounted road surface divider detection system 200 of the present invention can be a computer program (including program code) running on a computer device. For example, the vehicle-mounted road surface divider detection system of the present invention is an application software that can be used to execute the corresponding steps in the vehicle-mounted road surface divider detection method of the present invention.

[0135] In some embodiments, the vehicle-mounted road surface divider detection system 200 of the present invention can be implemented in a combination of hardware and software. As an example, the vehicle-mounted road surface divider detection system of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the vehicle-mounted road surface divider detection method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0136] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0137] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described vehicle-mounted road surface divider detection methods. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the vehicle-mounted road surface divider detection method shown in any embodiment of the present invention by calling the computer program.

[0138] In one alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0139] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0140] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0141] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0142] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0143] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0144] It should be noted that, Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0145] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described vehicle-mounted road surface divider detection methods.

[0146] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0147] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned vehicle-mounted road divider detection method.

[0148] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0150] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0151] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0152] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0153] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0154] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0155] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting road surface dividers on a vehicle, characterized in that, include: An initial light source direction vector is dynamically generated based on the vehicle sensor data at the current moment, and the initial light source direction vector is calibrated using the current vehicle attitude data to obtain a calibrated light source direction vector; the initial light source direction vector is either a natural light initial direction vector or an artificial light initial direction vector. The vehicle sensor data at the current moment includes: current geographical location data, current timestamp, current image frame, and current vehicle attitude data; Based on the calibration light source direction vector and the shadow geometric features formed by the road surface dividers extracted from the current image frame captured by the vehicle-mounted camera, the actual height value of the road surface dividers is calculated; the shadow geometric features include: the original shadow length value, the original shadow angle value, and the shadow depth value. The type of road divider is determined by combining the actual height value with the shadow morphology features formed by the road divider in the current image frame; the step of calculating the actual height value of the road divider based on the calibration light source direction vector and the shadow geometric features formed by the road divider extracted from the current image frame captured by the vehicle-mounted camera includes: The angle between the calibration light source direction vector and the road surface normal vector is determined as the light source incident angle value; The original shadow length value is corrected based on the shadow depth value to obtain the target shadow length value, and the original shadow angle value is corrected based on the intrinsic parameters of the vehicle camera to obtain the target shadow angle value; The actual height value is calculated based on the target shadow length value, the target shadow angle value, and the light source incident angle value.

2. The vehicle-mounted road surface divider detection method according to claim 1, characterized in that, The step of dynamically generating the initial light source direction based on the vehicle-mounted sensor data at the current moment includes: Based on the brightness distribution and highlight region spatial distribution characteristics in the current image frame, the dominance of natural light sources and artificial light sources is distinguished; wherein, the brightness distribution is: global average brightness; the highlight region spatial distribution characteristics include: highlight region proportion and spatial dispersion; the expression for distinguishing the dominance of natural light sources and artificial light sources is: ; in, The global average brightness, The percentage of the highlighted area. The spatial discreteness; Indicates the length of the image diagonal. , Indicates the image width. Indicates the image height; When natural light is dominant, the solar azimuth angle is calculated using an astronomical algorithm based on the current geographical location data and the current timestamp, in order to generate the initial direction vector of natural light in the world coordinate system; When artificial light sources are dominant, based on the spatial distribution characteristics of the high-light region, the direction of light is fitted by Hough transform to generate the initial direction vector of artificial light in the coordinate system of the vehicle camera.

3. The vehicle-mounted road surface divider detection method according to claim 2, characterized in that, The step of calibrating the initial light source direction vector using the current vehicle attitude data to obtain the calibrated light source direction vector includes: When the initial light source direction vector is the same as the initial natural light direction vector, a transformation from the world coordinate system to the vehicle camera coordinate system and an attitude compensation operation are performed on the initial natural light direction vector to obtain the calibration light source direction vector. When the initial light source direction vector is the same as the initial artificial light direction vector, an attitude compensation operation is performed on the initial artificial light direction vector to obtain the calibrated light source direction vector. The attitude compensation operation is achieved by constructing a rotation matrix based on the current vehicle attitude data.

4. The vehicle-mounted road surface divider detection method according to claim 1, characterized in that, The shadow morphological features include: shadow aspect ratio, shadow edge sharpness value, and shadow continuity; the step of determining the type of road divider by combining the actual height value with the shadow morphological features formed by the road divider in the current image frame includes: Based on the shadow aspect ratio, the shadow edge sharpness value, the shadow continuity and the actual height value, a feature vector to be measured is constructed; The feature vector to be tested is input into the trained classification decision model to obtain the predicted type probability distribution, and the type mapped by the maximum probability value in the predicted type probability distribution is determined as the type of the road surface separator.

5. A vehicle-mounted road surface divider detection system, used to implement the vehicle-mounted road surface divider detection method as described in claim 1, characterized in that, The vehicle-mounted road surface divider detection system includes: a processing module, a calculation module, and a detection module; The processing module is used to: dynamically generate an initial light source direction vector based on the vehicle sensor data at the current moment, and calibrate the initial light source direction vector using the current vehicle attitude data to obtain a calibrated light source direction vector; the initial light source direction vector is: a natural light initial direction vector or an artificial light initial direction vector; the vehicle sensor data at the current moment includes: current geographical location data, current timestamp, the current image frame, and the current vehicle attitude data; The calculation module is used to: calculate the actual height value of the road divider based on the calibration light source direction vector and the shadow geometric features formed by the road divider extracted from the current image frame captured by the vehicle camera; the shadow geometric features include: original shadow length value, original shadow angle value, and shadow depth value; The detection module is used to: determine the type of road divider by combining the actual height value with the shadow shape features formed by the road divider in the current image frame.

6. The vehicle-mounted road divider detection system according to claim 5, characterized in that, The processing module is specifically used for: Based on the brightness distribution and highlight region spatial distribution characteristics in the current image frame, the dominance of natural light sources and artificial light sources is distinguished; wherein, the brightness distribution is: global average brightness; the highlight region spatial distribution characteristics include: highlight region proportion and spatial dispersion; the expression for distinguishing the dominance of natural light sources and artificial light sources is: ； in, The global average brightness, The percentage of the highlighted area. The spatial discreteness; Indicates the length of the image diagonal. , Indicates the image width. Indicates the image height; When natural light is dominant, the solar azimuth angle is calculated using an astronomical algorithm based on the current geographical location data and the current timestamp, in order to generate the initial direction vector of natural light in the world coordinate system; When artificial light sources are dominant, based on the spatial distribution characteristics of the high-light region, the direction of light is fitted by Hough transform to generate the initial direction vector of artificial light in the coordinate system of the vehicle camera.

7. The vehicle-mounted road divider detection system according to claim 6, characterized in that, The processing module is specifically used for: When the initial light source direction vector is the same as the initial natural light direction vector, a transformation from the world coordinate system to the vehicle camera coordinate system and an attitude compensation operation are performed on the initial natural light direction vector to obtain the calibration light source direction vector. When the initial light source direction vector is the same as the initial artificial light direction vector, an attitude compensation operation is performed on the initial artificial light direction vector to obtain the calibrated light source direction vector. The attitude compensation operation is achieved by constructing a rotation matrix based on the current vehicle attitude data.

8. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the vehicle-mounted roadway divider detection method as described in any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer-readable storage medium to implement the vehicle-mounted roadway divider detection method as described in any one of claims 1 to 4.

Images