Road condition detection method, vehicle-mounted controller, vehicle and storage medium
Tracking the target vehicle in front of the vehicle through the front camera data and obtaining its vibration data, and combining the vibration data of the local vehicle for road conditions, the problem of low accuracy of road conditions detection in the existing technology is solved, and high accuracy in various scenarios and Reliability detection.
Patent Information
- Application Number
- CN202310595444.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-05-23
AI Technical Summary
The existing road condition detection methods cannot accurately detect the road conditions ahead of the vehicle, especially when the road surface is blocked or there are debris such as water, snow, fallen leaves and garbage, the detection accuracy is low.
By obtaining front camera data of the car, tracking the target's front vehicle and obtaining the vibration data of its front vehicle, combining the vibration data of the local vehicle for road conditions, using vibration signal prediction models to identify road conditions, reducing dependence on the target's front vehicle or high-precision map, and improving detection reliability and accuracy.
It realizes accurate detection of the road conditions ahead of the vehicle in various scenarios, reduces interference to external factors, expands application scenarios, and improves the accuracy and reliability of road conditions detection.
Smart Images

Figure CN119028117A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of road condition detection, and in particular to a road condition detection method, a vehicle-mounted controller, a vehicle and a storage medium. Background Art
[0002] There are two existing road condition detection methods: one is a vision-based road condition detection method, that is, a road image is captured by a camera, and a target detection or segmentation algorithm is used to detect targets such as speed bumps and manhole covers in the captured road image to achieve the purpose of road condition detection. This vision-based road condition detection method cannot avoid the problem of road obstruction. For example, detection cannot be achieved when the road is blocked by the vehicle in front. Moreover, debris such as accumulated water, snow, fallen leaves and garbage on the road surface will also interfere with the road condition detection results, resulting in low detection accuracy, and its application scenarios are greatly limited. The second is a road condition detection method based on the vehicle's sensing signal, that is, the road condition is detected by collecting the vehicle's speed and acceleration data collected by the inertial measurement device to detect whether the vehicle has passed through uneven roads such as speed bumps and manhole covers. In summary, the existing road condition detection method has the problem of being unable to accurately detect the road condition in front of the vehicle. Summary of the invention
[0003] The embodiments of the present invention provide a road condition detection method, a vehicle-mounted controller, a vehicle and a storage medium to solve the problem that the existing road condition detection method cannot accurately detect the road condition ahead of the vehicle.
[0004] A road condition detection method, comprising:
[0005] Get the front camera data of the vehicle;
[0006] Tracking a target front vehicle according to the front camera data, and acquiring front vehicle vibration data corresponding to the target front vehicle;
[0007] A road condition detection is performed based on the vibration data of the preceding vehicle and the vibration data of the own vehicle to obtain a road condition detection result.
[0008] Preferably, tracking the target vehicle ahead according to the vehicle ahead camera data comprises:
[0009] Performing target detection based on at least one of the vehicle front camera data to determine at least one front vehicle to be analyzed;
[0010] Obtaining the pixel distance between the front camera and each of the front vehicles to be analyzed and the front vehicle angle;
[0011] If the pixel distance and the front vehicle angle meet the preset screening conditions, the front vehicle to be analyzed is determined as a target front vehicle, and the target front vehicle is tracked.
[0012] Preferably, if the pixel distance and the front vehicle angle meet a preset screening condition, determining the front vehicle to be analyzed as a target front vehicle includes:
[0013] If the pixel distance and the angle of the preceding vehicle to be analyzed corresponding to the preceding vehicle to be analyzed meet the same lane condition, the preceding vehicle to be analyzed is determined as the preceding vehicle to be in the same lane;
[0014] If the pixel distance and the front vehicle angle corresponding to the front vehicle in the same lane meet the unobstructed condition, the front vehicle in the same lane is determined as the target front vehicle.
[0015] Preferably, if the pixel distance and the front vehicle angle satisfy the same lane condition, determining the front vehicle to be analyzed as the front vehicle in the same lane includes:
[0016] Determining the actual distance between the front camera and each of the front vehicles to be analyzed according to the pixel distance;
[0017] Decomposing the actual distance according to the front vehicle angle to determine the lateral distance between the front camera and each of the front vehicles to be analyzed;
[0018] If the lateral distance is less than a first distance threshold, the front vehicle to be analyzed is determined to be a front vehicle in the same lane.
[0019] Preferably, after decomposing the actual distance according to the front vehicle angle, the method further includes: determining the longitudinal distance between the front camera and each of the front vehicles to be analyzed;
[0020] If the front camera data of the vehicle in front of the same lane meets the unobstructed condition, determining the vehicle in front of the same lane as the target vehicle in front includes:
[0021] The longitudinal distances between the front camera and all the preceding vehicles in the same lane are compared, and the preceding vehicle in the same lane with the smallest longitudinal distance is determined as the target preceding vehicle.
[0022] Preferably, after determining the preceding vehicle in the same lane with the smallest longitudinal distance as the target preceding vehicle, the road condition detection method further comprises:
[0023] If the longitudinal distance between the front camera and the target front vehicle is less than a second distance threshold, a rear-end collision warning operation is performed.
[0024] Preferably, the tracking of the target preceding vehicle comprises:
[0025] Based on the continuous frames of the front camera data of the vehicle, a first license plate corresponding to the target front vehicle and a second license plate corresponding to the last tracked target are obtained;
[0026] If the first license plate and the second license plate are the same, it is determined that the target front vehicle is the last tracked target, and the front vehicle vibration data corresponding to the target front vehicle is obtained;
[0027] If the first license plate and the second license plate are different, it is determined that the target front vehicle is not the last tracking target, and the step of acquiring the front vehicle camera data is performed.
[0028] Preferably, the obtaining of the preceding vehicle vibration data corresponding to the target preceding vehicle includes:
[0029] Extracting feature points of the target front vehicle to determine target feature points corresponding to the target front vehicle;
[0030] Based on the target feature points, the continuous frames of the vehicle front camera data are analyzed to obtain the front vehicle vibration data corresponding to the target front vehicle.
[0031] Preferably, extracting feature points of the target front vehicle to determine target feature points corresponding to the target front vehicle includes:
[0032] Extracting feature points of the target front vehicle to obtain at least two original feature points corresponding to the target front vehicle;
[0033] Feature fusion is performed on at least two original feature points corresponding to the target front vehicle to obtain a target feature point corresponding to the target front vehicle.
[0034] Preferably, the analyzing the continuous frames of the vehicle front camera data based on the target feature points to obtain the front vehicle vibration data corresponding to the target front vehicle includes:
[0035] Based on the target feature point, the front camera data of the vehicle is identified in consecutive frames to determine the pixel coordinate change corresponding to the target feature point;
[0036] Based on the pixel coordinate change corresponding to the target feature point, the preceding vehicle vibration data corresponding to the target preceding vehicle is obtained.
[0037] Preferably, performing road condition detection according to the vibration data of the preceding vehicle and the vibration data of the own vehicle to obtain a road condition detection result includes:
[0038] Performing feature encoding on the vibration data of the preceding vehicle and the vibration data of the own vehicle respectively, and obtaining a first encoding feature corresponding to the vibration data of the preceding vehicle and a second encoding feature corresponding to the vibration data of the own vehicle;
[0039] splicing the first coding feature corresponding to the vibration data of the preceding vehicle and the second coding feature corresponding to the vibration data of the own vehicle to obtain a target coding feature;
[0040] A road condition detection is performed according to the target coding feature to obtain a road condition detection result.
[0041] A vehicle-mounted controller comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the above-mentioned road condition detection method is implemented when the processor executes the computer program.
[0042] A vehicle comprises the above-mentioned on-vehicle controller.
[0043] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the road condition detection method is implemented.
[0044] The above-mentioned road condition detection method, on-board controller, vehicle and storage medium track the target front vehicle based on the front camera data and obtain the corresponding front vehicle vibration data. The front vehicle vibration data can be obtained without communicating and exchanging data with the target front vehicle, thereby ensuring the reliability of the front vehicle vibration data acquisition, and can reduce the dependence on the target front vehicle or high-precision map, making it less affected by external factors, simple to implement and with a wide range of application scenarios; then, road condition detection is performed based on the front vehicle vibration data and the own vehicle vibration data, and the own vehicle vibration data is used to offset the interference of the vibration generated by the own vehicle passing through uneven roads on the front vehicle vibration data, which helps to ensure the accuracy of road condition detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0046] Figure 1 is a flow chart of a road condition detection method in one embodiment of the present invention;
[0047] Figure 2 is another flow chart of a road condition detection method in one embodiment of the present invention;
[0048] Figure 3 is another flow chart of a road condition detection method in one embodiment of the present invention;
[0049] Figure 4 is another flow chart of a road condition detection method in one embodiment of the present invention;
[0050] Figure 5 is another flow chart of a road condition detection method in one embodiment of the present invention;
[0051] Figure 6is another flow chart of a road condition detection method in one embodiment of the present invention;
[0052] Figure 7 is another flow chart of a road condition detection method in one embodiment of the present invention;
[0053] Figure 8 is another flow chart of a road condition detection method in one embodiment of the present invention;
[0054] Fig. 9 is another flow chart of a road condition detection method in one embodiment of the present invention;
[0055] Fig.10 is a schematic diagram of a front vehicle to be analyzed within the visual range of a front camera in one embodiment of the present invention;
[0056] Fig.11 It is a schematic diagram of road condition detection using a vibration signal prediction model in one embodiment of the present invention. DETAILED DESCRIPTION
[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] The road condition detection method provided by an embodiment of the present invention can be applied to a vehicle-mounted controller, which refers to a controller installed on a car. It can be an existing controller integrated with other functions on the vehicle, or it can be a dedicated controller on the vehicle specifically used for road condition detection.
[0059] In one embodiment, if Figure 1 As shown, a road condition detection method is provided, which is described by taking the method applied to a vehicle controller as an example, and includes the following steps:
[0060] S101: Acquire the front camera data of the vehicle;
[0061] S102: Tracking a target front vehicle according to the front camera data, and obtaining front vehicle vibration data corresponding to the target front vehicle;
[0062] S103: Performing road condition detection according to the vibration data of the preceding vehicle and the vibration data of the own vehicle to obtain a road condition detection result.
[0063] The front camera data refers to the camera data collected in real time in front of the vehicle, specifically the camera data collected by the front camera installed on the vehicle.
[0064] As an example, in step S101, the vehicle controller may obtain continuous multi-frame front camera data collected in real time by the front camera installed on the vehicle during the driving process. After obtaining the front camera data transmitted by the front camera in the form of a video stream, the vehicle controller may use a video stream parsing method to parse and process the front camera data in the form of a video stream, specifically using but not limited to the video processing tool of the open source library Opencv to parse and process the front camera data in the form of a video stream, so as to obtain the parsed continuous multi-frame front camera data. For example, the front camera data collected at the current moment may be determined as the current camera data, and the front camera data collected at the previous moment may be determined as the previous camera data.
[0065] The target front vehicle refers to the front vehicle determined after target detection and used to assist in road condition detection.
[0066] As an example, in step S102, the on-board controller may use a target detection algorithm to perform target detection on each of the front camera data in the front camera data and perform target tracking based on the continuous frame front camera data, and determine at least one front vehicle to be analyzed that is tracked by the continuous frame front camera data; then determine whether the front vehicle to be analyzed meets the preset screening conditions that can assist in road condition detection, and if the preset screening conditions are met, the front vehicle to be analyzed is determined as the target front vehicle; finally, vibration analysis is performed on the target front vehicle based on the front camera data, for example, the front vehicle vibration data corresponding to the target front vehicle can be analyzed and determined based on the pixel coordinate changes of a certain feature point of the target front vehicle in the continuous frame front camera data. The front vehicle to be analyzed here refers to the vehicle in the target detection frame corresponding to the target detection frame during the target detection process of the target detection algorithm, and is a vehicle that needs to be further analyzed to see if it meets the preset screening conditions, specifically the pre-set vehicles Car, Bus and Truck.
[0067] For example, the front camera data collected by the front camera is two-dimensional data. The front camera data of the same target front vehicle are respectively determined as the previous camera data and the current camera data. Based on the previous camera data and the current camera data, the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement of the target front vehicle from the previous moment to the current moment can be determined, so as to comprehensively determine the road condition passed by the target front vehicle based on the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement. For example, when the target front vehicle is traveling on a normal road condition (such as a flat road condition), the front vehicle X-direction vibration signal collected by the target front vehicle in the X-axis direction and the front vehicle Y-direction vibration signal collected in the Y-axis direction are smoothly changing, and the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement calculated based on the front vehicle X-direction vibration signal and the front vehicle Y-direction vibration signal are not obvious; when the target front vehicle passes through a speed bump, its left and right wheels will pass through the speed bump at the same time, and the front vehicle X-direction vibration displacement is not obvious while the front vehicle Y-direction vibration displacement is obvious; when the target front vehicle passes through a manhole cover or a road When a target front vehicle passes through a pothole, generally one side of the wheel passes by, and the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle are both obvious; therefore, the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle can be calculated respectively according to the two X-direction vibration signals and the two Y-direction vibration signals of the front vehicle determined by the front and rear two frames of front camera data, and according to different combinations of the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle, it is determined that the road condition passed by the target front vehicle can be but is not limited to any one of the normal road condition (i.e. the middle part where the flat road surface is located), the speed bump road condition and the manhole cover / pothole road condition.
[0068] Among them, the vibration data of the vehicle refers to the vibration data of the vehicle collected in real time, specifically the vibration data collected in real time by the vibration sensor set on the vehicle. Generally speaking, when the vehicle vibrates, there will be instantaneous acceleration changes in the vehicle. At this time, the vibration sensor can collect vibration signals in the three directions of X / Y / Z, that is, the vibration data of the vehicle collected by it includes the X-direction vibration signal of the vehicle, the Y-direction vibration signal of the vehicle and the Z-direction vibration signal of the vehicle. The X-direction vibration signal of the vehicle refers to the vibration signal of the vehicle along the X-axis direction collected by the vibration sensor in real time. The Y-direction vibration signal of the vehicle refers to the vibration signal of the vehicle along the Y-axis direction collected by the vibration sensor in real time. The Z-direction vibration signal of the vehicle refers to the vibration signal of the vehicle along the Z-axis direction collected by the vibration sensor in real time.
[0069] As an example, in step S103, after obtaining the vibration data of the preceding vehicle corresponding to the target preceding vehicle, the onboard controller may also obtain the vibration data of the vehicle collected in real time by the vehicle sensor, and then input the vibration data of the preceding vehicle and the vehicle into the pre-trained vibration signal prediction model, and predict the vibration data of the preceding vehicle and the vehicle through the vibration signal prediction model to obtain the road condition detection result output by the vibration signal prediction model. The vibration signal prediction model is a deep learning model with a multi-layer structure, specifically a deep learning model for road condition recognition based on the input vibration data.
[0070] During the training of the vibration signal prediction model, the vibration data of the vehicle and the preceding vehicle collected when the vehicle is driving through normal road conditions, speed bumps, manhole covers / road potholes, the vibration data of the vehicle, the preceding vehicle and their corresponding road condition categories are determined as training samples, and the training samples are input into a multi-layer deep learning model for model training. When the deep learning model reaches the model convergence condition, the trained vibration signal prediction model is obtained, so that the vibration signal prediction model can accurately identify the road condition category to which it belongs based on the actual input vibration data of the vehicle and the preceding vehicle. In this example, the deep learning model used to train the vibration signal prediction model can be an end-to-end model with time series information, which can be but not limited to LSTM, GRU or Transformer models. In this example, an end-to-end model with time series information is used to train the vibration signal prediction model, and model learning training is used to replace the traditional rule-based manual parameter adjustment prediction method, which has the problems of simple model deployment and a wide range of applications.
[0071] In this embodiment, the vibration sensor for collecting the vibration data of the vehicle and the front camera for collecting the front camera data are both installed on the vehicle. When the vehicle passes over an uneven road surface (such as when the vehicle passes over a speed bump, a manhole cover, or a pothole), the vibration of the vehicle will drive the vibration sensor and the front camera to vibrate together. Therefore, it is necessary to use the vibration data of the vehicle collected by the vibration sensor to offset the interference caused by the vibration generated when the vehicle passes over an uneven road surface (such as when the vehicle passes over a speed bump, a manhole cover, or a pothole), thereby ensuring the accuracy of predicting the road condition that the vehicle is about to enter based on the vibration data of the vehicle ahead.
[0072] In this embodiment, the target front vehicle is tracked based on the front camera data and its corresponding front vehicle vibration data is obtained. The front vehicle vibration data can be obtained without communicating and exchanging data with the target front vehicle, thereby ensuring the reliability of the front vehicle vibration data acquisition and reducing the dependence on the target front vehicle or high-precision map, making it less affected by external factors, simple to implement and with a wide range of application scenarios. Then, road condition detection is performed based on the front vehicle vibration data and the vibration data of the own vehicle, and the vibration data of the own vehicle is used to offset the interference of the vibration generated by the own vehicle passing through uneven roads on the vibration data of the front vehicle, which helps to ensure the accuracy of road condition detection.
[0073] In one embodiment, if Figure 2 As shown, step S102, that is, tracking the target front vehicle according to the front camera data, includes:
[0074] S201: performing target detection based on at least one front vehicle camera data to determine at least one front vehicle to be analyzed;
[0075] S202: Obtaining the pixel distance between the front camera and each front vehicle to be analyzed and the front vehicle angle;
[0076] S203: If the pixel distance and the front vehicle angle meet the preset screening conditions, the front vehicle to be analyzed is determined as the target front vehicle, and the target front vehicle is tracked.
[0077] As an example, in step S201, the vehicle controller may use a target detection algorithm to perform target detection on each of the front camera data, and determine a detected target in at least one target detection frame as the front vehicle to be analyzed.
[0078] In this example, the on-board controller can use a target detection algorithm based on deep learning to perform target detection on the front camera data, detect the detection targets of cars, buses and trucks, and determine at least one front vehicle to be analyzed in the frame of at least one target detection frame from the front camera data. The learning-based target detection algorithm here can use but is not limited to YOLO, SSD and Retinanet algorithms, and the target detection frame here is a rectangular frame where the detection targets of cars, buses and trucks are located in the front camera data of the car. It can be understood that the front camera data is the camera data collected within the visible range of the front camera of the car, and all vehicles within the visible range are collected. Considering that the feature points of smaller vehicles such as bicycles and motorcycles are not obvious and the motion trajectory is complex, which is not conducive to detection and tracking, this solution uses cars, buses and trucks as detection targets, so that the detection targets have obvious feature points and simple motion trajectories, which helps to ensure the accuracy and efficiency of subsequent road condition detection.
[0079] The front camera refers to a camera that captures video data from the front of the vehicle.
[0080] As an example, in step S202, the vehicle controller may use a distance calculation method to calculate the pixel distance between the front camera that captures the front camera data and the front vehicle to be analyzed. In this example, the pixel distance may be the pixel distance between the front camera and the target detection frame of the front vehicle to be analyzed in the frame. For example, the vehicle controller may use but is not limited to the Euclidean distance algorithm to calculate the pixel distance between the coordinate origin of the front camera and the front vehicle to be analyzed. The pixel distance here may be the distance between the coordinate origin of the front camera and the target point in the front vehicle to be analyzed, or the distance between the coordinate origin of the front camera and the target point in the target detection frame of the front vehicle to be analyzed in the frame. The target point here refers to the point determined in the target detection frame that needs to be used to calculate the pixel distance and the angle of the front vehicle. The target point may be but is not limited to the lower midpoint, upper midpoint or center point of the target detection frame. The center point here is the midpoint between the lower midpoint and the upper midpoint. Preferably, since the lower midpoint of the target detection frame is closest to the vehicle and has a smaller deformation, the error can be effectively reduced. Therefore, in this example, the coordinate origin of the front camera and the lower midpoint of the target detection frame are determined as the pixel distance between the two.
[0081] As an example, the on-board controller can calculate the front vehicle angle between the front camera that collects the front vehicle video data and the front vehicle to be analyzed, specifically, the front vehicle angle between the front camera and the target detection frame of the front vehicle to be analyzed in the frame. The front vehicle angle here can be the angle between the line between the coordinate origin of the front camera and the target point in the front vehicle to be analyzed and the front axis where the coordinate origin is located, or it can be the angle between the line between the coordinate origin of the front camera and the target point in the target detection frame of the front vehicle to be analyzed in the frame and the front axis where the coordinate origin is located. For example, when the pixel distance between the front camera and the target detection frame is the pixel distance between the coordinate origin of the front camera and the lower midpoint of the target detection frame, its front vehicle angle is the angle between the line between the coordinate origin of the front camera and the lower midpoint of the target detection frame and the front axis.
[0082] As an example, in step S203, after obtaining the pixel distance between the front camera and the target detection frame and the angle of the leading vehicle, the on-board controller can adopt a pre-set screening and analysis strategy to analyze and judge the pixel distance and the angle of the leading vehicle, and determine one of the at least one leading vehicles to be analyzed that meets the preset screening conditions as a target leading vehicle that can be used to assist in road condition detection.
[0083] In this embodiment, target detection is performed on the front camera data of the vehicle to determine at least one front vehicle to be analyzed. The pixel distance between the front camera of the vehicle and the front vehicle to be analyzed and the front vehicle angle are used to analyze and determine whether each front vehicle to be analyzed meets preset screening conditions that can be used to assist in road condition detection. When the preset screening conditions are met, the front vehicle to be analyzed can be determined as the target front vehicle to facilitate subsequent tracking of the target front vehicle, so as to use the front vehicle vibration data of the target front vehicle to accurately predict the road condition that the vehicle is about to enter, thereby ensuring the road condition detection effect.
[0084] In one embodiment, if Figure 3 As shown, step S203, that is, if the pixel distance and the front vehicle angle meet the preset screening conditions, the front vehicle to be analyzed is determined as the target front vehicle, including:
[0085] S301: If the pixel distance and the angle of the preceding vehicle to be analyzed meet the same lane condition, the preceding vehicle to be analyzed is determined as the preceding vehicle in the same lane;
[0086] S302: If the pixel distance and the angle of the preceding vehicle in the same lane corresponding to the preceding vehicle in the same lane satisfy the unobstructed condition, the preceding vehicle in the same lane is determined as the target preceding vehicle.
[0087] As an example, the preset screening conditions include the same lane condition and the no-blocking condition. The same lane condition is a pre-set condition for evaluating whether the front vehicle to be analyzed and the vehicle are in the same lane. The no-blocking condition is a pre-set condition for evaluating whether there are other vehicles blocking the front vehicle to be analyzed and the vehicle.
[0088] As an example, in step S301, the vehicle controller can determine whether the pixel distance and the angle of the front vehicle meet the preset same-lane condition based on the pixel distance between the front camera and each front vehicle to be analyzed and the angle of the front vehicle. If the same-lane condition is met, the front vehicle to be analyzed is determined as the same-lane front vehicle. In this example, the vehicle controller uses the target detection algorithm to perform target detection on the front camera data to determine at least one front vehicle to be analyzed, determine the pixel distance and the angle of the front vehicle between the front camera and each front vehicle to be analyzed, and analyze the pixel distance and the angle of the front vehicle to determine whether they meet the same-lane condition for determining that the vehicle and the front vehicle to be analyzed belong to the same lane. If they are met, the front vehicle to be analyzed is determined as the same-lane front vehicle.
[0089] As an example, in step S302, after determining the leading vehicle in the same lane from at least one leading vehicle to be analyzed, the on-board controller can determine whether the no-obstruction condition is met based on the pixel distance between the front camera and each leading vehicle in the same lane and the leading vehicle angle, that is, determine whether there is any other vehicle obstructing the vehicle in the same lane. If the no-obstruction condition is met, it is determined that there is no obstruction between the leading vehicle in the same lane and the vehicle, specifically, there is no obstruction between a leading vehicle in the same lane and the vehicle, and the leading vehicle in the same lane can be determined as the target leading vehicle.
[0090] In this embodiment, the front vehicle to be analyzed whose pixel distance and front vehicle angle satisfy both the same lane condition and the unobstructed condition is determined as the target front vehicle, which can ensure that the target front vehicle and the current vehicle are in the same lane, that is, the road condition that the target front vehicle travels through is the same as the road condition of the current vehicle; it also ensures that there are no other vehicles blocking the target front vehicle and the current vehicle, ensuring that the road condition that the current vehicle is about to enter is closest to the road condition of the target front vehicle, which helps to ensure the accuracy of road condition detection.
[0091] In one embodiment, if Figure 4 As shown, step S301, that is, if the pixel distance and the angle of the front vehicle meet the same lane condition, the front vehicle to be analyzed is determined as the front vehicle in the same lane, including:
[0092] S401: determining the actual distance between the front camera and each front vehicle to be analyzed according to the pixel distance;
[0093] S402: Decomposing the actual distance according to the angle of the front vehicle to determine the lateral distance between the front camera and each front vehicle to be analyzed;
[0094] S403: If the lateral distance is less than the first distance threshold, the front vehicle to be analyzed is determined to be a front vehicle in the same lane.
[0095] As an example, in step S401, the vehicle controller may determine the actual distance between the front camera and the front vehicle to be analyzed based on the calculated pixel distance between the front camera and the front vehicle to be analyzed. In this example, the vehicle controller may use a pre-set camera measurement calibration value to convert the pixel distance between the front camera and the front vehicle to be analyzed into the actual distance between the front camera and the front vehicle to be analyzed. The camera measurement calibration value is a pre-set calibration value, which is a calibration value used to implement the pixel distance and actual distance conversion process.
[0096] As an example, in step S402, the vehicle controller may decompose the actual distance between the front camera and the front vehicle to be analyzed based on the front vehicle angle between the front camera and the front vehicle to be analyzed, and obtain the lateral distance and longitudinal distance between the front camera and the front vehicle to be analyzed. For example, the front vehicle angle between the front camera and the front vehicle to be analyzed is α (e.g. Fig.10α2 and α3 in the figure), the actual distance between the front camera and the front vehicle to be analyzed is L (such as Fig.10 1-3), the lateral distance between the front camera and the front vehicle to be analyzed after decomposition and calculation is L*sinα; the longitudinal distance between the front camera and the front vehicle to be analyzed is L*cosα.
[0097] The first distance threshold is a pre-set distance threshold used to evaluate whether the same lane condition is met.
[0098] As an example, in step S403, after obtaining the lateral distance between the front camera and each front vehicle to be analyzed, the on-board controller can compare the lateral distance with a pre-set first distance threshold. When the lateral distance is less than the first distance threshold, it can be determined that the lateral distance between the two is small, and the judgment standard for determining that the two are traveling in the same lane is met. At this time, the corresponding front vehicle to be analyzed can be determined as the front vehicle in the same lane of the present vehicle; conversely, when the lateral distance is not less than the first distance threshold, it can be determined that the lateral distance between the two is large, and the judgment standard for determining that the two are traveling in the same lane is not met. It can be determined that the corresponding front vehicle to be analyzed is not the front vehicle in the same lane of the present vehicle.
[0099] In this embodiment, the actual distance between the front camera and the front vehicle to be analyzed is determined based on the pixel distance between the front camera and the front vehicle to be analyzed, and then the front vehicle angle between the front camera and the front vehicle to be analyzed is used to decompose the above actual distance to determine the lateral distance between the vehicle and the front vehicle to be analyzed during actual driving. The front vehicle to be analyzed whose lateral distance is less than the first distance threshold is determined as the front vehicle in the same lane, ensuring that the vehicle and the target front vehicle are traveling in the same lane, that is, ensuring that the road conditions they pass through are the same, which helps to ensure the accuracy of road condition detection based on the front vehicle vibration data of the target front vehicle.
[0100] In one embodiment, step S402, i.e., after decomposing the actual distance according to the angle of the front vehicle, further includes: determining the longitudinal distance between the front camera and each front vehicle to be analyzed;
[0101] Step 403, i.e., if the pixel distance and the angle of the preceding vehicle in the same lane meet the unobstructed condition, the preceding vehicle in the same lane is determined as the target preceding vehicle, including: comparing the longitudinal distances between the front camera and all preceding vehicles in the same lane, and determining the preceding vehicle in the same lane with the smallest longitudinal distance as the target preceding vehicle.
[0102] As an example, in step S402, the vehicle controller may decompose the actual distance between the front camera and the front vehicle to be analyzed based on the front vehicle angle between the front camera and the front vehicle to be analyzed, and obtain the lateral distance and longitudinal distance between the front camera and the front vehicle to be analyzed. For example, the front vehicle angle between the front camera and the front vehicle to be analyzed is α (e.g. Fig.10α2 and α3 in the figure), the actual distance between the front camera and the front vehicle to be analyzed is L (such as Fig.10 1-3), the lateral distance between the front camera and the front vehicle to be analyzed after decomposition and calculation is L*sinα; the longitudinal distance between the front camera and the front vehicle to be analyzed is L*cosα.
[0103] As an example, in step S403, after the vehicle controller has screened out at least one vehicle in front of the same lane according to the same lane condition, it can determine the longitudinal distance between the front camera and each vehicle in front of the same lane (i.e., obtain the longitudinal distance determined in the above step S402) based on the pixel distance corresponding to the front camera and each vehicle in front of the same lane and the angle of the front vehicle, compare the longitudinal distances between the front camera and all vehicles in front of the same lane, and determine the vehicle in front of the same lane with the smallest longitudinal distance as the target vehicle in front. Generally speaking, if the longitudinal distance between the front camera and a vehicle in front of the same lane is the smallest, it can be determined that there is no occlusion between the vehicle and the vehicle in front of the same lane by other vehicles in front of the same lane. Therefore, the vehicle in front of the same lane with the smallest longitudinal distance can be determined as the target vehicle in front, that is, the vehicle in front of the same lane closest to the vehicle is determined as the target vehicle in front.
[0104] In this embodiment, when the actual distance is decomposed according to the angle of the front vehicle, the lateral distance and longitudinal distance between the front camera and each front vehicle to be analyzed can be determined. After determining at least one front vehicle in the same lane according to the lateral distance, the longitudinal distance between the front camera and at least one front vehicle in the same lane can be compared, and the front vehicle in the same lane with the smallest longitudinal distance is determined as the target front vehicle, that is, the front vehicle in the same lane closest to the vehicle is determined as the target front vehicle. It can be understood that since the target front vehicle and the vehicle are in the same lane, it can be ensured that the road condition through which the target front vehicle travels is the same as the road condition of the vehicle. Among all the front vehicles in the same lane, the front vehicle in the same lane with the smallest longitudinal distance is determined as the target front vehicle to avoid the target front vehicle being blocked by other front vehicles in the same lane, thereby ensuring that the road condition that the vehicle is about to enter is closest to the road condition of the target front vehicle, which helps to ensure the accuracy of road condition detection.
[0105] In one embodiment, after determining the preceding vehicle in the same lane with the smallest longitudinal distance as the target preceding vehicle, the road condition detection method further includes: if the longitudinal distance between the front camera and the target preceding vehicle is less than a second distance threshold, executing a rear-end collision warning operation.
[0106] The second distance threshold is a preset threshold for evaluating whether there is a rear-end collision risk. The rear-end collision warning operation refers to an operation for giving an alarm when there is a rear-end collision risk, for example, a light reminder operation or a sound reminder operation may be performed.
[0107] As an example, when the on-board controller decomposes the actual distance according to the angle of the leading vehicle, it can determine the lateral distance and longitudinal distance between the front camera and each leading vehicle to be analyzed, determine at least one leading vehicle in the same lane according to the lateral distance, and determine the leading vehicle in the same lane with the smallest longitudinal distance as the target leading vehicle. It is also necessary to compare the longitudinal distance between the front camera and the target leading vehicle with a pre-set second distance threshold. If the longitudinal distance is less than the second distance threshold, it is determined that the longitudinal distance between the vehicle and the target leading vehicle in the same lane is small, and there is a greater risk of rear-end collision. Therefore, a rear-end collision alarm operation can be performed, such as a light reminder operation or a sound reminder operation, to remind the driver that there is a risk of rear-end collision and to drive carefully.
[0108] In one embodiment, if Figure 5 As shown, step S203, that is, tracking the target vehicle in front, includes:
[0109] S501: Based on the continuous frame front camera data, obtain a first license plate corresponding to the target front vehicle and a second license plate corresponding to the last tracked target;
[0110] S502: If the first license plate and the second license plate are the same, it is determined that the target front vehicle is the previous tracking target, and the front vehicle vibration data corresponding to the target front vehicle is obtained;
[0111] S503: If the first license plate and the second license plate are different, it is determined that the target front vehicle is not the last tracked target, and the front camera data of the vehicle is acquired.
[0112] The first license plate corresponding to the target vehicle in front refers to the license plate corresponding to the target vehicle in front identified in the vehicle front camera data. The second license plate corresponding to the last tracked target refers to the license plate corresponding to the last tracked target identified in the vehicle front camera data.
[0113] As an example, in step S501, when the vehicle-mounted controller tracks the target vehicle in front, it can identify the first license plate corresponding to the target vehicle in front that is currently being tracked and the second license plate corresponding to the last tracking target tracked at the last moment from the continuous frame vehicle front camera data. In this example, the vehicle-mounted controller can use a pre-set license plate recognition algorithm to perform license plate recognition on the vehicle front camera data to identify the first license plate corresponding to the target vehicle in front; and obtain the second license plate corresponding to the last tracking target tracked by the vehicle front camera data at the last moment from the vehicle-mounted cache, where the recognition process of the second license plate is the same as that of the first license plate. Generally speaking, when the vehicle front camera data is tracked in the last moment, the second license plate corresponding to the last tracking target will be identified and stored in the vehicle-mounted cache. Therefore, when the vehicle in front is tracked based on the continuous frame vehicle front camera data, the second license plate can be directly obtained from the vehicle-mounted cache without re-recognition, which can help save recognition time.
[0114] As an example, in step S502, the on-board controller may compare the first license plate of the target front vehicle with the second license plate corresponding to the previous tracking target. If the first license plate is the same as the second license plate, it can be determined that the target front vehicle is the previous tracking target, and the front vehicle vibration data corresponding to the target front vehicle can be obtained according to the execution in the subsequent step S102.
[0115] As an example, in step S503, the vehicle-mounted controller may compare the first license plate of the target front vehicle with the second license plate corresponding to the previous tracking target. If the first license plate is different from the second license plate, it can be determined that the target front vehicle is not the previous tracking target. The front camera data collected by the previous tracking target cannot be used together with the front camera data collected by the target front vehicle to calculate the front vehicle vibration data corresponding to the target front vehicle. Therefore, step S101 needs to be repeated, that is, obtaining the front camera data.
[0116] In this embodiment, by comparing the first license plate of the target front vehicle with the second license plate of the previous tracking target to see whether they are the same license plate, it is determined whether the target front vehicle is the previous tracking target. This can facilitate evaluation of whether the front vehicle vibration data of the target front vehicle can be calculated based on the front camera data corresponding to the target front vehicle and the front camera data corresponding to the previous tracking target, thereby ensuring the reliability of vibration detection based on the front camera data.
[0117] In one embodiment, if Figure 6 As shown, step S102, i.e. obtaining the preceding vehicle vibration data corresponding to the target preceding vehicle, includes:
[0118] S601: extracting feature points of the target vehicle ahead and determining target feature points corresponding to the target vehicle ahead;
[0119] S602: Analyze the continuous frames of vehicle front camera data based on the target feature points to obtain the front vehicle vibration data corresponding to the target front vehicle.
[0120] The target feature points refer to the feature points determined during the vibration detection process. As an example, when the target vehicle in front has a license plate, the four corner feature points of the license plate can be determined as its target feature points; when the target vehicle in front has no license plate, the four reserved installation positions corresponding to the license plate can be determined as its target feature points. As an example, fixed markers such as the lights or logos of the target vehicle in front can also be determined as its target feature points.
[0121] As an example, in step S601, after determining the target front vehicle, the onboard controller may extract feature points from the front vehicle camera data corresponding to the target front vehicle based on the front vehicle camera data to determine the target feature points corresponding to the target front vehicle. In this example, the target feature point corresponding to the target front vehicle may be one feature point in the target front vehicle or may be multiple feature points, and each target feature point corresponds to a feature point coordinate, including an X-axis coordinate of the feature point and a Y-axis coordinate of the feature point.
[0122] As an example, in step S602, after determining the target feature point corresponding to the target front vehicle, the onboard controller may determine the feature point coordinates of the target feature point in the continuous frame front camera data based on the target feature point; then, calculate the pixel coordinate change of the target feature point in the continuous frame front camera data according to the feature point coordinates of the target feature point, and determine the front vehicle vibration data corresponding to the target front vehicle according to the pixel coordinate change. For example, the onboard controller may calculate the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement in the two previous video frames according to the feature point coordinates of the target feature point in the current camera data and the feature point coordinates in the previous camera data, and determine the front vehicle vibration data corresponding to the continuous frame front camera data according to the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle.
[0123] In this embodiment, the front camera data collected by the front camera is two-dimensional data. The front and rear two frames of front camera data collected by the same target front vehicle are respectively determined as the previous camera data and the current camera data. Based on the previous camera data and the current camera data, the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement of the target front vehicle from the previous moment to the current moment can be determined. According to the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement; the vibration displacements in different directions determined at different moments are converted and processed to determine the front vehicle vibration data corresponding to the continuous frames of front camera data, so that the front vehicle vibration data can be used to analyze and determine the road condition passed by the target front vehicle, such as a flat road surface, a speed bump, a manhole cover / road pothole, etc.
[0124] In one embodiment, if Figure 7 As shown, step S601, i.e., extracting feature points of the target front vehicle to determine the target feature points corresponding to the target front vehicle, includes:
[0125] S701: extracting feature points of the target front vehicle to obtain at least two original feature points corresponding to the target front vehicle;
[0126] S702: Perform feature fusion on at least two original feature points corresponding to the target front vehicle to obtain a target feature point corresponding to the target front vehicle.
[0127] The original feature points refer to the feature points identified and determined from the front camera data of the vehicle.
[0128] As an example, in step S701, after determining the target vehicle in front, the onboard controller may extract feature points from the front camera data corresponding to the target vehicle in front, and obtain at least two original feature points corresponding to the target vehicle in front. In this example, the onboard controller may use a feature point detection method to detect the four corner feature points of the license plate or the feature points corresponding to the four reserved installation positions from the front camera data corresponding to the target vehicle in front, or detect at least two feature points corresponding to the headlights, vehicle logos or other fixed markers, and determine them as at least two original feature points corresponding to the target vehicle in front, and a feature point coordinate corresponding to each original feature point includes the feature point X-axis coordinate and the feature point Y-axis coordinate.
[0129] As an example, in step S702, after determining at least two original feature points corresponding to the target front vehicle, the on-board controller may perform feature fusion on the at least two original feature points. For example, the feature point coordinates corresponding to the at least two original feature points may be averaged to obtain the coordinate mean. The target feature point corresponding to the target front vehicle may be determined based on the coordinate mean.
[0130] In this embodiment, feature points of the target vehicle in front are extracted to obtain at least two original feature points instead of one original feature point. Since the positions of the at least two original feature points are mutually constrained, detecting at least two original feature points has a higher average accuracy than detecting one original feature point. Therefore, it is beneficial to improve the detection accuracy. In addition, detecting at least two original feature points only increases the computational complexity by a small amount in the last layer of the multi-layer network structure model, and the computational complexity of other layers remains unchanged. Generally, the number of model layers reaches more than 10 layers. Therefore, increasing the number of feature point detections has little effect on the time cost, but can improve the feature point detection accuracy.
[0131] In one embodiment, if Figure 8 As shown, step S602, i.e., analyzing the continuous frame front vehicle camera data based on the target feature points to obtain the front vehicle vibration data corresponding to the target front vehicle, includes:
[0132] S801: Based on the target feature points, identify the continuous frames of vehicle front camera data to determine the pixel coordinate changes corresponding to the target feature points;
[0133] S802: Based on the pixel coordinate change corresponding to the target feature point, the preceding vehicle vibration data corresponding to the target preceding vehicle is obtained.
[0134] As an example, in step S801, the vehicle-mounted controller can identify the continuous frames of front camera data where the target front vehicle is located based on the target feature points, and determine the feature point coordinates of the target feature points in each frame of the front camera data; then, based on the feature point coordinates of the continuous frames of the front camera data, determine the pixel coordinate changes corresponding to the target feature points.
[0135] As an example, in step S802, after the vehicle-mounted controller obtains the pixel coordinate changes corresponding to the target feature points at different times, it can perform coordinate conversion on the pixel coordinate changes to obtain the vibration data of the preceding vehicle. In this example, coordinate conversion of the target preceding vehicle refers to the process of converting the pixel coordinate changes of the target feature points of the target preceding vehicle at different times into the X-direction vibration signal and the Y-direction vibration signal of the preceding vehicle, that is, by converting the pixel coordinate changes corresponding to the target feature points at different times into the time series signal data formed by the vibration signals of the feature points vibrating in the x and y directions, wherein the sampling time interval of the feature point vibration signal is the inverse of the video frequency, so according to the sampling theorem, it is necessary to ensure that the signal sampling time interval is less than or equal to 0.5 times the time for the preceding vehicle to pass through the obstacle.
[0136] For example, the vehicle controller may determine the two frames of front camera data collected by the same target vehicle as the previous camera data and the current camera data, respectively, and may determine the feature point coordinates corresponding to the target feature point in the current camera data as the current feature coordinates, and the feature point coordinates in the previous camera data as the previous feature coordinates corresponding to the target feature point, the current feature coordinates include the current feature point X-axis coordinates and the current feature point Y-axis coordinates, the previous feature coordinates include the previous feature point X-axis coordinates and the previous feature point Y-axis coordinates, and the current feature coordinates and the previous feature coordinates both carry timestamps. Then, after obtaining the current feature coordinates and the previous feature coordinates corresponding to the target feature point, the vehicle controller may perform displacement calculation based on the current feature coordinates and the previous feature coordinates to determine the pixel coordinate change corresponding to the target feature point, where the pixel coordinate change includes the front vehicle X-axis vibration displacement and the front vehicle Y-axis vibration displacement, where the front vehicle X-axis vibration displacement and the front vehicle Y-axis vibration displacement both carry timestamps. Next, the on-board controller can perform coordinate conversion on the X-direction vibration displacement and Y-direction vibration displacement of the front vehicle carrying the timestamp, and specifically convert the X-direction vibration displacement and Y-direction vibration displacement of the front vehicle corresponding to the target feature point into front vehicle vibration data such as the X-direction vibration signal and the Y-direction vibration signal of the front vehicle.
[0137] In this embodiment, the front camera data and the previous camera data are identified based on the target feature points to determine their corresponding current feature coordinates and previous feature coordinates. The vibration displacement corresponding to the current moment can be calculated based on the current feature coordinates and the previous feature coordinates, so as to convert the vibration displacements at different moments into feature point vibration signals, thereby determining the vibration data of the preceding vehicle. The vibration displacement and coordinate conversion process has a small amount of calculation workload and does not require complicated processing, which helps to ensure the efficiency of obtaining the vibration data of the preceding vehicle.
[0138] In one embodiment, if Fig. 9 As shown, step S103, i.e., performing road condition detection according to the vibration data of the preceding vehicle and the vibration data of the vehicle, and obtaining the road condition detection result, includes:
[0139] S901: Feature encoding is performed on the vibration data of the preceding vehicle and the vibration data of the own vehicle respectively, and a first coding feature corresponding to the vibration data of the preceding vehicle and a second coding feature corresponding to the vibration data of the own vehicle are obtained;
[0140] S902: concatenating the first coding feature corresponding to the vibration data of the preceding vehicle and the second coding feature corresponding to the vibration data of the own vehicle to obtain a target coding feature;
[0141] S903: Perform road condition detection according to the target coding features and obtain road condition detection results.
[0142] As an example, the vehicle controller uses a pre-trained vibration signal prediction model to detect the road condition of the vibration data of the preceding vehicle and the vehicle itself. The vibration signal prediction model here is a multi-layered deep learning model, such as Fig.11 As shown, the deep learning model includes an encoder layer, a coding feature layer and a road condition prediction layer. The encoder layer includes multiple encoders, each of which is used to encode an input vibration signal and output a feature code. The encoder here is an encoder with timing information, for example, an encoder constructed by three LSTM layers. The coding feature layer is used to splice the coding features output by multiple encoder layers to form a target coding feature. The road condition prediction layer can receive the target coding feature input by the coding feature layer, perform prediction processing on the target coding feature, and determine the corresponding road condition detection result, that is, predict the road condition that the vehicle is about to enter, which can be any one of normal road conditions, speed bump road conditions, and manhole cover / road pit road conditions.
[0143] As an example, in step S901, the vehicle controller may input the vibration data of the preceding vehicle and the vibration data of the vehicle into the encoder layer of the vibration signal prediction model, and obtain the first encoding feature corresponding to the vibration data of the preceding vehicle and the second encoding feature corresponding to the vibration data of the vehicle, respectively. Here, the first encoding feature is the feature of the vibration data of the preceding vehicle after being encoded by the encoder, and the second encoding feature is the feature of the vibration data of the vehicle after being encoded by the encoder. Fig.11As shown, the vibration data of the preceding vehicle in the input encoder layer includes the X-direction vibration signal and the Y-direction vibration signal of the preceding vehicle, and two encoders are used to encode the X-direction vibration signal and the Y-direction vibration signal of the preceding vehicle respectively, so as to obtain the X-direction encoding feature of the preceding vehicle corresponding to the X-direction vibration signal and the Y-direction encoding feature of the preceding vehicle corresponding to the Y-direction vibration signal; the vibration data of the own vehicle in the input encoder layer includes the X-direction vibration signal, the Y-direction vibration signal and the Z-direction vibration signal of the own vehicle, and three encoders are used to encode the X-direction vibration signal, the Y-direction vibration signal and the Z-direction vibration signal of the own vehicle respectively, so as to obtain the X-direction encoding feature of the own vehicle corresponding to the X-direction vibration signal, the Y-direction encoding feature of the own vehicle corresponding to the Y-direction vibration signal and the Z-direction encoding feature of the own vehicle corresponding to the Z-direction vibration signal. For example, the vibration data of the input encoder layer include the vibration signal of the preceding vehicle in the X direction, the vibration signal of the preceding vehicle in the Y direction, the vibration signal of the own vehicle in the Z direction, the vibration signal of the own vehicle in the X direction and the vibration signal of the own vehicle in the Y direction. These vibration data can be respectively input into five encoders to obtain the coding features output by the five encoders, that is, the five coding features of the own vehicle in the X direction, the own vehicle in the Y direction, the own vehicle in the Z direction, the coding features of the preceding vehicle in the X direction and the coding features of the preceding vehicle in the Y direction.
[0144] As an example, in step S902, the on-board controller obtains the first coding feature corresponding to the vibration data of the preceding vehicle and the second coding feature corresponding to the vibration data of the vehicle itself, and inputs the first coding feature and the second coding feature into the coding feature layer of the vibration signal prediction model, so as to splice the first coding feature corresponding to the vibration data of the preceding vehicle and the second coding feature corresponding to the vibration data of the vehicle itself based on the pre-set coding order, and obtain the spliced target coding feature. The pre-set coding order here is the same as the coding order for splicing the training samples during the training of the vibration signal prediction model, so as to ensure that the spliced target coding feature can realize road condition prediction. For example, the splicing process can be performed according to the coding order of the vehicle's X-direction coding feature, the vehicle's Y-direction coding feature, the vehicle's Z-direction coding feature, the preceding vehicle's X-direction coding feature, and the preceding vehicle's Y-direction coding feature to obtain the target coding feature.
[0145] As an example, in step S903, after acquiring the target coding feature formed by splicing multiple coding features, the on-board controller inputs the target coding feature into the road condition prediction layer of the vibration signal prediction model, where the road condition prediction layer is a module for predicting road conditions based on the target coding feature. For example, the road condition prediction layer includes 5 LSTM layers and 1 fully connected layer, through which the target coding feature formed by splicing five coding features is detected for road conditions, and the road condition category that the vehicle is about to enter is determined, that is, the road condition category where the target preceding vehicle is currently located.
[0146] In this embodiment, feature encoding is performed on the vibration data of the preceding vehicle and the vibration data of the own vehicle respectively, and a first coding feature corresponding to the vibration data of the preceding vehicle and a second coding feature corresponding to the vibration data of the own vehicle are obtained. The first coding feature and the second coding feature are spliced to form a target coding feature, so that the target coding feature can reflect both the vibration conditions of the preceding vehicle and the vibration conditions of the own vehicle. By using the target coding feature to predict the road condition, it is possible to determine the road condition category of the target preceding vehicle at the current moment as the road condition category that the own vehicle is about to enter, and obtain the road condition detection result. It is possible to use the vibration data of the own vehicle to offset the interference of the preceding vehicle's vibration data caused by the uneven road surface passed by the preceding vehicle, thereby ensuring the accuracy of the road condition detection result.
[0147] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0148] In one embodiment, a vehicle-mounted controller is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the road condition detection method in the above embodiment is implemented, for example Figure 1 S101-S105 as shown, or Figures 2 to 8 To avoid repetition, it will not be described here.
[0149] In one embodiment, a vehicle is provided, which includes the vehicle-mounted controller in the above embodiment, and can implement the road condition detection method in the above embodiment, for example Figure 1 S101-S103 as shown, or Figures 2 to 9 To avoid repetition, it will not be described here.
[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the road condition detection method in the above embodiment is implemented, for example Figure 1 S101-S103 as shown, or Figures 2 to 9 To avoid repetition, it will not be described here.
[0151] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0152] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0153] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.
Claims
1. A road condition detection method, characterized in that: include: Get the front camera data of the vehicle; Tracking a target front vehicle according to the front camera data, and acquiring front vehicle vibration data corresponding to the target front vehicle; A road condition detection is performed based on the vibration data of the preceding vehicle and the vibration data of the own vehicle to obtain a road condition detection result.
2. The road condition detection method according to claim 1, characterized in that: The tracking of the target front vehicle according to the front vehicle camera data comprises: Performing target detection based on at least one of the vehicle front camera data to determine at least one front vehicle to be analyzed; Obtaining the pixel distance between the front camera and each of the front vehicles to be analyzed and the front vehicle angle; If the pixel distance and the front vehicle angle meet the preset screening conditions, the front vehicle to be analyzed is determined as a target front vehicle, and the target front vehicle is tracked.
3. The road condition detection method according to claim 2, characterized in that: If the pixel distance and the front vehicle angle meet a preset screening condition, determining the front vehicle to be analyzed as a target front vehicle includes: If the pixel distance and the angle of the preceding vehicle to be analyzed corresponding to the preceding vehicle to be analyzed meet the same lane condition, the preceding vehicle to be analyzed is determined as the preceding vehicle to be in the same lane; If the pixel distance and the front vehicle angle corresponding to the front vehicle in the same lane meet the unobstructed condition, the front vehicle in the same lane is determined as the target front vehicle.
4. The road condition detection method according to claim 3, characterized in that: If the pixel distance and the front vehicle angle satisfy the same lane condition, determining the front vehicle to be analyzed as the front vehicle in the same lane includes: Determining the actual distance between the front camera and each of the front vehicles to be analyzed according to the pixel distance; Decomposing the actual distance according to the front vehicle angle to determine the lateral distance between the front camera and each of the front vehicles to be analyzed; If the lateral distance is less than a first distance threshold, the front vehicle to be analyzed is determined to be a front vehicle in the same lane.
5. The road condition detection method according to claim 4, characterized in that: After decomposing the actual distance according to the front vehicle angle, the method further includes: determining the longitudinal distance between the front camera and each of the front vehicles to be analyzed; If the front camera data of the vehicle in front of the same lane meets the unobstructed condition, determining the vehicle in front of the same lane as the target vehicle in front includes: The longitudinal distances between the front camera and all the preceding vehicles in the same lane are compared, and the preceding vehicle in the same lane with the smallest longitudinal distance is determined as the target preceding vehicle.
6. The road condition detection method according to claim 5, characterized in that: After determining the preceding vehicle in the same lane with the smallest longitudinal distance as the target preceding vehicle, the road condition detection method further includes: If the longitudinal distance between the front camera and the target front vehicle is less than a second distance threshold, a rear-end collision warning operation is performed.
7. The road condition detection method according to claim 2, characterized in that: The tracking of the target front vehicle comprises: Based on the continuous frames of the front camera data of the vehicle, a first license plate corresponding to the target front vehicle and a second license plate corresponding to the last tracked target are obtained; If the first license plate and the second license plate are the same, it is determined that the target front vehicle is the last tracked target, and the front vehicle vibration data corresponding to the target front vehicle is obtained; If the first license plate and the second license plate are different, it is determined that the target front vehicle is not the last tracking target, and the step of acquiring the front vehicle camera data is performed.
8. The road condition detection method according to claim 1, characterized in that: The obtaining of the preceding vehicle vibration data corresponding to the target preceding vehicle includes: Extracting feature points of the target front vehicle to determine target feature points corresponding to the target front vehicle; Based on the target feature points, the continuous frames of the vehicle front camera data are analyzed to obtain the front vehicle vibration data corresponding to the target front vehicle.
9. The road condition detection method according to claim 8, characterized in that: The extracting feature points of the target front vehicle to determine the target feature points corresponding to the target front vehicle includes: Extracting feature points of the target front vehicle to obtain at least two original feature points corresponding to the target front vehicle; Feature fusion is performed on at least two original feature points corresponding to the target front vehicle to obtain a target feature point corresponding to the target front vehicle.
10. The road condition detection method according to claim 8, characterized in that: The analyzing the continuous frames of the vehicle front camera data based on the target feature points to obtain the front vehicle vibration data corresponding to the target front vehicle includes: Based on the target feature point, the front camera data of the vehicle is identified in consecutive frames to determine the pixel coordinate change corresponding to the target feature point; Based on the pixel coordinate change corresponding to the target feature point, the preceding vehicle vibration data corresponding to the target preceding vehicle is obtained.
11. The road condition detection method according to claim 1, characterized in that: The performing road condition detection according to the vibration data of the preceding vehicle and the vibration data of the own vehicle to obtain the road condition detection result includes: Performing feature encoding on the vibration data of the preceding vehicle and the vibration data of the own vehicle respectively, and obtaining a first encoding feature corresponding to the vibration data of the preceding vehicle and a second encoding feature corresponding to the vibration data of the own vehicle; splicing the first coding feature corresponding to the vibration data of the preceding vehicle and the second coding feature corresponding to the vibration data of the own vehicle to obtain a target coding feature; A road condition detection is performed according to the target coding feature to obtain a road condition detection result.
12. A vehicle-mounted controller, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the road condition detection method according to any one of claims 1 to 11 is implemented.
13. A vehicle, characterized in that: Includes the vehicle-mounted controller as claimed in claim 12.
14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the road condition detection method according to any one of claims 1 to 11 is implemented.
Citation Information
Patent Citations
Vehicle intelligent anti-collision pre-warning system based on VLC
CN107472244A
Vehicle driving abnormality detecting method based on vehicle networking dynamic comparison
CN109572594A
Vehicle driving road condition recognition method, device and equipment and storage medium
CN111862608A
Road supervision method and system based on vehicle vibration sense
CN112349109A
Automatic driving decision control system considering front vehicle encountering
CN112896188A
