Road condition detection method, vehicle-mounted controller, vehicle, and storage medium

Abstract

The application discloses a road condition detection method, a vehicle-mounted controller, a vehicle and a storage medium. The method comprises the following steps: acquiring front camera data of a vehicle; tracking a target front vehicle according to the front camera data of the vehicle, and acquiring front vehicle vibration data corresponding to the target front vehicle; and performing road condition detection according to the front vehicle vibration data and vehicle vibration data, and acquiring a road condition detection result. The method can guarantee the accuracy of road condition detection, has low cost, is simple to implement, and is suitable for a wide range of scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road condition detection, and in particular to a road condition detection method, a vehicle-mounted controller, a vehicle and a storage medium. BACKGROUND

[0002] There are two existing road condition detection methods: one is a vision-based road condition detection method, that is, a camera is used to capture a road surface image, and a target detection or segmentation algorithm is used to detect targets such as speed bumps and manhole covers in the captured road surface image, so as to achieve the purpose of road condition detection. This vision-based road condition detection method cannot avoid the problem of road surface obstruction, for example, it cannot detect when the road surface is obstructed by a preceding vehicle. In addition, accumulated water, accumulated snow, fallen leaves and garbage on the road surface will also interfere with the road condition detection result, resulting in low detection accuracy, and the application scenario is greatly limited. The second is a vehicle-based sensing signal road condition detection method, that is, the vehicle speed and acceleration data collected by the inertial measurement device are used for road condition detection to detect whether the vehicle has passed through a speed bump and a manhole cover and other uneven road surfaces. In summary, the existing road condition detection method cannot accurately detect the road conditions in front of the vehicle. SUMMARY

[0003] The embodiments of the present application provide a road condition detection method, a vehicle-mounted controller, a vehicle and a storage medium to solve the problem that the existing road condition detection method cannot accurately detect the road conditions in front of the vehicle.

[0004] A road condition detection method, comprising:

[0005] acquiring vehicle front camera data;

[0006] tracking a target preceding vehicle according to the vehicle front camera data, and acquiring preceding vehicle vibration data corresponding to the target preceding vehicle;

[0007] performing road condition detection according to the preceding vehicle vibration data and vehicle vibration data, and acquiring a road condition detection result.

[0008] Preferably, the tracking of the target preceding vehicle according to the vehicle front camera data comprises:

[0009] performing target detection according to at least one vehicle front camera data to determine at least one preceding vehicle to be analyzed;

[0010] acquiring a pixel distance and a preceding vehicle angle between the vehicle front camera and each preceding vehicle to be analyzed;

[0011] if the pixel distance and the preceding vehicle angle meet a preset screening condition, the preceding vehicle to be analyzed is determined as the target preceding vehicle, and the target preceding vehicle is tracked.

[0012] Preferably, if the pixel distance and the front vehicle angle meet a preset screening condition, the front vehicle to be analyzed is determined as a target front vehicle, comprising:

[0013] If the pixel distance and the front vehicle angle corresponding to the front vehicle to be analyzed meet a same lane condition, the front vehicle to be analyzed is determined as a same lane front vehicle.

[0014] If the pixel distance and the front vehicle angle corresponding to the same lane front vehicle meet an unobstructed condition, the same lane front vehicle is determined as a target front vehicle.

[0015] Preferably, if the pixel distance and the front vehicle angle meet a same lane condition, the front vehicle to be analyzed is determined as a same lane front vehicle, comprising:

[0016] According to the pixel distance, an actual distance between the front camera of the vehicle and each of the front vehicles to be analyzed is determined;

[0017] According to the front vehicle angle, the actual distance is decomposed to determine a lateral distance between the front camera of the vehicle and each of the front vehicles to be analyzed;

[0018] If the lateral distance is less than a first distance threshold, the front vehicle to be analyzed is determined as a same lane front vehicle.

[0019] Preferably, after the actual distance is decomposed according to the front vehicle angle, a longitudinal distance between the front camera of the vehicle and each of the front vehicles to be analyzed is determined;

[0020] If the front camera data of the same lane front vehicle meets an unobstructed condition, the same lane front vehicle is determined as a target front vehicle, comprising:

[0021] The longitudinal distances between the front camera of the vehicle and all the same lane front vehicles are compared, and the same lane front vehicle with the smallest longitudinal distance is determined as a target front vehicle.

[0022] Preferably, after the same lane front vehicle with the smallest longitudinal distance is determined as a target front vehicle, the road condition detection method further comprises:

[0023] If the longitudinal distance between the front camera of the vehicle and the target front vehicle is less than a second distance threshold, a rear-end warning operation is performed.

[0024] Preferably, the target front vehicle is tracked, comprising:

[0025] Based on the front camera data of the continuous frames, a first license plate corresponding to the target front vehicle and a second license plate corresponding to a last tracking target are obtained;

[0026] If the first license plate and the second license plate are the same, it is determined that the target front vehicle is the last tracking target, and then the front vehicle vibration data corresponding to the target front vehicle is acquired.

[0027] If the first license plate and the second license plate are different, it is determined that the target front vehicle is not the last tracking target, and then the front vehicle camera data is acquired.

[0028] Preferably, the acquisition of the front vehicle vibration data corresponding to the target front vehicle comprises:

[0029] Feature point extraction is performed on the target front vehicle to determine the target feature point corresponding to the target front vehicle.

[0030] Based on the target feature point, the continuous frame of the front vehicle camera data is analyzed to acquire the front vehicle vibration data corresponding to the target front vehicle.

[0031] Preferably, the feature point extraction on the target front vehicle to determine the target feature point corresponding to the target front vehicle comprises:

[0032] Feature point extraction is performed on the target front vehicle to acquire at least two original feature points corresponding to the target front vehicle.

[0033] Feature fusion is performed on the at least two original feature points corresponding to the target front vehicle to acquire the target feature point corresponding to the target front vehicle.

[0034] Preferably, the analysis of the continuous frame of the front vehicle camera data based on the target feature point to acquire the front vehicle vibration data corresponding to the target front vehicle comprises:

[0035] Based on the target feature point, the continuous frame of the front vehicle camera data is identified to determine the pixel coordinate change corresponding to the target feature point.

[0036] Based on the pixel coordinate change corresponding to the target feature point, the front vehicle vibration data corresponding to the target front vehicle is acquired.

[0037] Preferably, the road condition detection according to the front vehicle vibration data and the vehicle vibration data to acquire the road condition detection result comprises:

[0038] The front vehicle vibration data and the vehicle vibration data are respectively feature-encoded to acquire the first encoded feature corresponding to the front vehicle vibration data and the second encoded feature corresponding to the vehicle vibration data.

[0039] The first encoded feature corresponding to the front vehicle vibration data and the second encoded feature corresponding to the vehicle vibration data are spliced to acquire the target encoded feature.

[0040] According to the target coding feature, road condition detection is performed, and a road condition detection result is obtained.

[0041] A vehicle-mounted controller includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the road condition detection method when executing the computer program.

[0042] A vehicle includes the vehicle-mounted controller.

[0043] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the road condition detection method.

[0044] The road condition detection method, vehicle-mounted controller, vehicle, and storage medium track a target front vehicle based on front camera data of the vehicle and obtain corresponding front vehicle vibration data, without the need for communication and data interaction with the target front vehicle, thereby ensuring the reliability of the front vehicle vibration data acquisition, reducing the dependence on the target front vehicle or a high-precision map, making it less susceptible to external factors, and being simple to implement and widely applicable. Then, road condition detection is performed based on the front vehicle vibration data and vehicle vibration data, and the vehicle vibration data is used to offset the interference of the vehicle vibration caused by passing over uneven road surfaces on the front vehicle vibration data, which helps to ensure the accuracy of the road condition detection. BRIEF DESCRIPTION OF DRAWINGS

[0045] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0046] Figure 1 is a flowchart of a road condition detection method in an embodiment of the present application;

[0047] Figure 2 is another flowchart of a road condition detection method in an embodiment of the present application;

[0048] Figure 3 is another flowchart of a road condition detection method in an embodiment of the present application;

[0049] Figure 4 is another flowchart of a road condition detection method in an embodiment of the present application;

[0050] Figure 5 is another flowchart of a road condition detection method in an embodiment of the present application;

[0051] Figure 6is another flow chart of the road condition detection method in an embodiment of the present application;

[0052] Figure 7 is another flow chart of the road condition detection method in an embodiment of the present application;

[0053] Figure 8 is another flow chart of the road condition detection method in an embodiment of the present application;

[0054] Figure 9 is another flow chart of the road condition detection method in an embodiment of the present application;

[0055] Figure 10 is a schematic diagram of a front vehicle to be analyzed in the visible range of the front camera of the vehicle in an embodiment of the present application;

[0056] Figure 11 is a schematic diagram of the vibration signal prediction model for road condition detection in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0058] The road condition detection method provided by the embodiments of the present application can be applied to a vehicle-mounted controller, which refers to a controller arranged on a vehicle, which can be an existing controller integrated with other functions on the vehicle, or a special controller dedicated to road condition detection on the vehicle.

[0059] In an embodiment, as shown in Figure 1 , a road condition detection method is provided, which is described by taking the vehicle-mounted controller as an example, and includes the following steps:

[0060] S101: acquiring front camera data of the vehicle;

[0061] S102: tracking a target front vehicle according to the front camera data of the vehicle, and acquiring front vehicle vibration data corresponding to the target front vehicle;

[0062] S103: performing road condition detection according to the front vehicle vibration data and the vehicle vibration data, and acquiring a road condition detection result.

[0063] The front camera data of the vehicle refers to real-time collected camera data of the front of the vehicle, specifically camera data collected by a front camera arranged on the vehicle.

[0064] As an example, in step S101, the vehicle-mounted controller can obtain continuous multiple frames of front camera data collected by the front camera of the host vehicle in real time during the driving of the vehicle. After obtaining the front camera data transmitted by the front camera in the form of a video stream, the vehicle-mounted controller can analyze and process the front camera data in the form of a video stream by using a video stream analysis method. Specifically, but not limited to, the video stream analysis method can use the video processing tool of the open source library Opencv to analyze and process the front camera data in the form of a video stream, so as to obtain the continuous multiple frames of analyzed front camera data. For example, the front camera data collected at the current time can be determined as current camera data, and the front camera data collected at the last time can be determined as last camera data.

[0065] Among them, the target front vehicle refers to a front vehicle determined by target detection and used for assisting in road condition detection.

[0066] As an example, in step S102, the vehicle-mounted controller can use a target detection algorithm to perform target detection on each of the front camera data in the front camera data and perform target tracking based on the continuous frames of front camera data, to determine at least one front vehicle to be analyzed tracked by the continuous frames of front camera data. Then, it is judged whether the front vehicle to be analyzed meets the preset screening condition for assisting in road condition detection. If the preset screening condition is met, the front vehicle to be analyzed is determined as a target front vehicle. Finally, the target front vehicle is analyzed based on the front camera data. For example, the front vehicle vibration data corresponding to the target front vehicle can be determined based on the change of the pixel coordinates of a certain feature point of the target front vehicle in the continuous frames of front camera data. The front vehicle to be analyzed herein refers to the vehicle in the target detection box corresponding to the target detection algorithm in the target detection process. The vehicle needs to be further analyzed whether it meets the preset screening condition. Specifically, the vehicle is one of the pre-set vehicles, such as a car, a bus and a truck.

[0067] For example, the front camera data collected by the front camera is two-dimensional data, the front and rear two frames of front camera data collected by the same target front vehicle are determined as the last camera data and the current camera data respectively, and the X-direction vibration displacement and the Y-direction vibration displacement of the target front vehicle from the last time to the current time can be determined based on the last camera data and the current camera data, so as to comprehensively determine the road condition passed by the target front vehicle based on the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle. For example, when the target front vehicle is driving on a normal road (such as a flat road), the X-direction vibration signal of the front vehicle collected by the target front vehicle in the X-axis direction and the Y-direction vibration signal collected by the target front vehicle in the Y-axis direction are smoothly changed, and the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle calculated according to the X-direction vibration signal and the Y-direction vibration signal are not obvious; when the target front vehicle passes through a speed bump, the left and right wheels of the front vehicle will pass through the speed bump at the same time, the X-direction vibration displacement of the front vehicle is not obvious, and the Y-direction vibration displacement of the front vehicle is obvious; when the target front vehicle passes through a manhole cover or a road pit, generally only one side of the wheel passes through, and the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle are obvious; therefore, the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle can be calculated according to the two X-direction vibration signals and the two Y-direction vibration signals determined by the front and rear two frames of front camera data, and the road condition passed by the target front vehicle can be determined according to different combinations of the X-direction vibration displacement and the Y-direction vibration displacement of the front vehicle, which can be but not limited to any one of normal road condition (i.e. middle part of flat road), speed bump road condition and manhole cover / road pit road condition.

[0068] In the formula, the vehicle vibration data refers to the vibration data of the vehicle collected in real time, and specifically refers to the vibration data collected by the vibration sensor arranged on the vehicle in real time. Generally, when the vehicle vibrates, instantaneous acceleration change will occur in the vehicle, at this time, the vibration sensor can collect vibration signals in X / Y / Z directions, i.e. the vibration data collected by the vibration sensor includes the X-direction vibration signal of the vehicle, the Y-direction vibration signal of the vehicle and the Z-direction vibration signal of the vehicle. The X-direction vibration signal of the vehicle refers to the vibration signal of the vehicle collected by the vibration sensor in the X-axis direction in real time. The Y-direction vibration signal of the vehicle refers to the vibration signal of the vehicle collected by the vibration sensor in the Y-axis direction in real time. The Z-direction vibration signal of the vehicle refers to the vibration signal of the vehicle collected by the vibration sensor in the Z-axis direction in real time.

[0069] As an example, in step S103, after obtaining the front vehicle vibration data corresponding to the target front vehicle, the vehicle-mounted controller can also obtain the vehicle vibration data collected by the vehicle sensor in real time, and then input the front vehicle vibration data and the vehicle vibration data into the pre-trained vibration signal prediction model, and predict the front vehicle vibration data and the vehicle vibration data through the vibration signal prediction model to obtain the road condition detection result output by the vibration signal prediction model. The vibration signal prediction model is a deep learning model containing multiple layers, and specifically is a deep learning model for identifying road conditions according to input vibration data.

[0070] In the vibration signal prediction model training process, the vehicle vibration data and the front vehicle vibration data collected when the vehicle drives through normal road conditions, deceleration zone road conditions, and manhole / road pit road conditions are used as training samples, and the vehicle vibration data, the front vehicle vibration data, and the corresponding road condition categories are determined as the training samples. The training samples are input into a deep learning model with a multi-layer structure for model training. When the deep learning model reaches the model convergence condition, a trained vibration signal prediction model is obtained, so that the vibration signal prediction model can accurately identify the road condition category to which the actual input vehicle vibration data and front vehicle vibration data belong. In this example, the deep learning model used to train the vibration signal prediction model can be an end-to-end model with time sequence information, which can be but is not limited to an LSTM, GRU, or Transformer model. In this example, the end-to-end model with time sequence information is used to train the vibration signal prediction model, and the model learning training is used to replace the traditional rule-based manual parameter adjustment prediction method, which has the problems of simple model deployment and wide application range.

[0071] In this embodiment, the vibration sensor for collecting vehicle vibration data and the front vehicle camera for collecting front vehicle image data are both loaded on the vehicle. When the vehicle drives through uneven road surfaces (such as deceleration zone road conditions and manhole / road pit road conditions), the vehicle vibration will cause the vibration sensor and the front vehicle camera to vibrate together. Therefore, the vehicle vibration data collected by the vibration sensor is used to offset the interference of the vibration caused by the vehicle driving through uneven road surfaces (such as deceleration zone road conditions and manhole / road pit road conditions) on the front vehicle vibration data, so as to ensure the accuracy of predicting the road condition that the vehicle is about to enter based on the front vehicle vibration data.

[0072] In this embodiment, the front vehicle camera data is used to track the target front vehicle and obtain the corresponding front vehicle vibration data, without the need for communication and data interaction with the target front vehicle, which ensures the reliability of the front vehicle vibration data acquisition, reduces the dependence on the target front vehicle or high-precision map, and makes it less susceptible to external factors, simple to implement, and widely applicable. Then, the road condition detection is performed according to the front vehicle vibration data and the vehicle vibration data, and the vehicle vibration data is used to offset the interference of the vibration caused by the vehicle driving through uneven road surfaces on the front vehicle vibration data, which helps to ensure the accuracy of the road condition detection.

[0073] In an embodiment, as shown in FIG. 1A, step S102, i.e., tracking the target front vehicle according to the front vehicle image data, includes: Figure 2

[0074] S201: performing target detection according to at least one front vehicle image data to determine at least one front vehicle to be analyzed;

[0075] S202: obtaining a pixel distance and a front vehicle angle between the front vehicle camera and each front vehicle to be analyzed;​

[0076] S203: If the pixel distance and the front vehicle angle meet the preset screening condition, the front vehicle to be analyzed is determined as a target front vehicle, and the target front vehicle is tracked.

[0077] As an example, in step S201, the vehicle-mounted controller can perform target detection on each of the vehicle front camera data by using a target detection algorithm, and determine the detection target in the at least one target detection frame as the front vehicle to be analyzed.

[0078] In this example, the vehicle-mounted controller can perform target detection on the vehicle front camera data by using a deep learning-based target detection algorithm, detect the detection targets of the car Car, the bus Bus and the truck Truck, and determine at least one front vehicle to be analyzed in the at least one target detection frame from the vehicle front camera data. The learning-based target detection algorithm used herein can use, but is not limited to, algorithms such as YOLO, SSD and Retinanet. The target detection frame herein is a rectangular frame in which the car Car, the bus Bus and the truck Truck are located in the vehicle front camera data. Understandably, the vehicle front camera data is the camera data collected in the visual range of the vehicle front camera. The situation of all vehicles in the visual range is collected. Considering that the feature points of small vehicles such as bicycles and motorcycles are not obvious and the motion trajectory is complex, it is not conducive to detection and tracking. This scheme uses the car Car, the bus Bus and the truck Truck as the detection target, so that the detection target has obvious feature points and simple motion trajectory, which helps to ensure the accuracy and detection efficiency of subsequent road condition detection.

[0079] The vehicle front camera refers to a camera that captures vehicle front camera data.

[0080] As an example, in step S202, the vehicle-mounted controller can calculate the pixel distance between the front camera capturing the front camera data and the front vehicle to be analyzed. In this example, the pixel distance can be the pixel distance between the coordinate origin of the front camera and the target detection frame of the front vehicle to be analyzed in the frame. For example, the vehicle-mounted controller can calculate the pixel distance between the coordinate origin of the front camera and the target point in the front vehicle to be analyzed by using, but not limited to, the Euclidean distance algorithm. The pixel distance can be the distance between the coordinate origin of the front camera and the target point in the target detection frame of the front vehicle to be analyzed in the frame. The target point refers to the point in the target detection frame that needs to be used to calculate the pixel distance and the front vehicle angle. The target point can be, but not limited to, the lower edge midpoint, the upper edge midpoint, or the center point of the target detection frame. The center point is the midpoint between the lower edge midpoint and the upper edge midpoint. Preferably, the lower edge midpoint of the target detection frame is closest to the host vehicle, and the deformation is small, which can effectively reduce the error. Therefore, in this example, the pixel distance between the coordinate origin of the front camera and the lower edge midpoint of the target detection frame is determined.

[0081] As an example, the vehicle-mounted controller can calculate the front vehicle angle between the front camera capturing the front camera data and the front vehicle to be analyzed. Specifically, the front vehicle angle between the front camera and the target detection frame of the front vehicle to be analyzed in the frame. The front vehicle angle can be the angle between the line connecting the coordinate origin of the front camera and the target point in the front vehicle to be analyzed and the front axis where the coordinate origin is located. Alternatively, the front vehicle angle can be the angle between the line connecting the coordinate origin of the front camera and the target point in the target detection frame of the front vehicle to be analyzed in the frame and the front axis where the coordinate origin is located. For example, when the pixel distance between the front camera and the target detection frame is the pixel distance between the coordinate origin of the front camera and the lower edge midpoint of the target detection frame, the front vehicle angle is the angle between the line connecting the coordinate origin of the front camera and the lower edge midpoint of the target detection frame and the front axis.

[0082] As an example, in step S203, after obtaining the pixel distance between the front camera and the target detection frame and the front vehicle angle, the vehicle-mounted controller can analyze and determine the pixel distance and the front vehicle angle by using the pre-set screening analysis strategy. At least one front vehicle that meets the pre-set screening condition is determined as the target front vehicle that can be used to assist in road condition detection.

[0083] In this embodiment, by target detection on the front camera data, at least one front vehicle to be analyzed is determined, and whether each front vehicle to be analyzed meets the preset screening condition for assisting in road condition detection is analyzed and determined by using the pixel distance and the front vehicle angle between the front camera and the front vehicle to be analyzed. When the preset screening condition is met, the front vehicle to be analyzed can be determined as a target front vehicle, so as to subsequently track and process the target front vehicle, so as to accurately predict the road condition to be entered by the vehicle by using the front vehicle vibration data of the target front vehicle, and to ensure the road condition detection effect.

[0084] In an embodiment, as shown in FIG. 2, step S203, i.e., if the pixel distance and the front vehicle angle meet the preset screening condition, the front vehicle to be analyzed is determined as a target front vehicle, including: Figure 3

[0085] S301: If the pixel distance and the front vehicle angle corresponding to the front vehicle to be analyzed meet the same lane condition, the front vehicle to be analyzed is determined as a same lane front vehicle.

[0086] S302: If the pixel distance and the front vehicle angle corresponding to the same lane front vehicle meet the non-occlusion condition, the same lane front vehicle is determined as a target front vehicle.

[0087] As an example, the preset screening condition includes the same lane condition and the non-occlusion condition. The same lane condition is a condition for evaluating whether the front vehicle to be analyzed and the vehicle are located in the same lane, which is set in advance. The non-occlusion condition is a condition for evaluating whether there is other vehicle between the front vehicle to be analyzed and the vehicle, which is set in advance.

[0088] As an example, in step S301, the vehicle-mounted controller can determine whether the pixel distance and the front vehicle angle between the front camera and each front vehicle to be analyzed meet the preset same lane condition, and if the same lane condition is met, the front vehicle to be analyzed is determined as a same lane front vehicle. In this example, the vehicle-mounted controller uses a target detection algorithm to perform target detection on the front camera data to determine at least one front vehicle to be analyzed, determines the pixel distance and the front vehicle angle between the front camera and each front vehicle to be analyzed, and analyzes the pixel distance and the front vehicle angle to determine whether the same lane condition that the vehicle and the front vehicle to be analyzed belong to the same lane is met. If the same lane condition is met, the front vehicle to be analyzed is determined as a same lane front vehicle.

[0089] ​As an example, in step S302, after determining the same-lane front vehicle from the at least one front vehicle to be analyzed, the vehicle-mounted controller can determine whether the unobstructed condition is met based on the pixel distance between the front camera and each same-lane front vehicle and the front vehicle angle, i.e., determine whether there is another vehicle obstructing between the same-lane front vehicle and the host vehicle. If the unobstructed condition is met, it is determined that there is no obstruction between the same-lane front vehicle and the host vehicle, specifically, there is no obstruction of another same-lane front vehicle between the same-lane front vehicle and the host vehicle, and the same-lane front vehicle can be determined as the target front vehicle.

[0090] In this embodiment, the front vehicle to be analyzed that meets both the same-lane condition and the unobstructed condition in terms of the pixel distance and the front vehicle angle is determined as the target front vehicle, which can ensure that the target front vehicle is in the same lane as the host vehicle, i.e., the road conditions passed by the target front vehicle are the same as those of the host vehicle, and that there is no other vehicle obstructing between the target front vehicle and the host vehicle, ensuring that the road conditions to be entered by the host vehicle are closest to those of the target front vehicle, which helps to ensure the accuracy of the road condition detection.

[0091] In an embodiment, as shown in FIG. 4, Figure 4

[0092] S401: determining the actual distance between the front camera and each front vehicle to be analyzed according to the pixel distance;

[0093] S402: decomposing the actual distance according to the front vehicle angle to determine the lateral distance between the front camera and each front vehicle to be analyzed;

[0094] S403: if the lateral distance is less than a first distance threshold, determining the front vehicle to be analyzed as a same-lane front vehicle.

[0095] As an example, in step S401, the vehicle-mounted controller can determine the actual distance between the front camera and the front vehicle to be analyzed according to the calculated pixel distance between the front camera and the front vehicle to be analyzed. In this example, the vehicle-mounted controller can convert the pixel distance between the front camera and the front vehicle to be analyzed into the actual distance between the front camera and the front vehicle to be analyzed by using a pre-set camera measurement calibration value. The camera measurement calibration value is a pre-set calibration value used to realize the conversion between the pixel distance and the actual distance.

[0096] As an example, in step S402, the vehicle-mounted controller can decompose the actual distance between the front camera and the front vehicle to be analyzed based on the front vehicle angle between the front camera and the front vehicle to be analyzed to obtain the lateral distance and the longitudinal distance between the front camera and the front vehicle to be analyzed. For example, the front vehicle angle between the front camera and the front vehicle to be analyzed is a (as shown in FIG. 3). Figure 10 ​the actual distance between the front camera and the front vehicle to be analyzed is L (e.g. Figure 10 the actual distance between the front camera and the front vehicle to be analyzed is L (e.g.

[0097] The first distance threshold is a distance threshold preset for evaluating whether the same lane condition is met.

[0098] As an example, in step S403, after obtaining the lateral distance between the front camera and each front vehicle to be analyzed, the vehicle-mounted controller can compare the lateral distance with the first distance threshold preset, and when the lateral distance is less than the first distance threshold, it can be determined that the lateral distance between the two is small, reaching the determination standard for determining that the two are driving on the same lane, and at this time, the corresponding front vehicle to be analyzed can be determined as the same lane front vehicle of the host vehicle. Conversely, when the lateral distance is not less than the first distance threshold, it can be determined that the lateral distance between the two is large, and the determination standard for determining that the two are driving on the same lane is not reached, and the corresponding front vehicle to be analyzed can be determined as not being the same lane front vehicle of the host vehicle.

[0099] In this embodiment, the actual distance between the front camera and the front vehicle to be analyzed is determined according to the pixel distance between the front camera and the front vehicle to be analyzed, and the actual distance is decomposed using the front vehicle angle between the front camera and the front vehicle to be analyzed to determine the lateral distance between the host vehicle and the front vehicle to be analyzed in the actual driving process. The front vehicle to be analyzed with a lateral distance less than the first distance threshold is determined as the same lane front vehicle, ensuring that the host vehicle and the target front vehicle are driving on the same lane, i.e., ensuring that the two vehicles pass through the same road conditions, which helps to ensure the accuracy of road condition detection based on the front vehicle vibration data of the target front vehicle.

[0100] In an embodiment, step S402, after decomposing the actual distance according to the front vehicle angle, further comprises: determining the longitudinal distance between the front camera and each front vehicle to be analyzed;

[0101] Step 403, if the same lane front vehicle corresponding to the pixel distance and the front vehicle angle satisfies the unblocked condition, the same lane front vehicle is determined as the target front vehicle, comprising: comparing the longitudinal distances between the front camera and all same lane front vehicles, and determining the same lane front vehicle with the smallest longitudinal distance as the target front vehicle.

[0102] As an example, in step S402, the vehicle-mounted controller can decompose the actual distance between the front camera and the front vehicle to be analyzed based on the front vehicle angle between the front camera and the front vehicle to be analyzed, to obtain the lateral distance and the longitudinal distance between the front camera and the front vehicle to be analyzed. For example, the front vehicle angle between the front camera and the front vehicle to be analyzed is a (e.g. Figure 10L (as shown in FIG. 1B, where a2 and a3 are the angles between the actual distance and the horizontal direction and the actual distance and the vertical direction, respectively), the actual distance between the front camera and the front vehicle to be analyzed is L (as shown in FIG. 1B, where a2 and a3 are the angles between the actual distance and the horizontal direction and the actual distance and the vertical direction, respectively) Figure 10 L*sin a; and the longitudinal distance between the front camera and the front vehicle to be analyzed is L*cos a.

[0103] As an example, in step S403, after the vehicle controller filters at least one same-lane front vehicle according to the same-lane condition, the vehicle controller can determine the longitudinal distance between the front camera and each same-lane front vehicle according to the pixel distance and the front vehicle angle corresponding to each same-lane front vehicle (i.e., obtain the longitudinal distance determined in step S402), compare the longitudinal distances between the front camera and all same-lane front vehicles, and determine the same-lane front vehicle with the smallest longitudinal distance as the target front vehicle. Generally, the same-lane front vehicle with the smallest longitudinal distance between the front camera and the same-lane front vehicle can be determined to have no occlusion by other same-lane front vehicles between the host vehicle and the same-lane front vehicle, and thus the same-lane front vehicle with the smallest longitudinal distance can be determined as the target front vehicle, i.e., the same-lane front vehicle closest to the host vehicle.

[0104] In this embodiment, when the actual distance is decomposed according to the front vehicle angle, the lateral distance and the longitudinal distance between the front camera and each front vehicle to be analyzed can be determined, and after at least one same-lane front vehicle is determined according to the lateral distance, the longitudinal distances between the front camera and the at least one same-lane front vehicle can be compared, and the same-lane front vehicle with the smallest longitudinal distance can be determined as the target front vehicle, i.e., the same-lane front vehicle closest to the host vehicle. Understandably, since the target front vehicle is in the same lane as the host vehicle, the road conditions passed by the target front vehicle can be ensured to be the same as the road conditions of the host vehicle, and among all same-lane front vehicles, the same-lane front vehicle with the smallest longitudinal distance can be determined as the target front vehicle, avoiding the target front vehicle being occluded by other same-lane front vehicles, thereby ensuring that the road conditions to be entered by the host vehicle are closest to the road conditions of the target front vehicle, which helps to ensure the accuracy of the road condition detection.

[0105] In an embodiment, after the same-lane front vehicle with the smallest longitudinal distance is determined as the target front vehicle, the road condition detection method further includes: if the longitudinal distance between the front camera and the target front vehicle is less than a second distance threshold, performing a rear-end warning operation.

[0106] The second distance threshold is a threshold value for evaluating whether there is a rear-end risk, which is set in advance. The rear-end warning operation refers to an operation for warning when the vehicle has a rear-end risk, for example, a light reminding operation or a sound reminding operation.

[0107] As an example, when the vehicle-mounted controller decomposes the actual distance according to the front vehicle angle, the lateral distance and the longitudinal distance between the front camera of the vehicle and each front vehicle to be analyzed can be determined, at least one front vehicle in the same lane is determined according to the lateral distance, and the front vehicle in the same lane with the smallest longitudinal distance is determined as the target front vehicle. In addition, the longitudinal distance between the front camera of the vehicle and the target front vehicle is compared with the second distance threshold set in advance. If the longitudinal distance is smaller than the second distance threshold, it is determined that the longitudinal distance between the vehicle and the target front vehicle in the same lane is small, and there is a great risk of rear-end collision. Therefore, a rear-end collision warning operation can be performed, such as a light reminding operation or a sound reminding operation, to remind the driver of the rear-end collision risk and drive carefully.

[0108] In an embodiment, as shown in FIG. 2, step S203, i.e., tracking the target front vehicle, includes: Figure 5

[0109] S501: Based on the continuous frame front camera data, the first license plate corresponding to the target front vehicle and the second license plate corresponding to the last tracking target are obtained.

[0110] S502: If the first license plate and the second license plate are the same, it is determined that the target front vehicle is the last tracking target, and then the front vehicle vibration data corresponding to the target front vehicle is obtained.

[0111] S503: If the first license plate and the second license plate are different, it is determined that the target front vehicle is not the last tracking target, and then the front camera data is obtained.

[0112] The first license plate corresponding to the target front vehicle refers to the license plate corresponding to the target front vehicle recognized in the front camera data. The second license plate corresponding to the last tracking target refers to the license plate corresponding to the last tracking target recognized in the front camera data.

[0113] As an example, in step S501, when the vehicle-mounted controller performs front vehicle tracking on the target front vehicle, the first license plate corresponding to the target front vehicle tracked at the current time and the second license plate corresponding to the last tracking target tracked at the last time can be recognized from the continuous frame front camera data. In this example, the vehicle-mounted controller can use a pre-set license plate recognition algorithm to recognize the license plate of the target front vehicle from the front camera data, and obtain the second license plate corresponding to the last tracking target tracked at the last time from the vehicle-mounted cache. The recognition process of the second license plate is the same as that of the first license plate. Generally, when the front vehicle tracking is performed on the front camera data at the last time, the second license plate corresponding to the last tracking target is recognized and stored in the vehicle-mounted cache. Therefore, when the front vehicle tracking is performed based on the continuous frame front camera data, the second license plate can be directly obtained from the vehicle-mounted cache without the need for further recognition, which can help to save recognition time.

[0114] ​As an example, in step S502, the on-board controller may compare the first license plate of the target front vehicle with the second license plate corresponding to the previous tracking target. If the first license plate is the same as the second license plate, it can be determined that the target front vehicle is the previous tracking target, and the front vehicle vibration data corresponding to the target front vehicle can be obtained according to the execution in the subsequent step S102.

[0115] As an example, in step S503, the on-board controller may compare the first license plate of the target front vehicle with the second license plate corresponding to the previous tracking target. If the first license plate is different from the second license plate, it can be determined that the target front vehicle is not the previous tracking target. The front camera data collected by the previous tracking target cannot be used together with the front camera data collected by the target front vehicle to calculate the front vehicle vibration data corresponding to the target front vehicle. Therefore, step S101 needs to be repeated, that is, the front camera data needs to be obtained.

[0116] In this embodiment, by comparing the first license plate of the target front vehicle with the second license plate of the previous tracking target to see whether they are the same license plate, it is determined whether the target front vehicle is the previous tracking target. This can facilitate evaluation of whether the front vehicle vibration data of the target front vehicle can be calculated based on the front camera data corresponding to the target front vehicle and the front camera data corresponding to the previous tracking target, thereby ensuring the reliability of vibration detection based on the front camera data.

[0117] In one embodiment, if Figure 6 As shown, step S102, i.e., obtaining the preceding vehicle vibration data corresponding to the target preceding vehicle, includes:

[0118] S601: Extract feature points of the target vehicle in front and determine target feature points corresponding to the target vehicle in front;

[0119] S602: Analyze the continuous frames of vehicle front camera data based on the target feature points to obtain the vehicle front vibration data corresponding to the target vehicle front.

[0120] Target feature points are those identified during the vibration detection process. For example, if the target vehicle ahead has a license plate, the four corner feature points of the license plate can be used as target feature points. If the target vehicle ahead does not have a license plate, the four reserved mounting locations corresponding to the license plate can be used as target feature points. For example, fixed landmarks such as the vehicle ahead's headlights or logo can also be used as target feature points.

[0121] As an example, in step S601, after determining the target preceding vehicle, the onboard controller may perform feature point extraction on the preceding vehicle camera data corresponding to the target preceding vehicle based on the preceding vehicle camera data to determine a target feature point corresponding to the target preceding vehicle. In this example, the target feature point corresponding to the target preceding vehicle may be a single feature point or multiple feature points of the target preceding vehicle, each of which corresponds to a feature point coordinate, including an X-axis coordinate and a Y-axis coordinate of the feature point.

[0122] As an example, in step S602, after determining the target feature point corresponding to the target front vehicle, the vehicle-mounted controller can determine the feature point coordinates of the target feature point in the continuous frame of vehicle front camera data based on the target feature point; then, calculate the pixel coordinate change of the target feature point according to the feature point coordinates of the target feature point in the continuous frame of vehicle front camera data, and determine the front vehicle vibration data corresponding to the target front vehicle according to the pixel coordinate change. For example, the vehicle-mounted controller can calculate the X-direction vibration displacement and Y-direction vibration displacement of the front vehicle in the front and rear two frames of video frames according to the feature point coordinates of the target feature point in the current camera data and the feature point coordinates in the last camera data, and determine the front vehicle vibration data corresponding to the continuous frame of vehicle front camera data according to the X-direction vibration displacement and Y-direction vibration displacement of the front vehicle.

[0123] In this embodiment, the vehicle front camera collects two-dimensional vehicle front camera data, and the front and rear two frames of vehicle front camera data collected for the same target front vehicle are determined as the last camera data and the current camera data, respectively. The X-direction vibration displacement and Y-direction vibration displacement of the target front vehicle from the last time to the current time can be determined based on the last camera data and the current camera data, and the front vehicle vibration data corresponding to the continuous frame of vehicle front camera data can be determined according to the X-direction vibration displacement and Y-direction vibration displacement of the front vehicle. The vibration displacement in different directions determined at different times is converted and processed to determine the front vehicle vibration data corresponding to the continuous frame of vehicle front camera data, so as to analyze and determine the road conditions passed by the target front vehicle, such as flat road surface, speed bump, manhole / road pit, etc.

[0124] In an embodiment, as shown in Figure 7 Step S601, i.e., feature point extraction for the target front vehicle to determine the target feature point corresponding to the target front vehicle, includes:

[0125] S701: Extracting feature points for the target front vehicle to obtain at least two original feature points corresponding to the target front vehicle;

[0126] S702: Feature fusion is performed on the at least two original feature points corresponding to the target front vehicle to obtain the target feature point corresponding to the target front vehicle.

[0127] The original feature point refers to a feature point identified and determined from the vehicle front camera data.

[0128] As an example, in step S701, the vehicle-mounted controller can perform feature point extraction on the front vehicle camera data corresponding to the target front vehicle after determining the target front vehicle, to obtain at least two original feature points corresponding to the target front vehicle. In this example, the vehicle-mounted controller can use a feature point detection method to detect four corner feature points of a license plate or four feature points corresponding to the four reserved mounting positions, or detect at least two feature points corresponding to a vehicle light, a vehicle logo, or other fixed markers from the front vehicle camera data corresponding to the target front vehicle, and determine the at least two original feature points corresponding to the target front vehicle. Each original feature point corresponds to a feature point coordinate, including a feature point X-axis coordinate and a feature point Y-axis coordinate.

[0129] As an example, in step S702, after determining the at least two original feature points corresponding to the target front vehicle, the vehicle-mounted controller can perform feature fusion on the at least two original feature points, for example, perform mean value calculation on the feature point coordinates corresponding to the at least two original feature points to obtain a coordinate mean value, and determine a target feature point corresponding to the target front vehicle according to the coordinate mean value.

[0130] In this embodiment, at least two original feature points are obtained instead of one original feature point by performing feature point extraction on the target front vehicle. Since the positions of the at least two original feature points have mutual constraints, the average accuracy of detecting the at least two original feature points is higher than that of detecting one original feature point, thereby improving the detection accuracy. In addition, detecting the at least two original feature points only increases a small amount of calculation in the last layer of the multi-layer network structure model in terms of calculation complexity, and the calculation amount of other layers remains unchanged. Generally, the number of model layers reaches more than 10, so increasing the number of feature points detected has little effect on time consumption, but can improve the feature point detection accuracy.

[0131] In an embodiment, as shown in FIG. 6, Figure 8 Step S602, that is, based on the target feature point, analyzing the continuous frame front vehicle camera data to obtain front vehicle vibration data corresponding to the target front vehicle, includes:

[0132] S801: Based on the target feature point, identifying the continuous frame front vehicle camera data to determine the pixel coordinate change corresponding to the target feature point;

[0133] S802: Based on the pixel coordinate change corresponding to the target feature point, obtaining the front vehicle vibration data corresponding to the target front vehicle.

[0134] As an example, in step S801, the vehicle-mounted controller can identify the continuous frame front vehicle camera data in which the target front vehicle is located based on the target feature point, to determine the feature point coordinate of the target feature point in each frame of front vehicle camera data; and then, based on the feature point coordinates of the continuous frame front vehicle camera data, determine the pixel coordinate change corresponding to the target feature point.

[0135] As an example, in step S802, the vehicle-mounted controller can perform coordinate conversion on the pixel coordinate change of the target feature point after the pixel coordinate change of the target feature point corresponding to different time points is obtained, to obtain the front vehicle vibration data. In this example, the coordinate conversion of the target front vehicle refers to the process of converting the pixel coordinate change of the target feature point of the target front vehicle at different time points into the front vehicle X-direction vibration signal and the front vehicle Y-direction vibration signal, that is, converting the pixel coordinate change of the target feature point corresponding to different time points into a time sequence signal data formed by the feature point vibration signal of the x and y direction vibration, wherein the sampling time interval of the feature point vibration signal is the reciprocal of the video rate, and therefore according to the sampling theorem, the signal sampling time interval needs to be less than or equal to 0.5 times the time of the front vehicle passing through the obstacle.

[0136] For example, the vehicle-mounted controller can determine the front and rear two frames of vehicle front camera data collected by the same target front vehicle as the last camera data and the current camera data, respectively, can determine the feature point coordinates corresponding to the target feature point in the current camera data as the current feature coordinates, and can determine the feature point coordinates in the last camera data as the last feature coordinates corresponding to the target feature point. The current feature coordinates include the current feature point X-axis coordinate and the current feature point Y-axis coordinate, and the last feature coordinates include the last feature point X-axis coordinate and the last feature point Y-axis coordinate. Both the current feature coordinates and the last feature coordinates carry a timestamp. Then, after obtaining the current feature coordinates and the last feature coordinates corresponding to the target feature point, the vehicle-mounted controller can perform displacement calculation according to the current feature coordinates and the last feature coordinates to determine the pixel coordinate change of the target feature point. Here, the pixel coordinate change includes the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement. Both the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement carry a timestamp. Next, the vehicle-mounted controller can perform coordinate conversion on the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement carrying the timestamp, and specifically convert the front vehicle X-direction vibration displacement and the front vehicle Y-direction vibration displacement corresponding to the target feature point into front vehicle vibration data such as the front vehicle X-direction vibration signal and the front vehicle Y-direction vibration signal.

[0137] In this embodiment, the front vehicle camera data and the last camera data are identified based on the target feature point, the current feature coordinates and the last feature coordinates corresponding to the target feature point are determined, the vibration displacement corresponding to the current time point is calculated according to the current feature coordinates and the last feature coordinates, the vibration displacement at different time points is converted into a feature point vibration signal, and thus the front vehicle vibration data is determined. The vibration displacement and coordinate conversion process has less computational workload and does not need complex processing, which helps to ensure the efficiency of obtaining the front vehicle vibration data.

[0138] In an embodiment, as shown in FIG. 1, Figure 9 As shown in FIG. 1, step S103, i.e., performing road condition detection according to the front vehicle vibration data and the vehicle vibration data to obtain a road condition detection result, includes:

[0139] S901: Feature encoding is performed on the vibration data of the preceding vehicle and the vibration data of the own vehicle, respectively, to obtain a first coding feature corresponding to the vibration data of the preceding vehicle and a second coding feature corresponding to the vibration data of the own vehicle;

[0140] S902: Concatenate the first coding feature corresponding to the vibration data of the preceding vehicle and the second coding feature corresponding to the vibration data of the own vehicle to obtain a target coding feature;

[0141] S903: Perform road condition detection according to the target coding feature and obtain the road condition detection result.

[0142] As an example, the vehicle controller uses a pre-trained vibration signal prediction model to detect the road condition of the vibration data of the preceding vehicle and the vehicle itself. The vibration signal prediction model here is a multi-layer deep learning model, such as Figure 11 As shown, the deep learning model includes an encoder layer, a coding feature layer, and a road condition prediction layer. The encoder layer includes multiple encoders, each of which is used to encode an input vibration signal and output a feature code. The encoder here is an encoder with time series information, for example, it can be an encoder constructed by three LSTM layers. The coding feature layer is used to splice the coding features output by multiple encoder layers to form a target coding feature. The road condition prediction layer can receive the target coding feature input by the coding feature layer, perform prediction processing on the target coding feature, and determine the corresponding road condition detection result, that is, predict the road condition that the vehicle is about to enter, which can be any of normal road conditions, speed bump road conditions, manhole cover / road pothole conditions.

[0143] As an example, in step S901, the vehicle controller may input the vibration data of the preceding vehicle and the vibration data of the vehicle into the encoder layer of the vibration signal prediction model, and obtain the first coding feature corresponding to the vibration data of the preceding vehicle and the second coding feature corresponding to the vibration data of the vehicle, respectively. Here, the first coding feature is the feature of the vibration data of the preceding vehicle after being encoded by the encoder, and the second coding feature is the feature of the vibration data of the vehicle after being encoded by the encoder. Figure 11As shown, the front vehicle vibration data in the input encoder layer includes a front vehicle X-direction vibration signal and a front vehicle Y-direction vibration signal, two encoders are used to encode the front vehicle X-direction vibration signal and the front vehicle Y-direction vibration signal respectively, and the front vehicle X-direction vibration signal corresponds to a front vehicle X-direction encoding feature and the front vehicle Y-direction vibration signal corresponds to a front vehicle Y-direction encoding feature. The vehicle vibration data in the input encoder layer includes a vehicle X-direction vibration signal, a vehicle Y-direction vibration signal and a vehicle Z-direction vibration signal, three encoders are used to encode the vehicle X-direction vibration signal, the vehicle Y-direction vibration signal and the vehicle Z-direction vibration signal respectively, and the vehicle X-direction vibration signal corresponds to a vehicle X-direction encoding feature, the vehicle Y-direction vibration signal corresponds to a vehicle Y-direction encoding feature and the vehicle Z-direction vibration signal corresponds to a vehicle Z-direction encoding feature. For example, the vibration data of the input encoder layer includes a front vehicle X-direction vibration signal, a front vehicle Y-direction vibration signal, a vehicle Z-direction vibration signal, a vehicle X-direction vibration signal and a vehicle Y-direction vibration signal. These vibration data can be input into five encoders respectively, and the encoding features output by the five encoders are obtained, i.e., the vehicle X-direction encoding feature, the vehicle Y-direction encoding feature, the vehicle Z-direction encoding feature, the front vehicle X-direction encoding feature and the front vehicle Y-direction encoding feature are obtained.

[0144] As an example, in step S902, the vehicle-mounted controller obtains the first encoding feature corresponding to the front vehicle vibration data and the second encoding feature corresponding to the vehicle vibration data, inputs the first encoding feature and the second encoding feature into the encoding feature layer of the vibration signal prediction model, so as to splice the first encoding feature corresponding to the front vehicle vibration data and the second encoding feature corresponding to the vehicle vibration data based on the pre-set encoding order, and obtain the target encoding feature after splicing. The pre-set encoding order here is the same as the encoding order for splicing the training sample in the vibration signal prediction model training process, so as to ensure that the target encoding feature after splicing can realize road condition prediction. For example, the splicing process can be performed according to the encoding order of the vehicle X-direction encoding feature, the vehicle Y-direction encoding feature, the vehicle Z-direction encoding feature, the front vehicle X-direction encoding feature and the front vehicle Y-direction encoding feature, so as to obtain the target encoding feature.

[0145] As an example, in step S903, after obtaining the target encoding feature formed by splicing the plurality of encoding features, the vehicle-mounted controller inputs the target encoding feature into the road condition prediction layer of the vibration signal prediction model, and the road condition prediction layer here is a module for predicting the road condition according to the target encoding feature. For example, the road condition prediction layer includes 5 layers of LSTM layers and 1 layer of fully connected layers, and the target encoding feature formed by splicing the five encoding features is detected by the 5 layers of LSTM layers and the 1 layer of fully connected layers, so as to determine the road condition category of the vehicle to be driven into, i.e., the road condition category of the target front vehicle at the current time.

[0146] In the embodiment, the front vehicle vibration data and the vehicle vibration data are respectively encoded to obtain a first encoded feature corresponding to the front vehicle vibration data and a second encoded feature corresponding to the vehicle vibration data, the first encoded feature and the second encoded feature are spliced to form a target encoded feature, so that the target encoded feature can reflect the front vehicle vibration and the vehicle vibration, the road condition prediction is performed by using the target encoded feature, the road condition category in which the target front vehicle is currently located can be determined as the road condition category in which the vehicle is about to enter, the road condition detection result is obtained, the vehicle vibration data is used to offset the interference of the front vehicle vibration data caused by the uneven road surface passed by the front vehicle, and the accuracy of the road condition detection result is ensured.

[0147] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0148] In an embodiment, a vehicle-mounted controller is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the road condition detection method in the above embodiment when executing the computer program, for example Figure 1 S101-S105 shown in the above embodiment, or Figures 2 to 8 For the sake of brevity, details are not repeated here.

[0149] In an embodiment, a vehicle is provided, which includes the vehicle-mounted controller in the above embodiment, and the road condition detection method in the above embodiment can be implemented, for example Figure 1 S101-S103 shown in the above embodiment, or Figures 2 to 9 For the sake of brevity, details are not repeated here.

[0150] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the road condition detection method in the above embodiment, for example Figure 1 S101-S103 shown in the above embodiment, or Figures 2 to 9 For the sake of brevity, details are not repeated here.

[0151] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0153] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A road condition detection method, characterized in that: include: Get the front camera data of the vehicle; Tracking a target preceding vehicle according to the vehicle front camera data, and obtaining preceding vehicle vibration data corresponding to the target preceding vehicle; Performing a road condition detection based on the vibration data of the preceding vehicle and the vibration data of the own vehicle to obtain a road condition detection result includes: performing feature encoding on the vibration data of the preceding vehicle and the vibration data of the own vehicle respectively to obtain a first encoding feature corresponding to the vibration data of the preceding vehicle and a second encoding feature corresponding to the vibration data of the own vehicle; and concatenating the first encoding feature corresponding to the vibration data of the preceding vehicle and the second encoding feature corresponding to the vibration data of the own vehicle to obtain a target encoding feature; Perform road condition detection according to the target coding feature to obtain a road condition detection result.

2. The road condition detection method according to claim 1, wherein: Tracking the target vehicle ahead according to the vehicle-front camera data includes: Performing target detection based on at least one of the vehicle front camera data to determine at least one front vehicle to be analyzed; Obtaining the pixel distance and the front vehicle angle between the front camera and each of the front vehicles to be analyzed; If the pixel distance and the preceding vehicle angle meet a preset screening condition, the preceding vehicle to be analyzed is determined as a target preceding vehicle, and the target preceding vehicle is tracked.

3. The road condition detection method according to claim 2, wherein: If the pixel distance and the preceding vehicle angle meet a preset screening condition, determining the preceding vehicle to be analyzed as a target preceding vehicle includes: If the pixel distance and the angle of the preceding vehicle to be analyzed meet the same lane condition, the preceding vehicle to be analyzed is determined to be a preceding vehicle in the same lane; If the pixel distance and the preceding vehicle angle corresponding to the preceding vehicle in the same lane meet an unobstructed condition, the preceding vehicle in the same lane is determined as the target preceding vehicle.

4. The road condition detection method according to claim 3, wherein: If the pixel distance and the preceding vehicle angle satisfy the same lane condition, determining the preceding vehicle to be analyzed as a preceding vehicle in the same lane includes: Determining the actual distance between the front camera and each of the front vehicles to be analyzed based on the pixel distance; Decomposing the actual distance according to the front vehicle angle to determine the lateral distance between the front camera and each of the front vehicles to be analyzed; If the lateral distance is less than a first distance threshold, the preceding vehicle to be analyzed is determined to be a preceding vehicle in the same lane.

5. The road condition detection method according to claim 4, wherein: After decomposing the actual distance according to the front vehicle angle, the method further includes: determining a longitudinal distance between the front camera and each of the front vehicles to be analyzed; If the front camera data of the preceding vehicle in the same lane satisfies the unobstructed condition, determining the preceding vehicle in the same lane as the target preceding vehicle includes: The longitudinal distances between the front camera and all the preceding vehicles in the same lane are compared, and the preceding vehicle in the same lane with the smallest longitudinal distance is determined as the target preceding vehicle.

6. The road condition detection method according to claim 5, wherein: After determining the preceding vehicle in the same lane with the smallest longitudinal distance as the target preceding vehicle, the road condition detection method further includes: If the longitudinal distance between the front camera and the target vehicle in front is less than a second distance threshold, a rear-end collision warning operation is performed.

7. The road condition detection method according to claim 2, wherein: Tracking the target vehicle ahead includes: Based on the continuous frames of the vehicle front camera data, obtaining a first license plate corresponding to the target vehicle in front and a second license plate corresponding to the last tracked target; If the first license plate and the second license plate are the same, it is determined that the target preceding vehicle is the previous tracking target, and the preceding vehicle vibration data corresponding to the target preceding vehicle is obtained; If the first license plate and the second license plate are different, it is determined that the target front vehicle is not the previous tracking target, and the acquisition of the front vehicle camera data is performed.

8. The road condition detection method according to claim 1, wherein: The obtaining of the preceding vehicle vibration data corresponding to the target preceding vehicle includes: Extracting feature points of the target preceding vehicle to determine target feature points corresponding to the target preceding vehicle; Based on the target feature points, the continuous frames of the vehicle front camera data are analyzed to obtain the front vehicle vibration data corresponding to the target front vehicle.

9. The road condition detection method according to claim 8, wherein: The extracting feature points of the target preceding vehicle to determine the target feature points corresponding to the target preceding vehicle includes: Extracting feature points of the target preceding vehicle to obtain at least two original feature points corresponding to the target preceding vehicle; Feature fusion is performed on at least two original feature points corresponding to the target preceding vehicle to obtain a target feature point corresponding to the target preceding vehicle.

10. The road condition detection method according to claim 8, wherein: The analyzing the continuous frames of the vehicle front camera data based on the target feature points to obtain the front vehicle vibration data corresponding to the target front vehicle includes: Based on the target feature points, identifying the front camera data of the vehicle in consecutive frames, and determining the pixel coordinate changes corresponding to the target feature points; Based on the pixel coordinate changes corresponding to the target feature points, the preceding vehicle vibration data corresponding to the target preceding vehicle is obtained.

11. A vehicle-mounted controller comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the road condition detection method according to any one of claims 1 to 10 is implemented.

12. A vehicle, characterized in that: Including the vehicle-mounted controller according to claim 11.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the road condition detection method according to any one of claims 1 to 10 is implemented.

Images