Vehicle slip monitoring method and device, controller and vehicle

Abstract

The invention relates to a vehicle slip monitoring method and device, a controller and a vehicle. The method comprises the following steps: determining a first monitoring result based on visual data collected by a visual sensor; determining a second monitoring result based on motion data collected by a motion sensor; determining a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result are used for indicating the vehicle slip probability; and determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result. According to the invention, the slip prediction of the vehicle is carried out according to the multi-modal sensor data, the accuracy of the slip prediction result of the vehicle is improved, and the multi-modal sensor data comprises multi-dimensional monitoring of the visual data, the motion data and the auditory data, so that the single-side slip of the vehicle can be monitored, and the global slip of the vehicle can also be monitored.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, in particular to a vehicle slip monitoring method, device, controller and vehicle. BACKGROUND

[0002] In a complex and changeable road environment, such as rain, snow, ice, water and other low adhesion conditions, the vehicle is prone to lose control due to the sudden drop of tire and ground friction, which seriously threatens the safety of driving and riding. Therefore, it is necessary to accurately monitor the slip of the vehicle.

[0003] In the traditional technology, the wheel speed difference of the same shaft is determined by the wheel speed sensor to determine the single side slip. However, the traditional vehicle slip monitoring method has the problem of low monitoring accuracy. SUMMARY

[0004] Therefore, it is necessary to provide a vehicle slip monitoring method, device, controller and vehicle which can improve the monitoring accuracy.

[0005] In a first aspect, the present application provides a vehicle slip monitoring method, the method comprising:

[0006] determining a first monitoring result based on visual data collected by a visual sensor; determining a second monitoring result based on motion data collected by a motion sensor; determining a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result are used to indicate the probability of vehicle slip;

[0007] determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result.

[0008] In one embodiment, the determination of the slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result comprises:

[0009] determining weight information corresponding to the first monitoring result, the second monitoring result and the third monitoring result according to environmental information of the vehicle;

[0010] determining the slip prediction result of the vehicle according to the first monitoring result, the second monitoring result, the third monitoring result and the weight information.

[0011] In one embodiment, the determination of the weight information corresponding to the first monitoring result, the second monitoring result and the third monitoring result according to the environmental information of the vehicle comprises:

[0012] obtaining a weight basic value corresponding to the first monitoring result, the second monitoring result and the third monitoring result; the weight basic value is determined based on ideal environment information;

[0013] adjusting the weight basic value according to weather information to determine the weight information.

[0014] In one embodiment, the adjusting the preset weight basic value according to the weather information to determine the weight information comprises:

[0015] adjusting the preset weight basic value according to the weather information to determine the adjusted weight information;

[0016] obtaining current interference information of the vehicle;

[0017] determining a correction factor corresponding to the first monitoring result, the second monitoring result and the third monitoring result according to the interference information;

[0018] correcting the adjusted weight information according to the correction factor to determine the weight information.

[0019] In one embodiment, the determining the vehicle slip prediction result according to the first monitoring result, the second monitoring result, the third monitoring result and the weight information comprises:

[0020] weighting and summing the first monitoring result, the second monitoring result and the third monitoring result according to the weight information to determine a slip prediction probability;

[0021] when the slip prediction probability is greater than or equal to a probability threshold, determining that the slip prediction result is vehicle slip;

[0022] when the slip prediction probability is less than the probability threshold, determining that the slip prediction result is vehicle non-slip.

[0023] In one embodiment, the determining the first monitoring result based on visual data collected by a visual sensor comprises:

[0024] obtaining wheel speeds of each wheel of the vehicle collected by a wheel speed sensor;

[0025] determining a speed of the vehicle based on the visual data;

[0026] determining the first monitoring result according to the speed of the vehicle and the wheel speeds of each wheel.

[0027] In one embodiment, the determining the second monitoring result based on motion data collected by a motion sensor comprises:

[0028] acquire wheel speeds of each wheel of the vehicle collected by a wheel speed sensor, and determine wheel accelerations of each wheel based on the wheel speeds of each wheel;

[0029] determine an acceleration of the vehicle based on the motion data;

[0030] determine the second monitoring result according to the acceleration of the vehicle and the wheel accelerations of each wheel.

[0031] In one of the embodiments, the determining a third monitoring result based on the auditory data collected by the auditory sensor comprises:

[0032] filtering the auditory data to obtain first monitoring data;

[0033] performing frame windowing processing on the first monitoring data to divide a long signal into a short signal to obtain second monitoring data;

[0034] performing spectrum analysis processing on the monitoring data and determining a frequency band energy of each frame;

[0035] determining the third monitoring result according to the frequency band energy of each frame.

[0036] In one of the embodiments, in the case that the slip prediction result is that the vehicle slips, the method further comprises:

[0037] determining a slip rate of the vehicle according to the monitoring data of the wheel speed sensor and the visual data;

[0038] controlling other modules in the vehicle to perform warning according to the slip rate and a slip duration.

[0039] In a second aspect, the application further provides a slip monitoring device of a vehicle, comprising:

[0040] a first determining module configured to determine a first monitoring result based on visual data collected by a visual sensor, determine a second monitoring result based on motion data collected by a motion sensor, and determine a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result are used to indicate a vehicle slip probability;

[0041] a second determining module configured to determine a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result.

[0042] In a third aspect, the application further provides a vehicle-mounted controller, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0043] determining a first monitoring result based on visual data collected by a visual sensor; determining a second monitoring result based on motion data collected by a motion sensor; determining a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result being used to indicate a vehicle slip probability;

[0044] determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result.

[0045] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0046] determining a first monitoring result based on visual data collected by a visual sensor; determining a second monitoring result based on motion data collected by a motion sensor; determining a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result being used to indicate a vehicle slip probability;

[0047] determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result.

[0048] In a fifth aspect, the present application provides a computer program product comprising a computer program, the computer program being executed by a processor to implement the following steps:

[0049] determining a first monitoring result based on visual data collected by a visual sensor; determining a second monitoring result based on motion data collected by a motion sensor; determining a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result being used to indicate a vehicle slip probability;

[0050] determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result.

[0051] In a sixth aspect, the present application provides a vehicle comprising the vehicle-mounted controller according to the third aspect.

[0052] The vehicle slip monitoring method, device, controller and vehicle determine a first monitoring result based on visual data collected by a visual sensor, determine a second monitoring result based on motion data collected by a motion sensor, and determine a third monitoring result based on auditory data collected by an auditory sensor. The first monitoring result, the second monitoring result and the third monitoring result are used to indicate a vehicle slip probability. The vehicle slip prediction result is determined based on the first monitoring result, the second monitoring result and the third monitoring result. The vehicle slip prediction is performed based on multi-modal sensor data, which improves the accuracy of the vehicle slip prediction result. The multi-modal sensor data includes visual data, motion data and auditory data, which can be used to monitor single-side slip and global slip of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other related drawings can be obtained by those skilled in the art without creative labor.

[0054] Figure 1 An application environment diagram of the vehicle slip monitoring method in an embodiment;

[0055] Figure 2 A flowchart of the vehicle slip monitoring method in an embodiment;

[0056] Figure 3 A flowchart of the vehicle slip monitoring method in another embodiment;

[0057] Figure 4 A flowchart of the vehicle slip monitoring method in another embodiment;

[0058] Figure 5 A flowchart of the vehicle slip monitoring method in another embodiment;

[0059] Figure 6 A flowchart of the vehicle slip monitoring method in another embodiment;

[0060] Figure 7 A flowchart of the vehicle slip monitoring method in another embodiment;

[0061] Figure 8 A flowchart of the vehicle slip monitoring method in another embodiment;

[0062] Figure 9Flowchart of a method for monitoring skidding of a vehicle in another embodiment;

[0063] Figure 10 Flowchart of a method for monitoring skidding of a vehicle in another embodiment;

[0064] Figure 11 Flowchart of a method for monitoring skidding of a vehicle in another embodiment;

[0065] Figure 12 Block diagram of a device for monitoring skidding of a vehicle in an embodiment;

[0066] Figure 13 Internal structure diagram of an on-board controller in an embodiment. DETAILED DESCRIPTION

[0067] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0068] The method for monitoring skidding of a vehicle provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 . An on-board controller 102 communicates with a visual sensor 103, a motion sensor 104 and an auditory sensor 105. The on-board controller 102 acquires visual data collected by the visual sensor 103, motion data collected by the motion sensor 104 and auditory data collected by the auditory sensor 105, and determines a skidding prediction result of the vehicle according to the visual data, the motion data and the auditory data. The visual sensor 103 can be a vehicle-mounted camera, the motion sensor 104 can include an inertial measurement unit (IMU), and the auditory sensor 105 can be a microphone array.

[0069] In an embodiment, as shown in Figure 2 , a method for monitoring skidding of a vehicle is provided. The method is described by taking an on-board controller in Figure 1 as an example, which includes the following steps.

[0070] S201, determining a first monitoring result based on visual data collected by a visual sensor; determining a second monitoring result based on motion data collected by a motion sensor; and determining a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result are used to indicate a probability of skidding of the vehicle.

[0071] The visual sensor can be a front-view camera of the vehicle, the motion sensor can be an IMU (inertial measurement unit), and the auditory sensor can be a microphone array installed at the wheel end. Further, the visual data can be image data collected by the visual sensor, the motion data can be wheel acceleration or speed collected by the motion sensor, and the auditory data can be tire noise data collected by the auditory sensor during vehicle driving.

[0072] In the embodiments of the present application, the wheel speed sensor installed at the wheel end of the vehicle can also be used to obtain the wheel speed of each wheel, i.e., the speed of each wheel. Further, the first monitoring result is determined according to the speed of each wheel and the visual sensor, the second monitoring result is determined according to the speed of each wheel and the motion data, and the third monitoring result is determined by analyzing the auditory data.

[0073] Optionally, the same-axis wheel speed difference can be cross-verified according to the speed of each wheel, and if the wheel speed difference between each wheel is greater than a wheel speed difference threshold, it is determined that there is a unilateral slip probability, and this determination result is taken as the fourth monitoring result.

[0074] S202, determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result, and the third monitoring result.

[0075] As an optional implementation, if there is a vehicle slip probability greater than a preset slip probability in the first monitoring result, the second monitoring result, and the third monitoring result, it is determined that the slip prediction result is vehicle slip; or if there are at least two vehicle slip probabilities greater than the preset slip probability in the first monitoring result, the second monitoring result, and the third monitoring result, it is determined that the slip prediction result is vehicle slip; or if all vehicle slip probabilities in the first monitoring result, the second monitoring result, and the third monitoring result are greater than the preset slip probability, it is determined that the slip prediction result is vehicle slip.

[0076] As another optional implementation, the first monitoring result, the second monitoring result, and the third monitoring result are weighted and summed according to a preset weight value to obtain a target slip probability, and the slip prediction result of the vehicle is determined according to the target slip probability.

[0077] As another optional implementation, the first monitoring result, the second monitoring result, the third monitoring result, and the fourth monitoring result are weighted and summed according to a preset weight value to obtain a target slip probability, and the slip prediction result of the vehicle is determined according to the target slip probability.

[0078] The slip monitoring method of the vehicle determines a first monitoring result based on visual data collected by a visual sensor, determines a second monitoring result based on motion data collected by a motion sensor, and determines a third monitoring result based on auditory data collected by an auditory sensor. The first monitoring result, the second monitoring result, and the third monitoring result are used to indicate a vehicle slip probability. The vehicle slip prediction result is determined based on the first monitoring result, the second monitoring result, and the third monitoring result. The vehicle slip prediction is performed based on multi-modal sensor data, which improves the accuracy of the vehicle slip prediction result. The multi-modal sensor data includes visual data, motion data, and auditory data, which can be used to monitor single-side slip of the vehicle and global slip of the vehicle.

[0079] In one embodiment, an implementation of S202 is provided as shown in Figure 3 The determination of the vehicle slip prediction result based on the first monitoring result, the second monitoring result, and the third monitoring result includes:

[0080] S301, determining weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result according to environment information of the vehicle.

[0081] Optionally, the environment information of the vehicle can be at least one of current weather information, location information, and scene information.

[0082] In the embodiments of the present application, the weight information corresponding to the first monitoring result, the weight information corresponding to the second monitoring result, and the weight information corresponding to the third monitoring result are determined according to the environment information of the vehicle. Optionally, a corresponding relationship between the environment information and the weight information can be established in advance, so that the current weight information is determined according to the environment information of the vehicle.

[0083] S302, determining the vehicle slip prediction result according to the first monitoring result, the second monitoring result, the third monitoring result, and the weight information.

[0084] In the embodiments of the present application, the first monitoring result, the second monitoring result, and the third monitoring result can be weighted and summed according to the weight information to obtain the vehicle slip prediction result. Alternatively, the first monitoring result, the second monitoring result, and the third monitoring result can be sorted according to the weight information, and the monitoring result with the highest credibility is determined according to the order of the sorting result. The vehicle slip prediction result is determined according to the monitoring result.

[0085] In the above embodiments, the weight information corresponding to each monitoring result is determined according to the environment information of the vehicle, so that the credibility and accuracy of the weight information are improved.

[0086] In one embodiment, an implementation of S301 is provided as follows: Figure 4 As shown in the above, the "environmental information of the vehicle, the weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result is determined", which includes:

[0087] S401, acquiring a weight basic value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result; the weight basic value is determined based on ideal environmental information.

[0088] In the embodiments of the present application, the weight basic value can be determined based on the ideal environmental information by presetting a determination logic, or the weight basic value determined by a user based on the ideal environmental information is received in advance. Further, in the case of needing to monitor the vehicle slip, the weight basic value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result is read from the local.

[0089] Optionally, the ideal environmental information can include that the illumination intensity is greater than 500 lux and there is no occlusion, the acceleration fluctuation of the vehicle is less than 0.1g, and the environmental noise is less than 70dB; in the ideal environment, the weight information corresponding to the first monitoring result can be 0.5, the weight information corresponding to the second monitoring result can be 0.3, and the weight information corresponding to the third monitoring result is 0.2. Exemplarily, when the illumination intensity is greater than 500 lux and there is no occlusion, the weight information corresponding to the visual sensor is 0.5, the weight information corresponding to the motion sensor is 0.3, and the weight information corresponding to the auditory sensor is 0.2.

[0090] S402, adjusting the weight basic value according to the weather information to determine the weight information.

[0091] In the embodiments of the present application, a weight adjustment strategy corresponding to the weather information is established in advance, so that the current corresponding weight adjustment strategy is determined according to the current weather information, the weight basic value is adjusted according to the current corresponding weight adjustment strategy, and the weight information is determined.

[0092] Optionally, when there is a sensor failure, the weight information of the sensor is distributed to the monitoring result corresponding to the other sensors according to the weight information of the other sensors.

[0093] In the above application embodiments, the weight basic value is first determined according to the ideal environmental information, and then the weight basic value is adjusted according to the current weather information, which improves the matching degree and accuracy of the weight information and the current state, and the weight information does not need to be adjusted in the ideal environmental information, which improves the efficiency of determining the weight information.

[0094] In one embodiment, an implementation of S402 is provided as follows: Figure 5As shown, the above "adjusting the weight basic value according to the weather information to determine the weight information" includes:

[0095] S501, adjusting the preset weight basic value according to the weather information to determine the adjusted weight information.

[0096] The adjusted weight information includes first weight information, second weight information and third weight information, wherein the first weight information corresponds to the first monitoring result, the second weight information corresponds to the second monitoring result, and the third weight information corresponds to the third monitoring result.

[0097] In the embodiments of the present application, the current corresponding weight adjustment strategy is determined according to the current weather information, and the weight basic value is adjusted according to the current corresponding weight adjustment strategy to determine the adjusted weight information.

[0098] Exemplarily, when the weather or scene factors cause the visibility of the vehicle to change, the reliability of the visual sensor decreases, so it is necessary to lower the weight information of the first monitoring result of the visual sensor and raise the weight information of the monitoring result corresponding to other sensors. For example, when the wiper is activated and the contrast of the image collected by the visual sensor is less than the preset contrast (such as 0.3), the vehicle may be in heavy rain or fog weather, and the weight information of the first monitoring result is lowered by 0.2 from 0.5, and the weight information of the second monitoring result is raised to 0.6 from 0.3; when the suspended particulate matter is detected and the visibility is less than the preset distance (such as 50m), the vehicle may be in a sandstorm weather, and the weight information of the first monitoring result is lowered by 0.15 from 0.5, and the weight information of the third monitoring result is raised to 0.55 from 0.2; when the overexposure area of the image collected by the visual sensor is greater than the preset proportion (such as 50%), the vehicle may be in a strong backlight environment, and the weight information of the first monitoring result is lowered by 0.3 from 0.5, and the weight information of the second monitoring result is raised to 0.5 from 0.3.

[0099] Exemplarily, when the weather or the scene factor causes the vehicle to receive noise interference, the reliability of the auditory sensor decreases, thereby the weight information of the third monitoring result of the auditory sensor needs to be adjusted downward, and the weight information of the monitoring result corresponding to other sensors needs to be adjusted upward. For example, when the energy ratio of 1200Hz or above in the auditory data collected by the auditory sensor is greater than a preset ratio (such as 80%), the vehicle can be in a high-speed wind noise scene, the weight information of the third monitoring result is adjusted downward from 0.2 to 0.05, and the weight of the second monitoring result is adjusted upward from 0.3 to 0.45; when the energy of 200-500Hz in the auditory data collected by the auditory sensor suddenly increases and the windscreen wiper is activated, the vehicle can be in a scene of heavy rain hitting the vehicle body, the weight information of the third monitoring result is adjusted downward from 0.2 to 0.1, and the weight of the first monitoring result is adjusted upward from 0.5 to 0.6; when the positioning information of the vehicle is in a tunnel and the reverberation time of the auditory data is greater than a preset time length (such as 0.5s), the vehicle can be in a tunnel echo scene, the weight information of the third monitoring result is adjusted downward from 0.2 to 0.1, and the weight of the second monitoring result is adjusted upward from 0.3 to 0.4.

[0100] S502, obtain the interference information of the vehicle at present.

[0101] The interference information of the visual sensor can be visibility, the interference information of the auditory sensor can be signal-to-noise ratio, and the interference information of the motion sensor can be interference degree. Optionally, the interference degree is a vibration interference score with a value between 0 and 1.

[0102] In the embodiment of the application, the current visibility can be determined according to the image collected by the visual sensor; the signal-to-noise ratio can be determined according to the proportion of high-frequency noise in the auditory data collected by the auditory sensor and the total auditory data, for example, the signal-to-noise ratio can be the ratio of 800-1200hz signal to noise (total auditory data); and the current interference degree can be determined by receiving user input data.

[0103] S503, determine the correction factor corresponding to the first monitoring result, the second monitoring result and the third monitoring result according to the interference information.

[0104] In the embodiment of the application, the correction factor of the first monitoring result is a visual correction factor, which can be represented as k_v, and the visual correction factor can be linearly attenuated when the visibility is less than 100, that is, K_v = 0.5 × (visibility / 100); the correction factor of the second monitoring result is a motion correction factor, which can be represented as K_imu, and exemplarily, K_imu = 1-0.7×interference degree; and the correction factor of the third monitoring result is an auditory correction factor, which is constrained by a function to be between 0.2 and 1, and exemplarily, K_m= (signal-to-noise ratio-15) / 15.

[0105] S504, correcting the adjusted weight information according to the correction factor to determine the weight information.

[0106] In the embodiments of the present application, the weight information of the first monitoring result is W_v, the weight information corresponding to the second monitoring result is W_imu, and the weight information of the third monitoring result is k_v.

[0107] For example, W_v = k_v * first weight information; W_imu = k_imu * second weight information; and W_m = k_m * third weight information.

[0108] In the above embodiments, the preset weight basis value is adjusted according to the weather information, and the adjusted weight information is corrected according to the interference information to determine the weight information, so that the weight information is matched with the current environment and the interference of the state, and the accuracy of the weight information is further improved.

[0109] In one embodiment, an implementation of S302 is provided, as shown in Figure 6 According to the first monitoring result, the second monitoring result, and the third monitoring result and the weight information, the slip prediction result of the vehicle is determined.

[0110] S601, the first monitoring result, the second monitoring result, and the third monitoring result are weighted and summed according to the weight information to determine the slip prediction probability.

[0111] S602, when the slip prediction probability is greater than or equal to the probability threshold, the slip prediction result is determined as vehicle slip.

[0112] S603, when the slip prediction probability is less than the probability threshold, the slip prediction result is determined as vehicle not slip.

[0113] In the embodiments of the present application, the slip prediction probability is obtained by weighting and summing the first monitoring result, the second monitoring result, and the third monitoring result. For example, the slip prediction probability = W_v * first monitoring result + W_imu * second monitoring result + W_m * third monitoring result. Further, the probability threshold is set in advance. When the slip prediction probability is greater than or equal to the probability threshold, it indicates that there is a high probability of vehicle slip, and the slip prediction result is determined as vehicle slip. When the slip prediction probability is less than the probability threshold, it indicates that there is a small probability of vehicle slip, and the slip prediction result is determined as vehicle not slip.

[0114] Optionally, the probability threshold can be 80%.

[0115] In the above embodiments, the vehicle slip condition is evaluated in combination with the probability threshold, so that the reliability of the slip prediction result is higher.

[0116] In one embodiment, an implementation of S201 is provided, as shown in Figure 7 The "determining the first monitoring result based on the visual data collected by the visual sensor" includes:

[0117] S701, acquiring the wheel speeds of the vehicle collected by the wheel speed sensors.

[0118] In the embodiments of the present application, the wheel speed sensors are arranged on each wheel of the vehicle, and the wheel speed sensors collect the wheel speeds of the vehicle and send the wheel speeds to the vehicle controller.

[0119] Optionally, the wheel speed sensor can adopt the Hall effect or the magnetic resistance principle, and generate a pulse signal by monitoring the magnetic field change of the gear ring or the magnetic encoder rotating with the wheel; the magnetic pole outputs a square wave pulse every time it passes through the sensor, and the vehicle controller can determine the rotational speed and linear speed of the wheel in real time by counting the number of pulses in a unit time and combining the known wheel circumference.

[0120] S702, determining the speed of the vehicle based on the visual data.

[0121] In the embodiments of the present application, the actual ground displacement speed (the speed of the vehicle) can be determined by the optical flow algorithm, that is, the travel distance is indirectly calculated by estimating the motion vector of the pixel. Specifically, the image coordinates and the real coordinates need to be calibrated and mapped, for example, 1mm of the image is mapped to 10cm in the real world, and the specific mapping value is subject to the actual calibration result; therefore, under the assumption that the brightness is constant (the pixel gray scale is unchanged) in a very short time (Δt), the travel distance and the speed change of the vehicle can be determined by the displacement of the pixel in the very short time (Δt), for example, assuming that the pixel position after moving is (x+Δx, y+Δy), then Δx and Δy are mapped to the actual distance offset, and then divided by Δt, which is the speed of the actual vehicle in the time period. In addition, in order to reduce the error, the smaller the Δt is, the better, and the frame rate of the camera needs to be >30fps.

[0122] S703, determining the first monitoring result according to the speed of the vehicle and the wheel speeds.

[0123] Optionally, the average value of the wheel speeds can be determined first, and then the first monitoring result is determined according to the average value of the wheel speeds and the speed of the vehicle; or the speed of the vehicle and the wheel speeds can be compared to obtain a comparison result corresponding to each wheel speed, so as to determine the first monitoring result according to the multiple comparison results.

[0124] In the embodiment of the present application, the first difference value between the speed of the vehicle and the wheel speed of the vehicle is determined, when the first difference value is within a first threshold range, it is determined that the current vehicle has the possibility of slipping, and according to the numerical position of the first difference value within the first threshold range, the first monitoring result is determined.

[0125] For example, when the first threshold range is set to 3-10 km / h, it is linearly increased by 50-100%, for example, when the difference value is 3 km / h, the first monitoring result is 50%. When the first difference value is greater than 10 km / h, the first monitoring result is 100%.

[0126] In the above-mentioned embodiment of the present application, the reliable wheel speed of the vehicle is compared with the speed of the vehicle, so that the first monitoring result determined is more reliable.

[0127] In one embodiment, an implementation of S201 is provided, as shown in Figure 8 The above-mentioned "determining a second monitoring result based on the motion data collected by the motion sensor" includes:

[0128] S801, acquiring the wheel speed of each wheel of the vehicle collected by the wheel speed sensor, and determining the acceleration of each wheel based on the wheel speed of each wheel.

[0129] In the embodiment of the present application, each wheel of the vehicle is provided with a wheel speed sensor, the wheel speed sensor collects the wheel speed of each wheel of the vehicle, and sends the wheel speed of each wheel to the vehicle controller. Further, the vehicle controller can divide the speed difference (Δv) between the current time and the last time by the time interval (Δt) of the two measurements, i.e. acceleration a=Δv / Δt, according to the wheel speed of each wheel of the vehicle.

[0130] S802, determining the acceleration of the vehicle based on the motion data.

[0131] In the embodiment of the present application, the motion data can be the data collected by the IMU unit, the IMU unit is a device using an accelerometer and a gyroscope to measure the motion state of an object in three-dimensional space, the IMU unit collects the motion data of the vehicle during driving, and sends the motion data to the vehicle controller, the vehicle controller analyzes the motion data to determine the acceleration of the vehicle.

[0132] Optionally, the IMU unit at least includes a three-axis accelerometer and a three-axis gyroscope, the three-axis accelerometer is used to measure the linear acceleration of the vehicle in three directions (X, Y, Z), and the three-axis gyroscope is used to measure the angular velocity of the object rotating around the three axes.

[0133] S803, determining the second monitoring result according to the acceleration of the vehicle and the acceleration of each wheel.

[0134] Optionally, the average of the wheel acceleration can be determined first, and then the second monitoring result can be determined according to the average of the wheel acceleration and the acceleration of the vehicle; or the acceleration of the vehicle and the wheel acceleration can be compared to obtain a comparison result corresponding to each wheel acceleration, and then the second monitoring result can be determined according to the comparison results.

[0135] In the embodiment of the present application, the second difference between the acceleration of the vehicle and the wheel acceleration is determined, and when the second difference is within the second threshold range, it is determined that the current vehicle has the possibility of slipping, and the second monitoring result is determined according to the numerical position of the second difference within the second threshold range.

[0136] Exemplarily, when the second difference is greater than 0.4g, the second monitoring result is 100%, when the second difference is 0.3g, the second monitoring result is 50%, and when the second difference is 0.4g, the second monitoring result is 100%.

[0137] In the above application embodiment, the reliable wheel acceleration and the acceleration of the vehicle are compared, so that the determined second monitoring result is more reliable.

[0138] In one embodiment, an implementation of S201 is provided, as shown in Figure 9 The above "determining a third monitoring result based on the auditory data collected by the auditory sensor" includes:

[0139] S901, filtering the auditory data to obtain first monitoring data.

[0140] In the embodiment of the present application, when the tire is in contact with the road surface, two types of noise are mainly generated, which are tread pattern noise and stick-slip effect noise. The tread pattern noise is a low frequency of 200-500Hz, which is generated by the extrusion of the groove; the stick-slip effect noise is a high frequency of 800-1200Hz, which is caused by the micro sliding friction between the tire rubber and the road surface. It should be noted that when the wheel is normally rolling, the noise mechanism is continuous and uniform rubber deformation and release, and the energy is concentrated in 200-500Hz; when the wheel is slipping and idling, the noise mechanism is the high-frequency micro stick-slip vibration of the rubber and the road surface, and the energy suddenly increases in 800-1200Hz. That is, the tread pattern noise may be generated when the wheel is normally rolling, and the stick-slip effect noise may be generated when the wheel is slipping and idling.

[0141] In the embodiment of the present application, since the high-frequency audio of 800-1200hz is easy to cause attenuation in the air, the low-frequency noise may cover the high-frequency characteristics. Therefore, a pre-emphasis filter is used to improve the high-frequency gain and compensate for the transmission loss. That is, a high-pass filter is used to filter the auditory data to obtain the first monitoring data.

[0142] S902, frame windowing processing is performed on the first monitoring data, long signals are divided into short signals, and second monitoring data is obtained.

[0143] In the embodiment of the application, since the audible data includes many instantaneous audio signals when the vehicle slips, the long signals are divided into short time stable segments through frame processing to obtain the second monitoring data.

[0144] S903, frequency spectrum analysis processing is performed on the monitoring data, and the frequency band energy of each frame is determined.

[0145] In the embodiment of the application, the characteristic frequency band of 800-1200hz is identified, specifically, fast Fourier transform is performed on each frame of signal, and the power spectral density is calculated. The physical meaning is to quantify the distribution of signal frequency in the frequency domain. For example, assuming that the sampling rate is 44100hz, 0.1s of tire noise is collected, that is, 4410 sampling points are extracted, and further, whether there is a slip characteristic sampling point, that is, a component of 800-1200hz frequency, is analyzed, and 0.1s of data is further frame processed, for example, 0-20ms of data is analyzed to determine the amplitude and phase information of each frequency point. Since the frequency band of 800-1200hz needs to be concerned, the modulus value of the frequency point near the frequency band of 950hz is determined, for example, the modulus value average of the frequency point near the frequency band of 950hz can be determined as the frequency band energy of each frame.

[0146] S904, determining a third monitoring result according to the frequency band energy of each frame.

[0147] In the embodiment of the application, if the frequency band energy of each frame is greater than or equal to the energy threshold, it is determined that the third monitoring result is that the vehicle slips, that is, the slip probability included in the third monitoring result is 100%; if the frequency band energy of each frame is less than the energy threshold, it is determined that the third monitoring result is that the vehicle slips, that is, the slip probability included in the third monitoring result is 0.

[0148] It should be noted that since wind noise exists in the collection process, wind noise needs to be suppressed. A microphone can be arranged at the rearview mirror to perform coherent filtering, that is, the audio collected at the rearview mirror and the audio collected near the tire are analyzed for coherence. If the coherence is low, it is considered that the data is not wind noise and is input to the model for analysis.

[0149] In the above application embodiment, the audible data collected by the audible sensor is introduced to monitor the slip of the vehicle, so that the dimension of monitoring the slip of the vehicle is more extensive, and the accuracy of monitoring the slip of the vehicle is improved.

[0150] In one embodiment, as Figure 10As shown, in the case that the slip prediction result is that the vehicle slips, the slip monitoring method of the vehicle further includes:

[0151] S203, determining the slip rate of the vehicle according to the monitoring data of the wheel speed sensor and the visual data.

[0152] In the embodiment of the present application, the wheel speeds of each wheel of the vehicle are determined according to the monitoring data of the wheel speed sensor, and the vehicle speed is determined according to the visual data, and further, the slip rate of the vehicle is determined according to the wheel speed and the vehicle speed, for example, slip rate=(vehicle speed-wheel speed) / vehicle speed.

[0153] Optionally, the slip rate corresponding to each wheel can be determined, that is, four slip rates are determined according to the wheel speed and the vehicle speed of each wheel; or, the average value of the wheel speeds of each wheel can be determined, and the average value of the wheel speeds of each wheel is taken as the wheel speed, and the slip rate of the vehicle is determined, that is, one slip rate is determined.

[0154] S204, controlling other modules in the vehicle to give an alarm according to the slip rate and the slip duration.

[0155] As an optional implementation, if one slip rate is determined, when the slip rate is less than or equal to a slip rate threshold, the vehicle-mounted controller controls the instrument panel to display a yellow icon; and when the slip rate is greater than the slip rate threshold, the vehicle-mounted controller controls the voice module to output a voice to remind and suggest switching the four-wheel drive mode.

[0156] As another optional implementation, if four slip rates are determined, when there is a slip rate greater than the slip rate threshold, the vehicle-mounted controller controls the voice module to output a voice to remind and suggest switching the four-wheel drive mode; and when all the slip rates are less than or equal to the slip rate threshold, the vehicle-mounted controller controls the instrument panel to display a yellow icon.

[0157] Optionally, the slip rate threshold can be 10%.

[0158] In the embodiment of the present application, if the slip prediction results of more than N consecutive monitoring periods are all that the vehicle slips, the vehicle-mounted controller can control the voice module to output a voice to remind switching the sand or snow mode, and limit the torque.

[0159] In the above application embodiment, in the case that the slip prediction result is that the vehicle slips, other modules in the vehicle are controlled to give an alarm, and the vehicle slip condition is responded in time.

[0160] In one embodiment, a complete slip monitoring method of a vehicle is provided, as shown in the figure, the method includes: Figure 11

[0161] S1, acquiring the wheel speeds of each wheel of the vehicle collected by the wheel speed sensor.

[0162] ​S2, determine the speed of the vehicle based on the visual data.

[0163] S3, determine the first monitoring result according to the speed of the vehicle and the wheel speeds of each wheel.

[0164] S4, obtain the wheel speeds of each wheel of the vehicle collected by the wheel speed sensor, and determine the wheel accelerations of each wheel based on the wheel speeds of each wheel.

[0165] S5, determine the acceleration of the vehicle based on the motion data.

[0166] S6, determine the second monitoring result according to the acceleration of the vehicle and the wheel accelerations of each wheel.

[0167] S7, filter the auditory data to obtain the first monitoring data.

[0168] S8, perform frame windowing processing on the first monitoring data to divide long signals into short signals to obtain the second monitoring data.

[0169] S9, perform spectral analysis processing on the monitoring data and determine the frequency band energy of each frame.

[0170] S10, determine the third monitoring result according to the frequency band energy of each frame.

[0171] S11, obtain the weight base value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result.

[0172] S12, adjust the preset weight base value according to the weather information to determine the adjusted weight information.

[0173] S13, obtain the current interference information of the vehicle, and determine the correction factor corresponding to the first monitoring result, the second monitoring result, and the third monitoring result according to the interference information.

[0174] S14, correct the adjusted weight information according to the correction factor to determine the weight information.

[0175] S15, weight sum the first monitoring result, the second monitoring result, and the third monitoring result according to the weight information to determine the slip prediction probability.

[0176] S16, when the slip prediction probability is greater than or equal to the probability threshold, determine that the slip prediction result is that the vehicle slips; when the slip prediction probability is less than the probability threshold, determine that the slip prediction result is that the vehicle does not slip.

[0177] S17, in the case that the slip prediction result is that the vehicle slips, determine the slip rate of the vehicle according to the monitoring data of the wheel speed sensor and the visual data; and control other modules in the vehicle to perform warning according to the slip rate and the slip duration.

[0178] In the aforementioned vehicle skidding detection method, a first monitoring result is determined based on visual data collected by a visual sensor; a second monitoring result is determined based on motion data collected by a motion sensor; and a third monitoring result is determined based on auditory data collected by an auditory sensor. The first, second, and third monitoring results are used to indicate the probability of vehicle skidding. Based on the first, second, and third monitoring results, a vehicle skidding prediction result is determined. Predicting vehicle skidding based on multimodal sensor data improves the accuracy of the prediction results. Furthermore, the multimodal sensor data includes multi-dimensional monitoring of visual, motion, and auditory data, enabling monitoring not only unilateral skidding but also global skidding.

[0179] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0180] Based on the same inventive concept, this application also provides a vehicle skid detection device for implementing the vehicle skid detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more vehicle skid detection device embodiments provided below can be found in the limitations of the vehicle skid detection method described above, and will not be repeated here.

[0181] In one embodiment, such as Figure 12 As shown, a vehicle skidding detection device is provided, comprising: a first determining module 10 and a second determining module 11, wherein:

[0182] The first determining module 10 is used to determine a first monitoring result based on visual data collected by a visual sensor; determine a second monitoring result based on motion data collected by a motion sensor; and determine a third monitoring result based on auditory data collected by an auditory sensor. The first monitoring result, the second monitoring result, and the third monitoring result are used to indicate the probability of vehicle skidding.

[0183] The second determining module 11 is configured to determine the slip prediction result of the vehicle based on the first monitoring result, the second monitoring result, and the third monitoring result.

[0184] In an embodiment, the second determining module 11 includes a first determining unit and a second determining unit.

[0185] The first determining unit is configured to determine weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result according to environmental information of the vehicle.

[0186] The second determining unit is configured to determine the slip prediction result of the vehicle based on the first monitoring result, the second monitoring result, and the third monitoring result and the weight information.

[0187] In an embodiment, the first determining unit is specifically configured to obtain a weight basic value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result, and determine the weight information by adjusting the weight basic value based on weather information.

[0188] In an embodiment, the first determining unit is specifically configured to adjust a preset weight basic value based on the weather information to obtain adjusted weight information, obtain current interference information of the vehicle, determine a correction factor corresponding to the first monitoring result, the second monitoring result, and the third monitoring result based on the interference information, and correct the adjusted weight information based on the correction factor to obtain the weight information.

[0189] In an embodiment, the second determining unit is specifically configured to determine a slip prediction probability by weighted summing the first monitoring result, the second monitoring result, and the third monitoring result based on the weight information, determine that the slip prediction result is that the vehicle slips when the slip prediction probability is greater than or equal to a probability threshold, and determine that the slip prediction result is that the vehicle does not slip when the slip prediction probability is less than the probability threshold.

[0190] In an embodiment, the first determining module 10 includes a first obtaining unit, a third determining unit, and a fourth determining unit.

[0191] The first obtaining unit is configured to obtain wheel speeds of each wheel of the vehicle collected by a wheel speed sensor.

[0192] The third determining unit is configured to determine the speed of the vehicle based on the visual data.

[0193] The fourth determining unit is configured to determine the first monitoring result based on the speed of the vehicle and the wheel speeds of each wheel.

[0194] In one embodiment, the first determining module 10 comprises a second obtaining unit, a fifth determining unit and a sixth determining unit, wherein:

[0195] The second obtaining unit is configured to obtain wheel speeds of each wheel of the vehicle collected by the wheel speed sensor, and determine an acceleration of each wheel based on the wheel speeds.

[0196] The fifth determining unit is configured to determine an acceleration of the vehicle based on the motion data.

[0197] The sixth determining unit is configured to determine a second monitoring result according to the acceleration of the vehicle and the acceleration of each wheel.

[0198] In one embodiment, the first determining module 10 comprises a processing unit, a segmentation unit, an analysis unit and a seventh determining unit, wherein:

[0199] The processing unit is configured to perform filtering processing on the auditory data to obtain first monitoring data.

[0200] The segmentation unit is configured to perform frame segmentation and windowing processing on the first monitoring data, divide a long signal into a short signal, and obtain second monitoring data.

[0201] The analysis unit is configured to perform frequency spectrum analysis processing on the monitoring data, and determine a frequency band energy of each frame.

[0202] The seventh determining unit is configured to determine a third monitoring result according to the frequency band energy of each frame.

[0203] In one embodiment, the vehicle slip monitoring device further comprises a third determining module and a control module, wherein:

[0204] The third determining module is configured to determine a slip rate of the vehicle according to the monitoring data of the wheel speed sensor and the visual data.

[0205] The control module is configured to control other modules in the vehicle to perform warning according to the slip rate and the slip duration.

[0206] Each module in the vehicle slip monitoring device can be realized by software, hardware and a combination thereof in whole or in part. Each module can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to each module.

[0207] In one exemplary embodiment, a computer device is provided, which can be inserted into a controller, and an internal structure diagram thereof can be as shown in Figure 13As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for wired or wireless communication with external terminals, and wireless communication can be realized through WIFI, mobile cellular network, near field communication (NFC) or other technologies. The computer program is executed by the processor to realize a vehicle slip monitoring method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0208] Those skilled in the art can understand that, Figure 13 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0209] In one embodiment, a vehicle is provided, comprising an on-board controller as in the above embodiments.

[0210] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to realize the following steps:

[0211] Based on the visual data collected by the visual sensor, a first monitoring result is determined; based on the motion data collected by the motion sensor, a second monitoring result is determined; based on the auditory data collected by the auditory sensor, a third monitoring result is determined; the first monitoring result, the second monitoring result and the third monitoring result are used to indicate the probability of vehicle slip;

[0212] Based on the first monitoring result, the second monitoring result and the third monitoring result, a slip prediction result of the vehicle is determined.

[0213] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0214] According to the environmental information of the vehicle, the weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result is determined;

[0215] According to the first monitoring result, the second monitoring result, the third monitoring result, and the weight information, the slip prediction result of the vehicle is determined.

[0216] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0217] Obtain the weight basis value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result; the weight basis value is determined based on ideal environmental information;

[0218] Adjust the weight basis value according to the weather information to determine the weight information.

[0219] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0220] Adjust the preset weight basis value according to the weather information to determine the adjusted weight information;

[0221] Obtain the current interference information of the vehicle;

[0222] According to the interference information, the correction factor corresponding to the first monitoring result, the second monitoring result, and the third monitoring result is determined;

[0223] According to the correction factor, the adjusted weight information is corrected to determine the weight information.

[0224] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0225] According to the weight information, the first monitoring result, the second monitoring result, and the third monitoring result are weighted and summed to determine the slip prediction probability;

[0226] When the slip prediction probability is greater than or equal to the probability threshold, the slip prediction result is determined as the vehicle slipping;

[0227] When the slip prediction probability is less than the probability threshold, the slip prediction result is determined as the vehicle not slipping.

[0228] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0229] Obtain the wheel speed of each wheel of the vehicle collected by the wheel speed sensor;

[0230] Determine the speed of the vehicle based on the visual data;

[0231] determining a first monitoring result according to the speed of the vehicle and the wheel speeds of the wheels.

[0232] In an embodiment, the processor, when executing the computer program, further implements the following steps:

[0233] obtaining the wheel speeds of the wheels of the vehicle collected by the wheel speed sensor, and determining wheel accelerations of the wheels based on the wheel speeds of the wheels;

[0234] determining an acceleration of the vehicle based on the motion data;

[0235] determining a second monitoring result according to the acceleration of the vehicle and the wheel accelerations of the wheels.

[0236] In an embodiment, the processor, when executing the computer program, further implements the following steps:

[0237] filtering the hearing data to obtain first monitoring data;

[0238] performing frame windowing processing on the first monitoring data to divide a long signal into a short signal to obtain second monitoring data;

[0239] performing spectrum analysis processing on the monitoring data, and determining a frequency band energy of each frame;

[0240] determining a third monitoring result according to the frequency band energy of each frame.

[0241] In an embodiment, the processor, when executing the computer program, further implements the following steps:

[0242] determining a slip rate of the vehicle according to the monitoring data of the wheel speed sensor and the visual data;

[0243] controlling other modules in the vehicle to perform warning according to the slip rate and a slip duration.

[0244] In an embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the following steps:

[0245] determining a first monitoring result based on the visual data collected by the visual sensor, determining a second monitoring result based on the motion data collected by the motion sensor, and determining a third monitoring result based on the hearing data collected by the hearing sensor; the first monitoring result, the second monitoring result and the third monitoring result are used to indicate a slip probability of the vehicle;

[0246] determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result.

[0247] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0248] According to the environmental information of the vehicle, the weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result is determined;

[0249] According to the first monitoring result, the second monitoring result, the third monitoring result, and the weight information, the slip prediction result of the vehicle is determined.

[0250] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0251] Obtain the weight basis value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result; the weight basis value is determined based on ideal environmental information;

[0252] Adjust the weight basis value according to the weather information to determine the weight information.

[0253] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0254] Adjust the preset weight basis value according to the weather information to determine the adjusted weight information;

[0255] Obtain the current interference information of the vehicle;

[0256] According to the interference information, the correction factor corresponding to the first monitoring result, the second monitoring result, and the third monitoring result is determined;

[0257] According to the correction factor, the adjusted weight information is corrected to determine the weight information.

[0258] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0259] According to the weight information, the first monitoring result, the second monitoring result, and the third monitoring result are weighted and summed to determine the slip prediction probability;

[0260] When the slip prediction probability is greater than or equal to the probability threshold, the slip prediction result is determined as the vehicle slipping;

[0261] When the slip prediction probability is less than the probability threshold, the slip prediction result is determined as the vehicle not slipping.

[0262] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0263] Obtain the wheel speed of each wheel of the vehicle collected by the wheel speed sensor;

[0264] Determine the speed of the vehicle based on the visual data;

[0265] The first monitoring result is determined according to the speed of the vehicle and the wheel speeds of the wheels.

[0266] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0267] The wheel speeds of the wheels of the vehicle collected by the wheel speed sensor are acquired, and the wheel accelerations of the wheels are determined based on the wheel speeds of the wheels.

[0268] The acceleration of the vehicle is determined based on the motion data.

[0269] The second monitoring result is determined according to the acceleration of the vehicle and the wheel accelerations of the wheels.

[0270] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0271] The auditory data is filtered to obtain first monitoring data.

[0272] The first monitoring data is frame-windowed to divide a long signal into short signals to obtain second monitoring data.

[0273] The monitoring data is subjected to spectrum analysis processing, and the frequency band energy of each frame is determined.

[0274] The third monitoring result is determined according to the frequency band energy of each frame.

[0275] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0276] The slip rate of the vehicle is determined according to the monitoring data of the wheel speed sensor and the visual data.

[0277] The other modules in the vehicle are controlled to give an alarm according to the slip rate and the slip duration.

[0278] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by the processor, implements the following steps:

[0279] The first monitoring result is determined based on the visual data collected by the visual sensor; the second monitoring result is determined based on the motion data collected by the motion sensor; and the third monitoring result is determined based on the auditory data collected by the auditory sensor; the first monitoring result, the second monitoring result, and the third monitoring result are used to indicate the slip probability of the vehicle.

[0280] The slip prediction result of the vehicle is determined based on the first monitoring result, the second monitoring result, and the third monitoring result.

[0281] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0282] According to the environmental information of the vehicle, the weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result is determined;

[0283] According to the first monitoring result, the second monitoring result, the third monitoring result, and the weight information, the slip prediction result of the vehicle is determined.

[0284] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0285] Obtain the weight basis value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result; the weight basis value is determined based on ideal environmental information;

[0286] Adjust the weight basis value according to the weather information to determine the weight information.

[0287] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0288] Adjust the preset weight basis value according to the weather information to determine the adjusted weight information;

[0289] Obtain the current interference information of the vehicle;

[0290] According to the interference information, determine the correction factor corresponding to the first monitoring result, the second monitoring result, and the third monitoring result;

[0291] According to the correction factor, correct the adjusted weight information to determine the weight information.

[0292] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0293] According to the weight information, the first monitoring result, the second monitoring result, and the third monitoring result are weighted and summed to determine the slip prediction probability;

[0294] When the slip prediction probability is greater than or equal to the probability threshold, the slip prediction result is determined as the vehicle slipping;

[0295] When the slip prediction probability is less than the probability threshold, the slip prediction result is determined as the vehicle not slipping.

[0296] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0297] Obtain the wheel speed of each wheel of the vehicle collected by the wheel speed sensor;

[0298] Determine the speed of the vehicle based on the visual data;

[0299] The first monitoring result is determined according to the speed of the vehicle and the wheel speeds of the wheels.

[0300] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0301] The wheel speeds of the wheels of the vehicle collected by the wheel speed sensor are acquired, and the wheel accelerations of the wheels are determined based on the wheel speeds of the wheels.

[0302] The acceleration of the vehicle is determined based on the motion data.

[0303] The second monitoring result is determined according to the acceleration of the vehicle and the wheel accelerations of the wheels.

[0304] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0305] The auditory data is filtered to obtain first monitoring data.

[0306] The first monitoring data is frame-windowed to divide a long signal into short signals, to obtain second monitoring data.

[0307] The monitoring data is subjected to spectrum analysis processing, and the frequency band energy of each frame is determined.

[0308] The third monitoring result is determined according to the frequency band energy of each frame.

[0309] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0310] The slip rate of the vehicle is determined according to the monitoring data of the wheel speed sensor and the visual data.

[0311] The other modules in the vehicle are controlled to perform warning according to the slip rate and the slip duration.

[0312] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0313] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0314] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method of slip monitoring of a vehicle, characterized by, The method comprises: determining a first monitoring result based on visual data collected by a visual sensor, determining a second monitoring result based on motion data collected by a motion sensor, and determining a third monitoring result based on auditory data collected by an auditory sensor; the first monitoring result, the second monitoring result, and the third monitoring result are used to indicate a vehicle slip probability; determining a slip prediction result of the vehicle based on the first monitoring result, the second monitoring result, and the third monitoring result.

2. The method of claim 1, wherein, The determination of the slip prediction result of the vehicle based on the first monitoring result, the second monitoring result, and the third monitoring result comprises: determining weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result according to environmental information in which the vehicle is located; determining the slip prediction result of the vehicle according to the first monitoring result, the second monitoring result, the third monitoring result, and the weight information.

3. The method of claim 2, wherein, The determination of the weight information corresponding to the first monitoring result, the second monitoring result, and the third monitoring result according to the environmental information in which the vehicle is located comprises: obtaining a weight basic value corresponding to the first monitoring result, the second monitoring result, and the third monitoring result; the weight basic value is determined based on ideal environmental information; adjusting the weight basic value according to weather information to determine the weight information.

4. The method of claim 3, wherein, The adjustment of the preset weight basic value according to the weather information to determine the weight information comprises: adjusting the preset weight basic value according to the weather information to determine adjusted weight information; obtaining current interference information of the vehicle; determining a correction factor corresponding to the first monitoring result, the second monitoring result, and the third monitoring result according to the interference information; correcting the adjusted weight information according to the correction factor to determine the weight information.

5. The method of claim 2, wherein, The determination of the slip prediction result of the vehicle according to the first monitoring result, the second monitoring result, and the third monitoring result and the weight information comprises: weighting and summing the first monitoring result, the second monitoring result, and the third monitoring result according to the weight information to determine a slip prediction probability; when the slip prediction probability is greater than or equal to a probability threshold, determining that the slip prediction result is vehicle slip; when the slip prediction probability is less than the probability threshold, determining that the slip prediction result is vehicle non-slip.

6. The method of claim 1, wherein, The determination of the first monitoring result based on the visual data collected by the visual sensor comprises: obtaining wheel speeds of each wheel of the vehicle collected by a wheel speed sensor; determining a speed of the vehicle based on the visual data; determining the first monitoring result according to the speed of the vehicle and the wheel speeds of each wheel.

7. The method of claim 1, wherein, The determination of the second monitoring result based on the motion data collected by the motion sensor comprises: obtaining wheel speeds of each wheel of the vehicle collected by a wheel speed sensor, and determining wheel accelerations of each wheel based on the wheel speeds of each wheel; determining an acceleration of the vehicle based on the motion data; The second monitoring result is determined according to the acceleration of the vehicle and the respective wheel acceleration.

8. The method of claim 1, wherein, The third monitoring result is determined based on the auditory data collected by the auditory sensor, including: The auditory data is filtered to obtain first monitoring data; The first monitoring data is frame-windowed to divide long signals into short signals to obtain second monitoring data; The monitoring data is subjected to frequency spectrum analysis to determine the energy of each frame; The third monitoring result is determined according to the energy of each frame.

9. The method according to any one of claims 1 to 8, characterized in that, In the case that the slip prediction result is vehicle slip, the method further includes: The slip rate of the vehicle is determined according to the monitoring data of the wheel speed sensor and the visual data; Other modules in the vehicle are controlled to give an alarm according to the slip rate and the slip duration.

10. A slip monitoring device for a vehicle, characterized by The device includes: A first determining module configured to determine a first monitoring result based on the visual data collected by the visual sensor, determine a second monitoring result based on the motion data collected by the motion sensor, and determine a third monitoring result based on the auditory data collected by the auditory sensor; the first monitoring result, the second monitoring result and the third monitoring result are used to indicate the probability of vehicle slip; A second determining module configured to determine the slip prediction result of the vehicle based on the first monitoring result, the second monitoring result and the third monitoring result.

11. An in-vehicle controller comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 9.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 9.

13. A vehicle characterized by comprising: The vehicle-mounted controller according to claim 11. The vehicle-mounted controller according to claim 11.

Images