Rough road detection method, device, storage medium, electronic device and vehicle

Abstract

The present invention relates to a rough road detection method, device, storage medium, electronic device and vehicle, belonging to the field of vehicles, and capable of accurately determining a rough area on a road surface. A rough road detection method comprises: acquiring an image of the environment in front of a vehicle; extracting features from the image of the environment in front of the vehicle; matching the extracted features with a pre-trained feature model to determine a road surface boundary and a rough area on the road surface; determining a target road surface according to the road surface boundary, and determining the rough information of the rough area on the target road surface by a mathematical model comparison method.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicles, and in particular, to a rough road detection method, device, storage medium, electronic equipment and vehicle. Background Art

[0002] The existing rough road detection method includes: collecting sampling values ​​from vibration sensitive signals, filtering out periodic anomalies and random anomalies through a statistical module and calculating statistical signals, then calculating first-order and second-order derivatives based on rate or time through a differential module, and finally determining whether there is a rough road condition based on the statistical signal or the first-order and second-order derivatives through a comparison module.

[0003] Since the existing detection method mainly samples through vibration sensitive signals and obtains rough road information through a series of calculations and comparisons, its calculation difficulty is high and the accuracy is low. Summary of the invention

[0004] The purpose of the present disclosure is to provide a rough road surface detection method, device, storage medium, electronic device and vehicle, which can accurately determine the rough area on the road surface.

[0005] According to a first embodiment of the present disclosure, a method for detecting a rough road surface is provided, comprising: acquiring an image of the environment in front of a vehicle; performing feature extraction on the image of the environment in front of the vehicle; matching the extracted features with a pre-trained feature model to determine a road surface boundary and a rough area on the road surface; determining a target road surface according to the road surface boundary, and determining rough information of the rough area on the target road surface by a mathematical model comparison method.

[0006] Optionally, the extracting features from the image of the environment in front of the vehicle includes: extracting features from the image of the environment in front of the vehicle using a deep learning algorithm.

[0007] Optionally, matching the extracted features with a pre-trained feature model to determine the road surface boundary and the rugged area on the road surface includes: matching the extracted features with the pre-trained feature model, and marking the features of the road surface boundary and the rugged area after successful matching to determine the road surface boundary and the rugged range information on the road surface; obtaining calibration parameters of an imaging device that captures images of the environment in front of the vehicle; determining the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface based on the calibration parameters; and determining the rugged area according to the rugged range information, the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface.

[0008] Optionally, the method of determining the rugged information of the rugged area on the road surface by using a mathematical model comparison method includes: establishing an ideal flat road surface model based on the determined road surface boundary, and establishing an actual road surface model based on the determined road surface boundary and the rugged area on the road surface; comparing the ideal flat road surface model with the actual road surface model to obtain a difference function between the ideal flat road surface model and the actual road surface model; and determining the ruggedness and width of the rugged area using the difference function.

[0009] Optionally, the method further includes: extracting obstacle features based on the front environment image of the vehicle, matching the extracted obstacle features with a pre-trained obstacle feature model to determine obstacles on the road surface; determining the world coordinates of the obstacles on the road surface according to the calibration parameters; and determining obstacle information of the obstacles on the road surface based on the world coordinates of the obstacles on the road surface.

[0010] Optionally, after determining the ruggedness information of the rugged area on the target road surface by the mathematical model comparison method, the method further comprises: determining a driving strategy of the vehicle based on the determined road surface boundary information, obstacle information on the road surface and ruggedness information.

[0011] Optionally, determining the driving strategy of the vehicle based on the determined road boundary information, obstacle information on the road surface and rugged information includes: determining a drivable area based on the determined road boundary information, obstacle information on the road surface and rugged information; when the drivable area includes a rugged area and the vehicle is located within the lateral range of the rugged area, determining the acceleration of the vehicle to be a=V0-Vt / T, wherein Vt=V0-V0×R×(1-F), T=D / (V0-Vt), wherein V0 represents the current speed of the vehicle; Vt represents the recommended speed of the vehicle in the drivable area; T represents the shortest time from the current speed to the recommended speed; R represents the positional relationship between the vehicle and the rugged area, when the vehicle is outside the lateral range of the rugged area, R=0, when the vehicle is within the lateral range of the rugged area, R=1; F represents the ruggedness of the rugged area, and D represents the longitudinal distance between the vehicle and the rugged area.

[0012] According to a second embodiment of the present disclosure, a rough road surface detection device is provided, including: an acquisition module for acquiring an image of the environment in front of a vehicle; a feature extraction module for extracting features from the image of the environment in front of the vehicle; a matching module for matching the extracted features with a pre-trained feature model to determine a road surface boundary and a rough area on the road surface; and a rough information determination module for determining a target road surface based on the road surface boundary, and determining the rough information of the rough area on the target road surface by a mathematical model comparison method.

[0013] Optionally, the feature extraction module is further used to extract features from the front environment image of the vehicle using a deep learning algorithm.

[0014] Optionally, the matching module includes: a matching and labeling submodule, which is used to match the extracted features with the pre-trained feature model, and label the features of the road surface boundary and the rugged area after successful matching, so as to determine the road surface boundary and the rugged range information on the road surface; a calibration parameter acquisition submodule, which is used to acquire the calibration parameters of an imaging device that captures images of the environment in front of the vehicle; a world coordinate determination submodule, which is used to determine the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface based on the calibration parameters; and a rugged area determination submodule, which is used to determine the rugged area based on the rugged range information, the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface.

[0015] Optionally, the rugged information determination module includes: a mathematical modeling submodule, used to establish an ideal flat road surface model based on the determined road surface boundary, and to establish an actual road surface model based on the determined road surface boundary and the rugged area on the road surface; a comparison submodule, used to compare the ideal flat road surface model with the actual road surface model to obtain a difference function between the ideal flat road surface model and the actual road surface model; and a rugged information determination submodule, used to determine the ruggedness and width of the rugged area using the difference function.

[0016] Optionally, the feature extraction module is also used to extract obstacle features based on the front environment image of the vehicle; the matching module is also used to match the extracted obstacle features with a pre-trained obstacle feature model to determine the obstacles on the road surface, determine the world coordinates of the obstacles on the road surface according to the calibration parameters, and determine the obstacle information of the obstacles on the road surface based on the world coordinates of the obstacles on the road surface.

[0017] Optionally, the device further comprises: a driving strategy determination module, configured to determine the driving strategy of the vehicle based on the determined road boundary information, obstacle information and ruggedness information on the road surface.

[0018] Optionally, the driving strategy determination module includes: a drivable area determination submodule, used to determine the drivable area based on the determined road boundary information, obstacle information on the road surface and rugged information; an acceleration determination submodule, used to determine the acceleration of the vehicle as a=V0-Vt / T when the drivable area includes a rugged area and the vehicle is located within the lateral range of the rugged area, wherein Vt=V0-V0×R×(1-F), T=D / (V0-Vt), wherein V0 represents the current speed of the vehicle; Vt represents the recommended speed of the vehicle in the drivable area; T represents the shortest time from the current speed to the recommended speed; R represents the positional relationship between the vehicle and the rugged area, when the vehicle is located outside the lateral range of the rugged area, R=0, when the vehicle is located within the lateral range of the rugged area, R=1; F represents the ruggedness of the rugged area, and D represents the longitudinal distance between the vehicle and the rugged area.

[0019] According to a third embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the method according to the first embodiment of the present disclosure are implemented.

[0020] According to a fourth embodiment of the present disclosure, an electronic device is provided, including: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the steps of the method according to the first embodiment of the present disclosure.

[0021] According to a fifth embodiment of the present disclosure, a vehicle is provided, the vehicle comprising: the rough road surface detection device according to the second embodiment of the present disclosure.

[0022] By adopting the above technical solution, since the road surface features are first identified and matched, and then the rugged information of the rugged area is further determined through the mathematical model comparison method, the specific location, type, ruggedness, width and other information of the rugged area can be accurately known before the vehicle reaches the rugged area. Not only does the extracted road surface information become accurate and reliable, but it also enables driving decision suggestions to be executed based on the road surface information known in advance, which is conducive to avoiding driving risks in advance and greatly improves the utilization efficiency of road surface information.

[0023] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0025] Figure 1 A flowchart of a rough road surface detection method according to an embodiment of the present disclosure is shown.

[0026] Figure 2 An example diagram of road conditions after feature extraction is shown.

[0027] Figure 3 Another flow chart of a rough road surface detection method according to an embodiment of the present disclosure is shown.

[0028] Figure 4 A schematic block diagram of a rough road surface detection device according to an embodiment of the present disclosure is shown.

[0029] Figure 5 Another schematic block diagram of a rough road surface detection device according to an embodiment of the present disclosure is shown.

[0030] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0031] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0032] Figure 1 FIG. 1 is a flow chart showing a method for detecting a rough road surface according to an embodiment of the present disclosure. Figure 1 As shown, the method includes the following steps S11 to S14.

[0033] In step S11, an image of the environment in front of the vehicle is acquired.

[0034] The image of the environment in front of the vehicle may be an image of the environment in front of the vehicle collected by at least one of a monocular camera, a binocular camera, a multi-camera camera, a color camera, a grayscale camera, etc. The camera is preferably a high-definition camera so that the captured image is clearer. The viewing distance of the camera may be, for example, more than 1 km, and may simultaneously cover the current lane of the vehicle and the four lanes adjacent to the current lane, so that images in a longer distance and wider range in front of the vehicle can be captured, which is more conducive to serving as a reference for driving strategies. The camera may be installed behind the windshield of the vehicle.

[0035] The present disclosure does not limit the image format of the vehicle front environment image, such as grayscale image or color image. In order to facilitate the use of the next step, if the collected vehicle front environment image is a color image, the color image can be converted into a grayscale image. Of course, it is also feasible not to convert.

[0036] After the camera captures the image of the environment in front of the vehicle, the captured image of the environment in front of the vehicle can be acquired from the camera.

[0037] In step S12, feature extraction is performed on the environment image in front of the vehicle.

[0038] In this step, various deep learning algorithms can be used to extract features from the front environment image of the vehicle, such as Caffe, Tensorflow, etc., and there are also various methods for feature extraction, which are not limited by the present disclosure. For example, feature extraction can be performed based on the deep learning model of the SSD network of the Caffe framework. The deep learning model supports a large number of features and is one of the frameworks that can meet the deep feature extraction requirements of the present disclosure. Of course, if certain features required by the present disclosure cannot be found in the Caffe framework, sample training can be performed on the required features and added to the Caffe framework. The present disclosure does not limit the method of sample training.

[0039] In this step, the features to be extracted mainly include two categories: the first category is the road surface boundary; the second category is the rugged area of ​​the road surface.

[0040] The road surface boundary feature may be at least one of the following: a curb, a median strip, a green belt, a guardrail, a lane line, and the edge of other vehicles.

[0041] The feature of the rough road area may be at least one of the following: a speed bump, a manhole cover, a small pothole, an undulating road surface, a slope with a height difference, and other obstacles that may cause bumps in vehicle driving.

[0042] In step S13, the extracted features are matched with a pre-trained feature model to determine the road surface boundary and the rugged area on the road surface.

[0043] There are various ways to implement feature matching. One of the implementation methods includes: matching the extracted features with a pre-trained feature model, and marking the features of the road surface boundary and the rugged area after successful matching, to determine the road surface boundary and the rugged range information on the road surface; obtaining the calibration parameters of the imaging device that collects images of the environment in front of the vehicle; determining the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface based on the calibration parameters; and determining the rugged area according to the rugged range information, the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface.

[0044] In the specific implementation process, during the feature matching and labeling process, if the extracted feature matches a pre-trained feature model such as a lane line, then labeling is performed at the extracted feature, and the labeling made can be specific to a specific subcategory of each type of feature. For example, assuming that the major category of feature classification is road surface boundary, and the lane line is a subcategory under the major category of road surface boundary, then the extracted feature is labeled as a lane line rather than a road surface boundary.

[0045] In addition, since the number of camera eyes, installation position, installation angle, installation error, etc. will affect the final world coordinates, before determining the world coordinates of the road surface boundary and the rugged area on the road surface, it is necessary to calibrate the camera parameters to obtain various calibration parameters of the camera. For example, the calibration methods that can be used can include manual calibration, chessboard calibration, etc., so that the world coordinate system of the camera relative to the vehicle front environment can be established. Moreover, during the camera calibration process, the offset of the camera sensor in the three directions of x, y and z relative to the lens can also be determined, and these offsets are added to the calculation of the camera world coordinate system reconstruction, so that the world coordinate system can be accurately reflected by the camera coordinate system, so as to accurately reflect the distance, angle and width of the road surface boundary, rugged area, etc. relative to the vehicle in the reconstructed world coordinate system, so as to obtain the world coordinates of the road surface boundary, rugged area, etc.

[0046] by Figure 2 Take the road condition shown in the figure as an example. After performing feature matching, it can be determined that there are green belts, first lane lines, second lane lines, third lane lines, fourth lane lines, first obstacles, first rugged areas and vehicle 2 in the environment in front of the vehicle. The road boundary can be determined based on the green belt, first lane line and fourth lane line. Because there are solid lane lines in this example, the first lane line and the fourth lane line are selected as the road boundary according to the principle of solid lane line priority. If there are no solid lane lines, the edge of the green belt can be used as the road boundary, so that the road position can be preliminarily screened. After identifying the road position, obstacles, rugged areas, etc. on the road can be identified. In this example, the first obstacle, the first rugged area and vehicle 2 are identified on the road, and the world coordinates of these obstacles and rugged areas can also be determined with the help of the method described above.

[0047] In step S14, the target road surface is determined according to the road surface boundary, and the ruggedness information of the rugged area on the target road surface is determined by a mathematical model comparison method.

[0048] In step S13, only the world coordinates and types of various rugged areas can be obtained, such as speed bumps, manhole covers, small potholes, etc. However, the feature matching characteristics of deep learning determine that it is difficult to obtain information such as the ruggedness and width of the rugged area, such as the height of the speed bump, the height and convexity of the small pothole, etc. Because the ruggedness is different, the driving strategy of the vehicle is also different, so for the same rugged area type, different ruggedness information is very important. Therefore, it is necessary to further accurately determine the ruggedness, width and other information of the rugged area in step S14.

[0049] There are various ways to determine the rugged information of the rugged area. One implementation includes: establishing an ideal flat road model based on the determined road boundary, establishing an actual road model based on the determined road boundary, obstacles on the road surface, and rugged areas on the road surface; comparing the ideal flat road model with the actual road model to obtain a difference function between the ideal flat road model and the actual road model, wherein the difference function can reflect the difference between the ideal flat road model and the actual road model; and determining the ruggedness and width of the rugged area using the difference function.

[0050] In the specific implementation process, since the world coordinates of, for example, a rugged area have been obtained in step S13, the coordinate range of the rugged area can be determined through the world coordinates of each edge of the rugged area. Then, an ideal flat road surface model and an actual road surface model within the coordinate range can be established. Based on the comparison of these two models, information such as the ruggedness and width of the rugged area can be accurately determined, and the amount of calculation and false detection can be minimized.

[0051] By adopting the above technical solution, since the road surface features are first identified and matched, and then the rugged information of the rugged area is further determined through the mathematical model comparison method, the specific location, type, ruggedness, width and other information of the rugged area can be accurately known before the vehicle reaches the rugged area. Not only does the extracted road surface information become accurate and reliable, but it also enables driving decision suggestions to be executed based on the road surface information known in advance, which is conducive to avoiding driving risks in advance and greatly improves the utilization efficiency of road surface information.

[0052] In one embodiment, the method according to the embodiment of the present disclosure may further include: extracting obstacle features based on the front environment image of the vehicle, matching the extracted obstacle features with a pre-trained obstacle feature model, and determining obstacles on the road surface; determining the world coordinates of the obstacles on the road surface based on the calibration parameters; and determining obstacle information of the obstacles on the road surface based on the world coordinates of the obstacles on the road surface. The obstacle features may be at least one of the following: pedestrians, other vehicles, road warning signs, stone piers, animals, and other spilled items.

[0053] Through the above technical solution, obstacles on the road can be accurately determined in advance, which is conducive to avoiding driving risks in advance and greatly improves the utilization efficiency of road information.

[0054] Figure 3 FIG. 2 is another flow chart of a rough road surface detection method according to an embodiment of the present disclosure. Figure 3 As shown, after step S14, the rough road detection method according to the embodiment of the present disclosure may further include step S15, determining the driving strategy of the vehicle based on the determined road boundary information, obstacle information on the road surface and roughness information.

[0055] There are many ways to determine the vehicle driving strategy, one of which is as follows. First, obtain vehicle information, such as the chassis height of the vehicle, the maximum slope H0, the current speed V0 of the vehicle, the width K of the vehicle body, etc. Then, combined with the rugged information obtained in step S14, the longitudinal distance D between the vehicle and the rugged area, the positional relationship R between the vehicle and the rugged area, and the ruggedness F% of the rugged area can be obtained. The value of R can be 0 or 1. R = 0 means that the edge of the vehicle body is outside the lateral distance of the rugged area, which means that the rugged area will not affect the normal driving of the vehicle, for example Figure 2 The vehicle 2 in FIG. 1 is outside the lateral distance of the first rugged area. R=1 indicates that the edge of the vehicle body is within the lateral distance of the rugged area, which means that the rugged area will affect the normal driving of the vehicle. For example, Figure 2 The vehicle in the figure is located within the lateral distance of the first rugged area. The ruggedness F% can be, for example, a ratio relative to the maximum passable slope H0 of the vehicle. F=0% is a flat road, and more than 100% is an impassable area.

[0056] After obtaining the above information, the driving strategy can be determined next. For example, the recommended speed Vt=V0-V0×R×(1-F) in the drivable area can be determined by combining the position relationship R with the ruggedness F% and other information, and then the shortest time T=D / (V0-Vt) from the current speed to the recommended speed can be determined by the current speed and the recommended speed. Then, the recommended acceleration a=V0-Vt / T can be determined. In this way, it can be determined that the vehicle can be driven at the acceleration a determined above, so that before reaching the rugged area, the vehicle can be adjusted to a more comfortable speed for the driver or passengers in a suitable time, so that the vehicle can pass through the rugged area at this comfortable speed.

[0057] Next, combine Figure 2 Another implementation method for determining a vehicle driving strategy is described below. Figure 2As shown, the drivable area obviously includes two lanes determined by the second lane line, the third lane line, and the fourth lane line, and a part of the lane determined by the first lane line and the second lane line. Since the positional relationship between the vehicle and the first rugged area ahead is R=1 (that is, the vehicle will be affected by the first rugged area), the lane can be judged and changed by the longitudinal distance relationship between the vehicle and vehicle 2. For example, assuming that the longitudinal distance between the vehicle and vehicle 2 is detected to be D1, after the vehicle travels a distance exceeding D1 (when vehicle 2 is stationary), the vehicle can be controlled to change lanes and change to the lane determined by the third lane line and the fourth lane line for driving. Therefore, the positional relationship between the vehicle and the first rugged area can be changed to R=0, so that the vehicle can pass through the first rugged area without affecting the vehicle speed and driving experience.

[0058] Figure 4 FIG. 2 is a schematic block diagram of a rough road surface detection device according to an embodiment of the present disclosure. Figure 4 As shown, the rough road detection device includes: an acquisition module 41, used for acquiring an image of the environment in front of the vehicle; a feature extraction module 42, used for extracting features from the image of the environment in front of the vehicle; a matching module 43, used for matching the extracted features with a pre-trained feature model to determine the road surface boundary and the rough area on the road surface; a rough information determination module 44, used for determining the target road surface according to the road surface boundary, and determining the rough information of the rough area on the target road surface by a mathematical model comparison method.

[0059] By adopting the above technical solution, since the road surface features are first identified and matched, and then the rugged information of the rugged area is further determined through the mathematical model comparison method, the specific location, type, ruggedness, width and other information of the rugged area can be accurately known before the vehicle reaches the rugged area. Not only does the extracted road surface information become accurate and reliable, but it also enables driving decision suggestions to be executed based on the road surface information known in advance, which is conducive to avoiding driving risks in advance and greatly improves the utilization efficiency of road surface information.

[0060] Optionally, the feature extraction module 42 is further used to extract features from the image of the environment in front of the vehicle using a deep learning algorithm.

[0061] Optionally, the matching module 43 includes: a matching and labeling submodule, which matches the extracted features with the pre-trained feature model, and labels the features of the road surface boundary and the rugged area after successful matching, to determine the road surface boundary and the rugged range information on the road surface; a calibration parameter acquisition submodule, which is used to acquire the calibration parameters of an imaging device that captures images of the environment in front of the vehicle; a world coordinate determination submodule, which is used to determine the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface based on the calibration parameters; and a rugged area determination submodule, which is used to determine the rugged area based on the rugged range information, the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface.

[0062] Optionally, the rugged information determination module 44 includes: a mathematical modeling submodule, which is used to establish an ideal flat road surface model based on the determined road surface boundary, and to establish an actual road surface model based on the determined road surface boundary and the rugged area on the road surface; a comparison submodule, which is used to compare the ideal flat road surface model with the actual road surface model to obtain a difference function between the ideal flat road surface model and the actual road surface model; and a rugged information determination submodule, which is used to determine the ruggedness and width of the rugged area using the difference function.

[0063] Optionally, the feature extraction module 42 is also used to extract obstacle features based on the front environment image of the vehicle; the matching module 43 is also used to match the extracted obstacle features with a pre-trained obstacle feature model to determine the obstacles on the road surface, determine the world coordinates of the obstacles on the road surface according to the calibration parameters, and determine the obstacle information of the obstacles on the road surface based on the world coordinates of the obstacles on the road surface.

[0064] Figure 5 FIG. 2 is another schematic block diagram of a rough road surface detection device according to an embodiment of the present disclosure. Figure 5 As shown, the device further includes: a driving strategy determination module 45, which is used to determine the driving strategy of the vehicle based on the determined road boundary information, obstacle information and ruggedness information on the road surface.

[0065] Optionally, the driving strategy determination module 45 includes: a drivable area determination submodule, used to determine the drivable area based on the determined road boundary information, obstacle information on the road surface and rugged information; an acceleration determination submodule, used to determine the acceleration of the vehicle as a=V0-Vt / T when the drivable area includes a rugged area and the vehicle is located within the lateral range of the rugged area, wherein Vt=V0-V0×R×(1-F), T=D / (V0-Vt), wherein V0 represents the current speed of the vehicle; Vt represents the recommended speed of the vehicle in the drivable area; T represents the shortest time from the current speed to the recommended speed; R represents the positional relationship between the vehicle and the rugged area, when the vehicle is located outside the lateral range of the rugged area, R=0, when the vehicle is located within the lateral range of the rugged area, R=1; F represents the ruggedness of the rugged area, and D represents the longitudinal distance between the vehicle and the rugged area. In addition, the driving strategy determination module 45 can read vehicle information used in the driving strategy determination process, such as the vehicle's chassis height, maximum slope, current vehicle speed, vehicle body width, etc., through, for example, a CAN bus.

[0066] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0067] According to yet another embodiment of the present disclosure, a vehicle is provided. The vehicle includes the rough road surface detection device according to the embodiment of the present disclosure.

[0068] Figure 6 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 6 As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0069] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned rough road detection method. The memory 702 is used to store various types of data to support the operation of the electronic device 700, and these data may include, for example, instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module.

[0070] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned rough road detection method.

[0071] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned rough road detection method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions can be executed by the processor 701 of the electronic device 700 to complete the above-mentioned rough road detection method.

[0072] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0073] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0074] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A rough road detection method, characterized in that: include: Acquire an image of the environment in front of the vehicle; Performing feature extraction on the front environment image of the vehicle; Match the extracted features with the pre-trained feature model to determine the road surface boundaries and the rough areas on the road surface; Determine a target road surface according to the road surface boundary, and determine rugged information of a rugged area on the target road surface by a mathematical model comparison method; The step of matching the extracted features with a pre-trained feature model to determine the road surface boundary and the rough area on the road surface includes: Matching the extracted features with the pre-trained feature model, and marking the features of the road surface boundary and the features of the rugged area after successful matching, to determine the road surface boundary and the rugged range information on the road surface; Acquiring calibration parameters of an imaging device for acquiring images of the environment in front of the vehicle; Determine the world coordinates of the road surface boundary and the world coordinates of the rough area on the road surface based on the calibration parameters; determining the rugged area according to the rugged range information, the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface; Wherein, the determining the rugged information of the rugged area on the target road surface by using a mathematical model comparison method includes: An ideal flat road surface model is established based on the determined road surface boundary, and an actual road surface model is established based on the determined road surface boundary and a rugged area on the road surface; Comparing the ideal flat road surface model with the actual road surface model to obtain a difference function between the ideal flat road surface model and the actual road surface model; The difference function is used to determine the ruggedness and width of the rugged region.

2. The method according to claim 1, characterized in that The step of extracting features from the vehicle front environment image comprises: A deep learning algorithm is used to extract features from the image of the environment in front of the vehicle.

3. The method according to claim 1, characterized in that The method further comprises: Extracting obstacle features based on the front environment image of the vehicle, matching the extracted obstacle features with a pre-trained obstacle feature model, and determining obstacles on the road surface; Determine the world coordinates of the obstacle on the road surface according to the calibration parameters; Obstacle information of the obstacle on the road surface is determined based on the world coordinates of the obstacle on the road surface.

4. The method according to claim 3, characterized in that After determining the ruggedness information of the rugged area on the target road surface by the mathematical model comparison method, the method further includes: The driving strategy of the vehicle is determined based on the determined road boundary information, obstacle information and ruggedness information on the road surface.

5. The method according to claim 4, characterized in that The determining of the driving strategy of the vehicle based on the determined road boundary information, obstacle information and ruggedness information on the road surface includes: Determine a drivable area based on the determined road boundary information, obstacle information and ruggedness information on the road surface; In the case where the drivable area includes a rugged area and the vehicle is located within the lateral range of the rugged area, the acceleration of the vehicle is determined to be a = (V0-Vt) / T, where Vt = V0-V0×R×(1-F), T = D / (V0-Vt), Among them, V0 represents the current speed of the vehicle; Vt represents the recommended speed of the vehicle in the drivable area; T represents the shortest time from the current speed to the recommended speed; R represents the positional relationship between the vehicle and the rugged area, when the vehicle is outside the lateral range of the rugged area, R=0, when the vehicle is within the lateral range of the rugged area, R=1; F represents the ruggedness of the rugged area, and D represents the longitudinal distance between the vehicle and the rugged area.

6. A rough road detection device, characterized in that: include: An acquisition module is used to acquire an image of the environment in front of the vehicle; A feature extraction module, used for extracting features from the image of the environment in front of the vehicle; A matching module is used to match the extracted features with the pre-trained feature model to determine the road surface boundaries and the rough areas on the road surface; A roughness information determination module, used to determine a target road surface according to the road surface boundary, and determine the roughness information of a rough area on the target road surface by a mathematical model comparison method; Wherein, the matching module includes: A matching and labeling submodule, used to match the extracted features with the pre-trained feature model, and label the features of the road surface boundary and the rugged area after successful matching, to determine the road surface boundary and the rugged range information on the road surface; A calibration parameter acquisition submodule, used to acquire calibration parameters of an imaging device for collecting images of the environment in front of the vehicle; a world coordinate determination submodule, configured to determine the world coordinates of the road surface boundary and the world coordinates of the rough area on the road surface based on the calibration parameters; A rugged area determination submodule, configured to determine the rugged area according to the rugged range information, the world coordinates of the road surface boundary and the world coordinates of the rugged area on the road surface; Among them, the rugged information determination module includes: a mathematical modeling submodule, which is used to establish an ideal flat road surface model based on the determined road surface boundary, and to establish an actual road surface model based on the determined road surface boundary and the rugged area on the road surface; a comparison submodule, which is used to compare the ideal flat road surface model with the actual road surface model to obtain a difference function between the ideal flat road surface model and the actual road surface model; and a rugged information determination submodule, which is used to determine the ruggedness and width of the rugged area using the difference function.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 5 are implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 5.

9. A vehicle, characterized in that: The vehicle comprises: the rough road surface detection device according to claim 6.

Images