Driving road condition monitoring method, device and system

Abstract

The invention discloses a driving road condition monitoring method, device and system, and relates to the technical field of driving road condition monitoring. According to the technical scheme of the invention, the method comprises the steps: receiving the road condition data of aerial photography when an unmanned plane executes a road condition monitoring task of a vehicle, wherein the road condition data is the image data or video data of the environment where the vehicle is located; the road condition data are analyzed, monitoring results of at least one driving monitoring category can be obtained, and the driving monitoring category comprises a monitoring category for monitoring whether dangerous road conditions exist in the environment where the vehicle is located or not; and outputting corresponding road condition information on a display terminal on the vehicle according to a monitoring result. According to the technical scheme, the road condition data are obtained through the unmanned aerial vehicle from the aerial photography view angle, the limitation of driving road condition monitoring is reduced, whether dangerous road conditions exist or not can be determined after the shot road condition data are analyzed, the road condition data are output in time through the display terminal on the vehicle, more accurate and comprehensive road condition information is presented for the vehicle, and the driving safety is improved. And the safety of the vehicle in the running process is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of road condition monitoring, and in particular to a road condition monitoring method, device and system. Background Art

[0002] With the development of automobile electrification and intelligence, automatic monitoring gradually covers more and more driving scenarios. In each intelligent driving scenario, road information is collected mainly based on vehicle sensors, vehicle cameras, radars and other sensing electronic components. For example, the millimeter-wave radar on the front and rear bumpers of the car provides obstacle recognition within a range of 20m for the vehicle central processor, but due to its limited radiation angle range, multiple radars need to be deployed to more comprehensively identify the surrounding conditions of the entire vehicle. Therefore, the current road condition perception and recognition solutions all have perception limitations in angle or distance range, resulting in insufficient safety during vehicle driving. Summary of the invention

[0003] In order to solve the technical problem of how to reduce the limitations of road condition monitoring, the present invention provides a road condition monitoring method, device and system, which can reduce the limitations of road condition monitoring through unmanned aerial vehicles and improve the safety of vehicles during driving.

[0004] The embodiment of the present invention provides the following solution:

[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring a driving road condition, the method comprising:

[0006] Receiving road condition data taken by a drone when performing a road condition monitoring task for a vehicle, wherein the road condition data is image data or video data of the vehicle in the environment;

[0007] Parsing the road condition data to obtain at least one monitoring result of a driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether a vehicle is in a dangerous road condition in an environment;

[0008] The corresponding road condition information is output on the display terminal on the vehicle according to the monitoring results.

[0009] In an optional embodiment, before receiving the road condition data taken by the drone when performing the road condition monitoring task of the vehicle, the method further includes:

[0010] Obtaining information collected by on-board sensors on vehicles;

[0011] When it is determined based on the collected information that the vehicle has a risk of safe driving, a task execution instruction is output to the drone, wherein the task execution instruction is an instruction to control the drone to perform the road condition monitoring task.

[0012] In an optional embodiment, the safe driving risk includes a high-speed driving risk where the vehicle speed is greater than a speed threshold, and a low-speed driving risk where the vehicle speed is less than or equal to the speed threshold; when it is determined according to the collected information that the vehicle has a safe driving risk, outputting a task execution instruction to the drone includes:

[0013] When it is determined according to the collected information that the vehicle is at risk of high-speed driving, a first execution instruction is output to control the drone to take aerial photos of a first monitoring area of ​​the road where the vehicle is traveling;

[0014] When it is determined based on the collected information that the vehicle has a risk of low-speed driving, a second execution instruction is output to control the drone to take aerial photos of a second monitoring area around the vehicle, wherein the second monitoring area is smaller than the first monitoring area.

[0015] In an optional embodiment, the road condition data is parsed to obtain a monitoring result of whether there is a dangerous road condition in the environment where the vehicle is located, including:

[0016] Perform image recognition and analysis on road condition data to obtain the source of danger;

[0017] According to the coordinate position of the drone when performing the road condition monitoring task, the location of the danger source is analyzed to obtain the current distance between the danger source and the vehicle;

[0018] Based on the danger source and the current interval distance, determine whether the vehicle is in a dangerous road condition.

[0019] In an optional embodiment, the monitoring result is that the vehicle is in a dangerous road condition in the environment; the display terminal on the vehicle outputs corresponding road condition information according to the monitoring result, including:

[0020] Determine the source of the dangerous road condition and the current distance between the source of danger and the vehicle based on the monitoring results;

[0021] Obtaining a current display mode of the hazard source according to the current interval distance and a preset mapping relationship, wherein the preset mapping relationship is a relationship between different interval distances and corresponding display modes;

[0022] According to the current display mode, an instruction image for avoiding the danger source is displayed on the display terminal as road condition information.

[0023] In an optional embodiment, after determining the current interval distance between the danger source and the vehicle, the method further includes:

[0024] Determine whether the current interval distance is less than a distance threshold, where the distance threshold is the minimum distance for the vehicle to avoid the danger source based on the driver's control at the current speed;

[0025] When the current interval distance is less than the distance threshold, the vehicle is controlled to actively avoid the danger source;

[0026] When the current interval distance is greater than or equal to the distance threshold, the step of obtaining the current display mode of the danger source according to the current interval distance and the preset mapping relationship is executed.

[0027] In an optional embodiment, obtaining the current display mode of the danger source according to the current interval distance and the preset mapping relationship includes:

[0028] When the current interval distance is in the first distance interval in the preset mapping relationship, determining that the current display mode of the danger source is static display;

[0029] When the current interval distance is in the second distance interval in the preset mapping relationship, it is determined that the current display mode of the danger source is a flashing display, wherein the interval distance represented by the first distance interval is greater than the interval distance represented by the second distance interval.

[0030] In an optional embodiment, displaying an indication image for avoiding a danger source as road condition information on a display terminal according to a current display mode includes:

[0031] In the current display mode, a lane change image for avoiding a dangerous source and a warning image representing the dangerous source are determined as the indication image;

[0032] The lane change image and the warning image are projected via a display terminal, wherein the display terminal includes a head-up display and / or a digital beam headlight on the vehicle.

[0033] In a second aspect, an embodiment of the present invention further provides a device for monitoring a driving condition, the device comprising:

[0034] A receiving module is used to receive the road condition data taken by the drone when performing the road condition monitoring task of the vehicle, wherein the road condition data is the image data or video data of the vehicle in the environment;

[0035] An acquisition module is used to parse the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether there is a dangerous road condition in the environment where the vehicle is located;

[0036] The first output module is used to output corresponding road condition information on a display terminal on the vehicle according to the monitoring result.

[0037] In a third aspect, an embodiment of the present invention further provides a system for monitoring driving conditions, the system comprising:

[0038] The drone is used to perform the road condition monitoring task of the vehicle and send the road condition data to the outside, where the road condition data is the image data or video data of the vehicle in the environment;

[0039] The vehicle-mounted controller is connected to the UAV for communication, and is used to receive the road condition data and analyze the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether the vehicle has dangerous road conditions in the environment in which it is located;

[0040] The display terminal is connected to the vehicle controller for communication, and is used to output corresponding road condition information according to the monitoring results.

[0041] Compared with the prior art, the method, device and system for monitoring road conditions of the present invention have the following advantages:

[0042] The technical solution of the present invention receives the road condition data taken by the drone when performing the road condition monitoring task of the vehicle, and the road condition data is the image data or video data of the vehicle in the environment; the road condition data is parsed to obtain the monitoring result of at least one driving monitoring category, and the driving monitoring category includes the monitoring category of whether there are dangerous road conditions in the environment where the vehicle is located; and the corresponding road condition information is output on the display terminal on the vehicle according to the monitoring result. The technical solution obtains road condition data from the perspective of aerial photography by the drone, reduces the limitations of driving road condition monitoring, and can determine whether there are dangerous road conditions after parsing the road condition data taken, and output it in time through the display terminal on the vehicle, presenting more accurate and comprehensive road condition information to the vehicle, so that the vehicle can respond and avoid in time when there are dangerous road conditions in the environment, thereby improving the safety of the vehicle during driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flow chart of a method for monitoring road conditions provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of the principle of road condition monitoring provided by an embodiment of the present invention;

[0046] Figure 3 A logical schematic diagram of a method for monitoring a driving road condition provided by an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of the structure of a system for monitoring road conditions provided by an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of the structure of a vehicle monitoring system in the prior art provided by an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the structure of a device for monitoring road conditions provided by an embodiment of the present invention.

[0050] Explanation of the accompanying drawings: 1-vehicle, 2-drone, 3-hazard source, 4-first monitoring area, 5-second monitoring area, 6-image shooting path, 7-data transmission path. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention belong to the scope of protection of the embodiments of the present invention.

[0052] See also Figure 1 , Figure 1 A flow chart of a method for monitoring a driving condition provided by an embodiment of the present invention. The monitoring method can be applied to the control terminal of a vehicle. The control terminal can be a vehicle controller (VCU, Vehicle Control Unit) or other device capable of running the monitoring method, which is not specifically limited here. The monitoring method includes:

[0053] S11. Receive road condition data taken by a drone when performing a road condition monitoring task for a vehicle, wherein the road condition data is image data or video data of the vehicle in the environment in which it is located.

[0054] Specifically, the drone can be a vehicle-mounted drone configured to follow the vehicle, which is charged and recharged by the vehicle. When it is necessary to perform road condition monitoring tasks, the drone takes off from the vehicle to monitor the road conditions. Of course, the drone can also be a consumer-grade drone of the vehicle user, such as a drone connected to the user's mobile terminal device to take pictures or record videos. The drone that performs the road condition monitoring task can just be able to take pictures of the vehicle's environment, and there is no specific restriction here.

[0055] The aerial photography modes of the drone when performing the road condition monitoring task may include high-altitude bird's-eye view shooting, horizontal shooting, tracking shooting, surround shooting, etc. Through the aerial photography perspective, a more comprehensive and accurate road condition can be obtained, which can be represented by image data or video data. It should be noted that the drone's road condition monitoring task can be triggered by the vehicle user, for example, by a button integrated on the central control inside the vehicle; it can also be automatically triggered based on the vehicle's driving environment. After the road condition monitoring task is triggered, the drone takes off, shoots the vehicle's environment and outputs road condition data, which is sent to the vehicle via a preset communication method and received by the vehicle's built-in on-board controller. The road condition data described in the embodiment of the present invention includes the real-time traffic congestion status of the road, the passable status of the physical road surface, etc., and may also include road condition information around the parking lot, whether there are obstacles in the vehicle's parking space, etc.

[0056] Since the advantage of the road condition monitoring task is that it can obtain more comprehensive and accurate road condition data in a timely manner, the execution of the road condition monitoring task is triggered based on the driver's awareness during vehicle driving, which requires the driver to have rich driving experience. If the driver is inexperienced, the execution of the road condition monitoring task may not be triggered in time, resulting in a delay in the acquisition of road condition data. Based on this, in a specific implementation, before executing step S11, the method further includes:

[0057] The first step is to obtain the collected information from the vehicle's onboard sensors. The onboard sensors include onboard cameras, lidar, etc. Although the information obtained by this type of onboard sensor is limited, it can be implemented to obtain the current road conditions during the vehicle's driving process.

[0058] In the second step, when it is determined that the vehicle has a risk of safe driving based on the collected information obtained in real time, it means that the data obtained based on the on-board sensors can determine that there is a driving hazard. In order to further fully understand the road conditions, the task execution instruction is output to the drone. The task execution instruction is an instruction to control the drone to perform the road condition monitoring task. The risk of safe driving can be determined based on the fusion of collected information obtained by multiple types of on-board sensors. For example, based on the fusion calculation of the collected information obtained by the on-board camera and lidar, it is determined whether there are obstacles on the road during the vehicle's driving. In this way, the execution of the road condition monitoring task can be automatically triggered, and the drone can be controlled to obtain road condition data in a timely manner.

[0059] In actual applications, vehicles travel on various road conditions. If the same strategy is used to control the drone to perform road condition monitoring tasks, the acquired road condition data may not be the data required by the user. Based on this, in a specific implementation, the safe driving risk includes the high-speed driving risk when the vehicle speed is greater than the speed threshold, and the low-speed driving risk when the vehicle speed is less than or equal to the speed threshold. When it is determined that the vehicle has a safe driving risk based on the collected information, the task execution instruction is output to the drone, including:

[0060] When it is determined based on the collected information that the vehicle is at risk of high-speed driving, it means that the vehicle is dangerous at a higher speed and it is necessary to promptly detect whether the road environment ahead is safe. Then, the first execution instruction is output to control the drone to take aerial photos of the first monitoring area of ​​the road where the vehicle is traveling. Figure 2 , Figure 2 The first monitoring area 4 can be set as the range of 2km ahead of the road where the vehicle 1 is traveling. The drone 2 flies to a higher altitude to perform the road condition monitoring task, takes aerial photos of the road condition in the first monitoring area 4, outputs the road condition data, and sends it to the vehicle 1. Figure 2 As shown in FIG, when there is a danger source 3 in front of the road on which the vehicle is traveling, the drone 2 takes a picture of the danger source 3 on the image shooting path 6 at the curve position, and outputs the picture to the vehicle 1 via the data transmission path 7. The road condition data of the first monitoring area 4 includes dynamic obstacles and static obstacles within a range of 2 km, such as pedestrians, animals, other vehicles, etc., as well as the road and its surrounding green areas, static objects in the air, dynamic objects in the air, etc.

[0061] When it is determined based on the collected information that the vehicle 1 has a risk of driving at a low speed, it means that more attention needs to be paid to the road conditions in the surrounding area of ​​the vehicle 1, and then the second execution instruction is output to control the drone 2 to take aerial photos of the second monitoring area 5 around the vehicle, and the second monitoring area 5 is smaller than the first monitoring area 4. The second monitoring area 5 can be set to the range of 5m around the vehicle. The road condition data of the second monitoring area 5 includes dynamic obstacles and static obstacles within 5m around the vehicle, such as pedestrians, animals, other vehicles, etc.

[0062] At this point, the road condition data taken by the drone has been acquired, and step S12 is performed after the road condition data is acquired.

[0063] S12. Analyze the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether there is a dangerous road condition in the vehicle's environment.

[0064] Specifically, since the road condition data is video data or image data, the monitoring results of various driving monitoring categories can be determined by analyzing the road condition data. Driving monitoring categories can be set based on actual needs, such as monitoring categories for dangerous road conditions, monitoring categories for congested road conditions, monitoring categories for road conditions around parking spaces, etc. After analyzing the road condition data, the monitoring results of various driving monitoring categories are determined. The monitoring results can be characterized based on different driving monitoring categories. For example, for the monitoring results of dangerous road conditions, the monitoring results can be characterized as specific hazard categories, including falling rocks, floods, etc. For the monitoring results of congested road conditions, the monitoring results can be characterized as specific congestion levels, and a proportional correspondence relationship between the congestion length and the congestion level can be established to accurately characterize the monitoring results of the driving monitoring category.

[0065] It can be understood that since the driving monitoring category at least includes the monitoring category of whether there are dangerous road conditions in the vehicle's environment, it is possible to determine whether there is danger during the vehicle's driving process based on the monitoring results of this driving monitoring category, thereby achieving the purpose of timely and accurate detection of dangerous road conditions.

[0066] Exemplarily, step S12 includes sub-steps S12-1 to S12-3, which are specifically described as follows:

[0067] S12-1. Perform image recognition and analysis on the road condition data to obtain the source of danger. The road condition data can be implemented based on the convolutional neural network model. Before implementing the image analysis, the convolutional neural network model is trained with various training samples until the model can identify various different sources of danger. When analyzing the road condition data, the image data representing the road condition data is input into the trained convolutional neural network model, and the source of danger is determined based on the output of the model. When the road condition data is video data, multiple image frames can be input into the convolutional neural network model for analysis. Multiple image frames can be spaced by a preset number to ensure the accuracy of the analysis and reduce the redundancy of data processing.

[0068] S12-2. According to the coordinate position of the drone when performing the road condition monitoring task, the location of the danger source is analyzed to obtain the current distance between the danger source and the vehicle. The coordinate position includes the flight altitude of the drone and may also include the geographic coordinates between the location and the vehicle. The current distance between the danger source and the vehicle can be determined through the coordinate position.

[0069] S12-3, based on the hazard source and the current interval distance, determine the monitoring result of whether the vehicle has dangerous road conditions in the environment. The monitoring result can be directly characterized by the hazard source and the corresponding current interval distance. For example, when the hazard source is rockfall, the output hazard source is rockfall and the current interval distance between the rockfall and the vehicle. The current interval distance can be updated in real time based on the vehicle's driving speed. After obtaining the monitoring results of the driving monitoring category, enter step S13.

[0070] S13. Outputting corresponding road condition information on a display terminal on the vehicle according to the monitoring results.

[0071] Specifically, the display terminal can be one or more of the vehicle's LCD instrument, central control display, and vehicle rearview mirror, and can output corresponding road condition information based on the monitoring results. Through the road condition information, the driver can determine the actual road condition of the environment. If there are obstacles, etc., the driver can slow down in time, or stop to observe, thereby improving the safety of the vehicle during driving.

[0072] For the display of road condition information, corresponding processing can be performed based on the monitoring results. For example, when the monitoring results indicate that there are road conditions such as falling rocks and floods that seriously affect driving safety, the road condition information displayed on the display terminal is red, and prompts words such as "falling rocks" and "floods"; when the monitoring results indicate that there are road conditions such as accumulated water, the road condition information displayed on the display terminal is yellow, and prompts words such as "accumulated water"; when the monitoring results indicate that there are no foreign objects on the road where the vehicle is traveling, the display terminal displays normal road information in blue.

[0073] Exemplarily, the monitoring result is that the vehicle is in a dangerous road condition; step S13 includes sub-steps S13-1 to S13-3, which are specifically described as follows:

[0074] S13-1. Determine the source of the dangerous road condition and the current distance between the source and the vehicle based on the monitoring results. The severity of the danger to driving safety can be determined by the source of danger, and the time for the danger to driving safety can be estimated by the current distance.

[0075] When the current interval distance is less than the distance threshold, it means that the distance between the danger source and the vehicle is small, and the driver may not be able to make accurate and timely avoidance control, so the vehicle is controlled to actively avoid the danger source. When implementing active avoidance, the vehicle steering and / or braking system can be controlled by the on-board controller to enable the vehicle to avoid the danger source in time. The distance threshold is the minimum distance for the vehicle to avoid the danger source based on the driver's control at the current speed. The distance threshold can be set based on the experience of the technicians, or it can be determined through calibration experiments. When the current interval distance is greater than or equal to the distance threshold, it means that avoidance can be achieved through the driver's control, and then enter step S13-2.

[0076] S13-2, obtaining the current display mode of the danger source according to the current interval distance and the preset mapping relationship, wherein the preset mapping relationship is the relationship between different interval distances and corresponding display modes. The display mode includes different display colors, icons, etc., which can enable the driver to be alerted to the danger source through different display modes.

[0077] For example, when the current interval distance is in the first distance interval in the preset mapping relationship, it means that the danger source is far away from the vehicle, and the driver is gently prompted, and the current display mode of the danger source is determined to be static display. When the current interval distance is in the second distance interval in the preset mapping relationship, the interval distance represented by the first distance interval is greater than the interval distance represented by the second distance interval, which means that the danger source is close to the vehicle, and the current display mode of the danger source is determined to be flashing display. The first distance interval and the second distance interval can be freely set, for example, the first distance interval is set to 1-2km and the second distance interval is set to 0.5-1km.

[0078] S13-3, displaying the instruction image for avoiding the danger source as road condition information on the display terminal according to the current display mode. In different display modes, the display method of the danger source is different, and the driver can be reminded of different warning levels based on the display difference. The instruction image includes any image that reminds the driver to slow down, stop, etc., which can remind the driver to avoid the danger source.

[0079] Exemplarily, sub-step S13-3 includes:

[0080] In the first step, in the current display mode, a lane-changing image for avoiding a dangerous source and a warning image representing the dangerous source are determined as indication images. The lane-changing image is an image that guides the driver to change lanes to a corresponding lane, such as an indication arrow. The warning image is an image that warns the driver of the type of dangerous source, such as an image of falling rocks. Based on the lane-changing image and the warning image, a light signal for displaying the indication image can be generated.

[0081] The second step is to project lane change images and warning images through a display terminal, where the display terminal includes a head-up display and / or digital beam headlights on the vehicle. Head-up displays (or AR-HUD displays) are mostly used to display vehicle navigation images, and the light display technology of digital beam headlights (or DLP headlights) is mostly used in functions such as fun and interactivity. By displaying lane change images and warning images through a head-up display and / or digital beam headlights, the integration of virtual reality technology can be achieved, which not only brings users a closer to real display experience, but also reduces visual attention interference to a certain extent, further improving driving safety.

[0082] The following embodiment of the present invention will generally describe the method for monitoring the driving road condition. Figure 3 , Figure 3 The logical diagram of the method for monitoring the driving road condition specifically includes:

[0083] S301, obtaining the vehicle's immediate surroundings. When the vehicle's driving speed is less than or equal to a speed threshold, the vehicle's immediate surroundings are obtained through a drone.

[0084] S302, obtaining the road environment of the vehicle. When the vehicle's driving speed is greater than a speed threshold, the vehicle's road environment is obtained through a drone.

[0085] S303: inputting the traffic condition data of the surrounding environment or the road environment into the vehicle controller for processing by the vehicle central processor.

[0086] S304: Determine whether the vehicle is in driving danger based on the road condition data.

[0087] S305: When there is no driving danger for the vehicle, generate an electrical signal 4.

[0088] S306. When the danger source is at 1-2 km, generate electrical signal 3.

[0089] S307. When the danger source is at 0.5-1 km, an electrical signal 2 is generated.

[0090] S308. When the danger source is less than 0.5 km, generate electrical signal 1.

[0091] S309: Input the electrical signal into a display terminal of the vehicle, where the display terminal is a head-up display and / or a digital beam headlight on the vehicle.

[0092] S310: Adjust the LED brightness based on the input electrical signal.

[0093] S311. Based on the input electrical signal, adjust the angle of the micro-mirror.

[0094] S312: When the danger source is less than 0.5 km away, the steering / braking system of the vehicle is controlled to enable the vehicle to autonomously avoid the danger source.

[0095] Based on the same technical concept as the monitoring method, the embodiment of the present invention also provides a driving road condition monitoring system, which includes a drone, a vehicle-mounted controller and a display terminal. Figure 4 , Figure 4 A schematic diagram of the monitoring system.

[0096] The drone is used to perform the vehicle's road condition monitoring task and send road condition data to the outside, where the road condition data is image data or video data of the vehicle in its environment.

[0097] The vehicle-mounted controller is connected to the drone in communication, and the two can implement wireless communication. The vehicle-mounted controller is used to receive road condition data and analyze the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether the vehicle has dangerous road conditions in the environment. The analysis of the road condition data includes signal conversion, which is converted into a first display signal and a second display signal based on the road condition data.

[0098] The display terminal is connected to the vehicle controller for communication, and is used to output corresponding road condition information according to the monitoring results. When the display terminal includes multiple types of display devices, corresponding display is performed based on various display signals, the digital beam headlight projects the displayed image based on the first display signal, and the head-up display projects the displayed image based on the second display signal.

[0099] See also Figure 5 , Figure 5 The structure diagram of the vehicle monitoring system in the prior art. The system as a whole includes a perception layer, a decision layer and an execution layer. The perception layer includes multiple types of vehicle-mounted sensors. The perception process needs to rely on multiple multi-dimensional perception elements. It has the disadvantages of scattered perception elements, scattered software and complex signal sources. In addition, under the intelligent architecture based on the whole vehicle, the above perception elements are reusable, so higher requirements are put forward on the data processing capabilities of the vehicle-mounted central processor, resulting in higher physical costs, longer software development cycles, and higher update and iteration costs. In addition, there are many signal processing transmission nodes, and the risk of information delay, signal loss and misjudgment increases. In addition, all the perception areas that can be covered are limited. For example, the accuracy of laser radar detection of objects over 200m away is significantly reduced. Due to the limitation of the height of the whole vehicle, the vehicle-mounted camera can only detect objects at a distance of about 400m. According to the calculation of vehicles traveling on a highway at a speed of 120km / h, the road condition information can be recognized at most 10s in advance, which is difficult to deal with in extreme weather and extreme working conditions.

[0100] It can be understood that the driving road condition monitoring system of the embodiment of the present invention simplifies the traditional perception system components into a pure visual perception system, thereby simplifying the information processing solution; at the same time, the high-altitude aerial photography perspective of the drone can not only clearly detect the environmental information around the vehicle, but also detect the road condition data more than 2 km ahead of the driving route. The on-board central processing unit can process the road condition data to present more accurate real-time road information and monitoring results to the driver. In addition, visual information collection replaces the on-board perception detection components to reduce the physical cost and iteration cost of the perception system.

[0101] Based on the same technical concept as the monitoring method, the embodiment of the present invention also provides a monitoring device for driving road conditions, please refer to Figure 6 , Figure 6 The diagram is a schematic diagram of the structure of the monitoring device. The monitoring device includes:

[0102] The receiving module 601 is used to receive the road condition data taken by the drone when performing the road condition monitoring task of the vehicle, wherein the road condition data is the image data or video data of the vehicle in the environment;

[0103] The acquisition module 602 is used to analyze the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether there is a dangerous road condition in the environment where the vehicle is located;

[0104] The first output module 603 is used to output corresponding road condition information on a display terminal on the vehicle according to the monitoring result.

[0105] In an optional embodiment, the monitoring device further includes:

[0106] An acquisition module is used to acquire information collected by on-board sensors on the vehicle;

[0107] The second output module is used to output a task execution instruction to the UAV when it is determined based on the collected information that the vehicle has a risk of safe driving, wherein the task execution instruction is an instruction to control the UAV to perform a road condition monitoring task.

[0108] In an optional embodiment, the safe driving risk includes a high-speed driving risk where the vehicle speed is greater than a speed threshold, and a low-speed driving risk where the vehicle speed is less than or equal to the speed threshold; the second output module includes:

[0109] A first output submodule is used to output a first execution instruction when it is determined according to the collected information that the vehicle has a high-speed driving risk, so as to control the drone to take aerial photos of a first monitoring area of ​​the road where the vehicle is traveling;

[0110] The second output submodule is used to output a second execution instruction when it is determined according to the collected information that the vehicle has a risk of low-speed driving, so as to control the drone to take aerial photos of a second monitoring area around the vehicle, wherein the second monitoring area is smaller than the first monitoring area.

[0111] In an optional embodiment, the obtaining module includes:

[0112] The first acquisition submodule is used to perform image recognition and analysis on the road condition data to obtain the danger source;

[0113] The second acquisition submodule is used to analyze the position of the danger source according to the coordinate position of the UAV when performing the road condition monitoring task, so as to obtain the current interval distance between the danger source and the vehicle;

[0114] The third output submodule is used to determine the monitoring result of whether there is a dangerous road condition in the vehicle's environment according to the danger source and the current interval distance.

[0115] In an optional embodiment, the monitoring result is that the vehicle is in a dangerous road condition; the first output module includes:

[0116] A determination submodule, for determining the danger source of the dangerous road condition and the current distance between the danger source and the vehicle according to the monitoring result;

[0117] A third obtaining submodule is used to obtain the current display mode of the danger source according to the current interval distance and a preset mapping relationship, wherein the preset mapping relationship is the relationship between different interval distances and corresponding display modes;

[0118] The display submodule is used to display the instruction image for avoiding the danger source as the road condition information on the display terminal according to the current display mode.

[0119] In an optional embodiment, the first output module further includes:

[0120] A judgment submodule is used to judge whether the current interval distance is less than a distance threshold, wherein the distance threshold is the minimum distance for the vehicle to avoid the danger source based on the driver's control at the current speed;

[0121] A control submodule, used to control the vehicle to actively avoid the danger source when the current interval distance is less than the distance threshold;

[0122] The execution submodule is used to execute the step of obtaining the current display mode of the danger source according to the current interval distance and a preset mapping relationship when the current interval distance is greater than or equal to the distance threshold.

[0123] In an optional embodiment, the third obtaining submodule includes:

[0124] A first determining unit, configured to determine that the current display mode of the danger source is a static display when the current interval distance is in a first distance interval in a preset mapping relationship;

[0125] The second determination unit is used to determine that the current display mode of the danger source is a flashing display when the current interval distance is in a second distance interval in a preset mapping relationship, wherein the interval distance represented by the first distance interval is greater than the interval distance represented by the second distance interval.

[0126] In an optional embodiment, the display submodule includes:

[0127] A third determining unit, configured to determine, in a current display mode, a lane changing image for avoiding a dangerous source and a warning image representing the dangerous source as an indication image;

[0128] The display unit is used to project a lane change image and a warning image via a display terminal, wherein the display terminal includes a head-up display and / or a digital beam headlight on the vehicle.

[0129] The technical solution of the present invention The technical solution provided in the embodiment of the present invention has at least the following technical effects or advantages:

[0130] By receiving the road condition data taken by the drone when performing the road condition monitoring task of the vehicle, the road condition data is the image data or video data of the vehicle in the environment; parsing the road condition data, it is possible to obtain the monitoring result of at least one driving monitoring category, and the driving monitoring category includes the monitoring category of whether there are dangerous road conditions in the environment where the vehicle is located; and outputting the corresponding road condition information on the display terminal on the vehicle according to the monitoring result. This technical solution obtains road condition data from the perspective of aerial photography by the drone, reduces the limitations of driving road condition monitoring, and can determine whether there are dangerous road conditions after parsing the road condition data taken, and output it in time through the display terminal on the vehicle, presenting more accurate and comprehensive road condition information to the vehicle, so that the vehicle can respond and avoid in time when there are dangerous road conditions in the environment, thereby improving the safety of the vehicle during driving.

[0131] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (modules, systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0136] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for monitoring driving road conditions, characterized in that: The method comprises: Receiving road condition data taken by a drone when performing a road condition monitoring task for a vehicle, wherein the road condition data is image data or video data of the vehicle in the environment; Parsing the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether there is a dangerous road condition in the environment where the vehicle is located; The display terminal on the vehicle outputs corresponding road condition information according to the monitoring result.

2. The method for monitoring driving road conditions according to claim 1, characterized in that: Before receiving the road condition data taken by the drone when performing the road condition monitoring task of the vehicle, the method further includes: Acquiring information collected by on-board sensors on the vehicle; When it is determined that the vehicle has a risk of safe driving according to the collected information, a task execution instruction is output to the drone, wherein the task execution instruction is an instruction for controlling the drone to perform the road condition monitoring task.

3. The method for monitoring driving road conditions according to claim 2, characterized in that: The safe driving risk includes a high-speed driving risk where the vehicle speed is greater than a speed threshold, and a low-speed driving risk where the vehicle speed is less than or equal to the speed threshold; When it is determined according to the collected information that the vehicle has a risk of safe driving, outputting a task execution instruction to the drone includes: When it is determined according to the collected information that the vehicle has the risk of high-speed driving, a first execution instruction is output to control the drone to take aerial photos of a first monitoring area of ​​the road where the vehicle is traveling; When it is determined according to the collected information that the vehicle has the risk of low-speed driving, a second execution instruction is output to control the drone to take aerial photos of a second monitoring area around the vehicle, wherein the second monitoring area is smaller than the first monitoring area.

4. The method for monitoring driving road conditions according to claim 1, characterized in that: The road condition data is parsed to obtain a monitoring result of whether the vehicle has a dangerous road condition in the environment, including: Performing image recognition and analysis on the road condition data to obtain danger sources; According to the coordinate position of the drone when performing the road condition monitoring task, the position of the danger source is analyzed to obtain the current interval distance between the danger source and the vehicle; A monitoring result of whether there is a dangerous road condition in the environment where the vehicle is located is determined according to the danger source and the current interval distance.

5. The method for monitoring driving road conditions according to claim 1, characterized in that: The monitoring result is that the vehicle is in a dangerous road condition in the environment; and the display terminal on the vehicle outputs corresponding road condition information according to the monitoring result, including: Determine the danger source of the dangerous road condition and the current distance between the danger source and the vehicle according to the monitoring result; Obtaining the current display mode of the danger source according to the current interval distance and a preset mapping relationship, wherein the preset mapping relationship is the relationship between different interval distances and corresponding display modes; According to the current display mode, an instruction image for avoiding the danger source is displayed on the display terminal as road condition information.

6. The method for monitoring driving road conditions according to claim 5, characterized in that: After determining the current interval distance between the danger source and the vehicle, the method further includes: Determining whether the current interval distance is less than a distance threshold, wherein the distance threshold is a minimum distance for the vehicle to avoid the danger source based on driver control at the current vehicle speed; When the current interval distance is less than the distance threshold, controlling the vehicle to actively avoid the danger source; When the current interval distance is greater than or equal to the distance threshold, the step of obtaining the current display mode of the danger source according to the current interval distance and a preset mapping relationship is performed.

7. The method for monitoring driving road conditions according to claim 5, characterized in that: The obtaining of the current display mode of the danger source according to the current interval distance and the preset mapping relationship includes: When the current interval distance is in the first distance interval in the preset mapping relationship, determining that the current display mode of the danger source is static display; When the current interval distance is in the second distance interval in the preset mapping relationship, it is determined that the current display mode of the danger source is a flashing display, wherein the interval distance represented by the first distance interval is greater than the interval distance represented by the second distance interval.

8. The method for monitoring driving road conditions according to claim 5, characterized in that: The step of displaying the image indicating avoiding the danger source as road condition information on the display terminal according to the current display mode includes: In the current display mode, determining a lane-changing image for avoiding the danger source and a warning image representing the danger source as the indication image; The lane change image and the warning image are projected via the display terminal, wherein the display terminal includes a head-up display and / or a digital beam headlight on the vehicle.

9. A device for monitoring driving road conditions, characterized in that: The device comprises: A receiving module, used to receive road condition data taken by a drone when performing a road condition monitoring task for a vehicle, wherein the road condition data is image data or video data of the vehicle in the environment; An acquisition module, used for parsing the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether there is a dangerous road condition in the environment where the vehicle is located; The first output module is used to output corresponding road condition information on a display terminal on the vehicle according to the monitoring result.

10. A system for monitoring driving road conditions, characterized in that: The system comprises: A drone, used to perform a road condition monitoring task for a vehicle and send road condition data to the outside, wherein the road condition data is image data or video data of the vehicle in the environment; an on-board controller, which is in communication with the drone, and is used to receive the road condition data and parse the road condition data to obtain a monitoring result of at least one driving monitoring category, wherein the driving monitoring category includes a monitoring category of whether the vehicle has dangerous road conditions in the environment in which it is located; The display terminal is communicatively connected to the vehicle-mounted controller, and the display terminal is used to output corresponding road condition information according to the monitoring results.

Images