Traffic auxiliary path finding method and device and vehicle

Abstract

The invention provides a traffic auxiliary path finding method and device and a vehicle, and relates to the technical field of vehicles. The method is based on a vehicle carrying an unmanned aerial vehicle. The method comprises the following steps: in response to an active path-finding instruction, controlling the unmanned aerial vehicle to carry out path-finding on a target road section of a road where the vehicle is located; and according to the path finding result, generating optimal lane recommendation information for the vehicle. Based on the vehicle carrying the unmanned aerial vehicle, the unmanned aerial vehicle is controlled to take off by responding to the active path-finding instruction, the target road section is detected, the path-finding result about the target road section is obtained, and traffic information of the target road section is obtained timely and accurately; the optimal lane recommendation information is generated by identifying, analyzing and processing the path finding result, so that a lane selection suggestion and a path adjustment basis are provided for the user in advance, and the user can be assisted to complete reasonable lane changing operation or adjust a driving strategy before driving into the target road section, thereby improving the vehicle passing efficiency, the user travel efficiency, the driving experience and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a traffic assistance path exploration method and device and a vehicle. BACKGROUND

[0002] With the continuous acceleration of urbanization and the continuous rise of the number of motor vehicles, traffic congestion frequently occurs on urban roads, especially on highways and major roads during holidays and morning and evening peak hours, which seriously affects the efficiency of residents' travel and the capacity of roads. Although a variety of navigation and driving assistance systems for vehicles have been developed, there are still many deficiencies in dealing with complex congestion scenarios.

[0003] The mainstream navigation system for vehicles currently mainly relies on vehicle-mounted GPS, geographic information system (GIS) and traffic broadcast data to provide path planning and road condition information. However, such information usually has low update frequency and limited accuracy, making it difficult to achieve real-time dynamic perception and response to congested road sections. For example, the navigation system usually cannot timely and accurately identify the specific reasons for congestion in front of the vehicle, the traffic differences between lanes, or estimate the actual waiting time of the vehicle, which affects the timely and reasonable decision-making of the driver, and easily causes misjudgment, blind lane changing or stagnation in a lane with low traffic efficiency, thereby exacerbating traffic congestion, reducing traffic efficiency, and increasing driving risks. SUMMARY

[0004] The problem solved by the present application is how to timely and accurately obtain traffic information of the road in front of the vehicle and improve the efficiency of user travel.

[0005] To solve the above problems, the present application provides a traffic assistance path exploration method and device and a vehicle.

[0006] In a first aspect, the present application provides a traffic assistance path exploration method based on a vehicle carrying a drone; the traffic assistance path exploration method comprises: in response to an active path exploration instruction, controlling the drone to explore a target road section of a road where the vehicle is located; generating optimal lane recommendation information for the vehicle according to the exploration result.

[0007] Optionally, before the traffic assistance path exploration method controls the drone to explore the target road section of the road where the vehicle is located in response to the active path exploration instruction, the traffic assistance path exploration method comprises: generating the active path exploration instruction in response to prompt information about congestion on the road in front of the vehicle, or in response to an active path exploration trigger instruction input by a driver; wherein the prompt information is generated by a navigation system applied to the vehicle; the driver includes at least one of a user of the vehicle and an auxiliary driving control system.

[0008] Optionally, the controlling the UAV to explore the target section of the road where the vehicle is located comprises: controlling the UAV to explore the target section of the road where the vehicle is located, within a preset distance range in front of the vehicle.

[0009] Optionally, the controlling the UAV to explore the target section of the road where the vehicle is located in response to the active exploration instruction comprises: in response to the active exploration instruction, planning a flight path of the UAV for exploring the target section based on navigation information of the vehicle; wherein the target section is set by a driver or a navigation system applied to the vehicle; controlling the UAV to take off from the vehicle and explore the target section according to the flight path.

[0010] Optionally, the UAV is provided with an image acquisition mechanism and a spatial position measurement mechanism; the controlling the UAV to explore the target section of the road where the vehicle is located comprises: controlling the UAV to acquire road condition images of the target section through the image acquisition mechanism, and to measure spatial feature data of the target section through the spatial position measurement mechanism; performing data fusion and recognition analysis processing on the road condition images and the spatial feature data to determine traffic condition information of each lane of the target section, and taking the traffic condition information of each lane of the target section as the exploration result.

[0011] Optionally, the generating optimal lane recommendation information for the vehicle according to the exploration result comprises: determining a target optimal lane for the vehicle to change into on the target section based on a preset traffic flow speed estimation model and a preset optimal lane evaluation model according to the traffic condition information of each lane of the target section, and generating the optimal lane recommendation information corresponding to the target optimal lane.

[0012] Optionally, after the optimal lane recommendation information for the vehicle is generated, the traffic assistance exploration method further comprises: determining a lane change timing for the vehicle to change from a current lane into a target optimal lane based on the optimal lane recommendation information and current driving information of the vehicle, and generating a lane change timing prompt information; wherein the target optimal lane is a lane recommended by the optimal lane recommendation information.

[0013] Optionally, the generating optimal lane recommendation information for the vehicle according to the exploration result comprises: when a first target road section is congested, the unmanned aerial vehicle is controlled to continue to explore the first target road section before the vehicle drives off the first target road section; The optimal lane recommendation information is updated according to the updated exploration result. The lane change timing is determined according to the optimal lane recommendation information and the current driving information of the vehicle, and lane change timing prompt information is generated, which comprises: The lane change timing prompt information is updated according to the updated optimal lane recommendation information and the current driving information of the vehicle.

[0014] In a second aspect, the present application provides a traffic assistance exploration device, which comprises: An exploration response unit is configured to control an unmanned aerial vehicle to explore a target road section of a road on which a vehicle is located in response to an active exploration instruction. A lane recommendation unit is configured to generate optimal lane recommendation information for the vehicle according to an exploration result.

[0015] In a third aspect, the present application provides a vehicle comprising a memory and a processor. The memory is configured to store a computer program. The processor is configured to implement the traffic assistance exploration method of the first aspect when the computer program is executed.

[0016] In a fourth aspect, the present application provides a computer readable storage medium storing a computer program, which is read and run by a processor to implement the traffic assistance exploration method of the first aspect.

[0017] The traffic auxiliary path exploration method and device of the application and the vehicle have the following advantages: the application is based on a vehicle equipped with a UAV, and the UAV is controlled to take off and explore a target road section in response to an active path exploration instruction, so that path exploration results (such as real-time traffic condition information) about the target road section are obtained, and the traffic information of the road in front of the vehicle (such as the target road section being a road section located in front of the vehicle in the direction of travel of the vehicle) is obtained in a timely and accurate manner; the path exploration results are identified, analyzed and processed to generate optimal lane recommendation information, so as to provide lane selection suggestions and path adjustment basis for the user in advance, and assist the user in completing reasonable lane changing operations or adjusting driving strategies at a corresponding time (such as before entering the target road section or a congestion area of the target road section, or in the process of entering the target road section or the congestion area of the target road section), so as to effectively avoid lanes with low traffic efficiency or abnormal events, improve the overall traffic efficiency of the vehicle, improve the travel efficiency of the user, reduce the waiting time and driving burden caused by congestion, and improve the driving safety and driving experience. Moreover, the optimal lane recommendation information is generated based on the path exploration results to directly provide lane selection suggestions for the target road section to the user, on the one hand, to guide the user to make lane changing or path adjustment decisions at the appropriate time, avoid driving into lanes with low traffic efficiency, congestion or abnormal events due to information lag or inaccurate judgment, and effectively reduce traffic delays, frequent lane changing, driving risks and other problems caused by unreasonable lane selection, and improve the overall driving safety and road traffic efficiency; on the other hand, to avoid the understanding burden and decision difficulty caused by directly providing intermediate information (such as original road condition images or complex traffic parameters without analysis and processing) to the user, so as to improve the intuitiveness and practicality of information interaction, help the user quickly understand and adopt the corresponding suggestions, reduce driving interference and misjudgment risk, and further enhance the intelligence and user experience of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 FIG. 1 is a flowchart of a traffic auxiliary path exploration method according to an embodiment of the application; Figure 2 FIG. 2 is a sub-flowchart of step 100 according to an embodiment of the application; Figure 3 FIG. 3 is a structural block diagram of a traffic auxiliary path exploration device according to an embodiment of the application; Figure 4 FIG. 4 is a structural diagram of the communication connection between the memory and the processor of a vehicle according to an embodiment of the application. DETAILED DESCRIPTION

[0019] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein.

[0021] In combination Figure 1 As shown, the embodiments of the present application provide a traffic assistance path exploration method based on a vehicle carrying a UAV; the traffic assistance path exploration method comprises: Step 100, in response to an active path exploration instruction, controlling the UAV to explore a target road section of a road where the vehicle is located.

[0022] Specifically, in step 100, when receiving an instruction for triggering the UAV to take off for path exploration (denoted as an active path exploration instruction), such as the driver (including the user of the vehicle, the auxiliary driving control system) actively triggering or the vehicle automatically generating the instruction when meeting the corresponding conditions, the instruction is responded to, the UAV is controlled to take off and explore a target road section (such as the current road section of the vehicle or the congested road section in front of the vehicle, etc.) of the road where the vehicle is located, to obtain (or determine) the path exploration result (such as the real-time traffic condition information of the target road section) about the target road section, thereby providing corresponding data support for the driving decision of the vehicle (or the driver).

[0023] Step 200, generating optimal lane recommendation information for the vehicle according to the path exploration result.

[0024] Specifically, in step 200, based on the path exploration result of the UAV exploring the target road section in step 100, such as the real-time traffic condition information of the target road section, the relevant auxiliary decision information (denoted as optimal lane recommendation information) for guiding the optimal driving path selection and lane change of the vehicle is generated by identification, analysis and processing, which can be displayed to the user through the vehicle machine (such as through the display mechanism of the vehicle machine) or broadcast (such as through the audio interaction mechanism of the vehicle machine) to prompt the user to make corresponding lane change or path adjustment operation at the appropriate time. In some embodiments, the generated optimal lane recommendation information can also be directly used for the auxiliary driving control system (or the unmanned driving control system) of the vehicle as an input parameter of the corresponding path planning module or decision execution module, for realizing the automatic lane change control and path dynamic adjustment of the vehicle, so as to improve the traffic efficiency and environmental adaptability of the vehicle in the automatic driving state.

[0025] In summary, the method of the embodiment is based on a vehicle carrying a UAV, which controls the UAV to take off and detect a target road section in response to an active path exploration instruction, obtains path exploration results (such as real-time traffic condition information) about the target road section, and realizes timely and accurate acquisition of traffic information of a road in front of the vehicle (such as a target road section located in front of the vehicle in the direction of travel of the vehicle). By identifying, analyzing and processing the path exploration results, optimal lane recommendation information is generated to provide lane selection suggestions and path adjustment basis for the user in advance, which can assist the user to complete reasonable lane changing operation or adjust the driving strategy at a corresponding time (such as before entering the target road section or a congestion area of the target road section, or in the process of entering the target road section or the congestion area of the target road section), thereby effectively avoiding lanes with low traffic efficiency or abnormalities, improving the overall traffic efficiency of the vehicle, improving the travel efficiency of the user, reducing the waiting time and driving burden caused by congestion, and improving the driving safety and driving experience. Moreover, by generating optimal lane recommendation information based on the path exploration results, lane selection suggestions for the target road section are directly provided to the user. On the one hand, it is used to guide the user to make a decision on lane changing or path adjustment at the right time, avoid driving into a lane with low traffic efficiency, congestion or abnormal events due to information lag or inaccurate judgment, thereby effectively reducing traffic delays, frequent lane changing, driving risks and other problems caused by unreasonable lane selection, and improving the overall driving safety and road traffic efficiency. On the other hand, it avoids the understanding burden and decision difficulty caused by directly providing intermediate information (such as raw road condition images or complex traffic parameters without analysis and processing) to the user, thereby improving the intuitiveness and practicality of information interaction, helping the user to quickly understand and adopt the corresponding suggestions, reducing driving interference and misjudgment risk, and further enhancing the intelligence and user experience of the vehicle using the method of the embodiment.

[0026] In addition, by carrying a UAV, the vehicle can realize high-altitude real-time detection of the target road section. Compared with the traditional navigation system relying on vehicle-mounted sensors or traffic broadcast information, it effectively breaks through the limitations of limited vehicle-mounted sensing range, lagging road condition information update, inability to accurately identify congestion sources and lane traffic differences, etc. Through the wide range of real-time path exploration results (such as real-time traffic condition information of the target road section) obtained by the UAV from the aerial perspective, the real-time traffic state of the road in front of the vehicle can be grasped earlier and more comprehensively, thereby making up for the lack of micro-lane decision-making ability of the traditional navigation system which only has macro-path planning. At the same time, the method of the embodiment differs from the traffic monitoring technology in related technologies which relies on city-level UAVs or monitoring equipment, and realizes autonomous detection and local optimal path assistance capability at the individual vehicle level, with stronger real-time performance, pertinence and deployability, especially suitable for dynamic road condition response in holiday, peak period or sudden congestion environment, improving the efficiency, safety and experience of user travel, etc.

[0027] Optionally, in response to the active path exploration instruction, before controlling the UAV to explore the target road section of the road where the vehicle is located, the traffic assistance path exploration method comprises: The active path exploration instruction is generated in response to prompt information about congestion of the road ahead of the vehicle, or in response to an active path exploration trigger instruction input by the driver; wherein the prompt information is generated by a navigation system applied to the vehicle; the driver includes at least one of a user of the vehicle and an assisted driving control system.

[0028] Specifically, before controlling the UAV carried by the vehicle to take off and perform the path exploration operation, the corresponding active path exploration instruction needs to be received. For example, when the navigation system applied to the vehicle (such as a vehicle-mounted navigation system or an external navigation system in communication connection with the vehicle) detects that the road ahead of the vehicle is congested (or has a risk of congestion), the navigation system can automatically generate prompt information about congestion of the road ahead of the vehicle, and then trigger the generation of the active path exploration instruction. For example, the navigation system can determine whether there is abnormal congestion (or a risk of congestion) based on current traffic information data (such as average road section speed, historical travel time comparison, road state identification, etc.), and if so, output the corresponding prompt information; or when the driver of the vehicle (such as at least one of a user of the vehicle and an assisted driving control system) inputs an active path exploration trigger instruction, such as when the user inputs the corresponding input operation based on the human-computer interaction interface (such as a voice interface, a touch screen interaction interface, a physical button, etc.) provided by the vehicle, or when the assisted driving control system automatically generates an active path exploration trigger instruction based on its running state and environmental perception results during driving, the active path exploration instruction is triggered to realize the path exploration of the target road section by the UAV.

[0029] In this way, by introducing the generation process of the active path exploration instruction before controlling the UAV to perform the path exploration operation, and supporting automatic triggering based on the vehicle navigation system and active input triggering by the driver (including the user or the assisted driving control system), the method has higher scene adaptability and triggering flexibility.

[0030] Optionally, controlling the UAV to explore the target road section of the road where the vehicle is located comprises: Controlling the UAV to explore the target road section of the road where the vehicle is located, which is located within a preset distance range ahead of the vehicle.

[0031] In particular, for the target section, it is preferably a target section on the road where the vehicle is located and located within a preset distance range in front of the vehicle, so as to balance the forward-looking and real-time of the UAV path exploration. On the one hand, by detecting the front section in front of the vehicle (i.e. in the direction of travel) that has not yet entered in advance, sufficient response time can be reserved for subsequent driving path adjustment and lane selection, improving the forward-looking and safety of decision-making. On the other hand, the detection range is controlled within a reasonable distance, which can ensure that the collected traffic information still has high real-time and reference value, avoiding the interference of environmental changes and communication delay caused by too long detection distance. Moreover, the target section is on the road where the vehicle is located and located within a preset distance range in front of the vehicle, so as to ensure that the UAV detection area is highly related to the actual traffic path of the vehicle, avoid invalid detection of irrelevant sections, improve resource utilization efficiency and UAV operation efficiency, and reduce UAV flight task complexity and energy consumption. In some embodiments, by continuously and segmentally exploring the target section (which is also dynamically changing as the vehicle travels) by the UAV, the one-time large-scale detection mode of the UAV is replaced, which can reduce unnecessary long-time hovering flight and data collection redundancy, reduce communication delay and communication burden between the vehicle and the UAV, and at the same time enhance the real-time and pertinence of the detection data. Correspondingly, the road where the vehicle is located and the target section can be updated in combination with the real-time position of the vehicle and the update of the navigation path, and the UAV detection task area can be dynamically updated, so that it always focuses on the high-priority road section that the vehicle is about to enter, realizes fine scheduling of sensing resources and closed-loop execution of detection tasks, and thus improves the decision support capability and energy efficiency management capability of the whole vehicle (and the vehicle carrying the UAV) in complex traffic scenarios.

[0032] The upper and lower limits of the preset distance range are the distances from the farthest point and the closest point of the target road section to the vehicle, respectively, and are used to determine the target road section range and the UAV path exploration range. In some embodiments, the target road section is a congestion road section determined based on a navigation system. By using a UAV to explore the congestion road section, traffic condition information such as the traffic state, traffic density, and obstacle distribution of each lane in the road section can be collected in real time, which provides data support for generating timely and accurate optimal lane recommendation information, and effectively solves the problems of information update lag and insufficient resolution of traditional navigation systems. During the process of the vehicle passing through the congestion road section, the UAV can continuously collect road condition information of the road section, analyze the road condition in real time, and dynamically adjust the optimal lane recommendation. In other embodiments, the preset distance range can be updated in real time based on the current position of the vehicle, and can be adaptively adjusted according to factors such as the vehicle speed, road grade, navigation path, and traffic event density. For example, if the current speed is high, the farthest point distance can be extended to several kilometers to leave enough response window; if in urban low-speed road conditions, the farthest point distance can be appropriately shortened to ensure the effectiveness and timeliness of the obtained traffic information. In this way, by setting the distance interval between the closest point and the farthest point of the target road section as the dynamic detection range of the UAV, the detection task can be focused on the key road section to be passed through by the vehicle, the data relevance and detection efficiency are improved, the energy waste and data processing redundancy caused by an excessively wide detection area are reduced, and the controllability, intelligence, and practical value of the method are further enhanced.

[0033] Optionally, in combination with the method shown in Figure 1 , Figure 2 , the method further includes: Step 110, in response to the active path exploration instruction, planning a flight path of the UAV for exploring the target road section based on the navigation information of the vehicle; wherein the target road section is set by the driver or the navigation system applied to the vehicle.

[0034] Specifically, in step 110, upon receiving the active path exploration instruction, the target road section in front of the vehicle is analyzed and identified according to the navigation information (such as the navigation path, the current road, the vehicle positioning coordinates, the travel direction, the travel speed, etc.) of the current vehicle, thereby providing a basis for planning the flight path of the UAV for exploring the target road section. Wherein, for planning the flight path of the UAV for exploring the target road section, the process can be based on the navigation information of the vehicle, and comprehensively consider the geographical form of the target road section (such as road direction, intersection distribution, number of lanes), traffic risk distribution (such as congestion section, accident point, construction area), flight environment constraint (such as flight restricted area, building obstruction, signal interference area) and flight capability of the UAV itself (such as maximum endurance distance, flight height limit, positioning accuracy, etc.), to generate one or more (may choose one as the flight path) optimal UAV flight routes under the premise of safe accessibility, effective data collection and low path redundancy. In this way, by reasonably planning the UAV flight path, it is ensured that the UAV accurately covers the target road section area during the path exploration task execution process, avoids potential risk airspace, and efficiently completes the data collection task, thereby providing key perception support for subsequent traffic condition identification, optimal lane recommendation generation, etc.

[0035] For the target section, it is set by the driver or the navigation system applied to the vehicle. For example, when the target section is a congested section located within a certain distance range in front of the vehicle determined by the navigation system, the congested section can be automatically set as the target of the drone path exploration; wherein the navigation system can determine that there is obvious congestion on the current road and identify the corresponding section as the target based on the real-time traffic data analysis results, such as conditions such as sudden drop in average speed, increase in traffic density or existence of abnormal events. Or, when the user believes that there is a congestion risk in front of the road based on the actual driving experience (such as observing the dense or slow running vehicles in front of the road with the naked eye), the target section can also be manually specified through the vehicle interface, or a specific section in front of the vehicle can be selected as the target section through the human-computer interaction interface (such as voice interaction or touch input), triggering the active path exploration instruction for subsequent flight path planning and path exploration operation. In this way, the vehicle carrying the drone, in cooperation with the navigation system applied to the vehicle (for providing the above navigation information), can realize efficient identification and range determination of high-risk sections (such as congestion, accidents, construction, etc.) by combining the traffic data and path planning capabilities provided by the navigation system, and generate a target section and an exploration area accordingly, for guiding the drone to perform targeted flight path planning and detection task scheduling, which not only improves the timeliness and accuracy of target section identification, but also effectively reduces the complexity of user intervention operation, simplifies the path exploration task triggering process, and at the same time, by taking the high-risk section (or the congested section) as the priority detection target, the drone can concentrate resources on the most critical area for sensing and data collection within the limited flight time and energy budget, thereby maximizing the path exploration efficiency and information utilization value.

[0036] Step 120, control the drone to take off from the vehicle, and explore the target section according to the flight path.

[0037] Specifically, in step 120, a take-off instruction is sent to the drone to control it to take off from the vehicle mounting position (such as a roof platform or a special take-off and landing structure) and fly according to the flight path planned in step 110; and in the flight process, the traffic condition information of the target section such as the traffic flow state, obstacles, and lane passability is collected in real time based on the image acquisition mechanism, spatial position measurement mechanism and other sensing devices mounted on the drone, for forming the path exploration data required for subsequent lane recommendation analysis.

[0038] In this way, the target section can be reasonably planned and dynamically adapted before the drone takes off, improving the efficiency and safety of the drone flight, and ensuring the timeliness and accuracy of the path exploration results, providing reliable auxiliary decision basis for the vehicle.

[0039] Exemplarily, when the UAV is started to explore the road, a prompt information is sent to the user through the vehicle-mounted voice or display mechanism to prompt the user. When the UAV receives the active exploration instruction, it performs take-off preparation, including self-checking operation (such as battery power, motor state, sensor state checking, etc.), and flight path planning of the UAV based on the current navigation information of the vehicle to avoid known obstacles and no-fly zones; after completing the path planning (i.e., the take-off preparation is completed), the UAV takes off from the vehicle and flies to the target road section along the flight path to explore the road.

[0040] Optionally, a roof platform or a special automatic take-off and landing structure for the UAV is arranged on the vehicle to support the automatic take-off and landing of the UAV and ensure the safe storage and stable loading of the UAV during the driving of the vehicle. Exemplarily, the UAV adopts a fixed-wing type with vertical take-off and landing capability (such as a VTOL vertical take-off and landing composite wing UAV), and the vehicle is provided with a waterproof and shockproof integrated platform integrated on the roof to provide stable support and guidance for the UAV and realize the automatic take-off and precise landing of the UAV.

[0041] Optionally, the UAV is provided with a wireless communication module for wireless communication with the vehicle to support high-speed data transmission (such as 5G / V2X dual-mode communication) so as to transmit the data collected by the UAV back to the vehicle in real time. In some embodiments, the wireless communication between the UAV and the vehicle adopts end-to-end encryption (such as AES-256) to ensure the security of the data.

[0042] Optionally, the UAV is provided with a data communication module for wireless communication with the vehicle to transmit the collected data back to the vehicle in real time. The data communication module supports high-speed data transmission protocols, such as 5G communication, V2X communication or a combination thereof, to ensure low-latency and high-bandwidth data interaction between the UAV and the vehicle during high-speed driving of the vehicle. In some embodiments, the wireless communication link between the UAV and the vehicle adopts an end-to-end encryption transmission mechanism to improve the confidentiality and anti-interference ability of data transmission, such as encrypting the transmission content based on a preset encryption algorithm (such as AES-256 or other symmetric / asymmetric encryption algorithm) to prevent information leakage or malicious tampering during data transmission.

[0043] Optionally, in combination with the embodiments shown in Figure 1 , Figure 2 after the UAV is controlled to take off from the vehicle and explore the target road section according to the flight path, the UAV is controlled to explore the target road section of the road where the vehicle is located in response to the active exploration instruction, which further includes: Step 130, updating the flight path based on the navigation information.

[0044] Specifically, in step 130, considering that the vehicle can be in a state of motion, the traffic condition information of the target road section can change with the movement of the vehicle, for example, the road section in front can fluctuate in traffic situation due to the addition of vehicles, temporary traffic events (such as accidents, construction, police command), and other factors. Therefore, based on the real-time updated navigation information of the vehicle (such as vehicle positioning coordinates, speed, driving direction, navigation path, etc.), the flight path of the unmanned aerial vehicle can be adaptively and dynamically updated, so that the road exploration task of the unmanned aerial vehicle is always focused on the key area that is highly related to the driving path of the vehicle and has time-sensitive traffic conditions. For example, if the vehicle changes the driving route during driving, or the navigation system re-plans a new passing path, the new target road section can be quickly identified based on the new navigation path, and corresponding flight path adjustment instructions can be generated to guide the unmanned aerial vehicle to continuously detect the new target road section, avoiding wasting flight resources on the old path that has no practical significance.

[0045] In this way, the adaptation ability of the unmanned aerial vehicle road exploration to the dynamic operation condition of the vehicle is improved, the reliability of the auxiliary decision is avoided due to the lag of the navigation information or the change of the road condition without timely response, the real-time performance and decision reference value of the road exploration result are improved, and the intelligence, real-time performance and practicality of the method are further enhanced.

[0046] Optionally, the unmanned aerial vehicle is provided with an image acquisition mechanism and a spatial position measurement mechanism; the control of the unmanned aerial vehicle to explore the target road section of the road where the vehicle is located comprises: controlling the unmanned aerial vehicle to acquire the road condition image of the target road section through the image acquisition mechanism, and measuring the spatial feature data of the target road section through the spatial position measurement mechanism; performing data fusion and recognition analysis processing on the road condition image and the spatial feature data to determine the traffic condition information of each lane of the target road section, and taking the traffic condition information of each lane of the target road section as the road exploration result.

[0047] Specifically, the UAV carried by the vehicle is provided with an image acquisition mechanism and a spatial position measurement mechanism, which are respectively used to obtain image information and spatial feature data of the target section (i.e. three-dimensional spatial data used to represent the distribution of the geometric structure and obstacles of the target section). After receiving the active path exploration instruction and completing the flight path planning, the UAV is controlled to fly along the planned path, and the image acquisition mechanism (such as a high-resolution camera, an infrared imaging device, a visual sensor, etc.) is used to collect images of the target section in the airspace corresponding to the target section, so as to obtain the image (denoted as the road condition image) of the target section. At the same time, the spatial position measurement mechanism (such as an infrared ranging sensor, a laser radar, a structured light depth sensor, a binocular vision module, etc.) of the UAV is controlled to measure the spatial features of the target section, so as to obtain depth information or three-dimensional point cloud data related to the road structure, such as lane width, lane line spacing, obstacle relative height and distance, road undulation and curvature, and other spatial feature data. After obtaining the above road condition image and spatial feature data, subsequent data fusion and recognition analysis processing are performed. For example, the traffic state and occupancy of each lane can be detected based on the road condition image by using an image recognition algorithm, and the position and size of the obstacle can be accurately recognized and modeled by combining the ranging results in the spatial feature data, so as to comprehensively determine the traffic condition information (such as the vehicle density and flow rate of each lane of the target section) of each lane of the target section, and the information is taken as the output result of the UAV path exploration task, i.e. the path exploration result.

[0048] For data fusion and recognition analysis of the road condition image and spatial feature data, image recognition and target detection algorithms based on deep learning (such as convolutional neural network CNN, YOLO series, Faster R-CNN, etc.) can be used for lane line recognition, obstacle detection, and traffic density estimation of image data. At the same time, spatial modeling algorithms for point cloud or depth map processing (such as PointNet, 3D U-Net, VoxelNet, etc.) can be combined to extract key structural information (such as lane width, obstacle size and relative position, road curvature, etc.) in spatial feature data. To improve analysis accuracy and multi-source data understanding ability, a multi-modal fusion neural network model (such as a Transformer structure or an Attention mechanism-based fusion network) can be further used to realize high-dimensional fusion of image features and spatial features, and to identify traffic conditions based on fused features, such as classifying the traffic state of each lane on the target road section (such as smooth, slow, congested, blocked, etc.), identifying specific traffic events (such as construction areas, traffic accidents, temporary obstacles, etc.), and outputting traffic passing grades, passability probabilities or expected delay evaluation results for each lane. In some embodiments, a pre-trained neural network model (such as one trained based on a public data set or a self-built traffic image and point cloud data set) can be used, and the model can be fine-tuned or incrementally learned online during vehicle operation to adapt to different road environments, traffic rules or weather changes, thereby enhancing the generalization ability and robustness of the recognition process and improving the real-time performance and reliability of the auxiliary decision information generation.

[0049] In this way, through the collaborative application of the image acquisition and ranging functions of the UAV, the UAV can realize multi-modal perception of the target road section traffic scene, expanding from two-dimensional images to three-dimensional spatial structure recognition, not only improving the accuracy and precision of traffic condition recognition, but also making the lane selection or path adjustment recommendations ultimately recommended by the embodiment method more scenario-adaptive and reliable.

[0050] Optionally, the neural network model used for processing road condition images and spatial feature data can be deployed on a local edge computing platform of the vehicle; or deployed on a cloud computing platform and implemented through a communication link between the vehicle and the cloud for remote calling and data interaction processing of the model. The local edge computing platform of the vehicle can include an embedded processor (such as GPU, NPU or FPGA, etc.) dedicated to AI inference acceleration, which is used to realize fast processing and recognition of UAV return data on the vehicle side.

[0051] Optionally, according to the path exploration result, the optimal lane recommendation information for the vehicle is generated, including: According to the traffic condition information of each lane of the target road section, a target optimal lane for the vehicle to change into on the target road section is determined based on a preset traffic flow speed estimation model and a preset optimal lane evaluation model, and optimal lane recommendation information corresponding to the target optimal lane is generated.

[0052] Specifically, based on the path exploration results obtained after the unmanned aerial vehicle completes the path exploration task, which contain traffic condition information of each lane of the target road section, such as vehicle arrangement of each lane, lane traffic density, average driving speed, obstacle existence condition, etc., a preset traffic flow speed estimation model (denoted as a preset traffic flow speed estimation model) is used to predict the expected traffic flow speed of each lane in a future period of time, and a preset optimal lane evaluation model (denoted as a preset optimal lane evaluation model) is further combined to comprehensively score each lane of the target road section, so as to select one or more optimal lanes (denoted as target optimal lanes) as output, and generate corresponding optimal lane recommendation information. The preset traffic flow speed estimation model can be a prediction model constructed based on machine learning (such as deep neural network, graph neural network or time series modeling network), and the training data includes historical traffic flow, vehicle speed, time period distribution, lane type, road level, weather condition and other traffic big data, which are combined with the current collected real-time dynamic parameters for inference, so as to improve the accuracy and scene adaptability of the prediction. The preset optimal lane evaluation model can comprehensively consider evaluation factors including traffic flow speed prediction results of each lane of the target road section, matching degree of the current vehicle speed and the target optimal lane, vehicle lane changing cost, traffic stability, safety, etc., and use a multi-factor weighted algorithm, fuzzy logic reasoning, reinforcement learning strategy network or hybrid model for quantitative analysis and scoring. Further, an adaptive weight adjustment mechanism can be introduced to improve the robustness and decision reference value of the evaluation results in different scenarios; and through the preset optimal lane evaluation model, the optimal lane is predicted in advance to recommend a smooth transition lane, so as to select the best lane changing opportunity and realize lane smooth transition (lane changing).

[0053] Optionally, the optimal lane recommendation information includes the identification of the recommended lane (i.e. the target optimal lane), the recommended lane changing opportunity, auxiliary driving prompt sentences and other contents, and is provided to the user through the vehicle machine display interface or voice broadcast form, or is used as the input of the path decision module in the automatic driving mode to execute the automatic lane changing instruction or adjust the current driving path, so as to improve the driving efficiency and enhance the user experience.

[0054] Exemplarily, the preset traffic flow speed estimation model can adopt the following analysis and processing flow: first, taking the road condition image (such as a high-definition video stream, that is, a road condition image data with time continuity in a time length) collected by the unmanned aerial vehicle and the spatial feature data (such as laser radar or infrared ranging data) as input data, the image data is processed by a convolutional neural network (CNN) respectively, key image feature information representing the traffic state is extracted, for example, the distribution, type and driving speed of vehicles; at the same time, the ranging data is processed to obtain the spatial structure information of the target road section, such as three-dimensional point cloud or depth map data, and then the road geometric features are extracted, including lane width, relative position and size of obstacles, road undulation and curvature, etc. Then, the image features and spatial structure features are fused to generate a joint space-time feature map representing the traffic state of the target road section; then, based on the joint space-time feature map, a long short-term memory network (LSTM) or a time series modeling structure such as Transformer is used to model and dynamically analyze the trend of the traffic state changing with time, and historical traffic data is combined to predict and correct the traffic flow speed. Finally, the expected traffic flow speed information of each lane of the target road section in a future period of time is output, which can be km / h, and the prediction accuracy can reach ±0.1%. In addition, the congestion trend (such as intensification or mitigation) of the target road section in a future period of time can be predicted, thereby providing data support for subsequent lane recommendation strategies and path dynamic optimization.

[0055] Optionally, the traffic assistance route exploration method further comprises: generating road condition analysis information according to the route exploration result.

[0056] Specifically, the method of the embodiment can also generate road condition analysis information based on the route exploration result, such as the congestion length, reason and estimated flow speed of the target road section. These information can be displayed as independent information content on the car machine system interface, or broadcast to the user in the form of voice, prompting the user to pay attention to the traffic risk in front of him and assisting him to adjust the driving strategy more reasonably.

[0057] Optionally, when generating the optimal lane recommendation information for the vehicle according to the route exploration result, the driver preference information (such as not preferring to be close to a truck and avoiding frequent lane changing) can also be combined to generate the optimal lane recommendation information, and the information is fed back to the user through a graphical interface and a voice prompt mode.

[0058] Optionally, for the display interface provided by the display mechanism of the vehicle, it can adopt an intuitive graphical interface, which is divided into a map display area, a road condition information area, a recommended lane area, and a system status area (which is used to display relevant information of the vehicle or the external combustion engine, such as the working state of the UAV, the battery power, the signal strength, etc.). The display interface can display the current position of the vehicle, the surrounding road network structure, and the traffic conditions in front in a graphical manner, and different congestion degrees are marked with different colors (such as green for smooth, yellow for slow, and red for congestion), and the congestion length, congestion reason image, flow rate prediction, etc. are displayed in detail in the interface. The optimal recommended lane information is marked in the display interface based on the displayed map in the form of an arrow or highlighting, accompanied by a display of the recommended reason (such as the highest traffic efficiency of the lane). In addition, for the audio interaction mechanism of the vehicle, it can automatically broadcast corresponding prompt information according to the UAV operating state, optimal lane recommendation information, etc., such as "UAV detection complete", "suggest switching to the left lane", "UAV is returning, please pay attention to the screen", etc., and the prompt information content supports multi-language customization, improving the vehicle driving interaction experience.

[0059] Optionally, after generating the optimal lane recommendation information for the vehicle, the traffic assistance route exploration method further comprises: According to the optimal lane recommendation information and the current driving information of the vehicle, the lane change opportunity for the vehicle to change from the current lane to the target optimal lane is determined, and a lane change opportunity prompt information is generated; wherein the target optimal lane is the lane recommended by the optimal lane recommendation information.

[0060] Specifically, based on the lane recommended by the optimal lane recommendation information and the current driving information of the vehicle (such as the current vehicle speed, acceleration, lane, surrounding vehicle distance, surrounding environment, etc.), the lane change opportunity for the vehicle to change from the current lane to the target optimal lane is determined, to provide reasonable, safe and efficient operation opportunity support for the vehicle to change into the target optimal lane. Moreover, a prompt information about the lane change opportunity (denoted as lane change opportunity prompt information) is generated, which can be in the form of voice broadcast, visual reminder or vehicle navigation system interface to prompt the driver (or user) or provide to the assisted driving control system, to assist the vehicle to complete the lane change operation in time and accurately. In some embodiments, the lane change opportunity prompt information includes lane change suggestion time (or time range), required distance to change lane in advance, recommended lane change direction, lane change safety level evaluation result, etc., to enhance the explainability and execution rationality of the lane change operation.

[0061] Exemplarily, if the current lane of the vehicle is the same as the target optimal lane, no lane changing is needed. If the current lane of the vehicle is different from the target optimal lane, whether there is a lane changing time window satisfying the corresponding preset safety constraint condition can be determined (or predicted) in combination with the lateral distance, relative speed, traffic density of the adjacent lane, and inter-vehicle gap, and if so, corresponding lane changing time prompt information is generated. The lane changing time prompt information can be used as auxiliary information to prompt the user to perform the lane changing at a suitable time window, or as an input of the auxiliary driving control system to automatically perform the lane changing operation under the premise of safety. In some embodiments, the preset safety constraint condition includes a minimum safety distance threshold from the vehicle behind the adjacent lane, a speed difference limit, an evaluation value of the time required for lane changing, and an evaluation result of the feasibility of the lane changing trajectory, which can be dynamically updated based on the real-time traffic environment (such as the latest path exploration result).

[0062] Optionally, the optimal lane recommendation information for the vehicle is generated according to the path exploration result, including: When the first target section is congested, the unmanned aerial vehicle is controlled to continuously explore the first target section before the vehicle leaves the first target section. The optimal lane recommendation information is updated according to the updated path exploration result.

[0063] Specifically, considering the dynamic and variable traffic conditions, to further improve the real-time and accuracy of the recommended target optimal lane, especially in the case of traffic congestion on the target section (the target section currently detected by the unmanned aerial vehicle in traffic congestion is referred to as the first target section), if the first target section to be entered by the vehicle is congested (such as a speed significantly lower than the average speed, a density of vehicles in front of the vehicle exceeding a set threshold, etc.), the unmanned aerial vehicle carried on the vehicle is controlled to continuously explore the first target section dynamically before the vehicle leaves the first target section. Exemplarily, the flight path of the unmanned aerial vehicle can be adjusted in real time according to the driving direction, position, and speed of the vehicle, so that the unmanned aerial vehicle continuously detects the dynamic traffic state and road structure of the first target section before the vehicle leaves the first target section.

[0064] During the continuous exploration, the unmanned aerial vehicle updates the exploration result of the first target section based on the latest road condition images, spatial feature data, etc. acquired by the unmanned aerial vehicle, and synchronously updates the current optimal lane recommendation information based on the updated exploration result to reflect a more reasonable lane selection at the current time.

[0065] According to the optimal lane recommendation information and the current driving information of the vehicle, a lane changing time for the vehicle to change from the current lane to the target optimal lane is determined, and lane changing time prompt information is generated, including: According to the updated optimal lane recommendation information and the current driving information of the vehicle, the lane changing opportunity prompt information is updated.

[0066] Specifically, the lane changing opportunity and the target optimal lane are usually corresponding. On the basis of updating the optimal lane recommendation information, the current driving information (such as vehicle speed, acceleration, current lane, lateral distance between adjacent lanes, adjacent vehicle distance, and traffic change trend) of the vehicle is combined to re-determine the appropriate lane changing opportunity of the vehicle from the current lane into the target optimal lane, to realize the update of the lane changing opportunity and the synchronous update of the lane changing opportunity prompt information. In some embodiments, to ensure the continuous effectiveness of the lane changing suggestion, the lane changing opportunity prompt information can be updated synchronously after the optimal lane recommendation information is updated, so as to ensure that the driver or the auxiliary driving system can obtain timely and reliable lane changing decision support in a congested road section environment.

[0067] Optionally, for the update of the optimal lane recommendation information and the corresponding lane changing opportunity prompt information, a real-time updating manner can be adopted, for example, dynamic updating based on the real-time acquired path exploration result; or a certain update period can be set to limit the minimum update time interval of the optimal lane recommendation information and the lane changing opportunity prompt information, so as to avoid that the information is updated too frequently due to short-time fluctuation of the road condition or frequent refreshing of the detection data, and then too many lane changing prompts are caused, which affects the judgment of the driver or interferes with the stable operation of the auxiliary driving system. In some embodiments, the update period can be dynamically adjusted based on a preset rule. For example, when the traffic state of the first target road section changes dramatically (such as a sharp decrease in vehicle speed or a significant increase in congestion degree), the update period can be appropriately shortened to quickly respond to the change of the road condition; and when the traffic state of the target road section is relatively stable, the update period can be appropriately lengthened to reduce unnecessary lane changing prompts and lane changing operations, and to improve the user driving experience.

[0068] In combination with Figure 3 As shown in the figure, another embodiment of the present application provides a traffic assistance path exploration device, which comprises: A path exploration response unit is configured to control the UAV to perform path exploration on a target road section of a road where the vehicle is located in response to the active path exploration instruction. A lane recommendation unit is configured to generate optimal lane recommendation information for the vehicle according to the path exploration result.

[0069] The traffic assistance path exploration device of the present embodiment is used to implement the traffic assistance path exploration method described above, and has the same advantages as the traffic assistance path exploration method compared with the prior art, which will not be described here again.

[0070] In combination with Figure 4 As shown in the figure, another embodiment of the present application provides a vehicle, which comprises a memory 401 and a processor 402. The memory 401 is configured to store a computer program. The processor 402 is configured to, when executing the computer program, implement the traffic assistance path exploration method as above.

[0071] Alternatively, a vehicle comprises a memory 401 and a processor 402 coupled to the memory 401; the memory 401 is configured to store a computer program; and the processor 402 is configured to, when executing the computer program, perform the following operations: In response to the active path exploration instruction, control the UAV to explore a target road section of a road where the vehicle is located; According to the path exploration result, generate optimal lane recommendation information for the vehicle.

[0072] Another embodiment of the present application provides a computer readable storage medium, which stores a computer program; when the computer program is read and run by a processor, the traffic assistance path exploration method as above is implemented.

[0073] Alternatively, a non-volatile computer readable storage medium stores a computer program; when the computer program is executed by a processor, the processor performs the following operations: In response to the active path exploration instruction, control the UAV to explore a target road section of a road where the vehicle is located; According to the path exploration result, generate optimal lane recommendation information for the vehicle.

[0074] The technical solution of the embodiments of the present application or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various storage medium that can store program codes.

[0075] The computer readable storage medium of the present embodiment can be used to implement the traffic assistance path exploration method as above, and has the same advantages as the traffic assistance path exploration method as above compared with the prior art, which will not be repeated here.

[0076] Although the present application is disclosed as above, the protection scope of the present application is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications will fall within the protection scope of the present application.

Claims

1. A traffic-assisted route exploration method, characterized in that, Based on vehicles equipped with drones; The traffic-assisted route finding method includes: In response to an active route exploration command, the drone is controlled to explore the target road segment where the vehicle is located; Based on the route exploration results, optimal lane recommendation information is generated for the vehicle.

2. The traffic-assisted route finding method as described in claim 1, characterized in that, Before responding to an active route exploration command and controlling the drone to explore the target road segment where the vehicle is located, the traffic-assisted route exploration method includes: The active route exploration command is generated in response to a warning message about road congestion ahead of the vehicle, or in response to an active route exploration trigger command input by the driver; wherein the warning message is generated by a navigation system applied to the vehicle; and the driver includes at least one of the vehicle's user and the driver assistance control system.

3. The traffic-assisted route finding method as described in claim 1, characterized in that, The step of controlling the drone to scout the target road segment where the vehicle is located includes: The drone is controlled to explore the target road segment located within a preset distance range in front of the vehicle on the road where the vehicle is located.

4. The traffic-assisted route finding method as described in any one of claims 1-3, characterized in that, The step of responding to an active route exploration command and controlling the drone to explore the target road segment where the vehicle is located includes: In response to an active route exploration command, the drone plans a flight path to explore the target road segment based on the vehicle's navigation information; wherein the target road segment is set by the driver or the navigation system applied to the vehicle. The drone is controlled to take off from the vehicle and to explore the target road segment according to the flight path.

5. The traffic-assisted route finding method as described in any one of claims 1-3, characterized in that, The drone is equipped with an image acquisition mechanism and a spatial position measurement mechanism; controlling the drone to explore the target road segment where the vehicle is located includes: The drone is controlled to acquire road condition images of the target road segment through the image acquisition mechanism, and the spatial feature data of the target road segment is measured through the spatial position measurement mechanism. The road condition images and spatial feature data are fused and analyzed to determine the traffic condition information of each lane of the target road segment, and the traffic condition information of each lane of the target road segment is used as the road exploration result.

6. The traffic-assisted route finding method as described in claim 5, characterized in that, The step of generating optimal lane recommendation information for the vehicle based on the route exploration results includes: Based on the traffic condition information of each lane in the target road segment, and based on a preset traffic flow rate prediction model and a preset optimal lane evaluation model, the target optimal lane into which the vehicle is recommended to change is determined, and the optimal lane recommendation information corresponding to the target optimal lane is generated.

7. The traffic-assisted route finding method as described in any one of claims 1-3, characterized in that, After generating the optimal lane recommendation information for the vehicle, the traffic-assisted route finding method further includes: Based on the optimal lane recommendation information and the vehicle's current driving information, the timing for the vehicle to change lanes from its current lane to the target optimal lane is determined, and lane change timing prompt information is generated; wherein, the target optimal lane is the lane recommended by the optimal lane recommendation information.

8. The traffic-assisted route finding method as described in claim 7, characterized in that, The step of generating optimal lane recommendation information for the vehicle based on the route exploration results includes: When congestion occurs on the first target road segment, the drone is controlled to continuously explore the first target road segment before the vehicle leaves the first target road segment; The optimal lane recommendation information is updated based on the updated route exploration results. The step of determining the lane-changing timing for the vehicle to change from its current lane to the target optimal lane based on the optimal lane recommendation information and the vehicle's current driving information, and generating lane-changing timing prompt information, includes: The lane change timing prompt information is updated based on the updated optimal lane recommendation information and the vehicle's current driving information.

9. A traffic-aided route-finding device, characterized in that, include: The pathfinding response unit is used to respond to active pathfinding commands and control the UAV to explore the target road segment where the vehicle is located. The lane recommendation unit is used to generate optimal lane recommendation information for the vehicle based on the route exploration results.

10. A vehicle, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the traffic-assisted route finding method as described in any one of claims 1-8 when executing the computer program.

Images