A driving navigation system based on scene perception and dynamic multi-source fusion

Abstract

The present invention provides a driving navigation system based on scene perception and dynamic multi-source fusion. The present invention collects scenes inside and outside the car, converts satellite navigation maps and 3D digital visual maps, and finally realizes fusion navigation. The scene perception of the present invention is used to obtain scenes inside and outside the car, judge the driver's state, record the driver's bad habits, identify abnormal events in the car, or observe passengers to judge whether the passengers are sleeping or in other abnormal states, so as to remind the driver or adjust the passengers' needs. The digital map is used to mark the abnormal data outside the car obtained by scene perception, and at the same time has all the functions of the digital map of navigation in the prior art, navigation and traffic light marking; fusion navigation, dynamic navigation according to the scenes inside and outside the car, and the needs of the driver.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic navigation, and in particular to a driving navigation system based on scene perception and dynamic multi-source fusion. Background Art

[0002] At present, navigation technology is generally achieved through map data + rendering engine, path-finding algorithm + map slice loading, inertial navigation + correction filtering, TTS voice engine, intersection reminders, and real-time traffic data.

[0003] Although the existing technology can display road congestion and road navigation, the degree of congestion is determined by counting the traffic volume on the road section, and the vehicles all have GPS, which is equivalent to being able to report their own positions, which is an automatic congestion judgment mechanism. However, for accidents or maintenance sections on the road, it is mainly based on notifications from government departments and real-time uploads from drivers. For accidents and road maintenance in distant sections, it can be learned from the map, but for accidents or abnormal events that suddenly occur in very close places, within a few hundred meters or tens of meters, the vehicle's navigation still follows traffic rules. However, it is obvious that the vehicle needs to change lanes. At this time, it depends on the driver's subjective consciousness to change the road. However, if it is in an unfamiliar city, after changing lanes, whether the distance is farther or whether there are other accidents, the driver cannot know. Because the navigation system cannot automatically change the road if it does not know that there is an accident.

[0004] In addition, for the driver, if he is driving fatigue (drowsy) or has bad driving habits (such as looking at the mobile phone, smoking), he often needs to navigate on roads with fewer vehicles. The existing technology uses the shortest distance and the most time-saving navigation routes to facilitate the driver to reach the destination faster. For those with bad habits, other path options should be given, emphasizing that there are fewer vehicles and as few driving accidents as possible. Summary of the invention

[0005] The present invention provides a driving navigation system based on scene perception and dynamic multi-source fusion, so as to solve the problem that the navigation system in the prior art cannot accurately navigate and secondly cannot realize in-vehicle supervision.

[0006] A driving navigation system based on scene perception and dynamic multi-source fusion, comprising:

[0007] In-vehicle scene perception module: used to obtain real-time in-vehicle scene data with the help of at least one in-vehicle scene perception device;

[0008] The vehicle exterior scene perception module is used to obtain real-time vehicle exterior scene data with the help of at least one laser radar and a vehicle exterior scene perception device;

[0009] Satellite map module: used to obtain satellite positioning signals, generate an initial navigation map, and generate vehicle guide lines on the initial navigation map;

[0010] Digital map module: used to merge the real-time in-vehicle scene data and the real-time out-vehicle scene data into the initial navigation map to generate a 3D digital visual map;

[0011] Dynamic decision module: used to mark the requirements inside and outside the vehicle through the 3D digital visual map, and determine the corresponding navigation mode according to the requirement mark; wherein,

[0012] The navigation modes include: obstacle avoidance navigation mode for external vehicle needs, automatic driving navigation mode for internal vehicle needs, and conventional navigation mode;

[0013] Fusion navigation module: used to generate a global visual navigation route according to the navigation mode.

[0014] Further: the in-vehicle scene perception module includes:

[0015] The first perception device unit is used to pre-configure the in-vehicle scene perception device in the vehicle, wherein:

[0016] The in-vehicle scene sensing device includes: a temperature sensor, a light intensity sensor, an odor sensor and a multi-angle video monitor;

[0017] The multi-angle video monitor is installed on the roof, the bottom and the door of the vehicle respectively, and the multi-angle video monitor is an adaptive angle adjustment video monitor;

[0018] In-vehicle environment information acquisition unit: used to connect to the in-vehicle scene perception device and obtain real-time in-vehicle environment information through the vehicle terminal;

[0019] The real-time environmental information includes: temperature, light intensity and odor;

[0020] The vehicle occupant information collection unit is used to obtain the vehicle occupant information through the vehicle occupant monitoring device;

[0021] The in-car personnel information includes: passenger number information and passenger distribution information;

[0022] The vehicle occupant age determination unit is used to determine the age of the vehicle occupants and the driver and output age information;

[0023] The age groups include: infants, teenagers, young adults, middle-aged adults and the elderly;

[0024] Driver posture recognition unit: used to capture the driver's posture and obtain the driver's real-time motion information;

[0025] Passenger posture recognition unit: used to capture the passenger's posture and obtain the passenger's real-time movement information;

[0026] Scene data aggregation unit: used to aggregate the real-time environmental information, age group information, driver's real-time action information and passenger's real-time action information to generate in-vehicle scene data.

[0027] Further: the vehicle exterior scene perception module includes:

[0028] The second perception device configuration unit is used to configure the vehicle exterior scene perception device on the vehicle side in advance, wherein:

[0029] The vehicle exterior scene sensing device includes: an infrared ranging camera device and a high-precision image acquisition device;

[0030] The infrared ranging camera device and the high-precision image acquisition device are respectively arranged at the front and rear of the vehicle;

[0031] Lidar unit: used to obtain road information through Lidar, and perform real-time road simulation on different lanes to generate simulated lanes;

[0032] The real-time road simulation includes real-time simulation of different elements on the lane;

[0033] External vehicle sensing unit: used to obtain external vehicle sensing data in real time through external vehicle scene sensing equipment;

[0034] The vehicle exterior sensing data includes: high-precision images outside the vehicle, real-time distances of different elements, and traffic signs;

[0035] A perception correction unit: used for correcting the perception data of the simulated lane by using the external perception data of the vehicle to generate a high-precision simulation scene;

[0036] The vehicle exterior scene acquisition unit is used to generate real-time vehicle exterior scene data through the high-precision simulation scene.

[0037] Further: the satellite map module includes:

[0038] Trajectory capture unit: used to locate the vehicle in real time according to the satellite positioning signal and generate the vehicle's driving trajectory;

[0039] Initial navigation map unit: used to obtain the destination set by the user and generate the initial planned route;

[0040] A driving route unit: used for marking vehicle density on the initial planned route and using the driving trajectory as a driving line on the initial planned route;

[0041] A guidance route unit is used to determine whether the real-time driving trajectory changes the initial planned route according to the driving route, and to generate a second planned route after the initial planned route changes, and to generate a vehicle guidance route in the second planned route; wherein,

[0042] The number of the second planned routes is not less than 2;

[0043] Each of the second planned routes has a separate planning attribute; wherein,

[0044] The planning attributes include at least: shortest time, shortest mileage, smallest traffic volume and least number of accidents.

[0045] Further: the digital map module includes:

[0046] An in-vehicle scene analysis unit is used to analyze the in-vehicle scene data, determine the posture characteristics of different people in the vehicle, and judge the real-time in-vehicle needs through the posture characteristics to generate a first demand text;

[0047] The vehicle exterior scene analysis unit is used to analyze the vehicle exterior scene data, generate element features of different lanes outside the vehicle, generate lane data and element distribution data, and determine whether there is an avoidance requirement based on the element features, and generate a second requirement text;

[0048] A first fusion unit: configured to perform 3D conversion on the initial navigation map according to the lane data and the element distribution data to generate a 3D map;

[0049] The second fusion unit is used to determine the real-time demand information according to the first demand text and the second demand text, generate a marking box in the 3D map and fill in the real-time demand information, and generate a 3D digital visual map.

[0050] Further: the dynamic decision module includes:

[0051] Demand marking unit: used to mark the requirements inside and outside the vehicle according to the 3D digital visual map; wherein,

[0052] The in-vehicle and out-of-vehicle demand marks include: obstacle avoidance marks, passenger demand marks and driver demand marks;

[0053] Tag analysis unit: used to determine the demand type and demand object according to the vehicle internal and external demand tags, and set the navigation behavior; wherein,

[0054] The demand types include: driver demand, passenger demand and vehicle obstacle avoidance demand;

[0055] The navigation behavior includes: speed control behavior, stability adjustment behavior, driver reminder behavior, path regulation behavior, obstacle reminder behavior, lane switching behavior and vehicle automatic braking behavior;

[0056] Navigation mode decision unit: used to construct a navigation mode according to the navigation behavior, demand type and demand object, and set the navigation control parameters corresponding to the navigation mode; wherein

[0057] The construction of navigation mode includes:

[0058] When the requirement type is obstacle avoidance, an obstacle avoidance navigation mode for the vehicle's external requirements is generated.

[0059] When the demand objects are passengers and drivers, generating an automatic driving navigation mode for in-vehicle demand;

[0060] When there is no requirement type and requirement object, a normal navigation mode is generated.

[0061] Further: the obstacle avoidance navigation mode for generating off-vehicle requirements includes:

[0062] Determining real-time demand information according to the in-vehicle and out-vehicle demand tags;

[0063] Determine the corresponding lane and vehicle distance according to the real-time demand information;

[0064] generating a lane switching instruction and a speed adjustment instruction according to the lane and the 3D digital visual map;

[0065] Determining a lane to switch to according to the lane switching instruction;

[0066] According to the speed adjustment instruction, real-time speed adjustment is performed, and according to the real-time speed adjustment, the time and distance for switching lanes are determined, and corresponding external vehicle control parameters are generated. Through the external vehicle control parameters, an obstacle avoidance navigation mode required outside the vehicle is generated.

[0067] Further: the autonomous driving navigation mode for generating in-vehicle requirements includes:

[0068] According to the vehicle internal and external demand mark, the demand object is determined; wherein,

[0069] The demand object is a driver or a passenger;

[0070] Determine corresponding demand information according to the demand object;

[0071] According to the demand information, the automatic driving navigation mode required in the vehicle is set; wherein,

[0072] When the demand object is a driver, the automatic driving mode is started, and a corresponding voice command is generated according to the demand information;

[0073] When the demand object is a passenger, the semi-automatic driving mode is started and vehicle adjustment information is sent to the driver.

[0074] Further: the generating of the conventional navigation mode comprises:

[0075] According to the 3D digital visual map, displaying the destination on the 3D digital visual map;

[0076] Determine, based on the destination display, congested sections and low-traffic sections in the planned route;

[0077] According to the congested road section and the low-traffic road section, dynamic route guidance is performed on the 3D digital visual map, and the dynamic route guidance is used as a regular navigation mode.

[0078] Further: the fusion navigation module includes:

[0079] Navigation selection unit: used for selecting a corresponding navigation control scheme according to the navigation mode;

[0080] A global navigation unit: used to display global navigation route information in real time on a display device in the vehicle according to the navigation control scheme;

[0081] Visual display unit: used to simulate the user's location inside and outside the vehicle according to the global navigation route information, and to visualize the simulated scene.

[0082] The beneficial effects of the present invention are as follows: the present invention can collect the motion posture data of passengers and the motion posture data of the driver, so that the environment in the car can be adjusted and voice reminders can be given according to the motion postures of the driver and the passengers during navigation, and the speed of the car can also be assisted in adjustment. The most important thing about the above scheme is the collection of the environment in the car and the construction of the scene in the car, so as to obtain a clearer and more realistic simulation scene and more accurate scene data, and the real-time needs of the people in the car can be judged through the scene data.

[0083] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0084] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0086] Figure 1 A system composition diagram of a driving navigation system based on scene perception and dynamic multi-source fusion in an embodiment of the present invention;

[0087] Figure 2 It is a configuration flow chart of the obstacle avoidance navigation mode required outside the vehicle in an embodiment of the present invention;

[0088] Figure 3 The configuration flow chart of the automatic driving navigation mode for in-vehicle requirements in an embodiment of the present invention;

[0089] Figure 4 This is a configuration flow chart of the conventional navigation mode in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0091] As attached Figure 1 As shown, the present invention is a driving navigation system based on scene perception and dynamic multi-source fusion, comprising:

[0092] In-vehicle scene perception module: used to obtain real-time in-vehicle scene data with the help of at least one in-vehicle scene perception device;

[0093] The vehicle exterior scene perception module is used to obtain real-time vehicle exterior scene data with the help of at least one laser radar and a vehicle exterior scene perception device;

[0094] Satellite map module: used to obtain satellite positioning signals, generate an initial navigation map, and generate vehicle guide lines on the initial navigation map;

[0095] Digital map module: used to merge the real-time in-vehicle scene data and the real-time out-vehicle scene data into the initial navigation map to generate a 3D digital visual map;

[0096] Dynamic decision module: used to mark the requirements inside and outside the vehicle through the 3D digital visual map, and determine the corresponding navigation mode according to the requirement mark; wherein,

[0097] The navigation modes include: obstacle avoidance navigation mode for external vehicle needs, automatic driving navigation mode for internal vehicle needs, and conventional navigation mode;

[0098] Fusion navigation module: used to generate a global visual navigation route according to the navigation mode.

[0099] The principle of the above technical solution is: Figure 1 As shown, the present invention consists of six modules: scene perception is used to obtain scenes inside and outside the car, judge the driver's state, record the driver's bad habits, identify abnormal events in the car, or observe passengers to judge whether the passengers are sleeping or in other abnormal states, so as to remind the driver or adjust the passenger's needs. The digital map is used to mark the abnormal data outside the car obtained by scene perception, and at the same time has all the functions of the digital map of navigation in the prior art, navigation and traffic light marking; fusion navigation, dynamic navigation according to the scenes inside and outside the car, and the needs of the driver. When the driver has bad habits or driving fatigue in the car, roads with fewer vehicles are recommended first, and real-time dynamic navigation route adjustment is performed. For abnormalities outside the car, the navigation route is adjusted according to the cause of the abnormality. At the same time, there is also face recognition for driver authentication.

[0100] The present invention includes a variety of different navigation modes, and different navigation modes have different control methods. The advantage of the present invention over the prior art is that the present invention can identify user needs, which may be subjective behavioral needs or unconscious needs due to fatigue or stuffiness, and realize automated vehicle control needs based on these needs, so that the navigation system of the present invention and the intelligent driving system are integrated to achieve highly intelligent driving and intelligent control functions.

[0101] The beneficial effects of the above technical solution are: the present invention can achieve more accurate navigation, the present invention can achieve 3D visual navigation, and can adjust the vehicle speed and activate the required functions in real time according to the unconscious needs and active needs of the passengers in the car, and can display the navigation information visually in real time, and finally perform intelligent navigation that integrates intelligent driving and intelligent navigation.

[0102] Further: the in-vehicle scene perception module includes:

[0103] The first perception device configuration unit is used to pre-configure the in-vehicle scene perception device in the vehicle, wherein:

[0104] The in-vehicle scene sensing device includes: a temperature sensor, a light intensity sensor, an odor sensor and a multi-angle video monitor;

[0105] The multi-angle video monitor is installed on the roof, the bottom and the door of the vehicle respectively, and the multi-angle video monitor is an adaptive angle adjustment video monitor;

[0106] In-vehicle environment information acquisition unit: used to connect to the in-vehicle scene perception device and obtain real-time in-vehicle environment information through the vehicle terminal;

[0107] The real-time environmental information includes: temperature, light intensity and odor;

[0108] The vehicle occupant information collection unit is used to obtain the vehicle occupant information through the vehicle occupant monitoring device;

[0109] The in-car personnel information includes: passenger number information and passenger distribution information;

[0110] The vehicle occupant age determination unit is used to determine the age of the vehicle occupants and the driver and output age information;

[0111] The age groups include: infants, teenagers, young adults, middle-aged adults and the elderly;

[0112] Driver posture recognition unit: used to capture the driver's posture and obtain the driver's real-time motion information;

[0113] Passenger posture recognition unit: used to capture the passenger's posture and obtain the passenger's real-time movement information;

[0114] Scene data aggregation unit: used to aggregate the real-time environmental information, age group information, driver's real-time action information and passenger's real-time action information to generate in-vehicle scene data.

[0115] The principle of the above technical solution is:

[0116] When the present invention collects scenes in the car, it mainly analyzes the motion posture of people through multiple sensors and videos to determine whether the people in the car are in a normal sitting position, sleeping or other states. The judgment is divided into two aspects, one is the posture judgment of the driver, and the other is the posture judgment of the passengers in the car; when the present invention perceives, in addition to the posture of the human body, it also perceives the environment in the car, so as to ensure that the air conditioner can be automatically turned on when the temperature is too high. And it can identify the age of the passengers, so as to facilitate the recognition of the passenger's motion posture, because passengers of different age groups, or passengers of different ages, may have different movement amplitudes and movement methods. The in-car environment information also includes illumination and odor, because if the illumination is high and there are passengers in the car who need to sleep, the illumination is automatically adjusted, for example, the glass is changed to a dark mode. If the illumination in the car is too low, the illumination can also be increased. For the smell, the ventilation equipment in the car will be controlled to automatically ventilate.

[0117] The beneficial effect of the above technical solution is that the present invention can collect the motion posture data of passengers and the motion posture data of the driver, so that the environment in the car can be adjusted and voice reminders can be given according to the motion postures of the driver and the passengers during navigation, and the speed of the car can also be assisted in adjusting. The most important thing about the above solution is the collection of the in-car environment and the construction of the in-car scene, so as to obtain a clearer and more realistic simulation scene and more accurate scene data, and the real-time needs of the people in the car can be judged through the scene data.

[0118] Further: the vehicle exterior scene perception module includes:

[0119] The second perception device configuration unit is used to configure the vehicle exterior scene perception device on the vehicle side in advance, wherein:

[0120] The vehicle exterior scene sensing device includes: an infrared ranging camera device and a high-precision image acquisition device;

[0121] The infrared ranging camera device and the high-precision image acquisition device are respectively arranged at the front and rear of the vehicle;

[0122] Lidar unit: used to obtain road information through Lidar, and perform real-time road simulation on different lanes to generate simulated lanes;

[0123] The real-time road simulation includes real-time simulation of different elements on the lane;

[0124] External vehicle sensing unit: used to obtain external vehicle sensing data in real time through external vehicle scene sensing equipment;

[0125] The vehicle exterior sensing data includes: high-precision images outside the vehicle, real-time distances of different elements, and traffic signs;

[0126] A perception correction unit: used for correcting the perception data of the simulated lane by using the external perception data of the vehicle to generate a high-precision simulation scene;

[0127] The vehicle exterior scene acquisition unit is used to generate real-time vehicle exterior scene data through the high-precision simulation scene.

[0128] The principle of the above technical solution is:

[0129] The biggest problem with laser radar in the prior art is that it is impossible to know whether the laser radar is accurately measured, and whether an accurate result is obtained to judge whether the distance between different vehicles in the perceived scene area is accurate. In addition to laser radar in the prior art, there is generally a camera device to assist in scene perception, which is the basis of autonomous driving technology, but the general autonomous driving technology is combined with the navigation system. The present invention has made improvements in this regard. The laser radar technology of the present invention is to construct a lane-level three-dimensional scene map, realize road simulation, and generate a simulated lane. There are different elements on the simulated lane, mainly vehicles, pedestrians and other obstacles that may exist in the lane. However, it is only a laser radar, which is not necessarily accurate. The present invention uses the perception data of the vehicle side to correct the data perceived by the laser radar, because the laser radar perception can only perceive the distance, while the camera device can not only correct the distance, but also correct the authenticity of different elements. That is, pedestrians and vehicles are displayed more clearly on the lane simulated by the road, and abnormal phenomena on the road are also displayed, for example, there are pits or road collapses that need to be repaired in the lane, or car accidents and other phenomena.

[0130] The principle of the above technical solution is that the present invention is an intelligent navigation system combined with assisted driving function, because it is necessary to maintain various needs during driving, including driving needs, and better experience needs for riding and driving vehicles. Therefore, in terms of scene perception, the present invention builds external real-time scenes in a variety of ways to make the external real-time scenes look more real.

[0131] Further: the satellite map module includes:

[0132] The satellite map module includes:

[0133] Trajectory capture unit: used to locate the vehicle in real time according to the satellite positioning signal and generate the vehicle's driving trajectory;

[0134] Initial navigation map unit: used to obtain the destination set by the user and generate the initial planned route;

[0135] A driving route unit: used for marking vehicle density on the initial planned route and using the driving trajectory as a driving line on the initial planned route;

[0136] A guidance route unit is used to determine whether the real-time driving trajectory changes the initial planned route according to the driving route, and to generate a second planned route after the initial planned route changes, and to generate a vehicle guidance route in the second planned route; wherein,

[0137] The number of the second planned routes is not less than 2;

[0138] Each of the second planned routes has a separate planning attribute; wherein,

[0139] The planning attributes include at least: shortest time, shortest mileage, smallest traffic volume and least number of accidents.

[0140] The principle of the above technical solution is that the 3D digital visual map obtained by the present invention needs to be converted through two maps. The first is the initial navigation map obtained by the satellite. This map is based on the destination set by the user and the initial route is obtained by initial positioning. The initial navigation map will mark the vehicle density on the driving route, and will continuously update and generate planned routes based on the driver's driving. Each planned route has a vehicle guide line. The 3D digital visual map can not only see the navigation route, but also the real-time demand information inside and outside the vehicle, and real-time control and adjustment information. Of course, the vehicle to which the present invention is applicable is also a smart vehicle with a split-screen display of multiple screens.

[0141] The beneficial effect of the above technical solution is that the present invention determines the driving trajectory of the vehicle through the micro-signal, and plans the first planned route, that is, the initial driving route, which is the same navigation technology as the prior art, and the difference is that for route guidance, the present invention can judge whether it is necessary to replace the original planned route according to the driving trajectory and generate a guide line. Of course, the prior art also has the re-planning of the planned route after changing the initial driving trajectory. The difference between the present invention and the prior art is that the present invention will mark the vehicle density when planning the route, and different planning attributes are set. There are many routes, and there are different planning attributes for each route. These planning attributes are set by themselves, and can also be set to the route with the most restaurants or no epidemic.

[0142] Further: the digital map module includes:

[0143] An in-vehicle scene analysis unit is used to analyze the in-vehicle scene data, determine the posture characteristics of different people in the vehicle, and judge the real-time in-vehicle needs through the posture characteristics to generate a first demand text;

[0144] The vehicle exterior scene analysis unit is used to analyze the vehicle exterior scene data, generate element features of different lanes outside the vehicle, generate lane data and element distribution data, and determine whether there is an avoidance requirement based on the element features, and generate a second requirement text;

[0145] A first fusion unit: configured to perform 3D conversion on the initial navigation map according to the lane data and the element distribution data to generate a 3D map;

[0146] The second fusion unit is used to determine the real-time demand information according to the first demand text and the second demand text, generate a marking box in the 3D map and fill in the real-time demand information, and generate a 3D digital visual map.

[0147] The principle of the above technical solution is that the present invention generates visual data of 3D digital visual map mainly in that it can judge what demand is generated according to the in-car scene data and the out-car scene data in real time, and generate the corresponding demand text to be displayed in the annotation box on the 3D digital visual map. The present invention performs in-car analysis and out-car analysis respectively. The in-car analysis can judge the status of the passengers. For example, if the passenger is sleeping, the demand is to reduce the speed and keep the vehicle stable. Or if the driver is tired, the navigation system of the vehicle will give a fatigue reminder. Outside the car is the demand to avoid different obstacles and switch lanes. All these demands are displayed on the 3D digital visual map.

[0148] The beneficial effect of the above technical solution is that the 3D digital map generated by the present invention can mark different needs of users, including needs inside the car and needs outside the car. Through these needs, different vehicle intelligent navigation functions can be realized.

[0149] Further: the dynamic decision module includes:

[0150] Demand marking unit: used to mark the requirements inside and outside the vehicle according to the 3D digital visual map; wherein,

[0151] The in-vehicle and out-of-vehicle demand marks include: obstacle avoidance marks, passenger demand marks and driver demand marks;

[0152] Tag analysis unit: used to determine the demand type and demand object according to the vehicle internal and external demand tags, and set the navigation behavior; wherein,

[0153] The demand types include: driver demand, passenger demand and vehicle obstacle avoidance demand;

[0154] The navigation behavior includes: speed control behavior, stability adjustment behavior, driver reminder behavior, path regulation behavior, obstacle reminder behavior, lane switching behavior and vehicle automatic braking behavior;

[0155] Navigation mode decision unit: used to construct a navigation mode according to the navigation behavior, demand type and demand object, and set the navigation control parameters corresponding to the navigation mode; wherein

[0156] The construction of navigation mode includes:

[0157] When the requirement type is obstacle avoidance, an obstacle avoidance navigation mode for the vehicle's external requirements is generated.

[0158] When the demand objects are passengers and drivers, generating an automatic driving navigation mode for in-vehicle demand;

[0159] When there is no requirement type and requirement object, a normal navigation mode is generated.

[0160] The principle of the above technical solution is that the dynamic decision module is used to make navigation mode decisions. The present invention has three navigation modes, namely, obstacle avoidance navigation mode for external vehicle requirements, automatic driving navigation mode for internal vehicle requirements, and conventional navigation mode. These three modes are determined according to the demand tags after analyzing the demand tags to obtain the demand tags, namely, speed control behavior, stability adjustment behavior, driver reminder behavior, path control behavior, obstacle reminder behavior, lane switching behavior, and vehicle automatic braking behavior. Through these behaviors, navigation decisions are realized and the corresponding functions of navigation execution are set.

[0161] The beneficial effect of the above technical solution is that after the present invention converts the navigation map into a 3D digital visual map, it can mark the needs inside and outside the vehicle on the 3D digital visual map. These needs are reflected in the form of a demand box. For these needs, the corresponding setting parameters of the vehicle will be set, and the corresponding navigation control will be performed through the set parameters.

[0162] Further: As attached Figure 2 As shown, the obstacle avoidance navigation mode for generating external vehicle requirements includes:

[0163] Determine the corresponding lane and vehicle distance according to the real-time demand information;

[0164] generating a lane switching instruction and a speed adjustment instruction according to the lane and the 3D digital visual map;

[0165] Determining a lane to switch to according to the lane switching instruction;

[0166] According to the speed adjustment instruction, real-time speed adjustment is performed, and according to the real-time speed adjustment, the time and distance for switching lanes are determined, and corresponding external vehicle control parameters are generated. Through the external vehicle control parameters, an obstacle avoidance navigation mode required outside the vehicle is generated.

[0167] The principle of the above technical solution is that when performing obstacle avoidance navigation according to external needs, the vehicle's own position, the lane and distance where the obstacle is located will be determined based on the demand information, and the corresponding switching instructions will be generated through the 3D digital visual map. The vehicle's control parameters will be set according to the switching instructions, the vehicle will be controlled to avoid obstacles, automatically remind lane changes, and automatically control the vehicle to change lanes.

[0168] The beneficial effect of the above technical solution is that: when the present invention is in the dynamic obstacle avoidance navigation mode, the driver's active control of obstacle avoidance is not required, but the driver's control behavior is mainly used to adjust the vehicle's lane and speed to avoid obstacles and prevent accidents.

[0169] Further: As attached Figure 3 As shown, the autonomous driving navigation mode for generating in-vehicle requirements includes:

[0170] According to the vehicle internal and external demand mark, the demand object is determined; wherein,

[0171] The demand object is a driver or a passenger;

[0172] Determine corresponding demand information according to the demand object;

[0173] According to the demand information, the automatic driving navigation mode required in the vehicle is set; wherein,

[0174] When the demand object is a driver, the automatic driving mode is started, and a corresponding voice command is generated according to the demand information;

[0175] When the demand object is a passenger, the semi-automatic driving mode is started and vehicle adjustment information is sent to the driver.

[0176] The principle of the above technical solution is that in the automatic driving navigation mode of in-vehicle needs, the corresponding demand object can be determined according to the demand marks inside and outside the vehicle, and it can be judged whether the passenger has the need to adjust the vehicle or the driver has the need to adjust the vehicle. In this way, when there is a need, the driver can be reminded by voice, and the speed of the vehicle can be adjusted through the automatic driving mode and semi-automatic driving mode.

[0177] The principle of the above technical solution is that the innovation of the present invention lies in that a corresponding automatic navigation driving mode will be set for the demand information of the driver and the passenger. Through this driving mode, the vehicle can be controlled to perform semi-automatic driving, and corresponding voice commands can also be generated. The voice commands are used to broadcast the vehicle startup function.

[0178] Further: As attached Figure 4 As shown, generating a conventional navigation mode includes:

[0179] According to the 3D digital visual map, displaying the destination on the 3D digital visual map;

[0180] Determine, based on the destination display, congested sections and low-traffic sections in the planned route;

[0181] According to the congested road section and the low-traffic road section, dynamic route guidance is performed on the 3D digital visual map, and the dynamic route guidance is used as a regular navigation mode.

[0182] The principle of the above technical solution is that for the conventional navigation mode, a conventional navigation route is generated according to the destination, and conventional guided navigation is performed through marking congested sections and low-traffic sections.

[0183] The beneficial effect of the above technical solution is that: the present invention is similar to the navigation mode of the prior art in terms of conventional navigation mode, but the present invention is a dynamic navigation mode that can display congested sections and low-traffic sections in the planned route, so that the route can be dynamically adjusted in advance.

[0184] Further: the fusion navigation module includes:

[0185] Navigation selection unit: used for selecting a corresponding navigation control scheme according to the navigation mode;

[0186] A global navigation unit: used to display global navigation route information in real time on a display device in the vehicle according to the navigation control scheme;

[0187] Visual display unit: used to simulate the user's location inside and outside the vehicle according to the global navigation route information, and to visualize the simulated scene.

[0188] The principle of the above technical solution is: during the stage of integrated navigation, the parameters that need to be adjusted for the vehicle and the corresponding navigation control method will be determined according to the navigation mode, the global navigation route information will be displayed through the display screen inside the car, and the simulated scenes inside and outside the car will be visualized based on this route information.

[0189] The beneficial effect of the above technical solution is that after integrating navigation, the present invention can realize the visual display of simulated scenes. In this way, when navigating, if there is any danger or risk information inside or outside the car and the driver has not discovered it, it can be displayed in the simulated scene of the vehicle. However, this function is not available in the existing market.

[0190] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A driving navigation system based on scene perception and dynamic multi-source fusion, characterized in that: include: In-vehicle scene perception module: used to obtain real-time in-vehicle scene data with the help of at least one in-vehicle scene perception device; The vehicle exterior scene perception module is used to obtain real-time vehicle exterior scene data with the help of at least one laser radar and a vehicle exterior scene perception device; Satellite map module: used to obtain satellite positioning signals, generate an initial navigation map, and generate vehicle guide lines on the initial navigation map; Digital map module: used to merge the real-time in-vehicle scene data and the real-time out-vehicle scene data into the initial navigation map to generate a 3D digital visual map; Dynamic decision module: used to mark the requirements inside and outside the vehicle through the 3D digital visual map, and determine the corresponding navigation mode according to the requirement mark; wherein, The navigation modes include: obstacle avoidance navigation mode for external vehicle needs, automatic driving navigation mode for internal vehicle needs, and conventional navigation mode; Fusion navigation module: used to generate a global visual navigation route according to the navigation mode; The in-vehicle scene perception module includes: The first perception device configuration unit is used to pre-configure the in-vehicle scene perception device in the vehicle, wherein: The in-vehicle scene sensing device includes: a temperature sensor, a light intensity sensor, an odor sensor and a multi-angle video monitor; The multi-angle video monitor is installed on the roof, the bottom of the vehicle and the door respectively, and the multi-angle video monitor is an adaptive angle adjustment video monitor; In-vehicle environment information acquisition unit: used to connect to the in-vehicle scene perception device and obtain real-time in-vehicle environment information through the vehicle terminal; The real-time environmental information includes: temperature, light intensity and odor; The vehicle interior personnel information collection unit is used to obtain the vehicle interior personnel information through the vehicle interior monitoring equipment; wherein the vehicle interior personnel information includes: passenger number information and passenger distribution information; The vehicle occupant age determination unit is used to determine the age of the vehicle occupants and the driver and output age information; The age groups include: infants, teenagers, young adults, middle-aged adults and the elderly; Driver posture recognition unit: used to capture the driver's motion posture and obtain the driver's real-time motion information; Passenger posture recognition unit: used to capture the passenger's motion posture and obtain the passenger's real-time motion information; A scene data aggregation unit: used to aggregate the real-time environment information, age group information, driver's real-time action information and passenger's real-time action information to generate in-vehicle scene data; The vehicle exterior scene perception module comprises: The second sensing device configuration unit is used to configure the vehicle exterior scene sensing device on the vehicle side in advance, wherein the vehicle exterior scene sensing device includes: an infrared ranging camera device and a high-precision image acquisition device; The infrared ranging camera device and the high-precision image acquisition device are respectively arranged at the front and rear of the vehicle; Lidar unit: used to obtain road information through Lidar, and perform real-time road simulation on different lanes to generate simulated lanes; The real-time road simulation includes real-time simulation of different elements on the lane; The vehicle exterior sensing unit is used to obtain the vehicle exterior sensing data in real time through the vehicle exterior scene sensing device; wherein the vehicle exterior sensing data includes: high-precision images outside the vehicle, real-time distances of different elements and traffic signs; A perception correction unit: used for correcting the perception data of the simulated lane by using the external perception data of the vehicle to generate a high-precision simulation scene; The vehicle exterior scene acquisition unit is used to generate real-time vehicle exterior scene data through the high-precision simulation scene; The digital map module comprises: In-vehicle scene analysis unit: used to analyze the in-vehicle scene data, determine the posture characteristics of different people in the car, and judge the real-time in-vehicle needs through the posture characteristics to generate a first demand text; The vehicle exterior scene analysis unit is used to analyze the vehicle exterior scene data, generate element features of different lanes outside the vehicle, generate lane data and element distribution data, and determine whether there is an avoidance requirement based on the element features, and generate a second requirement text; The first fusion unit is used to convert the initial navigation map into 3D according to the lane data and element distribution data to generate a 3D map; The second fusion unit is used to determine the real-time demand information according to the first demand text and the second demand text, generate a marking box in the 3D map and fill in the real-time demand information, and generate a 3D digital visual map.

2. A driving navigation system based on scene perception and dynamic multi-source fusion as claimed in claim 1, characterized in that: The satellite map module includes: Trajectory capture unit: used to locate the vehicle in real time according to the satellite positioning signal and generate the vehicle's driving trajectory; Initial navigation map unit: used to obtain the destination set by the user and generate the initial planned route; A driving route unit: used for marking vehicle density on the initial planned route and using the driving trajectory as a driving line on the initial planned route; A guidance route unit is used to determine whether the real-time driving trajectory changes the initial planned route according to the driving route, and to generate a second planned route after the initial planned route changes, and to generate a vehicle guidance route in the second planned route; wherein, The number of the second planned routes is not less than 2; Each of the second planned routes has a separate planning attribute; wherein, The planning attributes include at least: shortest time, shortest mileage, smallest traffic volume and least number of accidents.

3. The driving navigation system based on scene perception and dynamic multi-source fusion as claimed in claim 1, characterized in that: The dynamic decision module includes: A demand marking unit: used to mark the vehicle's internal and external demand based on the 3D digital visual map; wherein the vehicle's internal and external demand marking includes: obstacle avoidance marking, passenger demand marking and driver demand marking; Tag analysis unit: used to determine the demand type and demand object according to the vehicle internal and external demand tags, and set the navigation behavior; wherein, The demand types include: driver demand, passenger demand and vehicle obstacle avoidance demand; The navigation behavior includes: speed control behavior, stability adjustment behavior, driver reminder behavior, path regulation behavior, obstacle reminder behavior, lane switching behavior and vehicle automatic braking behavior; Navigation mode decision unit: used to construct a navigation mode according to the navigation behavior, requirement type and requirement object, and Setting navigation control parameters corresponding to the navigation mode; wherein constructing the navigation mode comprises: When the requirement type is obstacle avoidance, an obstacle avoidance navigation mode for the vehicle's external requirements is generated. When the demand objects are passengers and drivers, an automatic driving navigation mode for in-vehicle demand is generated; when there is no demand type and demand object, a conventional navigation mode is generated.

4. A driving navigation system based on scene perception and dynamic multi-source fusion as claimed in claim 3, characterized in that: The obstacle avoidance navigation mode for generating external vehicle requirements includes: Determining real-time demand information according to the in-vehicle and out-vehicle demand tags; Determine the corresponding lane and vehicle distance according to the real-time demand information; generating a lane switching instruction and a speed adjustment instruction according to the lane and the 3D digital visual map; and determining a lane to be switched according to the lane switching instruction; According to the speed adjustment instruction, real-time speed adjustment is performed, and according to the real-time speed adjustment, the time and distance for switching lanes are determined, and corresponding external vehicle control parameters are generated. Through the external vehicle control parameters, an obstacle avoidance navigation mode required outside the vehicle is generated.

5. The driving navigation system based on scene perception and dynamic multi-source fusion as claimed in claim 3, characterized in that: The autonomous driving navigation mode for generating in-vehicle requirements includes: Determine the demand object according to the in-vehicle and out-of-vehicle demand mark; wherein the demand object is the driver or the passenger; Determine corresponding demand information according to the demand object; According to the demand information, the automatic driving navigation mode required in the vehicle is set; wherein, When the demand object is a driver, the automatic driving mode is started, and a corresponding voice command is generated according to the demand information; When the demand object is a passenger, the semi-automatic driving mode is started and vehicle adjustment information is sent to the driver.

6. The driving navigation system based on scene perception and dynamic multi-source fusion as claimed in claim 3, characterized in that: The generating of the conventional navigation mode comprises: According to the 3D digital visual map, displaying the destination on the 3D digital visual map; and determining the congested road sections and low-traffic road sections in the planned route according to the destination display; According to the congested road section and the low-traffic road section, dynamic route guidance is performed on the 3D digital visual map, and the dynamic route guidance is used as a regular navigation mode.

7. The driving navigation system based on scene perception and dynamic multi-source fusion as claimed in claim 1, characterized in that: The fusion navigation module includes: Navigation selection unit: used for selecting a corresponding navigation control scheme according to the navigation mode; A global navigation unit: used to display global navigation route information in real time on a display device in the vehicle according to the navigation control scheme; Visual display unit: used to simulate the user's location inside and outside the vehicle according to the global navigation route information, and to visualize the simulated scene.

Images