Intelligent driving obstacle avoidance method and system based on multi-source information fusion processing

Abstract

The invention provides an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing, and the method comprises the steps: collecting the real-time vehicle driving information and real-time environment image information of a vehicle during the driving of the vehicle, constructing an environment map, screening obstacles in a specified range of the vehicle, and generating a plurality of obstacle avoidance paths, the method comprises the steps of collecting real-time driver operation information of a vehicle, analyzing the response operation duration corresponding to each obstacle avoidance path executed by a driver, estimating the obstacle avoidance success rate of the corresponding obstacle avoidance path, collecting historical driving data of the vehicle, optimizing the target obstacle avoidance path with the highest obstacle avoidance success rate, constructing an optimized obstacle avoidance path, and obtaining the optimal obstacle avoidance path. According to the method, the driver is assisted to execute the optimized obstacle avoidance path, the optimized obstacle avoidance path is adjusted in real time according to the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance, and the obstacles encountered during vehicle driving can be constructed by collecting the information of the driver, the information of the vehicle and the information of the environment and then performing unified processing, so that avoidance can be performed in time.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving obstacle avoidance technology, and in particular to an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing. Background Art

[0002] With the development of artificial intelligence, more and more products are moving towards networking, intelligence and unmanned operation, especially in the automotive industry. Cars have been redefined as smart mobile terminals, shifting from mechanical products to electronic products. Smart driving vehicles and their related ecosystems have achieved unprecedented development, but unmanned operation is not something that can be achieved overnight. And with the development of artificial intelligence, more and more products are moving towards networking, intelligence and unmanned operation, especially in the automotive industry. Cars have been redefined as smart mobile terminals, shifting from mechanical products to electronic products. Against such a strong background, smart driving vehicles and their related ecosystems have achieved unprecedented development, but unmanned operation is not something that can be achieved overnight. It requires continuous accumulation of technology, iteration and gradual development, and there are still many core technologies that need to be solved.

[0003] Existing intelligent obstacle avoidance technology can only provide obstacle avoidance paths, but different drivers have different driving habits. The intelligently generated obstacle avoidance paths are difficult to adapt to different drivers, resulting in low obstacle avoidance quality.

[0004] Therefore, the present invention provides an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing. Summary of the invention

[0005] The present invention provides an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing, which can collect driver information, vehicle information and environmental information, and then uniformly process and construct obstacles encountered by the vehicle during driving, so as to avoid them in time.

[0006] The present invention provides an intelligent driving obstacle avoidance method based on multi-source information fusion processing, comprising:

[0007] Step 1: When the vehicle is traveling, real-time vehicle driving information and real-time environmental image information of the vehicle are collected to construct an environmental map of the vehicle, and obstacles within a specified range of the vehicle are screened in the environmental map;

[0008] Step 2: generating a plurality of obstacle avoidance paths according to the positional relationship between the obstacle and the vehicle, collecting the real-time driver operation information of the vehicle, and analyzing the driver's reaction operation time corresponding to executing each obstacle avoidance path;

[0009] Step 3: Estimate the obstacle avoidance success rate corresponding to the obstacle avoidance path according to the reaction operation duration, collect the historical driving data of the vehicle, optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path;

[0010] Step 4: Assist the driver to execute the optimized obstacle avoidance path, collect the real-time obstacle avoidance feedback of the vehicle, and adjust the optimized obstacle avoidance path in real time according to the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

[0011] In an implementable manner,

[0012] The said Step 1 includes:

[0013] Step 11: When the vehicle is in a driving state, collect the real-time device data generated by each vehicle device in the vehicle, perform time resampling on each piece of real-time device data respectively, generate several periodic device data of the vehicle, and align the timestamps corresponding to each piece of periodic device data respectively to generate the real-time driving information of the vehicle;

[0014] Step 12: Obtain the front image video and the rear image video of the vehicle, construct an environmental image pair of the vehicle according to the front image and the rear image corresponding to the same moment in the front image video and the rear image video, and use stereo vision technology to identify several stereo environmental objects included in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object;

[0015] Step 13: Draw a three-dimensional environmental model of the vehicle in a preset three-dimensional space according to the item specification parameters, perform dynamic rendering on the three-dimensional environmental model by using the front image video and the rear image video, generate a three-dimensional dynamic model of the vehicle, and construct a real-time environmental image of the vehicle according to the real-time dynamic characteristics of the three-dimensional dynamic model;

[0016] Step 14: Integrate the real-time vehicle driving information and the real-time environmental information, project the integration result to obtain the environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify the obstacles included within the specified range of the vehicle with the real-time driving position as the center.

[0017] In an implementable manner,

[0018] The said Step 2 includes:

[0019] Step 21: Determine the driving direction and speed of the vehicle according to the real-time vehicle driving information, determine the moving direction and speed of each obstacle according to the real-time environmental image information, deduce the collision points and collision times between the vehicle and each obstacle, and mark each collision point and collision time in the environmental map respectively;

[0020] Step 22: Determine several safe areas included in the environmental map and the corresponding safe time periods according to the collision points and collision times, simulate the continuous driving paths of the vehicle in the safe areas in the environmental map, and generate several obstacle avoidance paths;

[0021] Step 23: Divide each obstacle avoidance path into several areas to be operated according to the distribution of the safe areas in the environmental map, determine the corresponding vehicle driving conditions in each area to be operated respectively, and construct an obstacle avoidance operation process corresponding to the driver operating each obstacle avoidance path;

[0022] Step 24: According to the real-time driver operation information, simulate several operation pause characteristics corresponding to the driver executing each obstacle avoidance operation process, locate the pause positions corresponding to each operation pause characteristic in the corresponding obstacle avoidance operation process, screen out the target pause characteristics whose pause positions are not in the safe areas, and determine the reaction operation duration of the driver for the corresponding obstacle avoidance path based on several target pause characteristics corresponding to each obstacle avoidance operation process.

[0023] In an implementable manner,

[0024] It further includes:

[0025] Classify the obstacles into dynamic obstacles and static obstacles according to the real-time environmental image information;

[0026] Locate the image information corresponding to each dynamic obstacle in the real-time environmental image, and determine the moving direction and speed of each dynamic obstacle;

[0027] Regard the direction of the static obstacle pointing to the vehicle as the moving direction of the static obstacle, and record the moving speed of the static obstacle as 0.

[0028] In an implementable manner,

[0029] The said step 3 includes:

[0030] Step 31: Determine several operation danger points corresponding to the obstacle avoidance path according to the reaction operation duration. According to the path positions corresponding to each operation danger point, judge whether the path position is a safe position. If so, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding danger label.

[0031] Step 32: Determine several collision danger points of the obstacle avoidance path respectively according to the distance information between each obstacle avoidance path and different obstacles. Obtain the risk avoidance operation characteristics of the driver at the collision danger points corresponding thereto. According to the risk avoidance operation characteristics, judge whether the corresponding collision danger point is a safe position. If so, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding danger label.

[0032] Step 33: Count the several safety labels and danger labels corresponding to each obstacle avoidance path respectively, determine the obstacle avoidance success rate corresponding to the obstacle avoidance path, and screen the target obstacle avoidance path with the highest obstacle avoidance success rate. Obtain the historical driving data of the vehicle, and construct several driving habits of the driver according to the historical driving data.

[0033] Step 34: Use each driving habit to overall optimize the target obstacle avoidance path. At the same time, obtain the relevant driving habits corresponding to each danger label, and use the relevant driving habits to detail optimize the corresponding dangerous positions to generate the optimized obstacle avoidance path of the vehicle.

[0034] In an implementable manner,

[0035] It further includes:

[0036] When the target obstacle avoidance path does not contain danger labels, use each driving habit to overall optimize the target obstacle avoidance path respectively to generate the optimized obstacle avoidance path of the vehicle.

[0037] In an implementable manner,

[0038] The said step 4 includes:

[0039] Step 41: Convert the optimized obstacle avoidance path into a reminder voice and a reminder animation, use the reminder voice and the reminder animation to assist the driver to execute the optimized obstacle avoidance path, and at the same time collect the real-time obstacle avoidance operations of the driver.

[0040] Step 42: Generate the real-time obstacle avoidance feedback of the vehicle according to the real-time obstacle avoidance operations, use the real-time obstacle avoidance feedback to determine the real-time obstacle information within the specified range of the vehicle, and adjust the real-time obstacle avoidance direction of the optimized obstacle avoidance path according to the real-time obstacle information.

[0041] Step 43: Obtain the obstacle avoidance information within the specified range of the vehicle, construct the obstacle avoidance progress of the vehicle according to the obstacle avoidance information, and stop the assisted obstacle avoidance work when the obstacle avoidance progress is completed.

[0042] In an implementable manner,

[0043] It further includes:

[0044] When the driver issues an obstacle avoidance command, generate an obstacle avoidance path for the vehicle and assist the driver in avoiding obstacles.

[0045] The present invention provides an intelligent driving obstacle avoidance system based on multi-source information fusion processing, including:

[0046] An information acquisition module, configured to collect the real-time vehicle driving information and real-time environmental image information of the vehicle during vehicle driving to construct an environmental map of the vehicle, and screen obstacles within the specified range of the vehicle in the environmental map;

[0047] An obstacle avoidance analysis module, configured to generate several obstacle avoidance paths according to the positional relationship between the obstacle and the vehicle, and collect the real-time driver operation information of the vehicle to analyze the reaction operation duration of the driver for each obstacle avoidance path;

[0048] An obstacle avoidance optimization module, configured to estimate the obstacle avoidance success rate corresponding to the obstacle avoidance path according to the reaction operation duration, collect the historical driving data of the vehicle to optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path;

[0049] An obstacle avoidance assistance module, configured to assist the driver in executing the optimized obstacle avoidance path, and collect the real-time obstacle avoidance feedback of the vehicle, and perform real-time adjustment on the optimized obstacle avoidance path according to the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

[0050] In an implementable manner,

[0051] The information acquisition module includes:

[0052] An information processing unit, configured to, when the vehicle is in a driving state, collect the real-time device data generated by each vehicle device in the vehicle, perform time resampling on each real-time device data respectively to generate several periodic device data of the vehicle, and align the time stamps corresponding to each periodic device data respectively to generate the real-time driving information of the vehicle;

[0053] An image fusion unit, configured to obtain the front image video and the rear image video of the vehicle, construct an environmental image pair of the vehicle according to the front image and the rear image corresponding to the same moment in the front image video and the rear image video, and use stereoscopic vision technology to identify a plurality of stereoscopic environmental objects included in each environmental image pair, so as to obtain the object specification parameters corresponding to each stereoscopic environmental object;

[0054] A modeling and analysis unit, configured to draw a three-dimensional environmental model of the vehicle in a preset three-dimensional space according to the object specification parameters, perform dynamic rendering on the three-dimensional environmental model by using the front image video and the rear image video, generate a three-dimensional dynamic model of the vehicle, and construct a real-time environmental image of the vehicle according to the real-time dynamic characteristics of the three-dimensional dynamic model;

[0055] An obstacle recognition unit, configured to fuse the real-time vehicle driving information and the real-time environmental information, project the fusion result to obtain an environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and take the real-time driving position as the center to identify obstacles included within a specified range of the vehicle.

[0056] The achievable beneficial effects of the above technical solution are as follows: In order to assist the driver in better performing obstacle avoidance work and generating an obstacle avoidance method adapted to the driver, an environmental map of the vehicle is created by collecting the real-time vehicle driving information and the real-time environmental image information of the vehicle, and it is analyzed whether there are obstacles within the specified range of the vehicle. In this way, the distribution of obstacles near the vehicle can be monitored in real time, and then an obstacle avoidance path can be constructed according to the positional relationship between the obstacles and the vehicle. At this time, in order to improve the intelligence of the obstacle avoidance work, the reaction operation duration of the driver when executing each obstacle avoidance path is analyzed by combining the real-time driver operation information, so as to estimate the obstacle avoidance success rate of each obstacle avoidance path. The selected target obstacle avoidance path is optimized by inferring the driver's driving habits based on historical driving data, and the generated optimized obstacle avoidance path is used to assist the driver in performing obstacle avoidance work. At the same time, in order to avoid obstacle avoidance failure caused by sudden changes in the external environment, the optimized obstacle avoidance path is adjusted in real time according to the real-time obstacle avoidance feedback of the vehicle during the obstacle avoidance process, so as to assist the driver in completing the obstacle avoidance work. In this way, obstacle avoidance can be assisted based on various information during the vehicle driving process, and the driver can perform obstacle avoidance according to his own driving habits, which not only improves the quality of obstacle avoidance but also ensures the safety of the driver and the vehicle during the obstacle avoidance process.

[0057] Other features and advantages of the present invention will be described in the following specification, and part of them will become obvious from the specification or be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the written specification and the drawings.

[0058] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0059] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, but do not constitute a limitation to the present invention. In the accompanying drawings:

[0060] Figure 1 It is a schematic diagram of the working process of an intelligent driving obstacle avoidance method based on multi-source information fusion processing in an embodiment of the present invention;

[0061] Figure 2 It is a schematic diagram of the composition of an intelligent driving obstacle avoidance system based on multi-source information fusion processing in an embodiment of the present invention. Specific Embodiments

[0062] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0063] Embodiment 1

[0064] This embodiment provides an intelligent driving obstacle avoidance method based on multi-source information fusion processing, as Figure 1 shown, including:

[0065] Step 1: When the vehicle is driving, collect the real-time vehicle driving information and real-time environmental image information of the vehicle to construct an environmental map of the vehicle, and screen the obstacles within the specified range of the vehicle in the environmental map;

[0066] Step 2: Generate several obstacle avoidance paths according to the positional relationship between the obstacle and the vehicle, and collect the real-time driver operation information of the vehicle to analyze the reaction operation duration of the driver for each obstacle avoidance path;

[0067] Step 3: Estimate the obstacle avoidance success rate corresponding to the obstacle avoidance path according to the reaction operation duration, collect the historical driving data of the vehicle to optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path;

[0068] Step 4: Assist the driver to execute the optimized obstacle avoidance path, and collect the real-time obstacle avoidance feedback of the vehicle. Adjust the optimized obstacle avoidance path in real time according to the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

[0069] In this example, the real-time vehicle driving information represents the information generated during the vehicle's driving process, the real-time environmental image information represents the image information composed of the front image and the rear image in the vehicle's driving direction, and the real-time driver operation information represents the information generated when the driver operates the vehicle during driving;

[0070] In this example, the specified range is default: a range centered on the vehicle with a radius of 100 centimeters; the driver can adjust the length of the radius according to their own driving habits and driving needs;

[0071] In this example, the environmental map represents the map constructed according to the on-site environment around the vehicle;

[0072] In this example, the reaction operation duration represents the reaction duration required for the driver to execute an obstacle avoidance path;

[0073] In this example, the obstacle avoidance success rate represents the probability that the driver can complete the obstacle avoidance work when executing an obstacle avoidance path;

[0074] In this example, the real-time obstacle avoidance feedback represents the situation of the external environment collected by the driver during the process of executing the obstacle avoidance work.

[0075] The working principle and beneficial effects of the above technical solution: In order to assist the driver better in the obstacle avoidance work and generate an obstacle avoidance method suitable for this driver, the environmental map of the vehicle is created by collecting the real-time vehicle driving information and real-time environmental image information of the vehicle, and it is analyzed whether there are obstacles within the specified range of the vehicle. In this way, the distribution of obstacles near the vehicle can be monitored in real time, and then the obstacle avoidance path is constructed according to the positional relationship between the obstacles and the vehicle. At this time, in order to improve the intelligence of the obstacle avoidance work, the real-time driver operation information is combined to analyze the reaction operation duration when the driver executes each obstacle avoidance path, so as to estimate the obstacle avoidance success rate of each obstacle avoidance path. The selected target obstacle avoidance path is optimized by inferring the driver's driving habits based on historical driving data, and the generated optimized obstacle avoidance path is used to assist the driver in executing the obstacle avoidance work. At the same time, in order to avoid the obstacle avoidance failure caused by the sudden change of the external environment, the optimized obstacle avoidance path is adjusted in real time according to the real-time obstacle avoidance feedback of the vehicle during the obstacle avoidance process of the vehicle, assisting the driver to complete the obstacle avoidance work. In this way, auxiliary obstacle avoidance can be carried out based on various information during the vehicle driving process, and the driver can perform obstacle avoidance according to their own driving habits, which not only improves the quality of obstacle avoidance but also ensures the safety of the driver and the vehicle during the obstacle avoidance process.

[0076] Embodiment 2

[0077] Based on Embodiment 1, for the intelligent driving obstacle avoidance method based on multi-source information fusion processing, Step 1 includes:

[0078] Step 11: When the vehicle is in a driving state, collect the real-time device data generated by each vehicle device in the vehicle, perform time resampling on each piece of real-time device data respectively, generate several pieces of periodic device data of the vehicle, align the timestamps corresponding to each piece of periodic device data respectively, and generate the real-time driving information of the vehicle;

[0079] Step 12: Obtain the front image video and the rear image video of the vehicle, construct an environmental image pair of the vehicle according to the front image and the rear image corresponding to the same moment in the front image video and the rear image video, and use stereo vision technology to identify several stereo environmental items included in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental item;

[0080] Step 13: Draw a three-dimensional environmental model of the vehicle in a preset three-dimensional space according to the item specification parameters, perform dynamic rendering on the three-dimensional environmental model by using the front image video and the rear image video, generate a three-dimensional dynamic model of the vehicle, and construct a real-time environmental image of the vehicle according to the real-time dynamic features of the three-dimensional dynamic model;

[0081] Step 14: Integrate the real-time vehicle driving information and the real-time environmental information, project the integration result to obtain an environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify the obstacles included within the specified range of the vehicle with the real-time driving position as the center.

[0082] In this example, the vehicle device refers to the devices that make up the vehicle, and one vehicle device corresponds to one piece of real-time device data;

[0083] In this example, time resampling refers to the process of aggregating real-time device data in chronological order, and the period of each sampling is 1 minute;

[0084] In this example, timestamp alignment refers to the result of aligning the same moment in the timestamps of each piece of periodic device data;

[0085] In this example, the environmental image pair refers to the result of pairing the front image video and the rear image video collected by the vehicle at the same moment;

[0086] In this example, stereo vision technology refers to the process of simulating human binocular vision through two cameras;

[0087] In this example, the stereo environmental item refers to the result of converting the items included in the environmental image into a stereo effect;

[0088] In this example, the preset three-dimensional space refers to the space composed of the X-axis, Y-axis, and Z-axis;

[0089] In this example, the item specification parameters represent data used to describe the specifications of items in a three-dimensional environment.

[0090] The working principle and beneficial effects of the above technical solution are as follows: In order to better identify obstacles near the vehicle, when the vehicle is in a driving state, real-time device data generated by vehicle devices is resampled and timestamp-aligned to construct the vehicle's real-time driving information. By collecting data from each vehicle device, the accuracy and precision of the real-time driving information can be improved. Then, the front and rear image videos of the vehicle are fused, and stereo vision technology is used to identify three-dimensional environment items included in the environmental image. Thus, in a preset three-dimensional space, a three-dimensional environment model of the vehicle is drawn based on the item specification parameters of each three-dimensional environment item. Further, a three-dimensional dynamic model of the vehicle is constructed by means of dynamic rendering. In this way, a three-dimensional dynamic model can be created using the collected image information, improving the fit between the three-dimensional dynamic model and the environment. Then, based on the real-time dynamic characteristics of the three-dimensional dynamic model, the actual environmental image of the vehicle is determined. Furthermore, the real-time vehicle driving information and the real-time environmental information are fused and then projected to generate an environmental map of the vehicle. Finally, obstacles near the vehicle are identified in the environmental map. In this way, not only the effectiveness of obstacle recognition can be ensured, reducing recognition errors, but also the environment around the vehicle can be monitored in real time, enhancing the driver's driving safety awareness.

[0091] Embodiment 3

[0092] Based on Embodiment 1, for the intelligent driving obstacle avoidance method based on multi-source information fusion processing, Step 2 includes:

[0093] Step 21: Determine the driving direction and driving speed of the vehicle according to the real-time vehicle driving information, determine the moving direction and moving speed corresponding to each obstacle according to the real-time environmental image information, deduce the collision points and collision times between the vehicle and each obstacle, and mark each collision point and collision time in the environmental map respectively;

[0094] Step 22: Determine several safety regions included in the environmental map and the corresponding safety time periods for each safety region according to the collision points and collision times, simulate the continuous driving path of the vehicle in the safety regions in the environmental map, and generate several obstacle avoidance paths;

[0095] Step 23: Divide each of the obstacle avoidance paths into several areas to be operated according to the distribution of the safety areas in the environmental map, respectively determine the corresponding vehicle driving conditions in each area to be operated, and construct an obstacle avoidance operation process corresponding to the driver operating each of the obstacle avoidance paths.

[0096] Step 24: According to the real-time driver operation information, simulate several operation pause characteristics corresponding to the driver executing each obstacle avoidance operation process, locate the pause positions corresponding to each operation pause characteristic in the corresponding obstacle avoidance operation process, screen out the target pause characteristics whose pause positions are not in the safety area, and determine the reaction operation duration of the driver for the corresponding obstacle avoidance path based on several target pause characteristics corresponding to each obstacle avoidance operation process.

[0097] In this example, the collision point represents the position where the vehicle collides with an obstacle, and the collision moment represents the time when the vehicle collides with an obstacle.

[0098] In this example, the safety area represents the area where no collision will occur, and the safe time period means that when the vehicle travels to the corresponding safety area within this time period, no collision will occur.

[0099] In this example, the area to be operated indicates the area where the driver needs to perform obstacle avoidance operations.

[0100] In this example, the vehicle driving condition indicates the conditions that need to be executed when the vehicle drives safely in this operation area.

[0101] In this example, the operation pause characteristic indicates the characteristics presented when the driver maintains the original operation and pauses during the obstacle avoidance process.

[0102] The working principle and beneficial effects of the above technical solution: Since the driver will present several driving postures when driving a vehicle, the current state of the vehicle needs to be considered when avoiding obstacles, so as to analyze the obstacle avoidance effect. The most important of which is the reaction operation duration when the driver adjusts the vehicle to the obstacle avoidance state. Determine the driving direction and driving speed of the vehicle according to the real-time vehicle driving information, and at the same time determine the moving direction and moving speed of the obstacle according to the real-time image information, so as to deduce the collision point and collision moment when the vehicle collides with the obstacle, so as to determine several safety areas of the vehicle and the safe time period of each safety area in the environmental map. Construct the obstacle avoidance path by simulation, and further analyze the operation pause characteristics of the driver in combination with the real-time driver information. Since the situation of controlling the vehicle to maintain the original driving state may occur during the obstacle avoidance process, the reaction operation duration of the driver is determined according to whether the corresponding pause position is in the safety area, which lays the foundation for selecting a suitable obstacle avoidance path in the future.

[0103] Embodiment 4

[0104] Based on Embodiment 3, the intelligent driving obstacle avoidance method based on multi-source information fusion processing further includes:

[0105] Classify the obstacles into dynamic obstacles and static obstacles according to the real-time environmental image information;

[0106] Locate the image information corresponding to each dynamic obstacle in the real-time environmental image, and determine the moving direction and moving speed of each dynamic obstacle;

[0107] Regard the direction from the static obstacle to the vehicle as the moving direction of the static obstacle, and record the moving speed of the static obstacle as 0.

[0108] The working principle and beneficial effects of the above technical solution: Classify the obstacles into dynamic obstacles and static obstacles according to the actual situation. For unified analysis, regard the speed of the static obstacle as 0, which is convenient for subsequent obstacle avoidance analysis.

[0109] Embodiment 5

[0110] Based on Embodiment 1, in the intelligent driving obstacle avoidance method based on multi-source information fusion processing, Step 3 includes:

[0111] Step 31: Determine several operation risk points corresponding to the obstacle avoidance path according to the reaction operation duration, and judge whether the path position is a safe position according to the path position corresponding to each operation risk point. If so, set a corresponding safety label for the corresponding obstacle avoidance path, otherwise set a corresponding danger label;

[0112] Step 32: Determine several collision risk points of the obstacle avoidance path respectively according to the distance information between each obstacle avoidance path and different obstacles, obtain the risk avoidance operation characteristics of the driver corresponding to the collision risk points, and judge whether the corresponding collision risk point is a safe position according to the risk avoidance operation characteristics. If so, set a corresponding safety label for the corresponding obstacle avoidance path, otherwise set a corresponding danger label;

[0113] Step 33: Count the several safety labels and danger labels corresponding to each obstacle avoidance path respectively, determine the obstacle avoidance success rate corresponding to the obstacle avoidance path, and screen the target obstacle avoidance path with the highest obstacle avoidance success rate. Obtain the historical driving data of the vehicle, and construct several driving habits of the driver according to the historical driving data;

[0114] Step 34: Overall optimize the target obstacle avoidance path using each of the driving habits, and at the same time obtain the relevant driving habits corresponding to each of the hazard labels. Use the relevant driving habits to perform detailed optimization on the corresponding hazardous positions to generate the optimized obstacle avoidance path of the vehicle.

[0115] In this example, an operation hazard point refers to a position where a driver is prone to danger during the execution of an obstacle avoidance path due to a relatively long operation reaction time.

[0116] The working principle and beneficial effects of the above technical solution: By assigning hazard labels to the operation hazard points and collision hazard points when the driver executes each obstacle avoidance path to analyze the obstacle avoidance success rate of the corresponding obstacle avoidance path, then screening out the target obstacle avoidance path with the highest obstacle avoidance success rate, and then deducing the driving habits of the driver based on the historical driving data of the vehicle. Finally, use the driving habits to perform overall optimization and detailed optimization on the target obstacle avoidance path to generate an obstacle avoidance path that conforms to the driving habits of the driver, enables the driver to operate better, and ensures the obstacle avoidance effect, achieving high-quality obstacle avoidance.

[0117] Embodiment 6

[0118] Based on Embodiment 5, the intelligent driving obstacle avoidance method based on multi-source information fusion processing further includes:

[0119] When the target obstacle avoidance path does not contain hazard labels, overall optimize the target obstacle avoidance path using each of the driving habits to generate the optimized obstacle avoidance path of the vehicle.

[0120] The working principle and beneficial effects of the above technical solution: When the target obstacle avoidance path does not contain hazard labels, use the driving habits to perform overall optimization, which not only ensures the usability of the optimized obstacle avoidance path but also improves the generation efficiency of the optimized obstacle avoidance path.

[0121] Embodiment 7

[0122] Based on Embodiment 1, in the intelligent driving obstacle avoidance method based on multi-source information fusion processing, Step 4 includes:

[0123] Step 41: Convert the optimized obstacle avoidance path into a reminder voice and a reminder animation, use the reminder voice and the reminder animation to assist the driver in executing the optimized obstacle avoidance path, and at the same time collect the driver's real-time obstacle avoidance operations;

[0124] Step 42: Generate the real-time obstacle avoidance feedback of the vehicle according to the real-time obstacle avoidance operations, use the real-time obstacle avoidance feedback to determine the real-time obstacle information within the specified range of the vehicle, and adjust the real-time obstacle avoidance direction of the optimized obstacle avoidance path according to the real-time obstacle information;

[0125] Step 43: Obtain the obstacle avoidance information within the specified range of the vehicle, construct the obstacle avoidance progress of the vehicle according to the obstacle avoidance information, and stop the assisted obstacle avoidance work when the obstacle avoidance progress is completed.

[0126] In this example, during the process of obstacle avoidance assistance, the reminder voice and the reminder animation are played synchronously.

[0127] Working principle and beneficial effects of the above technical solution: In order to improve the assistance quality, when performing assisted obstacle avoidance, the driver is reminded to perform obstacle avoidance by combining voice and animation, and the real-time obstacle information near the vehicle is analyzed according to the driver's real-time obstacle avoidance operation, so as to adjust the obstacle avoidance method of the vehicle according to the actual situation. At the same time, the progress of this obstacle avoidance is monitored, and the assisted obstacle avoidance work is stopped when the obstacle avoidance is completed. In this way, not only can the obstacle avoidance quality be ensured, but also the obstacle avoidance path can be adjusted according to the actual situation, further effectively reducing the risk of collision.

[0128] Embodiment 8

[0129] Based on Embodiment 1, the intelligent driving obstacle avoidance method based on multi-source information fusion processing further includes:

[0130] When the driver issues an obstacle avoidance command, generate the obstacle avoidance path of the vehicle and assist the driver in performing obstacle avoidance.

[0131] Working principle and beneficial effects of the above technical solution: The driver can activate intelligent obstacle avoidance according to needs to assist the driver in driving smoothly.

[0132] Embodiment 9

[0133] This embodiment provides an intelligent driving obstacle avoidance system based on multi-source information fusion processing, as Figure 2 shown, including:

[0134] An information acquisition module, configured to collect the real-time vehicle driving information and real-time environmental image information of the vehicle during vehicle driving to construct an environmental map of the vehicle, and screen obstacles within the specified range of the vehicle in the environmental map;

[0135] An obstacle avoidance analysis module, configured to generate several obstacle avoidance paths according to the positional relationship between the obstacles and the vehicle, and collect the real-time driver operation information of the vehicle to analyze the reaction operation duration of the driver for each obstacle avoidance path;

[0136] An obstacle avoidance optimization module, configured to estimate the obstacle avoidance success rate corresponding to the obstacle avoidance path according to the reaction operation duration, collect the historical driving data of the vehicle to optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path;

[0137] An obstacle avoidance assistance module, which is used to assist the driver in executing the optimized obstacle avoidance path, collect the real-time obstacle avoidance feedback of the vehicle, and adjust the optimized obstacle avoidance path in real time according to the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

[0138] In this example, the real-time vehicle driving information represents the information generated during the vehicle's driving process, the real-time environmental image information represents the image information composed of the front image and the rear image in the driving direction of the vehicle, and the real-time driver operation information represents the information generated when the driver operates the vehicle during driving.

[0139] In this example, the specified range is default: a range with the vehicle as the center and a radius of 100 centimeters; the driver can adjust the length of the radius according to their own driving habits and driving needs.

[0140] In this example, the environmental map represents the map constructed according to the on-site environment around the vehicle.

[0141] In this example, the reaction operation duration represents the reaction duration required for the driver to execute an obstacle avoidance path.

[0142] In this example, the obstacle avoidance success rate represents the probability that the driver can complete the obstacle avoidance work when executing an obstacle avoidance path.

[0143] In this example, the real-time obstacle avoidance feedback represents the situation of the external environment collected by the driver during the execution of the obstacle avoidance work.

[0144] Working principle and beneficial effects of the above technical solution: In order to assist the driver in better obstacle avoidance and generate an obstacle avoidance method suitable for this driver, the real-time vehicle driving information and real-time environmental image information of the vehicle are collected to create an environmental map of the vehicle, and it is analyzed whether there are obstacles within the specified range of the vehicle. In this way, the distribution of obstacles near the vehicle can be monitored in real time, and then an obstacle avoidance path is constructed based on the positional relationship between the obstacles and the vehicle. At this time, in order to improve the intelligence of the obstacle avoidance work, the reaction operation duration of the driver when executing each obstacle avoidance path is analyzed by combining the real-time driver operation information, so as to estimate the obstacle avoidance success rate of each obstacle avoidance path. The selected target obstacle avoidance path is optimized by inferring the driving habits of the driver based on historical driving data, and the generated optimized obstacle avoidance path is used to assist the driver in performing the obstacle avoidance work. At the same time, in order to avoid obstacle avoidance failure caused by sudden changes in the external environment, the optimized obstacle avoidance path is adjusted in real time according to the real-time obstacle avoidance feedback of the vehicle during the obstacle avoidance process to assist the driver in completing the obstacle avoidance work. In this way, auxiliary obstacle avoidance can be carried out based on various information during the vehicle driving process, and the driver can perform obstacle avoidance according to his own driving habits, which not only improves the quality of obstacle avoidance but also ensures the safety of the driver and the vehicle during the obstacle avoidance process.

[0145] Embodiment 10

[0146] Based on Embodiment 9, for the intelligent driving obstacle avoidance system based on multi-source information fusion processing, the information acquisition module includes:

[0147] An information processing unit, configured to collect real-time device data generated by each vehicle device in the vehicle when the vehicle is in a driving state, perform time resampling on each piece of real-time device data respectively, generate a plurality of periodic device data of the vehicle, and align the timestamps corresponding to each piece of periodic device data respectively to generate the real-time driving information of the vehicle;

[0148] An image fusion unit, configured to obtain the front image video and the rear image video of the vehicle, construct an environmental image pair of the vehicle according to the front image and the rear image corresponding to the same moment in the front image video and the rear image video, and use stereo vision technology to identify a plurality of stereo environmental items included in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental item;

[0149] A modeling and analysis unit, configured to draw a three-dimensional environmental model of the vehicle in a preset three-dimensional space according to the item specification parameters, perform dynamic rendering on the three-dimensional environmental model by using the front image video and the rear image video to generate a three-dimensional dynamic model of the vehicle, and construct the real-time environmental image of the vehicle according to the real-time dynamic characteristics of the three-dimensional dynamic model;

[0150] An obstacle recognition unit is used to fuse the real-time vehicle driving information and the real-time environment information, project the fusion result to obtain an environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify obstacles contained within a specified range of the vehicle centered on the real-time driving position.

[0151] In this example, vehicle equipment refers to the equipment that makes up the vehicle, and one vehicle equipment corresponds to one piece of real-time equipment data;

[0152] In this example, time resampling refers to the process of aggregating real-time equipment data in chronological order, and the period of each sampling is 1 minute;

[0153] In this example, timestamp alignment refers to the result of aligning the same moment in the timestamps of equipment data for each period;

[0154] In this example, environmental image pair refers to the result of pairing the front image video and the rear image video collected by the vehicle at the same moment;

[0155] In this example, stereo vision technology refers to the process of simulating human binocular vision through two cameras;

[0156] In this example, stereo environmental object refers to the result of converting the objects contained in the environmental image into a three-dimensional effect;

[0157] In this example, the preset three-dimensional space refers to the space composed of the X-axis, Y-axis, and Z-axis;

[0158] In this example, object specification parameters refer to the data used to describe the specifications of stereo environmental objects.

[0159] Working principle and beneficial effects of the above technical solution: To better identify obstacles near the vehicle, when the vehicle is in a driving state, real-time device data generated by vehicle devices is resampled and timestamp-aligned to construct the vehicle's real-time driving information. By collecting data from each vehicle device, the accuracy and precision of the real-time driving information can be improved. Then, the front and rear image videos of the vehicle are fused, and stereo vision technology is used to identify the stereo environment objects included in the environmental image. Thus, in a preset three-dimensional space, a three-dimensional environment model of the vehicle is drawn based on the object specification parameters of each stereo environment object. Further, a three-dimensional dynamic model of the vehicle is constructed using dynamic rendering. In this way, the collected image information can be used to create a three-dimensional dynamic model, improving the fit between the three-dimensional dynamic model and the environment. Then, based on the real-time dynamic characteristics of the three-dimensional dynamic model, the actual environmental image of the vehicle is determined. Furthermore, the real-time vehicle driving information and real-time environmental information are fused and then projected to generate an environmental map of the vehicle. Finally, obstacles near the vehicle are identified in the environmental map. In this way, not only the effectiveness of obstacle recognition can be ensured, reducing recognition errors, but also the environment around the vehicle can be monitored in real time, improving the driving safety awareness of the driver.

[0160] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent driving obstacle avoidance method based on multi-source information fusion processing, characterized in that Including: Step 1: When the vehicle is in motion, collect the real-time vehicle driving information and real-time environmental image information of the vehicle to construct an environmental map of the vehicle, and screen for obstacles within a specified range of the vehicle in the environmental map; Step 2: Generate a number of obstacle avoidance paths based on the positional relationship between the obstacles and the vehicle, and collect the real-time driver operation information of the vehicle to analyze the reaction operation duration of the driver for each obstacle avoidance path; Step 3: Estimate the obstacle avoidance success rate corresponding to each obstacle avoidance path based on the reaction operation duration, collect the historical driving data of the vehicle to optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path; Step 4: Assist the driver to execute the optimized obstacle avoidance path, and collect the real-time obstacle avoidance feedback of the vehicle. Make real-time adjustments to the optimized obstacle avoidance path based on the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

2. The intelligent driving obstacle avoidance method based on multi-source information fusion processing according to claim 1, characterized in that The said Step 1 includes: Step 11: When the vehicle is in a driving state, collect the real-time device data generated by each vehicle device in the vehicle, perform time resampling on each real-time device data respectively to generate a number of periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data respectively to generate the real-time driving information of the vehicle; Step 12: Obtain the front image video and rear image video of the vehicle, construct an environmental image pair of the vehicle based on the front image and rear image corresponding to the same moment in the front image video and the rear image video, and use stereo vision technology to identify a number of stereo environmental objects included in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object; Step 13: Draw a three-dimensional environmental model of the vehicle in a preset three-dimensional space according to the item specification parameters, use the front image video and the rear image video to perform dynamic rendering on the three-dimensional environmental model to generate a three-dimensional dynamic model of the vehicle, and construct the real-time environmental image of the vehicle according to the real-time dynamic characteristics of the three-dimensional dynamic model; Step 14: Integrate the real-time vehicle driving information and the real-time environmental information, project the integration result to obtain the environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify the obstacles included within the specified range of the vehicle with the real-time driving position as the center.

3. An intelligent driving obstacle avoidance method based on multi-source information fusion processing according to claim 1, characterized in that, The said Step 2 includes: Step 21: Determine the driving direction and driving speed of the vehicle according to the real-time vehicle driving information, determine the moving direction and moving speed of each obstacle according to the real-time environmental image information, deduce the collision points and collision times between the vehicle and each obstacle, and mark each collision point and collision time in the environmental map respectively; Step 22: Determine a number of safe areas included in the environmental map and the corresponding safe time periods for each safe area according to the collision points and collision times, simulate the continuous driving paths of the vehicle in the safe areas in the environmental map, and generate a number of obstacle avoidance paths; Step 23: Divide each of the obstacle avoidance paths into several areas to be operated according to the distribution of the safety areas in the environmental map, respectively determine the corresponding vehicle driving conditions in each area to be operated, and construct an obstacle avoidance operation process corresponding to the driver operating each of the obstacle avoidance paths. Step 24: According to the real-time driver operation information, simulate several operation pause characteristics corresponding to the driver executing each obstacle avoidance operation process, locate the pause positions corresponding to each operation pause characteristic in the corresponding obstacle avoidance operation process, screen out the target pause characteristics whose pause positions are not in the safety area, and determine the reaction operation duration of the driver for the corresponding obstacle avoidance path based on the several target pause characteristics corresponding to each obstacle avoidance operation process.

4. The intelligent driving obstacle avoidance method based on multi-source information fusion processing according to claim 3, characterized in that, Further included: Classify the obstacles into dynamic obstacles and static obstacles according to the real-time environmental image information. Locate the image information corresponding to each dynamic obstacle in the real-time environmental image, and determine the moving direction and moving speed of each dynamic obstacle. Regard the direction of the static obstacle pointing to the vehicle as the moving direction of the static obstacle, and record the moving speed of the static obstacle as 0.

5. The intelligent driving obstacle avoidance method based on multi-source information fusion processing according to claim 1, characterized in that, The said step 3 includes: Step 31: Determine several operation risk points corresponding to the obstacle avoidance path according to the reaction operation duration, and judge whether the path position corresponding to each operation risk point is a safe position according to the path position corresponding to each operation risk point. If so, set a corresponding safety label for the corresponding obstacle avoidance path, otherwise set a corresponding danger label. Step 32: Determine several collision risk points of the obstacle avoidance path respectively according to the distance information between each obstacle avoidance path and different obstacles, obtain the risk avoidance operation characteristics of the driver corresponding to the collision risk points, and judge whether the corresponding collision risk point is a safe position according to the risk avoidance operation characteristics. If so, set a corresponding safety label for the corresponding obstacle avoidance path, otherwise set a corresponding danger label. Step 33: Count the several safety labels and danger labels corresponding to each obstacle avoidance path respectively, determine the obstacle avoidance success rate of the corresponding obstacle avoidance path, screen out the target obstacle avoidance path with the highest obstacle avoidance success rate, obtain the historical driving data of the vehicle, and construct several driving habits of the driver according to the historical driving data. Step 34: Use each driving habit to optimize the target obstacle avoidance path as a whole, and at the same time obtain the relevant driving habits corresponding to each danger label, and use the relevant driving habits to optimize the corresponding dangerous positions in detail to generate the optimized obstacle avoidance path of the vehicle.

6. The intelligent driving obstacle avoidance method based on multi-source information fusion processing according to claim 5, characterized in that, Further included: When the target obstacle avoidance path does not contain a danger label, use each driving habit to optimize the target obstacle avoidance path as a whole to generate the optimized obstacle avoidance path of the vehicle.

7. An intelligent driving obstacle avoidance method based on multi-source information fusion processing according to claim 1, characterized in that, The said step 4 includes: Step 41: Convert the optimized obstacle avoidance path into a reminder voice and a reminder animation, use the reminder voice and the reminder animation to assist the driver in executing the optimized obstacle avoidance path, and at the same time collect the real-time obstacle avoidance operation of the driver. Step 42: Generate real-time obstacle avoidance feedback for the vehicle according to the real-time obstacle avoidance operation, use the real-time obstacle avoidance feedback to determine real-time obstacle information within the specified range of the vehicle, and adjust the real-time obstacle avoidance direction of the optimized obstacle avoidance path according to the real-time obstacle information; Step 43: Obtain obstacle avoidance information within the specified range of the vehicle, construct the obstacle avoidance progress of the vehicle according to the obstacle avoidance information, and stop the auxiliary obstacle avoidance work when the obstacle avoidance progress is completed.

8. An intelligent driving obstacle avoidance method based on multi-source information fusion processing according to claim 1, characterized in that, It also includes: When the driver issues an obstacle avoidance instruction, generate an obstacle avoidance path for the vehicle and assist the driver in avoiding obstacles.

9. An intelligent driving obstacle avoidance system based on multi-source information fusion processing, characterized in that, It includes: An information collection module, which is used to collect real-time vehicle driving information and real-time environmental image information of the vehicle during vehicle driving to construct an environmental map of the vehicle, and screen obstacles within the specified range of the vehicle in the environmental map; An obstacle avoidance analysis module, which is used to generate several obstacle avoidance paths according to the positional relationship between the obstacles and the vehicle, and collect real-time driver operation information of the vehicle to analyze the reaction operation duration of the driver for each obstacle avoidance path; An obstacle avoidance optimization module, which is used to estimate the obstacle avoidance success rate corresponding to the obstacle avoidance path according to the reaction operation duration, collect historical driving data of the vehicle to optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path; An obstacle avoidance assistance module, which is used to assist the driver in executing the optimized obstacle avoidance path, collect real-time obstacle avoidance feedback of the vehicle, and make real-time adjustments to the optimized obstacle avoidance path according to the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

10. An intelligent driving obstacle avoidance system based on multi-source information fusion processing according to claim 9, characterized in that, The information collection module includes: An information processing unit, which is used to collect real-time device data generated by each vehicle device in the vehicle when the vehicle is in a driving state, perform time resampling on each real-time device data respectively, generate several periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data respectively to generate real-time driving information of the vehicle; An image fusion unit, which is used to obtain the front image video and the rear image video of the vehicle, construct an environmental image pair of the vehicle according to the front image and the rear image corresponding to the same moment in the front image video and the rear image video, and use stereovision technology to identify several stereo environmental items included in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental item; A modeling analysis unit, which is used to draw a three-dimensional environmental model of the vehicle in a preset three-dimensional space according to the item specification parameters, perform dynamic rendering on the three-dimensional environmental model by using the front image video and the rear image video, generate a three-dimensional dynamic model of the vehicle, and construct a real-time environmental image of the vehicle according to the real-time dynamic characteristics of the three-dimensional dynamic model; An obstacle recognition unit is configured to fuse the real-time vehicle driving information and the real-time environment information, project the fusion result to obtain an environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify obstacles included within a specified range of the vehicle centered on the real-time driving position.

Images