An intelligent driving obstacle avoidance method and system based on multi-source information fusion processing

Abstract

This invention provides an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing, comprising: collecting real-time vehicle driving information and real-time environmental image information to construct an environmental map while the vehicle is in motion; filtering obstacles within a specified range of the vehicle to generate several obstacle avoidance paths; collecting real-time driver operation information to analyze the driver's reaction time for each obstacle avoidance path; estimating the obstacle avoidance success rate of the corresponding obstacle avoidance path; collecting historical driving data of the vehicle to optimize the target obstacle avoidance path with the highest success rate; constructing an optimized obstacle avoidance path; assisting the driver in executing the optimized obstacle avoidance path; and adjusting the optimized obstacle avoidance path in real time based on real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance. The system can construct obstacles encountered by the vehicle while driving based on collected driver information, vehicle information, and environmental information, and then perform unified processing to avoid them in a timely manner.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving obstacle avoidance technology, and in particular to an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing. Background Technology

[0002] With the development of artificial intelligence, more and more products are moving towards connectivity, intelligence, and automation, especially in the automotive industry. Cars have been redefined as intelligent mobile terminals, shifting from mechanical products to electronic products. Intelligent driving vehicles and their related ecosystems have experienced unprecedented development. However, automation is not something that can be achieved overnight; it requires continuous technological accumulation and iteration, and many core technologies still need to be resolved.

[0003] Existing intelligent obstacle avoidance technology can only provide obstacle avoidance paths, but different drivers have different driving habits, and the intelligently generated obstacle avoidance paths are difficult to adapt to different drivers, resulting in low obstacle avoidance quality.

[0004] Therefore, the present invention provides an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing. Summary of the Invention

[0005] This invention discloses an intelligent driving obstacle avoidance method and system based on multi-source information fusion processing. It can construct obstacles encountered by the vehicle during driving by collecting information from the driver, vehicle, and environment, and then perform unified processing to avoid them in a timely manner.

[0006] This invention provides an intelligent driving obstacle avoidance method based on multi-source information fusion processing, comprising:

[0007] Step 1: Collect real-time vehicle driving information and real-time environmental image information of the vehicle while it is in motion to construct an environmental map of the vehicle, and filter obstacles within a specified range of the vehicle in the environmental map;

[0008] Step 2: Generate several obstacle avoidance paths based on the positional relationship between the obstacle and the vehicle, and collect real-time driver operation information of the vehicle to analyze the reaction operation time corresponding to each obstacle avoidance path executed by the driver;

[0009] Step 3: Estimate the obstacle avoidance success rate of the corresponding obstacle avoidance path based on the reaction operation time, collect the vehicle's historical driving data, optimize the target obstacle avoidance path with the highest success rate, and construct an optimized obstacle avoidance path;

[0010] Step 4: Assist the driver in executing the optimized obstacle avoidance path and collect real-time obstacle avoidance feedback from the vehicle. Adjust the optimized obstacle avoidance path in real time based on the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

[0011] In one feasible approach

[0012] Step 1 includes:

[0013] Step 11: When the vehicle is in motion, collect real-time device data generated by each vehicle device in the vehicle, resample each real-time device data by time to generate several periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data to generate real-time driving information of the vehicle.

[0014] Step 12: Acquire the front and rear video images of the vehicle, construct environmental image pairs of the vehicle based on the front and rear images corresponding to the same moment in the front and rear video images, and use stereo vision technology to identify several stereo environmental objects contained in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object.

[0015] Step 13: Draw a three-dimensional environment model of the vehicle in a preset three-dimensional space according to the specifications of the item, and dynamically render the three-dimensional environment model using the front and rear video images to generate a three-dimensional dynamic model of the vehicle. Construct a real-time environmental image of the vehicle based on the real-time dynamic features of the three-dimensional dynamic model.

[0016] Step 14: Fuse the real-time vehicle driving information and the real-time environment information, project the fusion result to obtain the vehicle's environment map, locate the vehicle's real-time driving position in the environment map, and identify obstacles within a specified range of the vehicle with the real-time driving position as the center.

[0017] In one feasible approach

[0018] Step 2 includes:

[0019] Step 21: Determine the driving direction and speed of the vehicle based on the real-time vehicle driving information, determine the moving direction and moving speed corresponding to each obstacle based on the real-time environmental image information, deduce the collision point and collision time between the vehicle and each obstacle, and mark each collision point and collision time in the environmental map respectively.

[0020] Step 22: Determine several safe zones contained in the environment map based on the collision point and collision time, and the safe time period corresponding to each safe zone. Simulate the continuous driving path of the vehicle in the safe zone in the environment map to generate several obstacle avoidance paths.

[0021] Step 23: Based on the distribution of the safe zone in the environmental map, each obstacle avoidance path is divided into several operational areas, and the vehicle driving conditions corresponding to each operational area are determined to construct the obstacle avoidance operation process corresponding to the driver operating each obstacle avoidance path.

[0022] Step 24: Based on the real-time driver operation information, simulate several operation pause features corresponding to each obstacle avoidance operation performed by the driver, locate the pause position corresponding to each operation pause feature in the corresponding obstacle avoidance operation, filter target pause features whose pause positions are not within the safe area, and determine the driver's reaction operation time to the corresponding obstacle avoidance path based on several target pause features corresponding to each obstacle avoidance operation.

[0023] In one feasible approach

[0024] Also includes:

[0025] Based on the real-time environmental image information, the obstacles are divided into dynamic obstacles and static obstacles;

[0026] Locate the image information corresponding to each dynamic obstacle in the real-time environmental image, and determine the movement direction and movement speed corresponding to each dynamic obstacle;

[0027] The direction in which the stationary obstacle points toward the vehicle is considered the direction of movement of the stationary obstacle, and the speed of movement of the stationary obstacle is recorded as 0.

[0028] In one feasible approach

[0029] Step 3 includes:

[0030] Step 31: Determine several operational hazard points corresponding to the obstacle avoidance path based on the reaction operation time. Determine whether the path position is a safe position based on the path position corresponding to each operational hazard point. If it is, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding hazard label.

[0031] Step 32: Determine several collision hazard points for each obstacle avoidance path based on the distance information between each obstacle avoidance path and different obstacles, obtain the driver's avoidance operation characteristics corresponding to the collision hazard points, and determine whether the corresponding collision hazard point is a safe location based on the avoidance operation characteristics. If it is, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding danger label.

[0032] Step 33: Statistically count the number of safety tags and danger tags corresponding to each obstacle avoidance path, determine the obstacle avoidance success rate of the corresponding obstacle avoidance path, filter the target obstacle avoidance path with the highest obstacle avoidance success rate, obtain the historical driving data of the vehicle, and construct several driving habits of the driver based on the historical driving data;

[0033] Step 34: Optimize the target obstacle avoidance path as a whole using each of the aforementioned driving habits, and at the same time obtain the relevant driving habits corresponding to each of the aforementioned hazard labels. Optimize the corresponding hazard locations in detail using the relevant driving habits to generate the optimized obstacle avoidance path for the vehicle.

[0034] In one feasible approach

[0035] Also includes:

[0036] When the target obstacle avoidance path does not contain hazard labels, the target obstacle avoidance path is optimized as a whole using each of the aforementioned driving habits to generate the vehicle's optimized obstacle avoidance path.

[0037] In one feasible approach

[0038] Step 4 includes:

[0039] Step 41: Convert the optimized obstacle avoidance path into a reminder voice and a reminder animation, and use the reminder voice and the reminder animation to assist the driver in executing the optimized obstacle avoidance path, while collecting the driver's real-time obstacle avoidance operations;

[0040] Step 42: Generate real-time obstacle avoidance feedback for the vehicle based on the real-time obstacle avoidance operation, use the real-time obstacle avoidance feedback to determine real-time obstacle information within a specified range of the vehicle, and adjust the real-time obstacle avoidance direction of the optimized obstacle avoidance path based on the real-time obstacle information;

[0041] Step 43: Obtain obstacle avoidance information within the specified range of the vehicle, construct the obstacle avoidance progress of the vehicle based on the obstacle avoidance information, and stop the assisted obstacle avoidance work when the obstacle avoidance progress is completed.

[0042] In one feasible approach

[0043] Also includes:

[0044] When the driver issues an obstacle avoidance command, an obstacle avoidance path is generated for the vehicle and assists the driver in avoiding obstacles.

[0045] This invention provides an intelligent driving obstacle avoidance system based on multi-source information fusion processing, comprising:

[0046] The information acquisition module is used to collect real-time vehicle driving information and real-time environmental image information of the vehicle while the vehicle is in motion to construct an environmental map of the vehicle, and to filter obstacles within a specified range of the vehicle in the environmental map.

[0047] The obstacle avoidance analysis module is used to generate several obstacle avoidance paths based on the positional relationship between the obstacle and the vehicle, and to collect real-time driver operation information of the vehicle to analyze the reaction operation time corresponding to each obstacle avoidance path executed by the driver.

[0048] The obstacle avoidance optimization module is used to estimate the obstacle avoidance success rate of the corresponding obstacle avoidance path based on the reaction operation time, collect the vehicle's historical driving data, optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path.

[0049] The obstacle avoidance assist module is used to assist the driver in executing the optimized obstacle avoidance path, and to collect real-time obstacle avoidance feedback from the vehicle. Based on the real-time obstacle avoidance feedback, the optimized obstacle avoidance path is adjusted in real time until the vehicle completes obstacle avoidance.

[0050] In one feasible approach

[0051] The information collection module includes:

[0052] The information processing unit is used to collect real-time device data generated by each vehicle device in the vehicle when the vehicle is in a driving state, resample each real-time device data by time to generate several periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data to generate real-time driving information of the vehicle.

[0053] The image fusion unit is used to acquire the front and rear image videos of the vehicle, construct the environmental image pairs of the vehicle based on the front and rear image images corresponding to the same moment in the front and rear image videos, and use stereo vision technology to identify several stereo environmental objects contained in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object.

[0054] The modeling and analysis unit is used to draw a three-dimensional environment model of the vehicle in a preset three-dimensional space according to the specifications of the item, dynamically render the three-dimensional environment model using the front and rear video images to generate a three-dimensional dynamic model of the vehicle, and construct a real-time environmental image of the vehicle based on the real-time dynamic features of the three-dimensional dynamic model.

[0055] An obstacle recognition unit is used to fuse the real-time vehicle driving information and the real-time environmental information, project the fusion result to obtain an environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify obstacles contained within a specified range of the vehicle with the real-time driving position as the center.

[0056] The beneficial effects of the above technical solution are as follows: To assist drivers in better obstacle avoidance and generate obstacle avoidance methods adapted to their specific needs, a vehicle environment map is created by collecting real-time vehicle driving information and real-time environmental image information. This map analyzes whether there are obstacles within a specified range of the vehicle. This allows for real-time monitoring of the distribution of obstacles near the vehicle. Then, based on the positional relationship between obstacles and the vehicle, an obstacle avoidance path is constructed. To improve the intelligence of obstacle avoidance, real-time driver operation information is combined to analyze the driver's reaction time when executing each obstacle avoidance path, thereby estimating the optimal obstacle avoidance path. The system improves obstacle avoidance success rate by inferring driver habits from historical driving data to optimize the selected obstacle avoidance path. This optimized path assists the driver in performing obstacle avoidance tasks. Furthermore, to prevent obstacle avoidance failure due to sudden changes in the external environment, the optimized path is adjusted in real-time based on the vehicle's obstacle avoidance feedback during the process, further assisting the driver in completing the obstacle avoidance task. This method utilizes various information from the vehicle's driving process to assist obstacle avoidance, helping the driver avoid obstacles according to their own driving habits. This improves the quality of obstacle avoidance and ensures the safety of both the driver and the vehicle during the obstacle avoidance process.

[0057] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0058] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0060] Figure 1 This is a schematic diagram illustrating the workflow of an intelligent driving obstacle avoidance method based on multi-source information fusion processing in an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of the composition of an intelligent driving obstacle avoidance system based on multi-source information fusion processing in an embodiment of the present invention. Detailed Implementation

[0062] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0063] Example 1

[0064] This embodiment provides an intelligent driving obstacle avoidance method based on multi-source information fusion processing, such as... Figure 1 As shown, it includes:

[0065] Step 1: Collect real-time vehicle driving information and real-time environmental image information of the vehicle while it is in motion to construct an environmental map of the vehicle, and filter obstacles within a specified range of the vehicle in the environmental map;

[0066] Step 2: Generate several obstacle avoidance paths based on the positional relationship between the obstacle and the vehicle, and collect real-time driver operation information of the vehicle to analyze the reaction operation time corresponding to each obstacle avoidance path executed by the driver;

[0067] Step 3: Estimate the obstacle avoidance success rate of the corresponding obstacle avoidance path based on the reaction operation time, collect the vehicle's historical driving data, optimize the target obstacle avoidance path with the highest success rate, and construct an optimized obstacle avoidance path;

[0068] Step 4: Assist the driver in executing the optimized obstacle avoidance path and collect real-time obstacle avoidance feedback from the vehicle. Adjust the optimized obstacle avoidance path in real time based on the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance.

[0069] In this example, real-time vehicle driving information refers to the information generated by the vehicle during driving, real-time environmental image information refers to the image information composed of the front and rear images of the vehicle's driving direction, and real-time driver operation information refers to the information generated when the driver operates the vehicle during driving.

[0070] In this example, the default defined range is a radius of 100 centimeters centered on the vehicle; the driver can adjust the length of the radius according to their driving habits and needs.

[0071] In this example, the environmental map represents a map based on the surrounding environment of the vehicle;

[0072] In this example, reaction time refers to the reaction time required for a driver to execute an obstacle avoidance path;

[0073] In this example, the obstacle avoidance success rate represents the probability that a driver can successfully avoid an obstacle when executing an obstacle avoidance path.

[0074] In this example, real-time obstacle avoidance feedback refers to the information about the external environment collected by the driver during the obstacle avoidance process.

[0075] The working principle and beneficial effects of the above technical solution are as follows: To assist drivers in better obstacle avoidance and generate obstacle avoidance methods adapted to their specific needs, a vehicle environment map is created by collecting real-time vehicle driving information and real-time environmental image information. This map analyzes whether there are obstacles within a specified range of the vehicle. This allows for real-time monitoring of the distribution of obstacles near the vehicle. Then, based on the positional relationship between obstacles and the vehicle, an obstacle avoidance path is constructed. To improve the intelligence of obstacle avoidance, real-time driver operation information is combined to analyze the driver's reaction time when executing each obstacle avoidance path, thereby estimating the optimal obstacle avoidance path. The system improves obstacle avoidance success rate by inferring driver habits from historical driving data to optimize the selected obstacle avoidance path. This optimized path assists the driver in performing obstacle avoidance tasks. Furthermore, to prevent obstacle avoidance failure due to sudden changes in the external environment, the optimized path is adjusted in real-time based on the vehicle's obstacle avoidance feedback during the process, further assisting the driver in completing the obstacle avoidance task. This method utilizes various information from the vehicle's driving process to assist obstacle avoidance, helping the driver avoid obstacles according to their own driving habits. This improves the quality of obstacle avoidance and ensures the safety of both the driver and the vehicle during the obstacle avoidance process.

[0076] Example 2

[0077] Based on Example 1, the intelligent driving obstacle avoidance method based on multi-source information fusion processing includes, in step 1:

[0078] Step 11: When the vehicle is in motion, collect real-time device data generated by each vehicle device in the vehicle, resample each real-time device data by time to generate several periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data to generate real-time driving information of the vehicle.

[0079] Step 12: Acquire the front and rear video images of the vehicle, construct environmental image pairs of the vehicle based on the front and rear images corresponding to the same moment in the front and rear video images, and use stereo vision technology to identify several stereo environmental objects contained in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object.

[0080] Step 13: Draw a three-dimensional environment model of the vehicle in a preset three-dimensional space according to the specifications of the item, and dynamically render the three-dimensional environment model using the front and rear video images to generate a three-dimensional dynamic model of the vehicle. Construct a real-time environmental image of the vehicle based on the real-time dynamic features of the three-dimensional dynamic model.

[0081] Step 14: Fuse the real-time vehicle driving information and the real-time environment information, project the fusion result to obtain the vehicle's environment map, locate the vehicle's real-time driving position in the environment map, and identify obstacles within a specified range of the vehicle with the real-time driving position as the center.

[0082] In this example, vehicle equipment refers to the equipment that makes up a vehicle, and one vehicle equipment corresponds to one real-time equipment data.

[0083] In this example, time resampling refers to the process of aggregating real-time device data in chronological order, with each sampling period being 1 minute.

[0084] In this example, timestamp alignment means aligning the same moment in the timestamps of the device data in each cycle;

[0085] In this example, the environmental image pair represents the result of pairing the front and rear video images of the vehicle captured at the same time.

[0086] In this example, stereoscopic vision technology refers to the process of simulating human binocular vision using two cameras;

[0087] In this example, 3D environment objects represent the result of converting objects contained in an environment image into a 3D effect;

[0088] In this example, the preset three-dimensional space is represented by the space composed of the X-axis, Y-axis, and Z-axis;

[0089] In this example, the item specification parameter represents the data used to describe the specifications of items in a three-dimensional environment.

[0090] The working principle and beneficial effects of the above technical solution are as follows: To better identify obstacles near the vehicle, when the vehicle is in motion, real-time device data generated by the vehicle's equipment is resampled and timestamped to construct real-time driving information. By collecting data from each vehicle device, the accuracy and precision of real-time driving information can be improved. Then, the front and rear video images of the vehicle are fused, and stereoscopic vision technology is used to identify stereoscopic environmental objects contained in the environmental images. Based on the object specifications of each stereoscopic environmental object, a 3D environmental model of the vehicle is drawn in a preset 3D space. Furthermore, dynamic rendering is used to construct a 3D dynamic model of the vehicle. In this way, the collected image information can be used to create a 3D dynamic model, improving the fit between the 3D dynamic model and the environment. Then, the real-time dynamic characteristics of the 3D dynamic model are used to determine the actual environmental image of the vehicle. The real-time vehicle driving information and real-time environmental information are then fused and projected to generate an environmental map of the vehicle. Finally, obstacles near the vehicle are identified in the environmental map. This method can not only ensure the effectiveness of obstacle identification and reduce identification errors, but also monitor the environment around the vehicle in real time, improving the driver's driving safety awareness.

[0091] Example 3

[0092] Based on Example 1, the intelligent driving obstacle avoidance method based on multi-source information fusion processing, step 2 includes:

[0093] Step 21: Determine the driving direction and speed of the vehicle based on the real-time vehicle driving information, determine the moving direction and moving speed corresponding to each obstacle based on the real-time environmental image information, deduce the collision point and collision time between the vehicle and each obstacle, and mark each collision point and collision time in the environmental map respectively.

[0094] Step 22: Determine several safe zones contained in the environment map based on the collision point and collision time, and the safe time period corresponding to each safe zone. Simulate the continuous driving path of the vehicle in the safe zone in the environment map to generate several obstacle avoidance paths.

[0095] Step 23: Based on the distribution of the safe zone in the environmental map, each obstacle avoidance path is divided into several operational areas, and the vehicle driving conditions corresponding to each operational area are determined to construct the obstacle avoidance operation process corresponding to the driver operating each obstacle avoidance path.

[0096] Step 24: Based on the real-time driver operation information, simulate several operation pause features corresponding to each obstacle avoidance operation performed by the driver, locate the pause position corresponding to each operation pause feature in the corresponding obstacle avoidance operation, filter target pause features whose pause positions are not within the safe area, and determine the driver's reaction operation time to the corresponding obstacle avoidance path based on several target pause features corresponding to each obstacle avoidance operation.

[0097] In this example, the collision point represents the location where the vehicle collides with the obstacle, and the collision time represents the time when the vehicle collides with the obstacle.

[0098] In this example, the safe zone represents an area where a collision will not occur, and the safe time period represents the time during which a vehicle will not collide when it travels to the corresponding safe zone.

[0099] In this example, the area to be operated is marked as the area where the driver needs to perform obstacle avoidance maneuvers;

[0100] In this example, the vehicle driving conditions identify the conditions that need to be performed for the vehicle to be driven safely in this operating area;

[0101] In this example, the operation pause feature identifies the characteristics that appear when a driver maintains the original operation during obstacle avoidance and pauses.

[0102] The working principle and beneficial effects of the above technical solution are as follows: Since drivers exhibit several driving postures while driving, obstacle avoidance requires consideration of the vehicle's current state to analyze its obstacle avoidance effectiveness. The most crucial factor is the driver's reaction time when adjusting the vehicle to obstacle avoidance mode. Real-time vehicle driving information determines the vehicle's direction and speed, while real-time image information determines the obstacle's direction and speed. This allows for the deduction of the collision point and time, identifying several safe zones for the vehicle and the safe time period for each zone on the environmental map. An obstacle avoidance path is constructed through simulation. Furthermore, real-time driver information is used to analyze the driver's pause characteristics. Since there may be instances where the vehicle maintains its original driving state during obstacle avoidance, the driver's reaction time is determined based on whether the corresponding pause location is within a safe zone, laying the foundation for selecting a suitable obstacle avoidance path.

[0103] Example 4

[0104] Based on Example 3, the intelligent driving obstacle avoidance method based on multi-source information fusion processing further includes:

[0105] Based on the real-time environmental image information, the obstacles are divided into dynamic obstacles and static obstacles;

[0106] Locate the image information corresponding to each dynamic obstacle in the real-time environmental image, and determine the movement direction and movement speed corresponding to each dynamic obstacle;

[0107] The direction in which the stationary obstacle points toward the vehicle is considered the direction of movement of the stationary obstacle, and the speed of movement of the stationary obstacle is recorded as 0.

[0108] The working principle and beneficial effects of the above technical solution are as follows: Based on the actual situation, obstacles are divided into dynamic obstacles and static obstacles. For the sake of unified analysis, the speed of static obstacles is regarded as 0, which facilitates subsequent obstacle avoidance analysis.

[0109] Example 5

[0110] Based on Example 1, the intelligent driving obstacle avoidance method based on multi-source information fusion processing, step 3 includes:

[0111] Step 31: Determine several operational hazard points corresponding to the obstacle avoidance path based on the reaction operation time. Determine whether the path position is a safe position based on the path position corresponding to each operational hazard point. If it is, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding hazard label.

[0112] Step 32: Determine several collision hazard points for each obstacle avoidance path based on the distance information between each obstacle avoidance path and different obstacles, obtain the driver's avoidance operation characteristics corresponding to the collision hazard points, and determine whether the corresponding collision hazard point is a safe location based on the avoidance operation characteristics. If it is, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding danger label.

[0113] Step 33: Statistically count the number of safety tags and danger tags corresponding to each obstacle avoidance path, determine the obstacle avoidance success rate of the corresponding obstacle avoidance path, filter the target obstacle avoidance path with the highest obstacle avoidance success rate, obtain the historical driving data of the vehicle, and construct several driving habits of the driver based on the historical driving data;

[0114] Step 34: Optimize the target obstacle avoidance path as a whole using each of the aforementioned driving habits, and at the same time obtain the relevant driving habits corresponding to each of the aforementioned hazard labels. Optimize the corresponding hazard locations in detail using the relevant driving habits to generate the optimized obstacle avoidance path for the vehicle.

[0115] In this example, the operational hazard point refers to the location where a driver is prone to danger due to a long reaction time while executing an obstacle avoidance path.

[0116] The working principle and beneficial effects of the above technical solution are as follows: By assigning hazard labels to the operational and collision hazard points when the driver executes each obstacle avoidance path, the success rate of the corresponding obstacle avoidance path is analyzed. Then, the target obstacle avoidance path with the highest success rate is selected. Next, the driver's driving habits are inferred based on the vehicle's historical driving data. Finally, the target obstacle avoidance path is optimized in both overall and detailed aspects using the driving habits to generate an obstacle avoidance path that conforms to the driver's driving habits, allowing the driver to operate it better, while ensuring the effectiveness of obstacle avoidance and achieving high-quality obstacle avoidance.

[0117] Example 6

[0118] Based on Example 5, the intelligent driving obstacle avoidance method based on multi-source information fusion processing further includes:

[0119] When the target obstacle avoidance path does not contain hazard labels, the target obstacle avoidance path is optimized as a whole using each of the aforementioned driving habits to generate the vehicle's optimized obstacle avoidance path.

[0120] The working principle and beneficial effects of the above technical solution are as follows: When the target obstacle avoidance path does not contain hazard labels, overall optimization is performed by aligning with driving habits, which not only ensures the usability of the optimized obstacle avoidance path, but also improves the generation efficiency of the optimized obstacle avoidance path.

[0121] Example 7

[0122] Based on Example 1, the intelligent driving obstacle avoidance method based on multi-source information fusion processing, step 4 includes:

[0123] Step 41: Convert the optimized obstacle avoidance path into a reminder voice and a reminder animation, and use the reminder voice and the reminder animation to assist the driver in executing the optimized obstacle avoidance path, while collecting the driver's real-time obstacle avoidance operations;

[0124] Step 42: Generate real-time obstacle avoidance feedback for the vehicle based on the real-time obstacle avoidance operation, use the real-time obstacle avoidance feedback to determine real-time obstacle information within a specified range of the vehicle, and adjust the real-time obstacle avoidance direction of the optimized obstacle avoidance path based on the real-time obstacle information;

[0125] Step 43: Obtain obstacle avoidance information within the specified range of the vehicle, construct the obstacle avoidance progress of the vehicle based on the obstacle avoidance information, and stop the assisted obstacle avoidance work when the obstacle avoidance progress is completed.

[0126] In this example, during obstacle avoidance assistance, the prompt voice and prompt animation are played simultaneously.

[0127] The working principle and beneficial effects of the above technical solution are as follows: In order to improve the quality of assistance, the driver is reminded to avoid obstacles by combining voice and animation during obstacle avoidance. The system analyzes real-time obstacle information near the vehicle based on the driver's real-time obstacle avoidance operations, and adjusts the vehicle's obstacle avoidance method according to the actual situation. At the same time, the progress of obstacle avoidance is monitored. Once the obstacle avoidance is completed, the obstacle avoidance assistance stops. This method not only ensures the quality of obstacle avoidance, but also adjusts the obstacle avoidance path according to the actual situation, further effectively reducing the risk of collision.

[0128] Example 8

[0129] Based on Example 1, the intelligent driving obstacle avoidance method based on multi-source information fusion processing further includes:

[0130] When the driver issues an obstacle avoidance command, an obstacle avoidance path is generated for the vehicle and assists the driver in avoiding obstacles.

[0131] The working principle and beneficial effects of the above technical solution are as follows: Drivers can activate intelligent obstacle avoidance as needed to assist them in driving smoothly.

[0132] Example 9

[0133] This embodiment provides an intelligent driving obstacle avoidance system based on multi-source information fusion processing, such as... Figure 2 As shown, it includes:

[0134] The information acquisition module is used to collect real-time vehicle driving information and real-time environmental image information of the vehicle while the vehicle is in motion to construct an environmental map of the vehicle, and to filter obstacles within a specified range of the vehicle in the environmental map.

[0135] The obstacle avoidance analysis module is used to generate several obstacle avoidance paths based on the positional relationship between the obstacle and the vehicle, and to collect real-time driver operation information of the vehicle to analyze the reaction operation time corresponding to each obstacle avoidance path executed by the driver.

[0136] The obstacle avoidance optimization module is used to estimate the obstacle avoidance success rate of the corresponding obstacle avoidance path based on the reaction operation time, collect the vehicle's historical driving data, optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path.

[0137] The obstacle avoidance assist module is used to assist the driver in executing the optimized obstacle avoidance path, and to collect real-time obstacle avoidance feedback from the vehicle. Based on the real-time obstacle avoidance feedback, the optimized obstacle avoidance path is adjusted in real time until the vehicle completes obstacle avoidance.

[0138] In this example, real-time vehicle driving information refers to the information generated by the vehicle during driving, real-time environmental image information refers to the image information composed of the front and rear images of the vehicle's driving direction, and real-time driver operation information refers to the information generated when the driver operates the vehicle during driving.

[0139] In this example, the default defined range is a radius of 100 centimeters centered on the vehicle; the driver can adjust the length of the radius according to their driving habits and needs.

[0140] In this example, the environmental map represents a map based on the surrounding environment of the vehicle;

[0141] In this example, reaction time refers to the reaction time required for a driver to execute an obstacle avoidance path;

[0142] In this example, the obstacle avoidance success rate represents the probability that a driver can successfully avoid an obstacle when executing an obstacle avoidance path.

[0143] In this example, real-time obstacle avoidance feedback refers to the information about the external environment collected by the driver during the obstacle avoidance process.

[0144] The working principle and beneficial effects of the above technical solution are as follows: To assist drivers in better obstacle avoidance and generate obstacle avoidance methods adapted to their specific needs, a vehicle environment map is created by collecting real-time vehicle driving information and real-time environmental image information. This map analyzes whether there are obstacles within a specified range of the vehicle. This allows for real-time monitoring of the distribution of obstacles near the vehicle. Then, based on the positional relationship between obstacles and the vehicle, an obstacle avoidance path is constructed. To improve the intelligence of obstacle avoidance, real-time driver operation information is combined to analyze the driver's reaction time when executing each obstacle avoidance path, thereby estimating the optimal obstacle avoidance path. The system improves obstacle avoidance success rate by inferring driver habits from historical driving data to optimize the selected obstacle avoidance path. This optimized path assists the driver in performing obstacle avoidance tasks. Furthermore, to prevent obstacle avoidance failure due to sudden changes in the external environment, the optimized path is adjusted in real-time based on the vehicle's obstacle avoidance feedback during the process, further assisting the driver in completing the obstacle avoidance task. This method utilizes various information from the vehicle's driving process to assist obstacle avoidance, helping the driver avoid obstacles according to their own driving habits. This improves the quality of obstacle avoidance and ensures the safety of both the driver and the vehicle during the obstacle avoidance process.

[0145] Example 10

[0146] Based on Embodiment 9, the intelligent driving obstacle avoidance system based on multi-source information fusion processing includes an information acquisition module comprising:

[0147] The information processing unit is used to collect real-time device data generated by each vehicle device in the vehicle when the vehicle is in a driving state, resample each real-time device data by time to generate several periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data to generate real-time driving information of the vehicle.

[0148] The image fusion unit is used to acquire the front and rear image videos of the vehicle, construct the environmental image pairs of the vehicle based on the front and rear image images corresponding to the same moment in the front and rear image videos, and use stereo vision technology to identify several stereo environmental objects contained in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object.

[0149] The modeling and analysis unit is used to draw a three-dimensional environment model of the vehicle in a preset three-dimensional space according to the specifications of the item, dynamically render the three-dimensional environment model using the front and rear video images to generate a three-dimensional dynamic model of the vehicle, and construct a real-time environmental image of the vehicle based on the real-time dynamic features of the three-dimensional dynamic model.

[0150] An obstacle recognition unit is used to fuse the real-time vehicle driving information and the real-time environmental information, project the fusion result to obtain an environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify obstacles contained within a specified range of the vehicle with the real-time driving position as the center.

[0151] In this example, vehicle equipment refers to the equipment that makes up a vehicle, and one vehicle equipment corresponds to one real-time equipment data.

[0152] In this example, time resampling refers to the process of aggregating real-time device data in chronological order, with each sampling period being 1 minute.

[0153] In this example, timestamp alignment means aligning the same moment in the timestamps of the device data in each cycle;

[0154] In this example, the environmental image pair represents the result of pairing the front and rear video images of the vehicle captured at the same time.

[0155] In this example, stereoscopic vision technology refers to the process of simulating human binocular vision using two cameras;

[0156] In this example, 3D environment objects represent the result of converting objects contained in an environment image into a 3D effect;

[0157] In this example, the preset three-dimensional space is represented by the space composed of the X-axis, Y-axis, and Z-axis;

[0158] In this example, the item specification parameter represents the data used to describe the specifications of items in a three-dimensional environment.

[0159] The working principle and beneficial effects of the above technical solution are as follows: To better identify obstacles near the vehicle, when the vehicle is in motion, real-time device data generated by the vehicle's equipment is resampled and timestamped to construct real-time driving information. By collecting data from each vehicle device, the accuracy and precision of real-time driving information can be improved. Then, the front and rear video images of the vehicle are fused, and stereoscopic vision technology is used to identify stereoscopic environmental objects contained in the environmental images. Based on the object specifications of each stereoscopic environmental object, a 3D environmental model of the vehicle is drawn in a preset 3D space. Furthermore, dynamic rendering is used to construct a 3D dynamic model of the vehicle. In this way, the collected image information can be used to create a 3D dynamic model, improving the fit between the 3D dynamic model and the environment. Then, the real-time dynamic characteristics of the 3D dynamic model are used to determine the actual environmental image of the vehicle. The real-time vehicle driving information and real-time environmental information are then fused and projected to generate an environmental map of the vehicle. Finally, obstacles near the vehicle are identified in the environmental map. This method can not only ensure the effectiveness of obstacle identification and reduce identification errors, but also monitor the environment around the vehicle in real time, improving the driver's driving safety awareness.

[0160] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An intelligent driving obstacle avoidance method based on multi-source information fusion processing, characterized in that, include: Step 1: Collect real-time vehicle driving information and real-time environmental image information of the vehicle while it is in motion to construct an environmental map of the vehicle, and filter obstacles within a specified range of the vehicle in the environmental map; Step 2: Generate several obstacle avoidance paths based on the positional relationship between the obstacle and the vehicle, and collect real-time driver operation information of the vehicle to analyze the reaction operation time corresponding to each obstacle avoidance path executed by the driver; Step 3: Estimate the obstacle avoidance success rate of the corresponding obstacle avoidance path based on the reaction operation time, collect the vehicle's historical driving data, optimize the target obstacle avoidance path with the highest success rate, and construct an optimized obstacle avoidance path; Step 4: Assist the driver in executing the optimized obstacle avoidance path and collect real-time obstacle avoidance feedback from the vehicle. Adjust the optimized obstacle avoidance path in real time based on the real-time obstacle avoidance feedback until the vehicle completes obstacle avoidance. Step 2 includes: Step 21: Determine the driving direction and speed of the vehicle based on the real-time vehicle driving information, determine the moving direction and moving speed corresponding to each obstacle based on the real-time environmental image information, deduce the collision point and collision time between the vehicle and each obstacle, and mark each collision point and collision time in the environmental map respectively. Step 22: Determine several safe zones contained in the environment map based on the collision point and collision time, and the safe time period corresponding to each safe zone. Simulate the continuous driving path of the vehicle in the safe zone in the environment map to generate several obstacle avoidance paths. Step 23: Based on the distribution of the safe zone in the environmental map, each obstacle avoidance path is divided into several operational areas, and the vehicle driving conditions corresponding to each operational area are determined to construct the obstacle avoidance operation process corresponding to the driver operating each obstacle avoidance path. Step 24: Based on the real-time driver operation information, simulate several operation pause features corresponding to each obstacle avoidance operation performed by the driver, locate the pause position corresponding to each operation pause feature in the corresponding obstacle avoidance operation, filter target pause features whose pause positions are not within the safe area, and determine the driver's reaction operation time to the corresponding obstacle avoidance path based on several target pause features corresponding to each obstacle avoidance operation.

2. The intelligent driving obstacle avoidance method based on multi-source information fusion processing as described in claim 1, characterized in that, Step 1 includes: Step 11: When the vehicle is in motion, collect real-time device data generated by each vehicle device in the vehicle, resample each real-time device data by time to generate several periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data to generate real-time driving information of the vehicle. Step 12: Acquire the front and rear video images of the vehicle, construct environmental image pairs of the vehicle based on the front and rear images corresponding to the same moment in the front and rear video images, and use stereo vision technology to identify several stereo environmental objects contained in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object. Step 13: Draw a three-dimensional environment model of the vehicle in a preset three-dimensional space according to the specifications of the item, and dynamically render the three-dimensional environment model using the front and rear video images to generate a three-dimensional dynamic model of the vehicle. Construct a real-time environmental image of the vehicle based on the real-time dynamic features of the three-dimensional dynamic model. Step 14: Fuse the real-time vehicle driving information and the real-time environmental image information, project the fusion result to obtain the environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify obstacles within a specified range of the vehicle with the real-time driving position as the center.

3. The intelligent driving obstacle avoidance method based on multi-source information fusion processing as described in claim 1, characterized in that, Also includes: Based on the real-time environmental image information, the obstacles are divided into dynamic obstacles and static obstacles; Locate the image information corresponding to each dynamic obstacle in the real-time environmental image, and determine the movement direction and movement speed corresponding to each dynamic obstacle; The direction in which the stationary obstacle points toward the vehicle is considered the direction of movement of the stationary obstacle, and the speed of movement of the stationary obstacle is recorded as 0.

4. The intelligent driving obstacle avoidance method based on multi-source information fusion processing as described in claim 1, characterized in that, Step 3 includes: Step 31: Determine several operational hazard points corresponding to the obstacle avoidance path based on the reaction operation time. Determine whether the path position is a safe position based on the path position corresponding to each operational hazard point. If it is, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding hazard label. Step 32: Determine several collision hazard points for each obstacle avoidance path based on the distance information between each obstacle avoidance path and different obstacles, obtain the driver's avoidance operation characteristics corresponding to the collision hazard points, and determine whether the corresponding collision hazard point is a safe location based on the avoidance operation characteristics. If it is, set a corresponding safety label for the corresponding obstacle avoidance path; otherwise, set a corresponding danger label. Step 33: Statistically count the number of safety tags and danger tags corresponding to each obstacle avoidance path, determine the obstacle avoidance success rate of the corresponding obstacle avoidance path, filter the target obstacle avoidance path with the highest obstacle avoidance success rate, obtain the historical driving data of the vehicle, and construct several driving habits of the driver based on the historical driving data; Step 34: Optimize the target obstacle avoidance path as a whole using each of the aforementioned driving habits, and at the same time obtain the relevant driving habits corresponding to each of the aforementioned hazard labels. Optimize the corresponding hazard locations in detail using the relevant driving habits to generate the optimized obstacle avoidance path for the vehicle.

5. The intelligent driving obstacle avoidance method based on multi-source information fusion processing as described in claim 4, characterized in that, Also includes: When the target obstacle avoidance path does not contain hazard labels, the target obstacle avoidance path is optimized as a whole using each of the aforementioned driving habits to generate the vehicle's optimized obstacle avoidance path.

6. The intelligent driving obstacle avoidance method based on multi-source information fusion processing as described in claim 1, characterized in that, Step 4 includes: Step 41: Convert the optimized obstacle avoidance path into a reminder voice and a reminder animation, and use the reminder voice and the reminder animation to assist the driver in executing the optimized obstacle avoidance path, while collecting the driver's real-time obstacle avoidance operations; Step 42: Generate real-time obstacle avoidance feedback for the vehicle based on the real-time obstacle avoidance operation, use the real-time obstacle avoidance feedback to determine real-time obstacle information within a specified range of the vehicle, and adjust the real-time obstacle avoidance direction of the optimized obstacle avoidance path based on the real-time obstacle information; Step 43: Obtain obstacle avoidance information within the specified range of the vehicle, construct the obstacle avoidance progress of the vehicle based on the obstacle avoidance information, and stop the assisted obstacle avoidance work when the obstacle avoidance progress is completed.

7. The intelligent driving obstacle avoidance method based on multi-source information fusion processing as described in claim 1, characterized in that, Also includes: When the driver issues an obstacle avoidance command, an obstacle avoidance path is generated for the vehicle and assists the driver in avoiding obstacles.

8. An intelligent driving obstacle avoidance system based on multi-source information fusion processing, characterized in that, include: The information acquisition module is used to collect real-time vehicle driving information and real-time environmental image information of the vehicle while the vehicle is in motion to construct an environmental map of the vehicle, and to filter obstacles within a specified range of the vehicle in the environmental map. The obstacle avoidance analysis module is used to generate several obstacle avoidance paths based on the positional relationship between the obstacle and the vehicle, and to collect real-time driver operation information of the vehicle to analyze the reaction operation time corresponding to each obstacle avoidance path executed by the driver. The obstacle avoidance optimization module is used to estimate the obstacle avoidance success rate of the corresponding obstacle avoidance path based on the reaction operation time, collect the vehicle's historical driving data, optimize the target obstacle avoidance path with the highest obstacle avoidance success rate, and construct an optimized obstacle avoidance path. The obstacle avoidance assist module is used to assist the driver in executing the optimized obstacle avoidance path, and to collect the real-time obstacle avoidance feedback of the vehicle. Based on the real-time obstacle avoidance feedback, the optimized obstacle avoidance path is adjusted in real time until the vehicle completes obstacle avoidance. The obstacle avoidance analysis module generates several obstacle avoidance paths based on the positional relationship between the obstacle and the vehicle, and collects real-time driver operation information to analyze the driver's reaction time for each obstacle avoidance path, including: The driving direction and speed of the vehicle are determined based on the real-time vehicle driving information. The moving direction and speed of each obstacle are determined based on the real-time environmental image information. The collision point and collision time between the vehicle and each obstacle are derived, and each collision point and collision time are marked on the environmental map. Based on the collision point and collision time, several safe zones contained in the environment map are determined, as well as the safe time period corresponding to each safe zone. The continuous driving path of the vehicle in the safe zone is simulated in the environment map to generate several obstacle avoidance paths. Based on the distribution of the safe zone in the environmental map, each obstacle avoidance path is divided into several operational areas, and the corresponding vehicle driving conditions in each operational area are determined to construct the obstacle avoidance operation process when the driver operates each obstacle avoidance path. Based on the real-time driver operation information, several operation pause features corresponding to each obstacle avoidance operation are simulated. The pause position corresponding to each operation pause feature is located in the corresponding obstacle avoidance operation. Target pause features whose pause positions are not within the safe area are filtered out. Based on several target pause features corresponding to each obstacle avoidance operation, the driver's reaction operation time to the corresponding obstacle avoidance path is determined.

9. The intelligent driving obstacle avoidance system based on multi-source information fusion processing as described in claim 8, characterized in that, The information collection module includes: The information processing unit is used to collect real-time device data generated by each vehicle device in the vehicle when the vehicle is in a driving state, resample each real-time device data by time to generate several periodic device data of the vehicle, and align the timestamps corresponding to each periodic device data to generate real-time driving information of the vehicle. The image fusion unit is used to acquire the front and rear image videos of the vehicle, construct the environmental image pairs of the vehicle based on the front and rear image images corresponding to the same moment in the front and rear image videos, and use stereo vision technology to identify several stereo environmental objects contained in each environmental image pair to obtain the item specification parameters corresponding to each stereo environmental object. The modeling and analysis unit is used to draw a three-dimensional environment model of the vehicle in a preset three-dimensional space according to the specifications of the item, dynamically render the three-dimensional environment model using the front and rear video images to generate a three-dimensional dynamic model of the vehicle, and construct a real-time environmental image of the vehicle based on the real-time dynamic features of the three-dimensional dynamic model. An obstacle recognition unit is used to fuse the real-time vehicle driving information and the real-time environmental image information, project the fusion result to obtain an environmental map of the vehicle, locate the real-time driving position of the vehicle in the environmental map, and identify obstacles contained within a specified range of the vehicle with the real-time driving position as the center.

Images