Unmanned driving system based on image processing

Abstract

The invention discloses an unmanned driving system based on image processing, and belongs to the technical field of unmanned driving. The unmanned driving system based on image processing comprises a self-inspection upgrading module, an image acquisition module, an environment perception module, a planning decision module, a behavior prediction module, a scene understanding module, a personalized driving module, a parking assisting module and a safety monitoring module. According to the invention, the self-checking upgrading module is arranged, various sensors of a software self-checking system are adopted, the unmanned driving system based on image processing is upgraded, real-time detection of hardware quality is carried out through image comparison based on high-definition camera data, system problems are avoided, and stable operation is ensured; the image collection module and the big data real-time collection sub-module detect and analyze road images uploaded by a user in real time based on big data, and the traffic accident collection sub-module receives real-time traffic conditions of a driving road section and remotely collects the images based on data of a traffic police service platform, so that the processing effect is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned driving, and in particular relates to an unmanned driving system based on image processing. Background Art

[0002] The unmanned driving system is an innovation that applies computer vision technology to unmanned driving. Its purpose is to enable autonomous vehicles to perceive and understand the road environment through image data collected by sensors such as cameras, so as to drive autonomously and make decisions. Such a system aims to improve the safety, accuracy and adaptability of unmanned vehicles so that they can travel in complex traffic environments. The core tasks of the system include image acquisition, preprocessing, feature extraction, target detection and tracking, road scene understanding, decision making, and real-time control. Through these steps, the unmanned driving system based on image processing realizes the perception and response of autonomous driving vehicles to complex traffic environments, thereby improving safety and adaptability.

[0003] The existing unmanned driving systems based on image processing have the following problems when in use: the system cannot perform self-checking, which poses a safety hazard and affects its use; at the same time, the image collection method is single and remote collection cannot be achieved, which limits its use. Summary of the invention

[0004] In order to solve the problems raised in the above background technology, the present invention provides an unmanned driving system based on image processing, which has the characteristics of self-checking and upgrading of the system and portable image collection.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an unmanned driving system based on image processing, the unmanned driving system based on image processing comprising a self-checking and upgrading module, an image acquisition module, an environment perception module, a planning and decision-making module, a behavior prediction module, a scene understanding module, a personalized driving module, a parking assistance module, and a safety monitoring module;

[0006] The self-check and upgrade module uses software self-check system sensors and system upgrades to perform real-time hardware quality detection through high-definition camera image comparison to generate system self-check results;

[0007] The image acquisition module is based on the big data and traffic police service platform, the big data real-time collection submodule and the traffic accident collection submodule, which obtain the real-time traffic conditions of the driving section and generate the driving road environment results;

[0008] The environmental perception module uses deep learning-based image processing algorithms and multi-sensor fusion technology to detect and classify traffic signs and signals on the road in real time, identify and analyze the surrounding environment, and generate environmental perception results;

[0009] The planning and decision-making module uses a reinforcement learning algorithm based on the environmental perception results to achieve autonomous driving decision-making and planning, optimize the driving path and avoid danger, and generate decision-making and planning results;

[0010] The behavior prediction module uses machine learning methods to predict the behavior of peripheral vehicles and pedestrians based on the environmental perception results and decision-making and planning results, and generates behavior prediction results;

[0011] The scene understanding module uses the image semantic analysis algorithm to perform scene understanding and semantic planning based on the environmental perception results and behavior prediction results, and generates scene understanding results and semantic planning results;

[0012] Based on the scene understanding results, the personalized driving module interacts with the behaviors, preferences and emotions of the driver and passengers. Through adaptive learning methods, it automatically adjusts the in-car settings to provide a personalized driving and riding experience and generates personalized setting results.

[0013] The parking assistance module realizes automatic parking based on the environmental perception results and personalized setting results, in conjunction with the regional perception system and vehicle control system;

[0014] The safety monitoring module uses image processing technology and environmental perception results to continuously monitor roads to support road maintenance and repair work and generate safety monitoring reports.

[0015] Furthermore, the image acquisition module includes a big data real-time collection submodule and a traffic accident collection submodule.

[0016] Furthermore, the self-checking and upgrading module includes a system sensor self-checking module, a system software upgrading module and a system hardware checking module.

[0017] Furthermore, the big data real-time collection submodule detects and analyzes the road images uploaded by users in real time based on big data to generate real-time road condition information;

[0018] The traffic accident collection submodule receives the real-time traffic conditions of the driving section based on the data of the traffic police service platform and generates the driving road environment information.

[0019] Furthermore, the system sensor self-check module performs self-checks on various sensors used in the system based on the system program and generates a sensor self-check report;

[0020] The system software upgrade module upgrades the unmanned driving system based on image processing based on the service platform data to generate a complete system;

[0021] The system hardware detection module performs real-time hardware detection through image comparison based on high-definition camera data and generates a hardware detection report.

[0022] Furthermore, the execution of the unmanned driving system based on image processing includes the following steps:

[0023] S1. Based on the self-check upgrade module, various sensors of the system are self-checked by software, and the system is upgraded. The hardware is checked in real time through high-definition camera image comparison to generate system self-check results;

[0024] S2. Based on the image collection module, the big data real-time collection submodule and the traffic accident collection submodule, the real-time traffic conditions of the driving section are obtained and the driving road environment results are generated;

[0025] S3, based on camera data, uses deep convolutional neural network to perform real-time detection and classification to generate environmental perception results;

[0026] S4. Based on the environmental perception results, the Dijkstra shortest path algorithm and fuzzy logic control algorithm are used to perform path calculation and driving strategy optimization to generate decision-making and planning results;

[0027] S5. Based on the environmental perception results, Kalman filter and long short-term memory network are used to perform trajectory tracking and behavior prediction to generate behavior prediction results;

[0028] S6. Based on the environmental perception results, the deep learning semantic analysis algorithm and semantic association network algorithm are used to perform scene understanding and semantic planning, and generate scene understanding and semantic planning results;

[0029] S7. Based on the behavior prediction results and scene understanding results, support vector machine and K-means clustering algorithm are used to analyze driver and passenger behavior and identify preferences to generate personalized setting results;

[0030] S8. Based on the environmental perception results and personalized setting results, convolutional neural network and Dijkstra algorithm are used to find parking positions and formulate parking strategies, and generate safety monitoring and automatic parking result reports.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention upgrades the unmanned driving system based on image processing by setting up a self-checking upgrade module and using various sensors of the software self-checking system. Based on high-definition camera data, the hardware quality is detected in real time through image comparison to avoid system problems and ensure stable operation.

[0033] 2. The image collection module and the big data real-time collection submodule of the present invention detect and analyze the road images uploaded by users in real time based on big data to generate real-time road condition information. The traffic accident collection submodule receives the real-time traffic conditions of the driving section based on the traffic police service platform data, collects images remotely, and improves the processing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the system framework of the present invention;

[0035] Figure 2 It is a flow chart of the image acquisition module of the present invention;

[0036] Figure 3 This is a flow chart of the self-check upgrade module of the present invention;

[0037] Figure 4 It is a schematic diagram of the execution steps of the system of the present invention; DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] See also Figure 1-4 , the present invention provides the following technical solutions: an unmanned driving system based on image processing, the unmanned driving system based on image processing includes a self-checking and upgrading module, an image acquisition module, an environment perception module, a planning and decision-making module, a behavior prediction module, a scene understanding module, a personalized driving module, a parking assistance module, and a safety monitoring module;

[0040] The self-check and upgrade module uses software self-check system sensors and system upgrades to perform real-time hardware quality detection through high-definition camera image comparison to generate system self-check results;

[0041] The image acquisition module is based on the big data and traffic police service platform, the big data real-time collection submodule and the traffic accident collection submodule, which obtain the real-time traffic conditions of the driving section and generate the driving road environment results;

[0042] The environmental perception module uses deep learning-based image processing algorithms and multi-sensor fusion technology to detect and classify traffic signs and signals on the road in real time, identify and analyze the surrounding environment, and generate environmental perception results;

[0043] The planning and decision-making module uses a reinforcement learning algorithm based on the environmental perception results to achieve autonomous driving decision-making and planning, optimize the driving path and avoid danger, and generate decision-making and planning results;

[0044] The behavior prediction module uses machine learning methods to predict the behavior of peripheral vehicles and pedestrians based on the environmental perception results and decision-making and planning results, and generates behavior prediction results;

[0045] The scene understanding module uses the image semantic analysis algorithm to perform scene understanding and semantic planning based on the environmental perception results and behavior prediction results, and generates scene understanding results and semantic planning results;

[0046] Based on the scene understanding results, the personalized driving module interacts with the behaviors, preferences and emotions of the driver and passengers. Through adaptive learning methods, it automatically adjusts the in-car settings to provide a personalized driving and riding experience and generates personalized setting results.

[0047] The parking assistance module realizes automatic parking based on the environmental perception results and personalized setting results, in conjunction with the regional perception system and vehicle control system;

[0048] The safety monitoring module uses image processing technology and environmental perception results to continuously monitor roads to support road maintenance and repair work and generate safety monitoring reports.

[0049] Furthermore, in the present invention, the image acquisition module includes a big data real-time collection submodule and a traffic accident collection submodule.

[0050] Furthermore, in the present invention, the self-checking and upgrading module includes a system sensor self-checking module, a system software upgrading module and a system hardware checking module.

[0051] Further in the present invention, the big data real-time collection submodule detects and analyzes the road images uploaded by users in real time based on the big data to generate real-time road condition information;

[0052] The traffic accident collection submodule receives the real-time traffic conditions of the driving section based on the data of the traffic police service platform and generates the driving road environment information.

[0053] Further in the present invention, the system sensor self-check module performs self-checks on various sensors used in the system based on the system program and generates a sensor self-check report;

[0054] The system software upgrade module upgrades the unmanned driving system based on image processing based on the service platform data to generate a complete system;

[0055] The system hardware detection module performs real-time hardware detection through image comparison based on high-definition camera data and generates a hardware detection report.

[0056] Further in the present invention, the execution of the unmanned driving system based on image processing includes the following steps:

[0057] S1. Based on the self-check upgrade module, various sensors of the system are self-checked by software, and the system is upgraded. The hardware is checked in real time through high-definition camera image comparison to generate system self-check results;

[0058] S2. Based on the image collection module, the big data real-time collection submodule and the traffic accident collection submodule, the real-time traffic conditions of the driving section are obtained and the driving road environment results are generated;

[0059] S3, based on camera data, uses deep convolutional neural network to perform real-time detection and classification to generate environmental perception results;

[0060] S4. Based on the environmental perception results, the Dijkstra shortest path algorithm and fuzzy logic control algorithm are used to perform path calculation and driving strategy optimization to generate decision-making and planning results;

[0061] S5. Based on the environmental perception results, Kalman filter and long short-term memory network are used to perform trajectory tracking and behavior prediction to generate behavior prediction results;

[0062] S6. Based on the environmental perception results, the deep learning semantic analysis algorithm and semantic association network algorithm are used to perform scene understanding and semantic planning, and generate scene understanding and semantic planning results;

[0063] S7. Based on the behavior prediction results and scene understanding results, support vector machine and K-means clustering algorithm are used to analyze driver and passenger behavior and identify preferences to generate personalized setting results;

[0064] S8. Based on the environmental perception results and personalized setting results, convolutional neural network and Dijkstra algorithm are used to find parking positions and formulate parking strategies, and generate safety monitoring and automatic parking result reports.

[0065] The structures and usage principles of the environmental perception module, planning and decision-making module, behavior prediction module, scene understanding module, personalized driving module, parking assistance module, and safety monitoring module in the present invention have been disclosed in an unmanned driving system and method based on image processing disclosed in Chinese patent application number 202311354406.5.

[0066] The working principle and use process of the present invention are as follows: Based on the self-check upgrade module, the present invention adopts various sensors of the software self-check system, system upgrade, and performs real-time detection of hardware quality through high-definition camera image comparison to generate system self-check results; Based on the image collection module, the big data real-time collection submodule and the traffic accident collection submodule, the real-time traffic conditions of the driving section are obtained to generate driving road environment results; Based on the camera data, a deep convolutional neural network is used to perform real-time detection and classification to generate environmental perception results; Based on the environmental perception results, the Dijkstra shortest path algorithm and the fuzzy logic control algorithm are used to perform path calculation and driving strategy optimization to generate decision-making and planning results; Based on the environmental perception Based on the results of environmental perception, Kalman filter and long short-term memory network are used to perform trajectory tracking and behavior prediction, and generate behavior prediction results. Based on the results of environmental perception, deep learning semantic analysis algorithm and semantic association network algorithm are used to perform scene understanding and semantic planning, and generate scene understanding and semantic planning results. Based on the behavior prediction results and scene understanding results, support vector machine and K-means clustering algorithm are used to analyze driver and passenger behavior and identify preferences, and generate personalized setting results. Based on the results of environmental perception and personalized setting results, convolutional neural network and Dijkstra algorithm are used to find parking positions and formulate parking strategies, and generate safety monitoring and automatic parking result reports.

[0067] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An unmanned driving system based on image processing, characterized in that: The unmanned driving system based on image processing includes a self-checking and upgrading module, an image acquisition module, an environment perception module, a planning and decision-making module, a behavior prediction module, a scene understanding module, a personalized driving module, a parking assistance module, and a safety monitoring module; The self-check upgrade module uses software self-check system various sensors, system upgrade, and real-time detection of hardware quality through high-definition camera image comparison to generate system self-check results; The image acquisition module is based on the big data and traffic police service platform, the big data real-time collection submodule and the traffic accident collection submodule, to obtain the real-time traffic conditions of the driving section and generate the driving road environment results; The environmental perception module uses deep learning-based image processing algorithms and multi-sensor fusion technology to detect and classify traffic signs and signals on the road in real time, identify and analyze the surrounding environment, and generate environmental perception results; The planning and decision-making module uses a reinforcement learning algorithm based on the environmental perception results to achieve autonomous driving decision-making and planning, optimize the driving path and avoid danger, and generate decision-making and planning results; The behavior prediction module uses a machine learning method to predict the behavior of peripheral vehicles and pedestrians based on the environmental perception results and the decision and planning results, and generates behavior prediction results; The scene understanding module uses an image semantic analysis algorithm to perform scene understanding and semantic planning based on the environmental perception results and behavior prediction results, and generates scene understanding results and semantic planning results; The personalized driving module interacts with the behaviors, preferences and emotions of the driver and passengers based on the scene understanding results, and automatically adjusts the in-vehicle settings through an adaptive learning method to provide a personalized driving and riding experience, thereby generating personalized setting results; The parking assistance module realizes automatic parking according to the environmental perception results and the personalized setting results in cooperation with the regional perception system and the vehicle control system; The safety monitoring module uses image processing technology and environmental perception results to continuously monitor roads to support road maintenance and repair work and generate safety monitoring reports.

2. The unmanned driving system based on image processing according to claim 1, characterized in that: The image acquisition module includes a big data real-time collection submodule and a traffic accident collection submodule.

3. The unmanned driving system based on image processing according to claim 1, characterized in that: The self-checking and upgrading module comprises a system sensor self-checking module, a system software upgrading module and a system hardware detecting module.

4. The unmanned driving system based on image processing according to claim 2, characterized in that: The big data real-time collection submodule detects and analyzes the road images uploaded by users in real time based on big data to generate real-time road condition information; The traffic accident collection submodule receives the real-time traffic conditions of the driving section based on the traffic police service platform data and generates the driving road environment information.

5. The unmanned driving system based on image processing according to claim 3, characterized in that: The system sensor self-check module performs self-checks on various sensors used in the system based on the system program and generates a sensor self-check report; The system software upgrade module upgrades the unmanned driving system based on image processing based on the service platform data to generate a system with complete content; The system hardware detection module performs real-time detection of hardware quality through image comparison based on high-definition camera data and generates a hardware detection report.

6. An unmanned driving system based on image processing according to any one of claims 1 to 5, characterized in that: The execution of the unmanned driving system based on image processing includes the following steps: S1. Based on the self-check upgrade module, various sensors of the system are self-checked by software, and the system is upgraded. The hardware is checked in real time through high-definition camera image comparison to generate system self-check results; S2. Based on the image collection module, the big data real-time collection submodule and the traffic accident collection submodule, the real-time traffic conditions of the driving section are obtained and the driving road environment results are generated; S3, based on camera data, uses deep convolutional neural network to perform real-time detection and classification to generate environmental perception results; S4. Based on the environmental perception results, the Dijkstra shortest path algorithm and fuzzy logic control algorithm are used to perform path calculation and driving strategy optimization to generate decision-making and planning results; S5. Based on the environmental perception results, Kalman filter and long short-term memory network are used to perform trajectory tracking and behavior prediction to generate behavior prediction results; S6. Based on the environmental perception results, the deep learning semantic analysis algorithm and semantic association network algorithm are used to perform scene understanding and semantic planning, and generate scene understanding and semantic planning results; S7. Based on the behavior prediction results and scene understanding results, support vector machine and K-means clustering algorithm are used to analyze driver and passenger behavior and identify preferences to generate personalized setting results; S8. Based on the environmental perception results and personalized setting results, convolutional neural network and Dijkstra algorithm are used to find parking positions and formulate parking strategies, and generate safety monitoring and automatic parking result reports.

Images