Automatic driving image quality detection method and system based on deep learning model
Through the self-driving image quality detection method based on deep learning models, the problem of poor recognition capabilities of traditional methods in complex environments is solved, accurate evaluation of image quality and autonomous driving control decisions are achieved, and the safety and real-time nature of the system are improved.
Patent Information
- Application Number
- CN202411973059.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-05-13
AI Technical Summary
Traditional autonomous driving image recognition methods are difficult to accurately identify lane lines and obstacles in complex environments, and lack effective image quality evaluation and multi-camera input fusion mechanisms, which cannot meet the high requirements of autonomous driving systems for real-time and accuracy.
The image quality detection method of autonomous driving based on deep learning models is adopted. By building a deep learning model, multi-camera input is processed, image quality evaluation is performed, and autonomous driving control decisions are made based on the evaluation results, including automatic downgrade or exit, and users are reminded.
It improves the image recognition capabilities of the autonomous driving system in complex environments, solves the problem of inaccurate image quality evaluation, enhances the safety and real-time nature of the system, and adapts to the high demands of the autonomous driving system.
Smart Images
Figure CN119992296A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving image quality detection method and system based on a deep learning model. Background Art
[0002] Image recognition is one of the key technologies in autonomous driving technology. Traditional image recognition methods rely on manual feature extraction and simple classifiers, which have significantly degraded performance when the image quality is poor. For example, in foggy, rainy or nighttime weather, the visibility and contrast of the image are reduced, making it difficult for traditional methods to accurately identify lane lines and obstacles. In addition, traditional methods lack effective fusion and quality assessment mechanisms when processing multi-camera inputs, and cannot adapt to the high requirements of autonomous driving systems for real-time performance and accuracy. Summary of the invention
[0003] In view of the above problems, the first purpose of the present invention is to provide an autonomous driving image quality detection method based on a deep learning model. The deep learning model is used to process multi-camera inputs and output accurate image quality assessments, which improves the image recognition capability of the autonomous driving system in complex environments and solves the problem of being unable to accurately assess image quality in various complex environments. It can automatically downgrade or exit when the image quality is poor and remind the user, thereby enhancing the safety of the system. The real-time and accuracy of the system are improved, and the high requirements of the autonomous driving system are met.
[0004] The second object of the present invention is to provide an autonomous driving image quality detection system based on a deep learning model.
[0005] To achieve the first objective, the first technical solution of the present invention is: an autonomous driving image quality detection method based on a deep learning model, comprising:
[0006] Step S01: Collect data, and pre-process, label, and classify the data to obtain a labeled image;
[0007] Step S02: constructing a deep learning model, using the annotated image to train the deep learning model, and deploying it to the automatic driving controller of the target intelligent driving vehicle;
[0008] Step S03: The autonomous driving controller receives real-time captured images, performs image quality assessment, and makes autonomous driving control decisions based on the image quality assessment results.
[0009] Preferably, the preprocessing in step S01 includes removing non-compliant images, and performing image denoising and contrast enhancement.
[0010] Preferably, in step S01, the annotations are marked as light or heavy according to the degree of impact on the safety of autonomous driving, and are classified according to the light or heavy annotation results.
[0011] Preferably, the labeling in step S01 also includes multi-person labeling and cross-validation.
[0012] Preferably, step S02 includes training the deep learning model according to a corresponding format of the target intelligent driving vehicle, saving and exporting the model.
[0013] Preferably, the method further includes optimizing the deep learning model, wherein the optimization uses INT8 quantization technology to convert floating-point parameters into integer parameters.
[0014] Preferably, in step S03, the real-time captured image includes a comprehensive view of the surrounding environment of the target intelligent driving vehicle, and the comprehensive view includes forward, rearward and side views.
[0015] Preferably, the method includes maintaining automatic driving if the image quality assessment result is that the image quality is good;
[0016] If the image quality assessment result is that the image quality has a slight problem, performing auxiliary improvement operations;
[0017] If the image quality assessment result is a serious image quality problem, the autonomous driving is downgraded or exited.
[0018] Preferably, a user interaction and reminder model is also included, which is used to remind the user to transfer the control right of the target intelligent driving vehicle according to the image quality assessment result.
[0019] To achieve the second purpose, the second technical solution of the present invention is: an autonomous driving image quality detection system based on a deep learning model, comprising:
[0020] Data collection module: used to collect data, and pre-process, label and classify the data to obtain labeled images;
[0021] A deep learning model construction and deployment module: used to construct a deep learning model, train the deep learning model using the annotated image, and deploy it to the automatic driving controller of the target intelligent driving vehicle;
[0022] The autonomous driving control decision module is used to perform image quality assessment based on the real-time captured image received by the autonomous driving controller, and make an autonomous driving control decision based on the image quality assessment result;
[0023] The user interaction and reminder module is used to remind the user to transfer the control of the target intelligent driving vehicle according to the image quality assessment results.
[0024] Beneficial effects of the above technical solution:
[0025] The automatic driving image quality detection method and system based on deep learning model provided by the present invention utilizes deep learning model to process multi-camera input and output accurate image quality assessment, thereby improving the image recognition capability of the automatic driving system in complex environments and solving the problem that image quality cannot be accurately assessed in various complex environments. It can automatically downgrade or exit when the image quality is poor and remind the user, thus enhancing the safety of the system. It improves the real-time performance and accuracy of the system and meets the high requirements of the automatic driving system. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0027] Figure 1 A flow chart of an autonomous driving image quality detection method based on a deep learning model provided for an embodiment of the present invention. DETAILED DESCRIPTION
[0028] The following is a further detailed description of the implementation methods of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.
[0029] The terms "first", "second", etc. (if any) in the specification and claims are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0030] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0031] Embodiment 1
[0032] An embodiment of the present invention provides an automatic driving image quality detection method based on a deep learning model. The specific process is as follows: Figure 1 As shown, including:
[0033] Building a deep learning model:
[0034] Build a deep learning model, collect data, pre-process the data, annotate the image, and then input it into the deep learning model to train the model. Specifically include: data enhancement, in order to improve the generalization ability and robustness of the model, the collected image data is subjected to a variety of augmentation operations. Preferred augmentation operations include but are not limited to rotation, color adjustment, brightness change, contrast enhancement, etc. Through these operations, more environmental changes are simulated so that the model can work stably under various conditions. In this embodiment, an original two-step architecture is adopted. In the first step, the network roughly classifies the image and divides the image into multiple categories such as glare, blur, occlusion, tunnel, rain and snow. The purpose of this step is to quickly identify the main problems in the image and provide a basis for subsequent fine classification. In the second step, the network subclassifies the image quality problems according to mild and severe standards. The purpose of this step is to accurately assess the extent of the image quality problem and provide a more detailed decision-making basis for the autonomous driving system.
[0035] Preferably, data collection is the cornerstone of building an effective deep learning model. Data collection is to automatically collect data related to image quality, which will cover various environmental conditions, such as rainy days, glare, etc., to ensure that the model can adapt to the changing actual situation. In this embodiment, an automated data collection process is adopted, and the automation of the data collection process is the key to achieving efficient data collection. In this embodiment, a series of preset conditions are used to trigger data collection. For example, the data collection mechanism is automatically activated by changes in ambient light and humidity detected by sensors. In addition, an image quality detector is used to assist in collecting data on image quality issues such as glare and blur. High-precision map information collects data on the tunnel environment, and determines whether the vehicle enters or leaves the tunnel area through GPS and vehicle location data. The collection of rainy day data is achieved by detecting the working status of the wiper. When the wiper is started, the system automatically records the relevant image data.
[0036] Data diversity and quality affect the generalization ability of the present invention, and the collected data needs to cover various extreme and edge cases. This includes glare under different lighting conditions, motion blur at different speeds, rainy day images under different rainfall amounts, etc. The diversity of data directly affects the robustness of the present invention. Therefore, in the data collection stage, it is necessary to focus on simulating various possible driving environments to ensure that the present invention can perform well in practical applications.
[0037] Data preprocessing: Perform preliminary data preprocessing on the collected data, such as denoising, contrast enhancement, etc., to improve the efficiency and effect of subsequent training. Perform preliminary screening of the data to exclude images that do not meet the labeling rules or are of too low quality, to ensure that the data input into the model training phase is of high quality and high relevance.
[0038] Preferably, image annotation is an indispensable part of deep learning model training, which directly determines the recognition ability and accuracy of the model. To ensure accurate classification and degree assessment of image quality problems. The present invention formulates annotation rules based on the degree of impact of image quality problems on autonomous driving safety. Problems are divided into two categories: mild and severe. Mild problems refer to problems that have little impact on the autonomous driving system but still require attention; severe problems refer to problems that seriously affect the performance of the autonomous driving system. For example, mild glare may only affect a local area of the image, while severe glare may cover the entire image, seriously affecting the system's perception of the environment.
[0039] The annotation is performed according to the annotation rules. The execution of the annotation process includes: first classifying the status of the entire image, and then distinguishing the degree of image quality problems. For example, for the forward camera, the severity of the image quality problem is judged according to the number of dashed lane segments starting from the lower edge of the image. For example, taking 5 white dashed lines as the threshold, if 5 white dashed lines can be clearly seen, and the obstacles in the self-lane within this range are clearly visible, it is considered mild; otherwise, if there are no lane lines in the front area or not all are clearly visible, it is considered severe. Or for problems such as blur, glare, too dark, overexposure, etc., a distinction is made between mild and severe based on whether it affects the visibility of the 70m self-lane ahead. For rear-facing and side cameras, a distinction is also made between mild and severe based on the clarity of the lane lines of the adjacent lanes and the obstacles in the lanes.
[0040] To ensure the accuracy and consistency of annotation, not only the above annotation rules are followed, but also multi-person annotation and cross-validation methods are adopted to reduce errors caused by subjectivity and improve the reliability of annotation.
[0041] Intelligent driving vehicle model deployment:
[0042] After the deep learning model is trained, it is deployed to the actual intelligent driving vehicle. Model export and optimization: After the model training is completed, the model is saved in the corresponding format of the target vehicle and exported to the corresponding format for running on the vehicle. It also includes optimizing the model to reduce the computational complexity and memory usage of the model. It can reduce the resource consumption of the model while maintaining the performance of the model.
[0043] INT8 quantization: In order to further improve the running efficiency of the model on the vehicle, the INT8 quantization technology is used to convert the floating point parameters in the model into integer parameters, thereby reducing the computational complexity and memory usage of the model. Through INT8 quantization, the model of the present invention can run efficiently in a resource-constrained vehicle environment while maintaining high performance.
[0044] Vehicle autonomous driving control decision:
[0045] After the model is deployed on the vehicle, the vehicle's autonomous driving controller will be responsible for the entire process on the vehicle side. After the model is deployed on the vehicle, the vehicle's autonomous driving controller will be responsible for the entire image quality detection process. It will evaluate the image quality in real time and remind the system to downgrade or exit when the quality is poor.
[0046] Vehicle-side image input and processing: The image data captured by each camera on the vehicle in real time is input into the model. These image data include a comprehensive view of the vehicle's surroundings, including the perspectives of the forward, rearward and side cameras, including pictures taken by fisheye cameras, wide-angle cameras, etc. The model processes the image in real time and outputs the evaluation results of the image quality. It is directly related to the safety and reliability of the autonomous driving system. Image quality judgment and response: Based on the output results of the model, the autonomous driving controller will judge the quality of the current image. If the image quality is good, the system will continue to operate normally; if the image quality is poor, the system will take different response measures according to the severity of the problem. For mild problems, the system may perform some auxiliary improvement operations, such as adjusting camera parameters, enhancing image contrast, etc.; for severe problems, the system may choose to downgrade the autonomous driving function or even completely exit the autonomous driving mode to ensure driving safety. User interaction and reminder: When the image quality is poor, the system will not only take corresponding technical measures, but also remind the driver to pay attention to the current driving environment through the vehicle's display system. This user interaction mechanism can ensure the driver's control over the vehicle while also improving the safety of the system.
[0047] The present invention also provides an autonomous driving image quality detection system based on a deep learning model, including a data collection module, a deep learning model construction and deployment module, an autonomous driving control decision module, and a user interaction and reminder module.
[0048] Among them, the data collection module is used to collect data, and pre-process, label and classify the data to obtain labeled images. The deep learning model construction and deployment module is used to build a deep learning model, train the deep learning model using the labeled images, and deploy it to the automatic driving controller of the target intelligent driving vehicle. The automatic driving control decision module is used to perform image quality assessment based on the real-time captured images received by the automatic driving controller, and make automatic driving control decisions based on the image quality assessment results. The user interaction and reminder module is used to remind the user to transfer the control of the target intelligent driving vehicle based on the image quality assessment results.
[0049] It can be seen that the automatic driving image quality detection method and system based on the deep learning model provided by the present invention utilizes the deep learning model to process multi-camera input and output accurate image quality evaluation, thereby improving the image recognition capability of the automatic driving system in complex environments and solving the problem of being unable to accurately evaluate image quality in various complex environments. It can automatically downgrade or exit when the image quality is poor and remind the user, thereby enhancing the safety of the system. It improves the real-time and accuracy of the system and meets the high requirements of the automatic driving system.
[0050] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made on the basis of the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the protection scope of the present invention.
Claims
1. A method for detecting image quality of autonomous driving based on a deep learning model, characterized in that: include: Step S01: Collect data, and pre-process, label, and classify the data to obtain a labeled image; Step S02: constructing a deep learning model, using the annotated image to train the deep learning model, and deploying it to the automatic driving controller of the target intelligent driving vehicle; Step S03: The autonomous driving controller receives real-time captured images, performs image quality assessment, and makes autonomous driving control decisions based on the image quality assessment results.
2. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 1, characterized in that: The preprocessing in step S01 includes removing non-compliant images, and performing image denoising and contrast enhancement.
3. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 1, characterized in that: In step S01, the annotations are marked as light or heavy according to the degree of impact on the safety of autonomous driving, and are classified according to the light or heavy annotation results.
4. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 1, characterized in that: The labeling in step S01 also includes multi-person labeling and cross-validation.
5. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 1, characterized in that: Step S02 includes training the deep learning model according to the corresponding format of the target intelligent driving vehicle, saving and exporting the model.
6. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 5, characterized in that: It also includes optimizing the deep learning model, and the optimization uses INT8 quantization technology to convert floating-point parameters into integer parameters.
7. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 1, characterized in that: In step S03, the real-time captured image includes a comprehensive view of the surrounding environment of the target intelligent driving vehicle, and the comprehensive view includes forward, rearward and side views.
8. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 1, characterized in that: Step S03 includes, if the image quality assessment result is that the image quality is good, maintaining the automatic driving; If the image quality assessment result is that the image quality has a slight problem, performing auxiliary improvement operations; If the image quality assessment result is a serious image quality problem, the autonomous driving is downgraded or exited.
9. The method for detecting image quality of autonomous driving based on a deep learning model according to claim 1, characterized in that: It also includes a user interaction and reminder model, which is used to remind the user to transfer control of the target intelligent driving vehicle based on the image quality assessment results.
10. An autonomous driving image quality detection system based on a deep learning model, characterized in that: include: Data collection module: used to collect data, and pre-process, label and classify the data to obtain labeled images; A deep learning model construction and deployment module: used to construct a deep learning model, train the deep learning model using the annotated image, and deploy it to the automatic driving controller of the target intelligent driving vehicle; The autonomous driving control decision module is used to perform image quality assessment based on the real-time captured image received by the autonomous driving controller, and make an autonomous driving control decision based on the image quality assessment result; The user interaction and reminder module is used to remind the user to transfer the control of the target intelligent driving vehicle according to the image quality assessment results.