Image quality enhancement system for autonomous driving

Abstract

The present invention relates to an image quality enhancement system for automatic driving, comprising: a road acquisition device for performing image signal acquisition operations on the road environment at the front end of a smart car; a network application device for intelligently analyzing the sharpening level of a processed image obtained after performing enhancement processing using the enhancement algorithm according to code data corresponding to the enhancement algorithm used, various screen content information of the order conversion image, and various configuration parameters of the order conversion image. Through the present invention, different types of enhancement algorithms can be intelligently analyzed using a feedforward neural network model to obtain different sharpening levels, and the enhancement algorithm corresponding to the sharpening level with the highest value is used as the optimal enhancement algorithm for matching the order conversion image obtained after the automatic driving scene image optimization processing to perform enhancement processing on the order conversion image, so as to obtain the final optimized image as auxiliary data of the automatic driving mode of the smart car.

Description

Technical Field

[0001] The present invention relates to the field of smart cars, and in particular to an image quality enhancement system for autonomous driving. Background Art

[0002] The automatic driving system uses advanced communication, computer, network and control technologies to achieve real-time and continuous control of the train. It uses modern communication means and directly faces the train, which can realize two-way data communication between the train and the ground, with fast transmission rate and large amount of information. The subsequent tracking train and control center can know the exact location of the preceding train in time, making operation management more flexible, control more effective, and more suitable for the needs of automatic train driving.

[0003] It can be seen that the automatic driving system has liberated manpower for various means of transportation and provided convenience for driving control operations. The automatic driving system generally relies on the image data of on-site monitoring. However, the image content of the image data corresponding to different times is different. If the same enhancement algorithm is used to perform the same enhancement processing on each image data corresponding to different times, the final image obtained is often not of the most optimized quality. Therefore, it is necessary to find an enhancement algorithm that can obtain the best image quality for each image data of the automatic driving at different times. Summary of the invention

[0004] In order to solve the technical problems in the prior art, the present invention provides an image quality enhancement system for autonomous driving, which uses a feedforward neural network model to perform intelligent analysis on different types of enhancement algorithms to obtain different sharpening levels, and uses the enhancement algorithm corresponding to the sharpening level with the highest value as the optimal enhancement algorithm to match the order conversion image obtained after the optimization processing of the autonomous driving scene image, so as to perform enhancement processing on the order conversion image to obtain the final optimized image used as auxiliary data for the autonomous driving mode of the smart car.

[0005] According to the present invention, there is provided a picture quality enhancement system for autonomous driving, the system comprising:

[0006] A mode detection device is provided in the smart car, and is used to send a first detection signal when detecting that the smart car is currently in the automatic driving mode, and is also used to send a second detection signal when detecting that the smart car is currently in the non-automatic driving mode;

[0007] A road surface acquisition device, arranged at the front end of the smart car and connected to the mode detection device, is used to start an image signal acquisition operation of the road surface environment at the front end of the smart car when receiving the first detection signal, so as to obtain and output a corresponding road surface environment image;

[0008] An order conversion device is connected to the road surface acquisition device and is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image, a distortion correction process, and a sharpening process based on a USM filter on the received road surface environment image order, so as to obtain and output a corresponding order conversion image;

[0009] A data capture device, arranged in the control console of the smart car and connected to the order conversion device, for receiving the order conversion image and acquiring various screen content information of the order conversion image, wherein the various screen content information of the order conversion image includes the resolution of the order conversion image, the number of foreground pixels, and various brightness values ​​corresponding to each pixel;

[0010] A configuration extraction device connected to the order conversion device, used to obtain various configuration parameters of the order conversion image, wherein the various configuration parameters of the order conversion image include a signal-to-noise ratio and a contrast ratio of the order conversion image;

[0011] A network application device, connected to the data capture device and the configuration extraction device respectively, for intelligently analyzing the sharpness level of the processed image obtained after the order conversion image is enhanced by the enhancement algorithm according to the code data corresponding to the enhancement algorithm used, the various screen content information of the order conversion image and the various configuration parameters of the order conversion image by using a feedforward neural network model, wherein the feedforward neural network model is a feedforward neural network after multiple learning;

[0012] Among them, the network application device is also used to perform intelligent analysis of different types of enhancement algorithms using a feedforward neural network model to obtain different sharpening levels, and use the enhancement algorithm corresponding to the sharpening level with the highest value as the optimal enhancement algorithm matching the order-converted image to perform enhancement processing on the order-converted image to obtain a final optimized image.

[0013] It can be seen that the present invention has at least the following three important inventive concepts:

[0014] First: obtaining various picture content information of the order conversion image obtained after targeted optimization processing of the image signal of the front road environment of the smart car, the various picture content information of the order conversion image includes the resolution of the order conversion image, the number of foreground pixels and the brightness values ​​corresponding to each pixel, and obtaining various configuration parameters of the order conversion image at the same time, the various configuration parameters of the order conversion image include the signal-to-noise ratio and contrast of the order conversion image, so as to provide various reliable basic information for the processing effects of various enhancement algorithms of the order conversion image;

[0015] Second: using a feedforward neural network model to intelligently analyze the sharpness level of a processed image obtained after the order conversion image is enhanced using the enhancement algorithm according to the code data corresponding to the enhancement algorithm used, various screen content information of the order conversion image, and various configuration parameters of the order conversion image, wherein the feedforward neural network model is a feedforward neural network after multiple learning;

[0016] Third: Different types of enhancement algorithms are respectively subjected to intelligent analysis using a feedforward neural network model to obtain different sharpening levels, and the enhancement algorithm corresponding to the sharpening level with the highest value is used as the optimal enhancement algorithm matching the order conversion image to perform enhancement processing on the order conversion image, so as to obtain the final optimized image used as auxiliary data for the automatic driving mode of the smart car. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The embodiments of the present invention will be described below with reference to the accompanying drawings, wherein:

[0018] Figure 1 Schematic diagram of the internal structure of an image quality enhancement system for autonomous driving according to the first embodiment of the present invention.

[0019] Figure 2 FIG. 4 is a schematic diagram of the internal structure of an image quality enhancement system for autonomous driving according to a second embodiment of the present invention.

[0020] Figure 3 Schematic diagram of the internal structure of an image quality enhancement system for autonomous driving according to the third embodiment of the present invention. DETAILED DESCRIPTION

[0021] An embodiment of the image quality enhancement system for autonomous driving of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Figure 1 This is a schematic diagram of the internal structure of an image quality enhancement system for autonomous driving according to a first embodiment of the present invention, wherein the system includes:

[0023] A mode detection device is provided in the smart car, and is used to send a first detection signal when detecting that the smart car is currently in the automatic driving mode, and is also used to send a second detection signal when detecting that the smart car is currently in the non-automatic driving mode;

[0024] Specifically, a SOC device may be selected to implement the mode detection device, which is arranged in the smart car and is used to send a first detection signal when detecting that the smart car is currently in the automatic driving mode, and is also used to send a second detection signal when detecting that the smart car is currently in the non-automatic driving mode;

[0025] A road surface acquisition device, arranged at the front end of the smart car and connected to the mode detection device, is used to start an image signal acquisition operation of the road surface environment at the front end of the smart car when receiving the first detection signal, so as to obtain and output a corresponding road surface environment image;

[0026] An order conversion device is connected to the road surface acquisition device and is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image, a distortion correction process, and a sharpening process based on a USM filter on the received road surface environment image order, so as to obtain and output a corresponding order conversion image;

[0027] A data capture device, arranged in the control console of the smart car and connected to the order conversion device, for receiving the order conversion image and acquiring various screen content information of the order conversion image, wherein the various screen content information of the order conversion image includes the resolution of the order conversion image, the number of foreground pixels, and various brightness values ​​corresponding to each pixel;

[0028] A configuration extraction device connected to the order conversion device, used to obtain various configuration parameters of the order conversion image, wherein the various configuration parameters of the order conversion image include a signal-to-noise ratio and a contrast ratio of the order conversion image;

[0029] A network application device, connected to the data capture device and the configuration extraction device respectively, for intelligently analyzing the sharpness level of the processed image obtained after the order conversion image is enhanced by the enhancement algorithm according to the code data corresponding to the enhancement algorithm used, the various screen content information of the order conversion image and the various configuration parameters of the order conversion image by using a feedforward neural network model, wherein the feedforward neural network model is a feedforward neural network after multiple learning;

[0030] The network application device is further used to perform intelligent analysis on different types of enhancement algorithms using a feedforward neural network model to obtain different sharpening levels, and use the enhancement algorithm corresponding to the sharpening level with the highest value as the optimal enhancement algorithm matching the order conversion image to perform enhancement processing on the order conversion image to obtain a final optimized image;

[0031] The road surface acquisition device is further used to stop the image signal acquisition operation of the road surface environment at the front end of the smart car when receiving the second detection signal;

[0032] Among them, the received road environment image is sequentially processed to retain the junction of two parts with obvious color or brightness contrast in the image, distortion correction processing and sharpening processing based on USM filter to obtain and output the corresponding sequential conversion image, including: the sequential conversion device includes a first conversion unit, which is used to perform the processing of retaining the junction of two parts with obvious color or brightness contrast in the image on the received image signal.

[0033] Figure 2 FIG. 4 is a schematic diagram of the internal structure of an image quality enhancement system for autonomous driving according to a second embodiment of the present invention.

[0034] and Figure 1 different, Figure 2 The image quality enhancement system for autonomous driving may also include the following components:

[0035] a signal sharpening mechanism, connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively, for performing signal sharpening on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively, so as to obtain signal sharpened images corresponding to the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device;

[0036] The signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device to obtain the signal-sharpened images corresponding to the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively, including: the first level signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively in the horizontal direction;

[0037] The signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device to obtain the signal-sharpened images corresponding to the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively. The second level signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively, in the vertical direction.

[0038] The signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device to obtain the signal-sharpened images corresponding to the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively, and further includes: the third level signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device by using the spatial differential method;

[0039] And wherein, signal sharpening is performed on the respective output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device to obtain respective signal-sharpened images corresponding to the respective output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, and further includes: the fourth level signal sharpening performs edge sharpening on the respective output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device.

[0040] Figure 3 Schematic diagram of the internal structure of an image quality enhancement system for autonomous driving according to the third embodiment of the present invention.

[0041] and Figure 1 different, Figure 3 The image quality enhancement system for autonomous driving may also include the following components:

[0042] A timing service device, connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively, for respectively providing the road surface acquisition device, the sequence conversion device, the data capture device and the network application device with the timing services required by each of them;

[0043] Wherein, the timing service device is respectively connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, and is used to respectively provide the road surface acquisition device, the sequence conversion device, the data capture device and the network application device with the timing services required by each device, including: the timing service device has a built-in quartz oscillation unit, and is respectively connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, and is used to respectively provide the road surface acquisition device, the sequence conversion device, the data capture device and the network application device with the timing services required by each device;

[0044] And wherein, the timing service device is respectively connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, and is used to respectively provide the road surface acquisition device, the sequence conversion device, the data capture device and the network application device with the respective required timing services, including: using a SOC device to implement the timing service device.

[0045] In addition, in the image quality enhancement system for autonomous driving, the received road environment image is sequentially subjected to a process of retaining the junction of two parts with obvious color or light-dark contrast in the image, a distortion correction process, and a sharpening process based on a USM filter to obtain and output a corresponding sequential conversion image, and further includes: the sequential conversion device includes a second conversion unit for performing a distortion correction process on the received image signal;

[0046] Among them, the received road environment image is sequentially processed to retain the junction of two parts with obvious color or light and dark contrast in the image, distortion correction processing and sharpening processing based on USM filter to obtain and output the corresponding sequential conversion image, including: the sequential conversion device also includes a third conversion unit, which is used to perform sharpening processing based on USM filter on the received image signal.

[0047] The image quality enhancement system for autonomous driving of the present invention is used to address the technical problem in the prior art that it is difficult to obtain the best enhancement algorithm that best matches the optimized image content of each frame of the front road image used for autonomous driving. Different types of enhancement algorithms are respectively subjected to intelligent analysis using a feedforward neural network model to obtain different sharpening levels, and the enhancement algorithm corresponding to the highest numerical sharpening level is used as the optimal enhancement algorithm to match the order conversion image obtained after the optimization processing of the autonomous driving scene image, so as to perform enhancement processing on the order conversion image, so as to obtain the final optimized image used as auxiliary data for the autonomous driving mode of the smart car, thereby improving the image quality of each frame of monitoring screen obtained by the vehicle in the autonomous driving state.

[0048] Those skilled in the art will readily appreciate the additional advantages and changes. Therefore, the present invention in its broader aspects is not limited to the specific details and representative embodiments shown and described herein. Therefore, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

Claims

1. A picture quality enhancement system for autonomous driving, characterized in that: The system comprises: A mode detection device is provided in the smart car, and is used to send a first detection signal when detecting that the smart car is currently in the automatic driving mode, and is also used to send a second detection signal when detecting that the smart car is currently in the non-automatic driving mode; A road surface acquisition device, arranged at the front end of the smart car and connected to the mode detection device, is used to start an image signal acquisition operation of the road surface environment at the front end of the smart car when receiving the first detection signal, so as to obtain and output a corresponding road surface environment image; An order conversion device is connected to the road surface acquisition device and is used to perform a process of retaining the junction of two parts with obvious color or light-dark contrast in the image, a distortion correction process, and a sharpening process based on a USM filter on the received road surface environment image order, so as to obtain and output a corresponding order conversion image; A data capture device, arranged in the control console of the smart car and connected to the order conversion device, for receiving the order conversion image and acquiring various screen content information of the order conversion image, wherein the various screen content information of the order conversion image includes the resolution of the order conversion image, the number of foreground pixels, and various brightness values ​​corresponding to each pixel; A configuration extraction device connected to the order conversion device, used to obtain various configuration parameters of the order conversion image, wherein the various configuration parameters of the order conversion image include a signal-to-noise ratio and a contrast ratio of the order conversion image; A network application device, connected to the data capture device and the configuration extraction device respectively, for intelligently analyzing the sharpness level of the processed image obtained after the order conversion image is enhanced by the enhancement algorithm according to the code data corresponding to the enhancement algorithm used, the various screen content information of the order conversion image and the various configuration parameters of the order conversion image by using a feedforward neural network model, wherein the feedforward neural network model is a feedforward neural network after multiple learning; Among them, the network application device is also used to perform intelligent analysis of different types of enhancement algorithms using a feedforward neural network model to obtain different sharpening levels, and use the enhancement algorithm corresponding to the sharpening level with the highest value as the optimal enhancement algorithm matching the order-converted image to perform enhancement processing on the order-converted image to obtain a final optimized image.

2. The image quality enhancement system for autonomous driving according to claim 1, characterized in that: The road surface acquisition device is also used to stop the image signal acquisition operation of the road surface environment at the front end of the smart car when receiving the second detection signal; Among them, the received road environment image is sequentially processed to retain the junction of two parts with obvious color or brightness contrast in the image, distortion correction processing and sharpening processing based on USM filter to obtain and output the corresponding sequential conversion image, including: the sequential conversion device includes a first conversion unit, which is used to perform the processing of retaining the junction of two parts with obvious color or brightness contrast in the image on the received image signal.

3. The image quality enhancement system for autonomous driving according to claim 2, characterized in that: The system further comprises: a signal sharpening mechanism, connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively, for performing signal sharpening on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, respectively, so as to obtain signal sharpened images corresponding to the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device; Among them, signal sharpening is performed on the respective output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device to obtain respective signal-sharpened images corresponding to the respective output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, including: the first level signal sharpening performs horizontal sharpening on the respective output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device.

4. The image quality enhancement system for autonomous driving according to claim 3, characterized in that: Signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively to obtain signal-sharpened images corresponding to the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively, and also includes: second-level signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively in the vertical direction.

5. The image quality enhancement system for autonomous driving according to claim 4, characterized in that: Signal sharpening is performed on the output image signals of the road surface acquisition device, the order conversion device, the data capture device and the network application device respectively to obtain signal-sharpened images corresponding to the output image signals of the road surface acquisition device, the order conversion device, the data capture device and the network application device respectively, and also includes: third-level signal sharpening is performed on the output image signals of the road surface acquisition device, the order conversion device, the data capture device and the network application device respectively by using spatial differentiation method.

6. The image quality enhancement system for autonomous driving according to claim 5, characterized in that: Signal sharpening is performed on the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively to obtain signal-sharpened images corresponding to the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively, and also includes: fourth-level signal sharpening is performed on the edge of the output image signals of the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively.

7. The image quality enhancement system for autonomous driving according to any one of claims 3 to 6, characterized in that: The system further comprises: The timing service device is connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively, and is used to provide the required timing services to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device respectively.

8. The image quality enhancement system for autonomous driving according to claim 7, characterized in that: The timing service device is respectively connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, and is used to respectively provide the road surface acquisition device, the sequence conversion device, the data capture device and the network application device with the timing services required by each of them. The timing service device includes: the timing service device has a built-in quartz oscillation unit, which is respectively connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, and is used to respectively provide the road surface acquisition device, the sequence conversion device, the data capture device and the network application device with the timing services required by each of them.

9. The image quality enhancement system for autonomous driving according to claim 7, characterized in that: The timing service device is respectively connected to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, and is used to respectively provide the required timing services to the road surface acquisition device, the sequence conversion device, the data capture device and the network application device, including: using a SOC device to implement the timing service device.

Images