Camera image quality diagnosis method and device, electronic equipment and medium

Abstract

The invention provides a camera image quality diagnosis method and device, electronic equipment and a medium. The method comprises the following steps: acquiring a camera image; when an abnormal pixel point exists in the camera image, a diagnosis result is output based on the position of the abnormal pixel point of the camera image, the diagnosis result comprises a result representing that the camera image is a normal image or an abnormal image, and the abnormal image represents that the abnormal pixel point of the camera image is in a designated area; the normal image indicates that abnormal pixel points of the camera image are not in the designated area, and the designated area comprises an area where a moving target is located and / or an area where a lane line is located in the camera image. According to the technical scheme, the utilization rate of the camera image is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a camera image quality diagnosis method, device, electronic equipment and medium. Background Art

[0002] In the field of intelligent driving, autonomous driving systems mainly rely on various sensors, especially image sensors, to perceive the surrounding environment and realize functions such as positioning, planning and control. Therefore, image quality information is crucial to the system.

[0003] However, in practical applications, image sensors need to operate in various weather and lighting conditions. Operating in extreme conditions such as direct sunlight, reflections, heavy rain, and fog may lead to a decrease in the quality of collected images, making it difficult to meet the requirements for the recognition and classification of perceived objects, which seriously threatens driving safety. Summary of the invention

[0004] In view of this, an object of an embodiment of the present application is to provide a camera image quality diagnosis method, which can improve the problem of image quality degradation affecting driving safety.

[0005] In order to achieve the above technical objectives, the technical solutions adopted in this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a camera image quality diagnosis method, the method comprising:

[0007] Get camera image;

[0008] When there are abnormal pixels in the camera image, a diagnosis result is output based on the position of the abnormal pixels in the camera image, and the diagnosis result includes a result characterizing that the camera image is a normal image or an abnormal image. The abnormal image indicates that the abnormal pixels in the camera image are within a specified area, and the normal image indicates that the abnormal pixels in the camera image are not within the specified area. The specified area includes an area where a moving target and / or a lane line in the camera image are located.

[0009] Furthermore, between acquiring the camera image and outputting the diagnosis result based on the position of the abnormal pixel point of the camera image, the method includes:

[0010] The camera image is input into a segmentation model, and the segmentation model is used to determine whether there are abnormal pixels in the camera image, wherein the segmentation model is used to perform pixel segmentation on the camera image to obtain single pixels, and classify the single pixels as normal pixels or abnormal pixels. When there are abnormal pixels in the camera image, the step of outputting a diagnostic result based on the position of the abnormal pixels in the camera image is executed.

[0011] Furthermore, before outputting the diagnosis result based on the position of the abnormal pixel point of the camera image, the method further includes:

[0012] Acquire a training data set, wherein the training data set includes a plurality of training images;

[0013] Annotating pixels in the training image to obtain the training image with an annotation result, wherein the annotation result includes a label indicating that the annotated pixel is an abnormal pixel or a normal pixel, and the abnormal pixel of the training image represents a pixel of the training image having a resolution lower than a first threshold and / or an exposure value higher than a second threshold;

[0014] The segmentation model is trained using the labeled training images.

[0015] Further, after outputting the diagnosis result based on the position of the abnormal pixel point of the camera image, the method further includes:

[0016] The camera images are stored in a storage list, and when the number of abnormal images in the storage list exceeds a first preset number, an alarm is issued.

[0017] Further, after outputting the diagnosis result based on the position of the first abnormal pixel, the method further includes:

[0018] If the number of the camera images stored in the storage list exceeds a second preset number, the camera images that meet specified conditions are deleted, and the specified conditions include that the storage time exceeds a preset time or a maximum storage time, and the second preset number is greater than or equal to the first preset number.

[0019] In a second aspect, an embodiment of the present application provides a camera image quality diagnosis device, comprising:

[0020] An acquisition module configured to acquire a camera image;

[0021] The diagnostic module is configured to output a diagnostic result based on the position of the abnormal pixel point of the camera image when there is an abnormal pixel point in the camera image, and the diagnostic result includes a result characterizing that the camera image is a normal image or an abnormal image, the abnormal image indicates that the abnormal pixel point of the camera image is within a specified area, and the normal image indicates that the abnormal pixel point of the camera image is not within the specified area, and the specified area includes an area where a moving target and / or an area where a lane line is located in the camera image.

[0022] Furthermore, it also includes a visual processing module, which is configured to input the camera image into a segmentation model, and the segmentation model determines whether there are abnormal pixels in the camera image, wherein the segmentation model is used to perform pixel segmentation on the camera image to obtain a single pixel, and classify the single pixel as a normal pixel or an abnormal pixel.

[0023] Furthermore, it also includes a training module, which is configured to obtain a training data set, wherein the training data set includes multiple training images; annotate the pixels in the training image to obtain the training image with annotation results, wherein the annotation results include labels indicating that the annotated pixels are abnormal pixels or normal pixels, and the abnormal pixels of the training image represent pixels of the training image with a resolution lower than a first threshold and / or an exposure value higher than a second threshold; and use the annotated training image to train the segmentation model.

[0024] In a third aspect, an embodiment of the present application proposes an electronic device, which includes a processor and a memory coupled to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device executes the above method.

[0025] In a fourth aspect, an embodiment of the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the above method.

[0026] The invention adopting the above technical solution has the following advantages:

[0027] In the technical solution of the present application, by identifying abnormal pixels and outputting the diagnosis results at the positions of the abnormal pixels, abnormal images can be identified before using the camera images, thereby avoiding potential misleading information caused by analyzing abnormal images, which is beneficial to reducing the probability of the autonomous driving system making an erroneous response. At the same time, when the existence of abnormal pixels in the camera image is identified, the diagnosis results are not output immediately, but the diagnosis results are output in combination with the positions of the abnormal pixels, thereby avoiding the abandonment of problematic camera images that do not affect use, thereby improving the utilization rate of camera images.

[0028] In the technical solution provided in the present application, the designated area in the camera image, such as the area where the moving target is located and / or the area where the lane line is located, when abnormal pixels appear in the designated area, the diagnosis result of the abnormal image is output, which is conducive to improving driving safety.

[0029] In the technical solution proposed in this application, when the number of abnormal images in the storage list exceeds a preset value, an alarm can be issued to remind relevant personnel to deal with it in time. This helps to timely discover and solve problems with vehicle-mounted cameras, such as surface stains, etc., to avoid potential safety hazards.

[0030] In the technical solution of the present application, the entire diagnostic process mainly relies on the automatic recognition and judgment of the segmentation model, which reduces the influence of human intervention and subjective judgment and improves the objectivity and consistency of the diagnostic results. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The present application may be further described by the non-limiting embodiments given in the accompanying drawings. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings may be obtained based on these drawings without creative effort.

[0032] Figure 1 A structural diagram of an electronic device provided in an embodiment of the present application.

[0033] Figure 2 A flowchart of a camera image quality diagnosis method provided in an embodiment of the present application.

[0034] Figure 3 A schematic diagram of camera positions provided in an embodiment of the present application.

[0035] Figure 4 Flowcharts of S110 and S121 provided in the embodiments of the present application.

[0036] Figure 5 Flowcharts of S110, S122 and S123 provided in the embodiments of the present application.

[0037] Figure 6 A flow chart of a camera image quality diagnosis method provided in an embodiment of the present application.

[0038] Figure 7 A block diagram of a camera image quality diagnosis device provided in an embodiment of the present application.

[0039] Icon: 100-electronic device, 101-processing module, 102-storage module; 200-acquisition module; 300-diagnosis module; 400-visual processing module. DETAILED DESCRIPTION

[0040] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that in the drawings or descriptions, similar or identical parts use the same figure numbers, and the implementation methods not shown or described in the drawings are forms known to ordinary technicians in the relevant technical field. In the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0041] Existing Solution 1: "Video Image Quality Diagnosis Method and System for Security Monitoring" uses the BP neural network model to analyze the images taken by the monitoring equipment, and uses the diagnosis module to determine whether the video image is abnormal and derive the cause of the abnormality, which is then sent to the management module for storage and statistics of the cause of the abnormality.

[0042] Existing Solution 2: "Video Image Quality Diagnosis System and Method" integrates the video image quality diagnosis system into the video surveillance system in the form of a dedicated diagnosis network, effectively reducing the bandwidth consumption of the video surveillance system and the connection port consumption of the monitoring equipment. It also proposes a non-real-time image quality diagnosis system, which uses the peak and valley periods of network bandwidth usage, reasonably uses bandwidth, and further optimizes bandwidth.

[0043] For the existing solution 1, although the image abnormality and the cause of the abnormality are diagnosed, the image quality abnormality is often not the whole image abnormality, and the abnormality of some non-important areas will not affect the use of the camera; for the existing solution 2, in order to save bandwidth, a non-real-time method is used, which will only be started during the low-peak period of bandwidth usage, which is obviously not suitable for autonomous driving systems that have high requirements for real-time performance.

[0044] Please refer to Figure 1 , an embodiment of the present application provides an electronic device 100 that may include a processing module 101 and a storage module 102. The processing module 101 stores a computer program, and when the computer program is executed by the processing module 101, the electronic device can perform the corresponding steps in the following camera image quality diagnosis method.

[0045] In this embodiment, the processing module 101 may be an integrated circuit chip having signal processing capabilities. The above-mentioned processing module may be a general-purpose processor. For example, the processor may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and may implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0046] The storage module 102 may be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc. In this embodiment, the storage module is used to store a program, and the processing module executes the program after receiving an execution instruction.

[0047] Understandably, Figure 1 The electronic device structure shown in FIG. 1 is only a schematic diagram of a structure. The electronic device may also include Figure 1 More components are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0048] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding processes of each step in the method described later, and will not be elaborated here.

[0049] Please refer to Figure 2 The present application also provides a camera image quality diagnosis method. The camera image quality diagnosis method may include the following steps:

[0050] Step 110, acquiring a camera image;

[0051] Step 120: When there are abnormal pixels in the camera image, a diagnosis result is output based on the position of the abnormal pixels in the camera image. The diagnosis result includes whether the camera image is a normal image or an abnormal image.

[0052] The following is a detailed description of the various steps of the camera image quality diagnosis method, as follows:

[0053] In step 110, it can be understood that the camera image refers to an image captured by a vehicle-mounted camera. Figure 3As shown, in this embodiment, 6 cameras are evenly arranged around the vehicle to capture a 360° viewing angle video around the vehicle. After receiving the video captured by the camera, the video file is converted into a plurality of frames of images, which are camera images. After receiving the camera image, the diagnosis result is output through step 120.

[0054] In step 120, if there are no abnormal pixels in the camera image, the camera image is directly output as a normal image. When there are abnormal pixels, the camera image is judged to be an abnormal image based on the position of the abnormal pixels in the camera image. This is because there are often camera images with abnormal pixels, and not all positions of the camera image are abnormal. If the position of the abnormal pixel does not affect the vehicle's judgment of the road conditions ahead during automatic driving, there is no need to judge the camera image as an abnormal image. The camera image can be used as a normal image, and the system can also use the visual system to determine driving information such as road conditions based on the camera image. For example, if the camera image is a photo representing the road surface, and the abnormal pixel only appears on the asphalt road surface, and does not block the road sign information (such as a deceleration sign), then the camera image is a normal image.

[0055] The diagnosis result can be input into the corresponding camera image with a label representing a normal image or an abnormal image.

[0056] The abnormal pixel in this embodiment refers to a pixel that shows abnormality in the camera image, and its characteristics or performance are significantly different from those of the surrounding normal pixels. For example, the significant difference is manifested as a sudden change in pixel value, abnormal color or inconsistent brightness.

[0057] Exemplarily, abnormal pixels include the following types:

[0058] 1. Bright spot: The abnormal pixel is overly bright when displayed, contrasting with the surrounding pixels, and is commonly seen as a fault point on a monitor or LED screen.

[0059] 2. Dark spots: Abnormal pixels fail to light up when displayed, in contrast to the surrounding pixels, and are also fault points of the monitor or LED screen.

[0060] 3. Monochrome dots or color dots: Abnormal pixels can only display a single color or multiple colors, which are inconsistent with the display colors of surrounding pixels. This may be caused by a failure of the display panel or a problem with the driving circuit.

[0061] Therefore, based on the above steps 110 and 120, in at least one embodiment, the camera image quality diagnosis method is as follows: Figure 4 As shown, the specific steps include:

[0062] Step 110: Acquire camera image;

[0063] Step 121: When there are abnormal pixels in the camera image, and the abnormal pixels in the camera image are located in an important area, the diagnosis result is an abnormal image, and the important area includes an area where a moving target is located and / or an area where a lane line is located in the camera image.

[0064] In step 121, during autonomous driving, the recognition of moving targets and lane lines is crucial.

[0065] Moving targets generally include pedestrians and moving vehicles. The trajectories of pedestrians and vehicles have a greater impact on the autonomous driving solution. For example, before passing through a sidewalk, it is necessary to predict the trajectories of pedestrians on the sidewalk so that the sidewalk can be passed safely. At the same time, in the autonomous driving solution, the position of the vehicle also plays a decisive role in the autonomous driving solution.

[0066] Lane line recognition also has a significant impact on autonomous driving solutions. For example, when high-precision maps fail, the vehicle's driving path generally follows the lane lines.

[0067] Therefore, if the position of the abnormal pixel point of the camera image is located in the area where the moving target is located or in the area where the lane line is located, it will affect the decision of the autonomous driving solution. Therefore, it is necessary to define such camera images as abnormal images and not consider the camera images when designing autonomous driving decisions.

[0068] Therefore, it can be understood that as long as the abnormal pixel points of the camera image are located in the area that affects the autonomous driving decision, the camera image with the abnormal pixel points of the camera image is an abnormal image.

[0069] In this embodiment, the position of the moving target is determined by the dynamic target module, and the position of the lane line is determined by the static lane line module. Therefore, when executing step 121, it is necessary to call the dynamic target module to determine the position of the dynamic target in the camera image, and call the static lane line module to determine the position of the lane line.

[0070] The dynamic target module determines the position of the dynamic target in the image by using a combination of multiple technologies and algorithms. Examples thereof include the following methods:

[0071] 1. Image Preprocessing

[0072] First, the camera image needs to be preprocessed, including removing noise, enhancing image contrast and clarity, and performing image compression to improve image quality and reduce the amount of computation for subsequent processing.

[0073] 2. Object Detection

[0074] Object detection is a key step in determining the location of dynamic objects in camera images. It uses deep learning algorithms (such as YOLO, SSD, etc.) to identify and locate various objects in images, including motor vehicles, pedestrians, cyclists, etc. These algorithms usually divide the image into multiple grid cells and predict the location and size of possible objects in each cell.

[0075] 3. Target Tracking

[0076] When a target is detected, it is tracked to obtain real-time position and velocity information. Target tracking algorithms can be implemented by extracting target features (such as color, shape, edges, etc.) or building a state model of the target (such as Kalman filter, etc.). These algorithms can continuously update the position and velocity of the target and handle possible occlusion or deformation of the target.

[0077] Sensor Fusion

[0078] In order to improve the accuracy and robustness of target positioning, autonomous driving systems usually fuse data from multiple sensors. For example, the image data captured by the camera can be fused with the point cloud data obtained by the LiDAR. By combining the data of these two sensors, richer scene information can be obtained and the errors that may be caused by a single sensor can be reduced.

[0079] 5. High-precision map matching

[0080] High-precision maps are another important component of autonomous driving. They contain physical and semantic information about the road and can be used to assist in target positioning. By matching real-time road physical information with prefabricated high-precision maps, positioning errors can be further reduced and the vehicle's driving trajectory can be pulled back to the correct road.

[0081] 6. Algorithm Optimization and Post-Processing

[0082] Finally, the target positioning algorithm needs to be optimized and post-processed. This includes using more advanced deep learning models to improve the accuracy of target detection and using more sophisticated matching algorithms to reduce map matching errors. In addition, data fusion algorithms can be used to integrate data from different sensors to produce more accurate positioning results.

[0083] In the static lane line module, determining the lane lines in the camera image relies on the comprehensive application of multiple technologies and algorithms. Exemplarily, the following methods are included:

[0084] 1. Image Preprocessing

[0085] First, the images captured by the camera need to be preprocessed to improve the accuracy and robustness of lane detection. Image preprocessing includes operations such as removing noise and enhancing image contrast and clarity. These operations can be achieved through digital image processing techniques, such as using filters to remove noise and using histogram equalization to enhance contrast.

[0086] 2. Lane Detection Algorithm

[0087] Lane detection algorithms are a key step in determining lane lines in camera images. These algorithms are usually based on computer vision and machine learning techniques and are able to identify and extract lane lines from images. Here are some common lane detection algorithms:

[0088] Feature-based method: This method identifies lane lines by extracting features from the image (such as edges, colors, textures, etc.). Common feature extraction methods include Canny edge detection, Hough transform, etc. Then, by matching these features with predefined lane line models (such as straight lines, curves, etc.), the position and shape of the lane lines can be determined.

[0089] Machine learning-based methods: This method uses machine learning algorithms (such as support vector machines, random forests, etc.) to classify and identify images. First, a large amount of lane line image data needs to be collected, annotated, and preprocessed. Then, this data is used to train a machine learning model so that it can recognize lane lines in the image. In the detection stage, the camera image is input into the trained model to obtain the position and shape of the lane line.

[0090] Deep learning-based methods: In recent years, deep learning has made significant progress in the field of lane detection. Deep learning methods use deep learning models such as convolutional neural networks (CNNs) to extract features and classify images. Similar to machine learning-based methods, deep learning methods also require a large amount of labeled data for training. However, due to their powerful feature extraction capabilities and generalization performance, deep learning methods have shown higher accuracy and robustness in lane detection.

[0091] 3. Lane Tracking and Optimization

[0092] Once the lane line is detected, it needs to be tracked to obtain real-time position and shape information. The lane line tracking algorithm can be implemented by combining historical detection results with current detection results. In addition, state estimation methods such as Kalman filters can be used to smooth the position and shape of the lane line to reduce the impact of noise and errors.

[0093] 4. Sensor Fusion and Multi-Source Information Integration

[0094] In order to improve the accuracy and robustness of lane detection, autonomous driving systems usually fuse data from multiple sensors. For example, the image data captured by the camera can be fused with the point cloud data obtained by the LiDAR. By combining the data of these two sensors, richer scene information can be obtained and the errors that may be caused by a single sensor can be reduced. In addition, the vehicle's dynamic model, GPS positioning information, etc. can be combined to further improve the accuracy and reliability of lane detection.

[0095] 5. Algorithm Optimization and Post-Processing

[0096] Finally, the lane detection algorithm needs to be optimized and post-processed. This includes using more advanced deep learning models to improve detection accuracy and using more sophisticated matching algorithms to reduce errors. In addition, the performance and computational efficiency of deep learning models can be optimized through data enhancement, model pruning and other techniques. In the post-processing stage, morphological operations, non-maximum suppression and other techniques can be used to further refine the position and shape of lane lines.

[0097] In at least one embodiment, Figure 5 As shown, the camera image quality diagnosis method includes the following steps:

[0098] Step 110: Acquire camera image;

[0099] Step 122: inputting the camera image into a segmentation model, and determining by the segmentation model whether there are abnormal pixels in the camera image, wherein the segmentation model is used to perform pixel segmentation on the camera image to obtain single pixels, and classify the single pixels as normal pixels or abnormal pixels;

[0100] Step 123: When there are abnormal pixels in the camera image, and the abnormal pixels in the camera image are located in the important area, the diagnosis result is an abnormal image.

[0101] In step 122, the segmentation model may be trained based on the following method.

[0102] A training data set is obtained, where the training data set includes a plurality of training images. Exemplarily, the obtaining method may be to collect the training images in an image library.

[0103] Label each pixel in the training image, and the labeling result is abnormal pixel or normal pixel. Abnormal pixel represents pixel with reduced visibility, lost visibility and / or overexposure. Visibility can be represented by resolution. For example, a first threshold is set. When the resolution of the pixel is lower than the first threshold, it is marked as reduced visibility or lost visibility. The exposure is represented by the exposure value. When the exposure value is higher than the second threshold, it is marked as overexposure.

[0104] The segmentation model is trained using the labeled training images.

[0105] The segmentation model in this embodiment uses a CNN (convolutional neural network) model. More specifically, training the segmentation model requires preparing a number of data collected by six cameras, defining annotation rules, and performing manual annotation to train the model. The annotation rules require that each pixel be classified, and there are seven categories in total: normal, slightly blurred, heavily blurred, slightly blocked, heavily blocked, slightly glared, and heavily glared. The seven categories can be represented by setting the resolution and exposure value. For example, when the resolution is m and the exposure value is n, it is marked as slightly blurred.

[0106] Among them, the fuzzy annotation scenarios are as follows: due to the fogging of the lens caused by rain, snow, fog and other weather conditions, or the adhesion or dirtiness of transparent objects on the lens, the local / global visibility of the image semantic area is reduced; the occlusion annotation scenario: refers to the adhesion, dirtiness or other unknown factors of opaque foreign objects (such as mud) on the lens, which causes the local / global visibility loss of the image semantic area; the glare annotation scene: due to the overexposure of local pixel areas caused by direct light / reflection, the local visibility of the image semantic area is lost, and no useful information can be seen. After annotating a certain amount of data according to the above annotation rules, use CNN (convolutional neural network) to train the data.

[0107] In this embodiment, the camera images of six cameras can be looped through and respectively input into the segmentation model for inference. The model outputs a camera image of a certain size, and each pixel on the camera image represents a category.

[0108] In this embodiment, the result of the segmentation model classifies the category of the pixel points of each training image of the training image. When using the segmentation model, if the pixel points of all camera images are normal, the target is directly output normally. If there is an abnormality, the following operations are performed: at the same time, the input of other modules is received, such as the dynamic target module (output vehicles and pedestrians), the static lane line module, etc., and the input target is traversed in a loop (here to the moving target and the lane line), and it is determined whether the position of the target on the image overlaps with the first abnormal pixel. If not, the target is output normally. If so, it is determined to be an abnormal image.

[0109] In at least one embodiment, Figure 5 As shown, the camera image quality diagnosis method includes the following steps:

[0110] Step 110: Acquire camera image;

[0111] Step 124: When there are abnormal pixels in the camera image and the abnormal pixels are located in the important area, the diagnosis result is an abnormal image;

[0112] Step 125: The camera images are stored in a storage list. When the number of abnormal images in the storage list exceeds a first preset number, an alarm is issued. The storage list is used to store the camera images in chronological order.

[0113] That is, after the camera captures the video, the video is converted into a number of camera images in frame order. After each camera image passes through step 124, it is stored in the storage list. Therefore, it can be seen that the camera images are stored in chronological order. The storage list has a certain length. In the storage list, if the number of abnormal images exceeds the first preset value, it means that the mirror of the camera may be contaminated, and an alarm is issued. The alarm is not limited to text, sound, etc. When the camera images in the storage list exceed the second preset value, the oldest camera image is deleted and the latest camera image is stored. The camera image with the longest storage time can also be deleted. The second preset value can be the upper limit of the images that can be stored in the storage list. The first preset value and the second preset value can also be flexibly set according to the bench test results, which is not limited in this embodiment.

[0114] In this embodiment, a voting device is set up, and a list of a certain length is maintained in the voting device list (equivalent to a storage list). The list stores the diagnostic results of each frame. For each new frame result, it is first determined whether the list exceeds a certain fixed length. If so, the oldest result in the list is deleted and the latest result is saved to ensure a relatively real-time effect. The number and area of ​​abnormalities in the fixed-length results are counted. If the abnormality of a certain area exceeds the threshold set in advance, the vote is passed and an abnormal alarm is directly issued. If the vote is not passed, the output is normal.

[0115] In at least one embodiment, Figure 6 As shown, the camera image quality diagnosis method includes the following steps:

[0116] S1: Get camera image from camera list;

[0117] S2: Input the camera image into the segmentation model and classify each pixel in the camera image. If all pixels are normal, the whole image is normal and is output normally.

[0118] S3: If there are abnormal pixels, determine whether the position of the abnormal pixels affects the target, which refers to whether the position of the abnormal pixels is in the same position as the moving target and the lane line;

[0119] S4: If yes, the abnormal image is input into the voting device, and an abnormal alarm is generated if the vote is passed, and a normal output is generated if the vote is not passed;

[0120] S5: If S3 is negative, the output is normal.

[0121] In this embodiment, although the camera image is diagnosed as an abnormal image in S3, since the number of abnormal images in the storage list does not exceed the preset threshold, the abnormal image can be output normally. At the same time, the abnormal image can also be deleted and not used in the autonomous driving strategy.

[0122] Please refer to Figure 7 The present application also provides a camera image quality diagnosis device, wherein the direction of the arrow indicates the direction of the signal flow, and the camera image quality diagnosis device includes at least one software function module that can be stored in a storage module in the form of software or firmware or fixed in an operating system (OS). The processing module is used to execute the executable module stored in the storage module, such as the software function module and computer program included in the camera image quality diagnosis device.

[0123] The camera image quality diagnosis device includes an acquisition module 200 and a diagnosis module 300, and the functions of each unit may be as follows:

[0124] The acquisition module 200 is configured to acquire a camera image, and the diagnosis module 300 is configured to output a diagnosis result based on the position of the abnormal pixel point of the camera image when there are abnormal pixels in the camera image, and the diagnosis result includes whether the camera image is a normal image or an abnormal image. The abnormal image indicates that the abnormal pixel point of the camera image is within a specified area, and the normal image indicates that the abnormal pixel point of the camera image is not within the specified area. The specified area includes the area where the moving target and / or the area where the lane line is located in the camera image.

[0125] When the abnormal pixel point of the camera image is located in the important area, the diagnosis module 300 outputs the diagnosis result that the camera image is an abnormal image, and the important area includes the area where the moving target is located and / or the area where the lane line is located in the camera image.

[0126] This embodiment also includes a visual processing module 400, which is configured to input the camera image into a segmentation model, and the segmentation model determines whether there are abnormal pixels in the camera image, wherein the segmentation model is used to perform pixel segmentation on the camera image to obtain single pixels, and classify the single pixels as normal pixels or abnormal pixels.

[0127] This embodiment also includes a training module 500, which is configured to obtain a training data set, wherein the training data set includes multiple training images; annotate the pixels in the training image to obtain the training image with an annotation result, wherein the annotation result includes a label indicating that the annotated pixel is an abnormal pixel or a normal pixel, and the abnormal pixel of the training image represents a pixel of the training image with a resolution lower than a first threshold and / or an exposure value higher than a second threshold; and train the segmentation model using the annotated training image.

[0128] The embodiment of the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer executes the camera image quality diagnosis method described in the above embodiment.

[0129] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by hardware, and can also be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0130] In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of the module, program segment or code contains one or more executable instructions for implementing the specified logical function. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0131] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A camera image quality diagnosis method, characterized in that: The method comprises: Get camera image; When there are abnormal pixels in the camera image, a diagnosis result is output based on the position of the abnormal pixels in the camera image, and the diagnosis result includes a result characterizing that the camera image is a normal image or an abnormal image. The abnormal image indicates that the abnormal pixels in the camera image are within a specified area, and the normal image indicates that the abnormal pixels in the camera image are not within the specified area. The specified area includes an area where a moving target and / or a lane line in the camera image are located.

2. The method according to claim 1, characterized in that Between acquiring the camera image and outputting the diagnosis result based on the position of the abnormal pixel point of the camera image, the method includes: The camera image is input into a segmentation model, and the segmentation model is used to determine whether there are abnormal pixels in the camera image, wherein the segmentation model is used to perform pixel segmentation on the camera image to obtain single pixels, and classify the single pixels as normal pixels or abnormal pixels. When there are abnormal pixels in the camera image, the step of outputting a diagnostic result based on the position of the abnormal pixels in the camera image is executed.

3. The method according to claim 2, characterized in that Before outputting a diagnosis result based on the position of the abnormal pixel point of the camera image, the method further includes: Acquire a training data set, wherein the training data set includes a plurality of training images; Annotating pixels in the training image to obtain the training image with an annotation result, wherein the annotation result includes a label indicating that the annotated pixel is an abnormal pixel or a normal pixel, and the abnormal pixel of the training image represents a pixel of the training image having a resolution lower than a first threshold and / or an exposure value higher than a second threshold; The segmentation model is trained using the labeled training images.

4. The method according to claim 1, characterized in that: After outputting the diagnosis result based on the position of the abnormal pixel point of the camera image, the method further includes: The camera images are stored in a storage list, and when the number of abnormal images in the storage list exceeds a first preset number, an alarm is issued.

5. The method according to claim 4, characterized in that After outputting a diagnosis result based on the position of the first abnormal pixel, the method further includes: If the number of the camera images stored in the storage list exceeds a second preset number, the camera images that meet specified conditions are deleted, and the specified conditions include that the storage time exceeds a preset time or a maximum storage time, and the second preset number is greater than or equal to the first preset number.

6. A camera image quality diagnostic device, characterized in that: include: An acquisition module configured to acquire a camera image; The diagnostic module is configured to output a diagnostic result based on the position of the abnormal pixel point of the camera image when there is an abnormal pixel point in the camera image, and the diagnostic result includes a result characterizing that the camera image is a normal image or an abnormal image, the abnormal image indicates that the abnormal pixel point of the camera image is within a specified area, and the normal image indicates that the abnormal pixel point of the camera image is not within the specified area, and the specified area includes an area where a moving target and / or an area where a lane line is located in the camera image.

7. The device according to claim 6, characterized in that: It also includes a visual processing module, which is configured to input the camera image into a segmentation model, and determine by the segmentation model whether there are abnormal pixels in the camera image, wherein the segmentation model is used to perform pixel segmentation on the camera image to obtain a single pixel, and classify the single pixel as a normal pixel or an abnormal pixel.

8. The device according to claim 7, characterized in that: It also includes a training module, which is configured to obtain a training data set, wherein the training data set includes multiple training images; annotate the pixels in the training image to obtain the training image with annotation results, wherein the annotation results include labels indicating that the annotated pixels are abnormal pixels or normal pixels, and the abnormal pixels of the training image represent pixels of the training image with a resolution lower than a first threshold and / or an exposure value higher than a second threshold; and train the segmentation model using the annotated training image.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory coupled to each other, wherein the memory stores a computer program. When the computer program is executed by the processor, the electronic device executes the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 5.

Images