Image quality detection method and device, equipment and storage medium

Abstract

The invention provides an image quality detection method and device, equipment and a storage medium. The method comprises the steps that a semantic segmentation model is used for segmenting a to-be-detected image, the pixel-level category weight of each pixel in the to-be-detected image is determined, and the semantic segmentation model comprises a depth separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network which are cascaded; the image to be detected is divided into a plurality of blocks, the block-level weight of each block is determined according to the occurrence frequency of target semantic information in each block, and each block comprises a plurality of pixels; determining an image quality condition score of each block according to the pixel level category weight of each pixel and the block level weight of each block; and determining a quality diagnosis result of the to-be-detected image according to the image quality condition scores of all the blocks included in the key domain in the to-be-detected image.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an image quality detection method and apparatus, equipment and storage medium. Background Art

[0002] Vehicles are usually equipped with multiple cameras to capture environmental image information around the vehicle so that the vehicle can perform intelligent assisted driving based on the images captured by the cameras. Since vehicles are often exposed to complex and diverse working conditions, the camera's field of view is frequently blocked by objects such as dirt, water mist, leaves, and paper scraps, making the information in the field of view blurry or even invisible. Cameras at various viewing angles find it difficult to collect clear and reliable driving data, resulting in many assisted driving functions being unable to be activated due to the lack of real-time data they rely on. Therefore, accurately judging the quality of the image captured by the current camera is an important part of the implementation of intelligent driving, and safe and reliable intelligent driving.

[0003] In related technologies, one way to determine image quality is to use a statistical model to calculate the pixel distribution of the image, calculate statistical indicators such as variance, contrast, and saturation, and determine the threshold for abnormal diagnosis based on empirical values. When the statistical indicator does not meet the preset threshold, the image quality is determined to be low. This method is simple and convenient, but its application scenarios are limited, the threshold setting is highly subjective, and it is difficult to adapt to complex and changing driving environments.

[0004] Alternatively, it is also possible to use deep learning methods, relying on the multi-layer convolution filters of the convolutional neural network (CNN), and annotated true value data to drive the image quality recognition model to learn deeper image quality characterization features, and output reliable image quality classification results under the supervision of a large amount of data. However, deep learning is still unable to deal with abnormal conditions in local areas. During driving, many abnormal conditions are often limited to a certain local area of ​​the camera. If only the global image classification method is relied upon for judgment, it will not be able to accurately reflect the local abnormal conditions. At the same time, in the process of model training, using the area ratio as a threshold will make the full-image annotation judgment rule more subjective, and the model training will not be able to accurately distinguish local scenes, thereby increasing the probability of misjudgment.

[0005] Therefore, how to effectively locate and identify these local abnormal areas has become a key issue that needs to be urgently solved in the current field of image quality diagnosis. Summary of the invention

[0006] The embodiments of the present application provide an image quality detection method and apparatus, a device and a storage medium. The following introduces various aspects involved in the embodiments of the present application.

[0007] In a first aspect, an image quality detection method is provided, the method comprising: using a semantic segmentation model to segment an image to be detected, determining a pixel-level category weight of each pixel in the image to be detected, the pixel-level category weight being used to characterize the probability that the type of each pixel is a target feature type, the semantic segmentation model comprising a cascaded depth-separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network; dividing the image to be detected into a plurality of blocks, determining a block-level weight of each block according to the frequency of occurrence of target semantic information in each block, each block comprising a plurality of pixels; determining an image quality status score of each block according to the pixel-level category weight of each pixel and the block-level weight of each block; and determining a quality diagnosis result of the image to be detected according to the image quality status scores of all blocks contained in a key domain in the image to be detected.

[0008] According to the above technical means, with the help of a semantic segmentation model including a deep separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network, pixel-level and block-level weights are integrated. First, the weight is determined for each pixel classification, and then the block weight is determined in combination with the block target semantic information frequency. The two together determine the block image quality score, and finally the quality diagnosis result is obtained based on the key domain block score. This allows the detection process to fully consider the image details and semantic information, thereby comprehensively and accurately evaluating the image quality.

[0009] In some embodiments, the use of a semantic segmentation model to segment the image to be detected includes: extracting features of the image to be detected by the deep separable variable group convolutional neural network to obtain multiple first feature maps of different scales; using the bidirectional feature pyramid network to perform feature fusion on the multiple first feature maps to obtain a second feature map; and using the segmentation head network to perform segmentation based on the second feature map to determine the pixel-level category weight of each pixel in the image to be detected.

[0010] According to the above technical means, the cascaded deep separable variable group convolutional neural network, bidirectional feature pyramid network and segmentation head network are used to perform semantic segmentation on the image to be detected. The feature information at different levels can be fully utilized, so that the model can consider global and local features at the same time, realize pixel-level semantic segmentation, and improve the recognition ability of defects such as blur, occlusion and glare.

[0011] In some embodiments, the use of the bidirectional feature pyramid network to perform feature fusion on the multiple first feature maps to obtain a second feature map includes: inputting the multiple first feature maps into the bidirectional feature pyramid network to obtain multiple third feature maps; performing coordinate convolution on the multiple third feature maps to obtain multiple fourth feature maps; and performing feature concatenation on the multiple fourth feature maps to obtain the second feature map.

[0012] Based on the above technical means, through the feature fusion strategy of the bidirectional feature pyramid network, combined with coordinate convolution and feature concatenation technology, the feature map information from different dimensions of the backbone network is effectively integrated and utilized, thereby improving the model's understanding and processing capabilities of complex scenes.

[0013] In some embodiments, determining the pixel-level category weight of each pixel in the image to be detected includes: using the segmentation head network to determine the pixel-level classification result of the image to be detected, the pixel-level classification result includes the probability that each pixel is a feature of a different category; performing an argmax operation on the pixel-level classification result to assign a feature category label with the highest probability to each pixel; and determining the probability of the feature category label of each pixel as the pixel-level category weight.

[0014] According to the above technical means, the segmentation head network is used to determine the pixel-level classification results of the image to be detected, and the probability of each pixel being a feature of a different category is obtained. By performing an argmax operation on the pixel-level classification results, each pixel is assigned a feature category label with the highest probability, and the probability of the label is determined as the pixel-level category weight, which plays a key role in the subsequent combination of block-level weights to evaluate image quality.

[0015] In some embodiments, the image quality status score of each of the blocks is the product of the block-level weight of each of the blocks and the weighted average of the pixel-level category weights of all pixels of the image to be detected; the image quality status score is used to characterize the probability of the target feature existing in each of the blocks.

[0016] According to the above technical means, the image quality status score of each block is defined as the product of the block-level weight and the weighted average of the pixel-level category weights of all pixels in the image to be detected. This calculation method integrates the pixel-level and block-level information in the score calculation, fully considering the impact of both on image quality, avoiding errors caused by a single factor, thereby improving the accuracy of the quality status assessment of each block and providing reliable intermediate data for the final accurate judgment of the quality of the image to be detected.

[0017] In some embodiments, determining the quality diagnosis result of the image to be detected based on the image quality status scores of all blocks included in the key domain of the image to be detected includes: determining the number of target blocks whose image quality status scores among all blocks included in the key domain are greater than a first score threshold; when the number of target blocks is greater than the first number threshold, determining that the image to be detected is an abnormal image; when the number of target blocks is less than or equal to the first number threshold, determining that the image to be detected is a normal image.

[0018] According to the above technical means, by determining the number of target blocks whose image quality scores in all blocks included in the key domain are greater than the first score threshold, it is determined whether the image to be detected is an abnormal image or a normal image. This judgment method based on the score of a specific block in the key domain focuses on the more important key domain part of the image, making the image quality diagnosis more objective and accurate, effectively distinguishing normal and abnormal images, and improving the credibility of the detection results.

[0019] In some embodiments, the target semantic information includes at least one of the following features in the road environment around the vehicle: traffic lights, signs, lane lines, and dynamic targets; the block-level weight of each of the blocks is positively correlated with the frequency of occurrence of the target semantic information.

[0020] According to the above technical means, by dividing the image to be detected into multiple blocks, each block can be analyzed and processed independently. This regionalized processing method not only improves the efficiency of image processing, but also allows for more refined processing based on the characteristics of each block. The block-level weight is determined based on the frequency of occurrence of the target semantic information in each block, so that more attention is paid to important areas in the image. This weight distribution method helps to more accurately identify and utilize key information in the image in subsequent processing.

[0021] In a second aspect, an image quality detection device is provided, comprising: a segmentation unit, used to segment the image to be detected by using a semantic segmentation model, and determine the pixel-level category weight of each pixel in the image to be detected, wherein the semantic segmentation model comprises a cascaded depth-separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network; a first determination unit, used to divide the image to be detected into a plurality of blocks, and determine the block-level weight of each block according to the occurrence frequency of target semantic information in each block, wherein each block comprises a plurality of pixels; a second determination unit, used to determine the image quality status score of each block according to the pixel-level category weight of each pixel and the block-level weight of each block; and a third determination unit, used to determine the quality diagnosis result of the image to be detected according to the image quality status scores of all blocks contained in the key domain of the image to be detected.

[0022] According to a third aspect, an image quality detection device is provided, comprising a processor memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method described in the first aspect.

[0023] According to a fourth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program implements the method described in the first aspect when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic flow chart of an image quality detection method provided in an embodiment of the present application;

[0025] Figure 2 A flowchart of a method for segmenting an image to be detected using a semantic segmentation model;

[0026] Figure 3 This is an example diagram of a semantic segmentation model;

[0027] Figure 4 A schematic flow chart of an image quality detection method provided in another embodiment of the present application;

[0028] Figure 5 A flow chart of the image quality detection method provided in the embodiment of the present application;

[0029] Figure 6 A schematic structural diagram of an image detection device provided in an embodiment of the present application;

[0030] Figure 7 A schematic structural diagram of an image detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0032] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application. Before introducing the technical solution of the embodiment of this application, the image quality detection method of the related art and the problems existing therein are first described in detail with examples.

[0034] A vehicle is usually equipped with multiple cameras to capture the image information of the environment around the vehicle, so that the vehicle can perform intelligent assisted driving based on the images collected by the cameras. The multiple cameras in the vehicle can usually include front-view, left-view, right-view and rear-view cameras, among which the front-view camera is responsible for the field of view information in front of the vehicle, the left-view and right-view cameras are responsible for the field of view information on the side of the vehicle, and the rear-view camera is responsible for capturing information on the rear side of the vehicle; through the above multiple cameras, global information around the vehicle can be obtained.

[0035] With the improvement of vehicle computing performance, vehicles have the ability to support the deployment of more and more complex deep learning algorithm models, thus enabling the gradual implementation of higher-level intelligent driving solutions. Taking the BEV (Bird's Eye View) series of algorithms as an example, it uses the image captured by the vehicle camera as input, converts the camera coordinate system to the BEV perspective, extracts features, and identifies the driving semantic information around the vehicle, such as pedestrians, vehicles, signs and markings, etc., and then inputs the above information into the back-end model to complete the planning of the vehicle's driving trajectory and realize high-level intelligent assisted driving.

[0036] However, the complexity of the real world poses considerable challenges to the above process. As vehicles are often exposed to complex and diverse working conditions, the camera's field of view is frequently blocked by dirt, water mist, leaves, paper scraps and other objects, making the information in the field of view blurry or even invisible. It is difficult for cameras at various viewing angles to collect clear and reliable driving data, resulting in many assisted driving functions being unable to be activated due to the lack of real-time data they rely on. At the same time, glare and backlighting caused by direct sunlight or lighting will seriously affect the camera's imaging quality, resulting in the loss of semantic information in the collected image information, further hindering the normal functioning of intelligent driving functions.

[0037] Therefore, accurately judging the quality of the image captured by the current camera is an important part of implementing smart driving, especially safe and reliable smart driving.

[0038] In order to solve the above problems, computer vision algorithms are usually used in related technologies. One way to determine the image quality is to use a statistical model to count the pixel distribution of the image, calculate statistical indicators such as variance, contrast and saturation, and determine the threshold for abnormal diagnosis based on empirical values. When the statistical indicators do not meet the preset threshold, the image quality is determined to be low. Specifically, in some related technologies, the image to be detected can be converted into a grayscale image, combined with the number of interval pixels in the grayscale image, and the brightness coefficient of the grayscale image is calculated, and the number of interval pixels in the grayscale image is compared with the set threshold to complete the diagnosis of image quality. This method is simple and convenient, but its application scenarios are limited, the threshold setting is highly subjective, and it is difficult to adapt to complex and changeable driving environments.

[0039] In some related technologies, image quality can also be determined by methods other than the image domain. For example, the image domain is converted to the frequency domain through Fourier transform for feature analysis, and the frequency domain feature descriptor is constructed by spectrum contrast, spectrum texture and direction features. The image is classified by combining machine learning methods such as principal component analysis (PCA) and support vector machine (SVM), thereby identifying outliers.

[0040] In addition, based on deep learning methods, we can rely on the multi-layer convolution filters of the convolutional neural network CNN and annotated true value data to drive the image quality recognition model to learn deeper image quality representation features, and output reliable image quality classification results under the supervision of a large amount of data, which greatly reduces the difficulty of artificially constructing features and has better generalization than traditional methods.

[0041] Although deep learning models have achieved remarkable results in global image quality classification, they are still unable to handle abnormal situations in local areas. During driving, many abnormal situations are often limited to a certain local area of ​​the camera. If we only rely on the global image classification method for judgment, it will not be able to accurately reflect the local abnormal situation. At the same time, in the process of model training, using the area ratio as a threshold will make the global image annotation judgment rule more subjective, and the model training will not be able to accurately distinguish local scenes, thereby increasing the probability of misjudgment.

[0042] Therefore, how to effectively locate and identify these local abnormal areas has become a key issue that needs to be solved in the current field of image quality diagnosis. Effectively locating specific abnormal areas can more accurately determine whether the current abnormal state affects the intelligent assisted driving function. For example, abnormal conditions such as blur, occlusion and glare in non-critical areas such as edges or corners can be ignored, and the confidence of abnormal areas that affect important semantic elements such as ground lane lines, pedestrians, vehicles, and signs and markings can be increased, so as to achieve image quality diagnosis with smaller granularity.

[0043] In view of the above technical problems, the embodiments of the present application provide an image quality detection method and apparatus, a device and a storage medium. The technical solution of the application will be described in detail below in conjunction with the accompanying drawings.

[0044] Figure 1 is a schematic flow chart of an image quality detection method provided in an embodiment of the present application, Figure 1 The method includes steps S110-S140.

[0045] like Figure 1 As shown, in step S110, the image to be detected is segmented using a semantic segmentation model to determine the category weight of each pixel in the image to be detected. The semantic segmentation model includes a cascaded depth separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network.

[0046] The image to be detected in the embodiment of the present application includes environmental information around the vehicle. The image to be detected may be an original image of the environment in which the vehicle is currently located, captured by cameras around the vehicle, which are deployed on the vehicle; or, the image may be an image processed from the original image. For example, the image to be detected may be an image obtained by cropping the original image. For another example, the image to be detected may also be one of multiple color channels of the original image captured by the camera. For another example, the image to be detected may also be an image obtained by grayscale processing of the original image.

[0047] It can be understood that the camera that collects the above-mentioned original image can be any one of the multiple cameras deployed on the vehicle, and the embodiment of the present application does not specifically limit this.

[0048] The image to be detected is input into the above-mentioned semantic segmentation model, and the key features therein are extracted to obtain the classification result of each pixel in the image; the classification result can be characterized by the aforementioned category weight, which represents the probability that the type of each pixel point is the target feature type; the target feature type mentioned here can be, for example, any one or more of the blur, occlusion and glare mentioned above.

[0049] The semantic segmentation model in the embodiment of the present application uses a modular design, including a cascaded depth-separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network. This approach has strong versatility and flexibility, and the network structure of each part can be set according to needs.

[0050] The deep separable variable group convolutional neural network constitutes the backbone network of the semantic segmentation model, which is used to extract features of different scales in the image to be detected; the bidirectional feature pyramid network constitutes the neck network, which is located between the backbone network and the head, and is responsible for further processing and integration of the features extracted by the backbone network. The role of the neck is similar to a bridge connecting the backbone network and the head; in the technical solution of the embodiment of the present application, the bidirectional feature pyramid network is used to fuse the features of different scales extracted by the backbone network; the segmentation head network constitutes the head network, which is the top part of the entire network structure and is responsible for performing specific tasks. In the technical solution of the embodiment of the present application, the segmentation head network processes the feature map output by the neck network to obtain the semantic segmentation result of the image to be detected.

[0051] In step S120 , the image to be detected is divided into a plurality of blocks, and the block-level weight of each block is determined according to the frequency of occurrence of target semantic information in each block.

[0052] In a specific implementation, for example, the image to be detected can be divided into multiple grid areas to determine whether each grid area contains important information about the vehicle's driving (i.e., the aforementioned target semantic information). The target semantic information mentioned here is a feature that has a greater impact on the vehicle's intelligent assisted driving function.

[0053] In an embodiment of the present application, the block-level weight of each block can be determined by relying on a machine learning model, and the machine learning model can be, for example, a deep neural network and a convolutional neural network.

[0054] In some embodiments, the target semantic information includes at least one of the following features in the vehicle's environment: traffic lights, signs, lane lines, and dynamic targets.

[0055] The block-level weight of each block mentioned above is positively correlated with the frequency of occurrence of the target semantic information in the block, that is, the higher the frequency of occurrence of the target semantic information in each block, the greater the block-level weight of the block.

[0056] In some embodiments, the value range of the block-level weight is (0, 1].

[0057] According to the above technical means, by dividing the image to be detected into multiple blocks, each block can be analyzed and processed independently. This regionalized processing method not only improves the efficiency of image processing, but also allows for more refined processing based on the characteristics of each block. The block-level weight is determined based on the frequency of occurrence of the target semantic information in each block, so that more attention is paid to important areas in the image. This weight distribution method helps to more accurately identify and utilize key information in the image in subsequent processing.

[0058] It should also be noted that each block in the embodiment of the present application includes multiple pixels, that is, the dimension of the block is greater than the pixel dimension.

[0059] In the embodiment of the present application, the shapes of different blocks may be the same or different, and the number of pixels contained in different blocks may be the same or different, which is not limited in the embodiment of the present application.

[0060] In step S130, the image quality status score of each block is determined according to the pixel category weight of each pixel and the block level weight of each block.

[0061] The image quality status score not only takes into account the classification information at the pixel level, but also the overall information at the block level, thus providing more comprehensive and accurate data support for image quality assessment.

[0062] In some embodiments, the image quality status score of each block may be the product of a weighted average of pixel category weights of all pixels in the image to be detected and a block-level weight of the block.

[0063] According to the above technical means, the image quality status score of each block is defined as the product of the block-level weight and the weighted average of the pixel-level category weights of all pixels in the image to be detected. This calculation method integrates the pixel-level and block-level information in the score calculation, fully considering the impact of both on image quality, avoiding errors caused by a single factor, thereby improving the accuracy of the quality status assessment of each block and providing reliable intermediate data for the final accurate judgment of the quality of the image to be detected.

[0064] In step S140, a quality diagnosis result of the image to be detected is determined according to the image quality status scores of all blocks included in the key domain in the image to be detected.

[0065] In the embodiment of the present application, the above-mentioned key domain is a part of all blocks of the image to be detected, and the key domain can be determined according to the different degrees of influence on the driving field of view. Taking the image captured by the front-view camera as an example, the upper area of ​​the image is usually the sky, the middle area includes pedestrians and other objects in the road environment, and the lower part of the image contains key features such as signs and markings on the road. If the occlusion or blur occurs in the upper part of the image, the impact on driving is small, and when it appears in the lower middle part of the image, it will have a greater impact. In this case, all areas in the lower middle part of the image can be determined as key domains. For another example, when the camera that collects the image is a fisheye lens, it can increase the field of view, but there will be large distortion and blur at the edge, and the assisted driving algorithm usually crops this image; when performing quality inspection on this type of image, the edge area can be discarded, that is, all areas except the edge area are set as key domains.

[0066] It can also be understood that the determination of the key domain is not fixed, but can be determined according to the driving state of the vehicle and the different scenes it is in. For example, when the vehicle is driving forward, the key domain in the rear view image can be set to be less to save computing resources; when the vehicle is driving backward (i.e., reversing), it is necessary to set the key domain in the rear view and side view images to be more to comprehensively determine the quality of the rear view image.

[0067] There are many methods for determining the quality detection result of the image to be detected based on the image quality status scores of all blocks in the key domain. For example, as a possible implementation method, the score of each key domain can be accumulated, and according to the size of the accumulated sum and the size of the preset threshold, when the accumulated sum is greater than the preset threshold, the image to be detected is determined to be an abnormal frame, otherwise it is a normal frame; or, as another implementation method, the scores of multiple key domains can be weighted and summed according to the different importance of each key domain, and then compared with the preset threshold to determine whether the image to be detected is available.

[0068] According to the above technical means, with the help of a semantic segmentation model including a deep separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network, pixel-level and block-level weights are integrated. First, the weight is determined for each pixel classification, and then the block weight is determined in combination with the block target semantic information frequency. The two together determine the block image quality score, and finally the quality diagnosis result is obtained based on the key domain block score. This allows the detection process to fully consider the image details and semantic information, thereby comprehensively and accurately evaluating the image quality.

[0069] In some embodiments, the determining of the quality diagnosis result of the image to be detected according to the image quality status scores of all blocks included in the key domain in the image to be detected further includes:

[0070] Determine the number of target blocks whose image quality status scores of all blocks included in the key domain are greater than the first score threshold. For each block, according to the method for determining the image quality status score provided above, the larger the image quality status score, the greater the probability that target features such as occlusion, blur and glare exist in the image.

[0071] When the number of target blocks whose image quality status scores are greater than the first score threshold is greater than the first number threshold, the image to be detected is determined to be an abnormal image, that is, there is less reliable driving data in the collected image at this time.

[0072] When the number of target blocks whose image quality status scores are greater than the first score threshold is less than or equal to the first number threshold, the image to be detected is determined to be a normal image. At this time, there are fewer abnormal areas in the image, which can meet the requirements of the intelligent driving algorithm.

[0073] According to the above technical means, by determining the number of target blocks whose image quality scores in all blocks included in the key domain are greater than the first score threshold, it is determined whether the image to be detected is an abnormal image or a normal image. This judgment method based on the score of a specific block in the key domain focuses on the more important key domain part of the image, making the image quality diagnosis more objective and accurate, effectively distinguishing normal and abnormal images, and improving the credibility of the detection results.

[0074] Combine the following Figure 2 and Figure 3 The technical solution of the embodiment of the present application is further described. Figure 2 The flowchart of the method for segmenting the image to be detected using the semantic segmentation model is shown. Figure 3 is an example of the semantic segmentation model 300. Figure 2 The method includes steps S210-S230.

[0075] In step S210, feature extraction is performed on the image to be detected by using a depth-separable variable convolutional neural network to obtain a plurality of first feature maps of different scales.

[0076] In an embodiment of the present application, the above-mentioned depth-separable variable convolutional neural network can be VargNetV2. The model exemplarily includes a plurality of cascaded downsampling layers, and the image to be detected is continuously downsampled multiple times. In each downsampling process, the size of the area that the convolution kernel can cover on the input feature map gradually increases, that is, the receptive field increases, so that more global information can be captured. In multiple consecutive downsamplings, feature maps of different scales can be generated, that is, the aforementioned multiple first feature maps, and this multi-scale feature can provide richer contextual information.

[0077] by Figure 3 Take the model 300 as an example, the model 300 includes a backbone network 310, a neck network 320 and a segmentation head network 330, wherein the backbone network 310 includes 6 downsampling layers, and the input feature map of each downsampling layer is the output feature map of the previous downsampling layer. As the downsampling proceeds layer by layer, the scale of the output feature map gradually increases. Each downsampling layer of the backbone network 310 can output feature maps of different scales respectively, and the multiple feature maps of different sizes or some of the feature maps therein are the aforementioned multiple first feature maps. For example, in Figure 3 In the example, the five feature maps output by the 2nd to 6th downsampling layers are used as the above-mentioned multiple first feature maps.

[0078] In step S220, a plurality of first feature maps are fused using a bidirectional pyramid network to obtain a second feature map.

[0079] The bidirectional pyramid network may be, for example, Bi-FPN, which can perform pyramid fusion on feature maps of different scales and make full use of feature information at different levels. The feature map fusion method based on the BI-FPN network is further described below.

[0080] The fusion process of the feature map of the BI-FPN network includes top-down fusion, bottom-up fusion and lateral connection.

[0081] In the top-down path, starting from the feature map of the highest layer, it is fused with the upsampled feature map of the next layer. The upsampling operation usually uses methods such as interpolation to enlarge the low-resolution feature map to the same size as the feature map of the previous layer, and then merge the two by element-by-element addition or other fusion methods to obtain a new fused feature map. This is done step by step downward, and each layer fuses the semantic information of the upper layer and the detailed information of the current layer.

[0082] The bottom-up path starts from the lowest layer of feature maps and fuses them with the downsampled feature maps of the previous layer based on the top-down fusion. The downsampling operation generally reduces the resolution of the feature map through methods such as pooling, and then merges it with the feature map of the next layer to further enrich the semantic information of the feature map.

[0083] In addition to the above-mentioned top-down and bottom-up fusion paths, there will be horizontal connections at each layer to fuse the original feature map of the same layer with the fused feature map again to make full use of the feature information of the layer and enhance the feature expression capability.

[0084] exist Figure 3 In the example, the input of the neck network 320 is the first feature map output by the 2nd to 6th downsampling layers of the aforementioned backbone network 310, and the second feature map is output through the above-mentioned top-down path, bottom-up path and lateral connection.

[0085] In step S230, segmentation is performed based on the plurality of second feature maps using a segmentation head network to determine a pixel-level category weight for each pixel in the image to be detected.

[0086] The segmentation head network 330 generally includes a series of convolutional layers and upsampling layers, which work together to classify each pixel based on the information in the second feature map and determine its pixel-level category weight in the image to be detected. This process ensures that each pixel can be accurately classified into its semantic category, achieving accurate segmentation of the image content.

[0087] According to the above technical means, the cascaded deep separable variable group convolutional neural network, bidirectional feature pyramid network and segmentation head network are used to perform semantic segmentation on the image to be detected. The feature information at different levels can be fully utilized, so that the model can consider global and local features at the same time, realize pixel-level semantic segmentation, and improve the recognition ability of defects such as blur, occlusion and glare.

[0088] In some embodiments, the step S220 of fusing the plurality of first feature maps using a bidirectional pyramid network to obtain a second feature map further includes:

[0089] The multiple first feature maps are input into a bidirectional feature pyramid network to obtain multiple third feature maps; coordinate convolution is performed on the multiple third feature maps to obtain multiple fourth feature maps; and feature concatenation is performed on the multiple fourth feature maps to obtain multiple second feature maps.

[0090] According to the foregoing description, the multiple first feature maps are feature maps of different scales output by the backbone network. Inputting these feature maps into the bidirectional pyramid network can fully utilize the complementarity of these multi-level feature maps.

[0091] In the bidirectional feature pyramid network, the input first feature map is processed through a series of upsampling, downsampling and lateral connection operations to generate multiple third feature maps. These operations are designed to enhance the network's perception of features of different scales and promote the communication and fusion of cross-scale features. The third feature map not only retains the important information of the original feature map, but also integrates contextual information from other scales.

[0092] Coordinate convolution is a special convolution operation that introduces position information (i.e., coordinates) on the basis of traditional convolution, allowing the network to more accurately locate and perceive objects in the image. Through coordinate convolution, the feature representation of each pixel not only contains the local information around it, but also incorporates its position information in the image.

[0093] By concatenating the features of multiple fourth feature maps, feature maps of different sizes and levels are spliced ​​into a second feature map. The second feature map has a richer feature representation. It not only includes multi-scale features from low-level to high-level, but also enhances the ability to capture position information through coordinate convolution, providing strong feature support for subsequent segmentation.

[0094] Based on the above technical means, through the feature fusion strategy of the bidirectional feature pyramid network, combined with coordinate convolution and feature concatenation technology, the feature map information from different dimensions of the backbone network is effectively integrated and utilized, thereby improving the model's understanding and processing capabilities of complex scenes.

[0095] In some embodiments, determining the pixel-level category weight of each pixel in the image to be detected in the aforementioned step S230 further includes:

[0096] The segmentation head network is used to determine the pixel-level classification results of the image to be detected. The pixel-level classification results include the probability that each pixel is a feature of a different category; an argmax operation is performed on the pixel-level classification results to assign a feature category label with the highest probability to each pixel; the probability of the feature category label of each pixel is determined as the pixel-level category weight.

[0097] In an embodiment of the present application, the segmentation head network extracts and classifies the second feature map, and the output result is a multidimensional matrix, in which each element corresponds to a pixel in the image, and the value of each element represents the probability that the pixel belongs to a feature of a different category. The pixel-level classification result is subjected to an argmax operation. The role of the argmax function here is to select the category label with the highest probability from the probability distribution of each pixel. In other words, for each pixel in the image, the network will assign it a most likely category label, which is the category corresponding to the index with the maximum value selected by the argmax function from the probability distribution. Finally, the probability of determining the feature category label of each pixel is the pixel-level category weight. These weights represent the network's confidence in the pixel classification results.

[0098] According to the above technical means, the segmentation head network is used to determine the pixel-level classification results of the image to be detected, and the probability of each pixel being a feature of a different category is obtained. By performing an argmax operation on the pixel-level classification results, each pixel is assigned a feature category label with the highest probability, and the probability of the label is determined as the pixel-level category weight, which plays a key role in the subsequent combination of block-level weights to evaluate image quality.

[0099] Figure 4 is a schematic flow chart of an image quality detection method provided by another embodiment of the present application. Figure 4 The method provided in the embodiments of the present application is further described. Figure 4 The method includes steps S410-S450.

[0100] In step S410, an image to be detected is collected by a vehicle-mounted camera.

[0101] Multiple cameras deployed on the vehicle are used to collect multiple images around the vehicle. For example, seven cameras can be set on the vehicle to collect 7V images of the surrounding view or four cameras can be used to collect 4V images of the surrounding view. Each frame of the collected image is recorded as {I n ,n∈[1,4]or[1,7]}, different cameras have different resolutions, and the current frame is input into the subsequent image quality segmentation network Seg_QAN.

[0102] In step S420, an image quality segmentation network Seg_QAN is constructed. The image quality anomaly detection network is constructed based on a semantic segmentation model.

[0103] Among them, the backbone network of Seg_QAN uses VargNetV2 built based on deeply separable variable group convolution. The neck network is a multi-scale feature fusion network. More specifically, the multi-scale feature fusion network is a bidirectional feature pyramid network (Bi-FPN), which can deeply fuse features of different scales from top to bottom and then from bottom to top. For example, the backbone network uses different convolution steps (such as stride2 to stride64) to perform convolution operations on the input image, output feature information of different receptive fields, and the feature fusion network performs fusion and outputs fusion features of stride4 to stride64. Next, in order to enhance the robustness of the position information of pixel-level features, each size of feature first passes through a coordinate convolution coordconv, and then passes through the segmentation projection network to segment the feature map.

[0104] The forward process of the above model is:

[0105] Forward:{Adam{CE_loss[gt,SegHead(Coconv(Bi-FPN(B(I N ))))]}}(1)

[0106] Among them, Adam{} represents the model training optimizer, CE_loss[] is the cross entropy loss used for pixel-level classification of the semantic segmentation model, gt is the mask of the pixel-level segmentation annotation of the current frame, SegHead represents the segmentation head network of the model, coconv is the coordinate convolution operation, Bi-FPN is the neck network of the model, that is, the bidirectional feature pyramid network, and B is the backbone network backbone, which uses VargNetV2.

[0107] During the training of the above model, the losses are calculated for the multi-scale features of the model from stride4 to stride64:

[0108] train:CE_loss[gt,feat i ],i∈{4,8,16,32,64}(2)

[0109] Among them, feat. i represents the multi-scale output features from the i-th layer.

[0110] When calculating the cross entropy loss of different categories, in order to alleviate the problem of data imbalance between different categories, before calculating the loss, the weight of calculating the loss is determined by automatically counting the proportion of the number of different abnormal categories in the pixels:

[0111] class_weights c =max ([1-num_pixelsc / object_num],weight_min),c∈[1,class_num] (3)

[0112] Among them, class_weights c is the calculated weight value of category c, max() represents the maximum value operation, num_pixels c is the number of abnormal class pixels, weight_min is the lower limit of the class weight in the loss calculation, and class_num is the number of image quality segmentation classes. It should be noted that in the model reasoning process, the intermediate level features are used, such as the stride8 feature.

[0113] The aforementioned coordinate convolution Coconv() mainly records the pixel positions of features of different scales, and then fuses them in series with the current features in the form of coordinate positions:

[0114] Coconv:{Concat(feat i ,x,y)}(4)

[0115] Concat(.) represents a feature concatenation operation. x and y are the coordinate values ​​of the coordinate matrix generated by the width and height of the aligned feature map. Coordinate convolution can effectively avoid the loss of position information.

[0116] After the coordinate convolution, the segmentation head network uses the Mask of the corresponding layer feature output using the depth-separable convolution w*,h* ,w * and h * The width and height of the Mask are increased to the original input size by upsampling operation up_sample() during model inference. w,h Finally, after argmax, we get the pixel-level category score (i.e., the pixel-level category weight in the previous article) S w,h , w and h are the width and height of the original image, and the formula for the above process is as follows:

[0117] S w,h =argmax(up_sample(Mask)) (5)

[0118] Step S430, divide the current frame image into multiple blocks, and assign block-level weights to different blocks. Specifically, by setting the grid_size value, the current frame image I nIt is divided into m×n blocks. The block-level weight is determined according to the frequency of occurrence of important semantic information in different areas during driving (such as traffic lights, signs, lane lines, and dynamic targets, etc.). The more the above semantic information appears in the area, the greater the weight, which is expressed as a m,n ∈[0,1].

[0119] In step S440, the image quality status score R_qas of each region is calculated hierarchically based on the pixel-level category weight and the block-level weight. m,n , thus obtaining the image quality score of the local abnormality of the current frame. By comparing the threshold thr_re, the diagnosis result of the area can be obtained. The image quality score calculation formula of each area can be expressed as follows:

[0120]

[0121] Among them, n c Represents the number of pixels of category c.

[0122] In step S450, a key region K is defined in the current frame, and the diagnosis results of multiple regions in the key region are combined to vote to obtain the image quality diagnosis result of the final global frame image. The process is identified as follows:

[0123]

[0124] Among them, sum() is the number of votes for the abnormal area in the statistical key domain K. If the comprehensive vote is greater than QA_thr, it is an abnormal frame, otherwise it is a normal frame.

[0125] Figure 5 is a flow chart of an image quality detection method provided in an embodiment of the present application, the method comprising the steps

[0126] like Figure 5 As shown, in this method, step S501, obtain the current frame image (also called the image to be detected) collected by the camera; step S502, use the semantic segmentation network Seg_QAN to perform semantic segmentation on the current frame image; step S503, obtain the label image of the pixel-level features; step S504, upsample the label image of the pixel-level features to the size of the original image; step S505, combine the segmentation results of the semantic segmentation network, and assign a pixel-level category weight S to each pixel. w,h Step S506: divide the current frame image into m*n area blocks, and assign a block-level weight a to each area block. m,n Step S507: Calculate the scores R_qas of different categories in different regions based on the pixel-level category weights and block-level weights. m,nIn step S508, voting is performed to count whether the number of abnormalities in each area within the key domain is greater than a preset threshold value QA_thr. When the number of abnormalities is greater than the above threshold value, the process goes to step S509 to report abnormal diagnosis, otherwise, the process goes to step S510 to report normal diagnosis.

[0127] The embodiment of the present application provides a new type of image quality anomaly detection method based on the semantic segmentation method, and constructs an anomaly detection network (Seg_QAN) at the pixel-level granularity. The network is supervised by the segmented and annotated abnormal data (blur, occlusion, glare) collected in a large number of daily driving conditions, learns to locate local abnormal areas, and can accurately output the classification information of each pixel. At the same time, a hierarchical diagnosis strategy is formulated, which hierarchically integrates the segmented local classification information through the method of regional weighted statistics. Different weights are assigned to different areas according to the impact on the driving field of view. After the weighted strategy, the image quality anomaly diagnosis results of each local area of ​​the current frame are output, and each local abnormal condition is weighed, and the diagnosis of the global image is comprehensively output, and the image quality diagnosis of the current frame is performed from the bottom up from the local to the global.

[0128] The technical solution of this application not only applies the semantic segmentation method in image quality diagnosis for the first time, and the network can output pixel-level abnormality classification results, but also formulates a hierarchical diagnosis strategy in downstream image quality diagnosis, which outputs more reliable global quality diagnosis results after comprehensively considering the contribution of key domain and non-key domain information. It is of great significance to help achieve more fine-grained image quality detection, improve image information utilization, function switching efficiency, and promote the implementation of more flexible and intelligent intelligent driving solutions.

[0129] Combination of the above Figure 1-Figure 5 , the method embodiment of the present application is described in detail, and the device embodiment of the present application will be described in detail in conjunction with the accompanying drawings. It should be understood that the description of the device embodiment corresponds to the method embodiment, so the part not described in detail can refer to the device embodiment above.

[0130] Figure 6 It is a schematic structural diagram of the image detection device provided in an embodiment of the present application. Figure 6 The device 600 in the embodiment comprises:

[0131] The segmentation unit 610 is used to segment the image to be detected by using a semantic segmentation model, and determine the pixel-level category weight of each pixel in the image to be detected. The pixel-level category weight is used to characterize the probability that the type of each pixel is the type of the target feature. The semantic segmentation model includes a cascaded deep separable variable group convolutional neural network, a bidirectional feature pyramid network and a segmentation head network; the image to be detected includes environmental information around the vehicle, and the target feature is any one of blur, occlusion and glare.

[0132] The first determination unit 620 is used to divide the image to be detected into multiple blocks, and determine the block-level weight of each block according to the occurrence frequency of the target semantic information in each block, wherein each block includes multiple pixels.

[0133] The second determining unit 630 is used to determine the image quality status score of each block according to the pixel-level category weight of each pixel and the block-level weight of each block.

[0134] The third determining unit 640 is used to determine the quality diagnosis result of the image to be detected according to the image quality status scores of all blocks included in the key region in the image to be detected.

[0135] In some embodiments, the segmentation unit is also used to: extract features from the image to be detected through a deep separable variable group convolutional neural network to obtain multiple first feature maps of different scales; use a bidirectional feature pyramid network to fuse the multiple first feature maps to obtain a second feature map; use a segmentation head network to perform segmentation based on the second feature map to determine the pixel-level category weight of each pixel in the image to be detected.

[0136] In some embodiments, a bidirectional feature pyramid network is used to perform feature fusion on multiple first feature maps to obtain a second feature map, including: inputting multiple first feature maps into the bidirectional feature pyramid network to obtain multiple third feature maps; performing coordinate convolution on the multiple third feature maps to obtain multiple fourth feature maps; and performing feature concatenation on the multiple fourth feature maps to obtain a second feature map.

[0137] In some embodiments, determining the pixel-level category weight of each pixel in the image to be detected includes: using a segmentation head network to determine the pixel-level classification result of the image to be detected, the pixel-level classification result includes the probability that each pixel is a feature of a different category; performing an argmax operation on the pixel-level classification result to assign a feature category label with the highest probability to each pixel; and determining the probability of the feature category label of each pixel as the pixel-level category weight.

[0138] In some embodiments, the image quality status score of each block is the product of the block-level weight of each block and the weighted average of the pixel-level category weights of all pixels in the image to be detected; the image quality status score is used to characterize the probability of the target feature existing in each block.

[0139] In some embodiments, the third determination unit is also used to: determine the number of target blocks in all blocks included in the key domain whose image quality status scores are greater than the first score threshold; when the number of target blocks is greater than the first number threshold, determine that the image to be detected is an abnormal image; when the number of target blocks is less than or equal to the first number threshold, determine that the image to be detected is a normal image.

[0140] In some embodiments, the target semantic information includes at least one of the following features in the vehicle's surrounding environment: traffic lights, signs, lane lines, and dynamic targets; the block-level weight of each block is positively correlated with the frequency of occurrence of the target semantic information.

[0141] Based on the image quality detection method provided in the above embodiment, the embodiment of the present application also provides a device for performing image quality detection, which may be a vehicle or a host computer. Figure 7 The schematic diagram of the structure of the device for performing image quality detection provided in the embodiment of the present application is as follows. The device 700 includes a processor 710 and a memory 720. The memory 720 stores a computer program, which can be executed by the processor 710, so that the processor 710 executes the method described in the method embodiment above. The memory 720 can be independent of the processor 710 or integrated in the processor 710.

[0142] The device 700 may further include a transceiver 730. The processor 710 may communicate with other devices or chips through the transceiver 730. For example, the processor 710 may transmit and receive data with other devices or chips through the transceiver 730.

[0143] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0144] The embodiment of the present application provides a storage medium, which stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement some or all of the steps in the above method. The storage medium can be transient or non-transient.

[0145] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0146] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0147] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0148] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0149] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0150] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0151] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0152] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0153] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0154] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0155] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0156] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a vehicle-mounted terminal (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0157] The methods disclosed in several method embodiments provided in the embodiments of the present application can be arbitrarily combined without conflict to obtain new method embodiments.

[0158] The features disclosed in several product embodiments provided in the embodiments of the present application can be arbitrarily combined without conflict to obtain new product embodiments.

[0159] The features disclosed in several method or device embodiments provided in the embodiments of the present application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0160] The above is only an implementation method of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. An image quality detection method, characterized in that: The method comprises: Using a semantic segmentation model, the image to be detected is segmented to determine a pixel-level category weight of each pixel in the image to be detected, wherein the pixel-level category weight is used to characterize the probability that the type of each pixel is the type of the target feature, and the semantic segmentation model includes a cascaded deep separable variable group convolutional neural network, a bidirectional feature pyramid network, and a segmentation head network; Dividing the image to be detected into a plurality of blocks, and determining a block-level weight of each block according to the occurrence frequency of target semantic information in each block, wherein each block includes a plurality of pixels; Determine the image quality status score of each block according to the pixel-level category weight of each pixel and the block-level weight of each block; The quality diagnosis result of the image to be detected is determined according to the image quality status scores of all blocks included in the key domain of the image to be detected.

2. The image quality detection method according to claim 1, characterized in that: The method of using the semantic segmentation model to segment the image to be detected includes: Extracting features of the image to be detected by using the depth-separable variable group convolutional neural network to obtain a plurality of first feature maps of different scales; Using the bidirectional feature pyramid network to perform feature fusion on the multiple first feature maps to obtain a second feature map; The segmentation head network is used to perform segmentation based on the second feature map to determine a pixel-level category weight for each pixel in the image to be detected.

3. The image quality detection method according to claim 2, characterized in that: The step of using the bidirectional feature pyramid network to perform feature fusion on the plurality of first feature maps to obtain a second feature map includes: Inputting the plurality of first feature maps into the bidirectional feature pyramid network to obtain a plurality of third feature maps; Performing coordinate convolution on the multiple third feature maps to obtain multiple fourth feature maps; The plurality of fourth feature maps are feature-concatenated to obtain the second feature map.

4. The image quality detection method according to claim 2, characterized in that: Determining the pixel-level category weight of each pixel in the image to be detected includes: Determine a pixel-level classification result of the image to be detected by using the segmentation head network, wherein the pixel-level classification result includes a probability that each pixel is a feature of a different category; Performing an argmax operation on the pixel-level classification result to assign a feature category label with the highest probability to each pixel; The probability of determining the feature category label of each pixel is the pixel-level category weight.

5. The image quality detection method according to any one of claims 1 to 4, characterized in that: The image quality status score of each block is the product of the block-level weight of each block and the weighted average of the pixel-level category weights of all pixels of the image to be detected; The image quality status score is used to represent the probability of the target feature existing in each of the blocks.

6. The image quality detection method according to any one of claims 1 to 4, characterized in that: The step of determining the quality diagnosis result of the image to be detected according to the image quality status scores of all blocks included in the key domain of the image to be detected includes: Determine the number of target blocks whose image quality status scores are greater than a first score threshold among all blocks included in the key domain; When the number of the target blocks is greater than a first number threshold, determining that the image to be detected is an abnormal image; When the number of the target blocks is less than or equal to the first number threshold, it is determined that the image to be detected is a normal image.

7. The image quality detection method according to any one of claims 1 to 4, characterized in that: The target semantic information includes at least one of the following features in the road environment around the vehicle: traffic lights, signboards, lane lines, and dynamic targets; The block-level weight of each of the blocks is positively correlated with the occurrence frequency of the target semantic information.

8. An image quality detection device, characterized in that: include: A segmentation unit, used to segment the image to be detected by using a semantic segmentation model, and determine a pixel-level category weight of each pixel in the image to be detected, wherein the pixel-level category weight is used to characterize the probability that the type of each pixel is the type of the target feature, and the semantic segmentation model includes a cascaded deep separable variable group convolutional neural network, a bidirectional feature pyramid network, and a segmentation head network; A first determination unit is used to divide the image to be detected into a plurality of blocks, and determine a block-level weight of each block according to the occurrence frequency of target semantic information in each block, wherein each block includes a plurality of pixels; A second determining unit, configured to determine an image quality status score of each of the blocks according to the pixel-level category weight of each pixel and the block-level weight of each block; The third determining unit is used to determine the quality diagnosis result of the image to be detected according to the image quality status scores of all blocks included in the key domain of the image to be detected.

9. An image quality detection device, characterized in that: The device comprises a processor memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images