Shielding detection method and device, electronic equipment and storage medium

Abstract

The invention provides a shielding detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original image, and carrying out the preprocessing of the original image, obtaining a to-be-detected image, the preprocessing comprises image enhancement and / or image graying, the original image comprises image information collected by image collection equipment, and the image information comprises image information collected by image collection equipment; saliency detection is performed on the to-be-detected image to obtain growth information, the growth information comprises at least one growth region and at least one initial seed point in the growth region, and region growth is performed according to a preset gray scale threshold value and the difference between each sub-pixel value and a plurality of neighborhood pixel values in the corresponding growth neighborhood, a seed pixel value and a neighborhood pixel value are obtained based on the initial seed point and the to-be-detected image, and a shielding detection result of the image acquisition device is determined according to segmentation features of a target region in the region segmentation image; according to the method, the accuracy and the fine granularity of the occlusion detection are improved in a manner of matching the saliency detection with the region growth.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a blocking detection method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the development of intelligent cockpit technology, a driver monitoring system (DMS) plays an important role in improving driving safety. The DMS usually includes functions such as fatigue monitoring, distraction monitoring, abnormal behavior recognition, and emotion recognition. However, the functions of the DMS are based on vision and mainly rely on image acquisition by an infrared camera in the vehicle. The effectiveness of the functions of the DMS depends largely on the state of the infrared camera, especially whether the infrared camera is blocked, which will directly affect the accuracy and reliability of the DMS. Therefore, effective blocking detection of the infrared camera is crucial to ensure that the functions of the DMS are running normally.

[0003] In related technologies, a deep learning model needs to collect a large amount of image data of full blocking and partial blocking of an infrared camera for training the model. Moreover, the model needs to collect image data again for different types and different installation methods of the infrared camera, which is time-consuming and laborious. In a scene with low complexity of image features, the deep learning model is not the optimal solution. SUMMARY

[0004] The present application provides a blocking detection method, device, electronic device, and storage medium to solve the technical problem of complex blocking detection of the image acquisition device.

[0005] The present application provides a blocking detection method, which includes: obtaining an original image and pre-processing the original image to obtain a to-be-detected image, the pre-processing including image enhancement and / or image grayscale, the original image containing image information collected by an image acquisition device; performing saliency detection on the to-be-detected image to obtain growth information, the growth information including at least one growth region and at least one initial seed point in the growth region; performing region growing according to a preset gray threshold and a difference between each seed pixel value and a plurality of neighborhood pixel values in a corresponding growth neighborhood to obtain a region segmentation image, the seed pixel value and the neighborhood pixel value being obtained based on the initial seed point and the to-be-detected image; and determining a blocking detection result of the image acquisition device according to a segmentation feature of a target region in the region segmentation image.

[0006] In some embodiments of the present application, the occlusion detection result of the image acquisition device is determined according to the segmentation features of the target regions in the region-segmented image, including: if the total number of regions in the region-segmented image is less than a preset first threshold, the occlusion state in the occlusion detection result is determined as device occlusion; if the total number of regions in the region-segmented image is greater than or equal to the preset first threshold, the occlusion state in the occlusion detection result is determined as device non-occlusion.

[0007] In some embodiments of the present application, the occlusion detection result of the image acquisition device is determined according to the segmentation features of the target regions in the region-segmented image, further including: if the occlusion state is device occlusion, the shape features of at least one target region are determined, the shape features including circularity and / or rectangularity; if at least one of the circularity of a target region is greater than or equal to a preset second threshold, and the rectangularity of the target region is greater than or equal to a preset third threshold, the target region is determined as an occlusion region; if there is an occlusion region, the occlusion proportion is determined according to the total area of the image to be detected and the occlusion area of at least one occlusion region; wherein the occlusion detection result further includes the occlusion proportion and / or at least one occlusion region.

[0008] In some embodiments of the present application, the region growing is performed according to a preset gray scale threshold and the difference between each sub-pixel value and a plurality of neighborhood pixel values in the corresponding growing neighborhood, including: determining the growing neighborhood corresponding to each initial seed point according to a preset first neighborhood size, the growing neighborhood including a plurality of neighborhood pixel points in the image to be detected; if the absolute value of the difference between the seed pixel value of an initial seed point and the neighborhood pixel value of a neighborhood pixel point is less than the preset gray scale threshold, the neighborhood pixel point is determined as a same-class seed point of the initial seed point, so as to determine new same-class seed points according to the growing neighborhood corresponding to the same-class seed points; at least one target region is determined according to each initial seed point and the corresponding same-class seed point.

[0009] In some embodiments of the present application, the saliency detection is performed on the image to be detected to obtain growth information, including: performing local contrast saliency detection on the image to be detected to obtain a saliency image, and binarizing the saliency image to obtain a binary image; dividing the binary image into at least one growth region, and determining at least one initial pixel point with an initial pixel value of zero in each growth region as an initial seed point of the corresponding growth region.

[0010] In some embodiments of the present application, the local contrast saliency detection is performed on the image to be detected to obtain a saliency image, and the saliency image is binarized to obtain a binary image, including: performing mean filtering on any to-be-detected pixel value in the image to be detected according to a preset second neighborhood size to obtain a mean filtering image; determining a saliency image based on the difference between the mean filtering image and the image to be detected; performing maximum inter-quantile threshold binarization on the saliency image to obtain a binary image.

[0011] In some embodiments of the present application, the original image is preprocessed to obtain an image to be detected, including: obtaining an image to be enhanced, the image to be enhanced being a gray-scale format original image; performing gamma transformation on the image to be enhanced to obtain a transformed image, and performing bilateral filtering on the transformed image to obtain an image to be detected.

[0012] The present application provides an occlusion detection device, the device comprising: an image acquisition module, configured to acquire an original image, and pre-process the original image to obtain an image to be detected, the pre-processing including image enhancement and / or image graying, the original image containing image information collected by an image collection device; a growth information determination module, configured to perform saliency detection on the image to be detected to obtain growth information, the growth information including at least one growth region and at least one initial seed point in the growth region; a region growing module, configured to perform region growing according to a preset gray scale threshold, and the difference between each seed pixel value and a plurality of neighborhood pixel values in a corresponding growth neighborhood, to obtain a region segmentation image, the seed pixel value and the neighborhood pixel value being obtained based on the initial seed point and the image to be detected; an occlusion detection module, configured to determine an occlusion detection result of the image collection device according to the segmentation features of a target region in the region segmentation image.

[0013] The present application provides an electronic device, comprising: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the occlusion detection method as described in any of the above embodiments.

[0014] The present application also provides a computer-readable storage medium having a computer program stored thereon, when the computer program is executed by a processor of a computer, the computer executes the occlusion detection method as described in any of the above embodiments.

[0015] The beneficial effects of this application are as follows: The occlusion detection method, device, electronic device and storage medium proposed in this application determine the initial seed points for random growth through the results of saliency detection, so as to perform region growth without the need to introduce human prior knowledge, and have stronger scene adaptability. Thus, the occlusion detection results of the image acquisition device can be accurately and quickly determined by the idea of ​​image segmentation. The accuracy and fineness of occlusion detection are improved by combining saliency detection with region growth. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] In the attached diagram: Figure 1 A schematic diagram of an exemplary system architecture provided in an embodiment of this application; Figure 2 This is a flowchart illustrating an occlusion detection method provided in one embodiment of this application; Figure 3(a) is the original image of the device without obstruction provided in one embodiment of this application; Figure 3(b) is the original image corresponding to the complete occlusion of the device provided in one embodiment of this application; Figure 3(c) is the original image corresponding to partial occlusion of the device provided in one embodiment of this application; Figure 4(a) is a transformed image corresponding to the unobstructed device provided in one embodiment of this application; Figure 4(b) is a transformed image corresponding to full occlusion of the device provided in one embodiment of this application; Figure 4(c) is a transformed image corresponding to partial occlusion of the device provided in one embodiment of this application; Figure 5(a) is the image to be detected when the device is not obstructed in one embodiment of this application; Figure 5(b) shows the image to be detected corresponding to full occlusion of the device provided in one embodiment of this application; Figure 5(c) is a detection image corresponding to partial occlusion of the device provided in one embodiment of this application; Figure 6(a) is a saliency image corresponding to the unobstructed device provided in one embodiment of this application; Figure 6(b) is a saliency image corresponding to full occlusion of the device provided in one embodiment of this application; Figure 6(c) is a saliency image corresponding to partial occlusion of the device provided in an embodiment of this application; FIG. 7(a) is an initial seed point corresponding to a non-occlusion of the device provided in an embodiment of the present application; FIG. 7(b) is an initial seed point corresponding to a full-occlusion of the device provided in an embodiment of the present application; FIG. 7(c) is an initial seed point corresponding to a partial-occlusion of the device provided in an embodiment of the present application; FIG. 8(a) is a region segmentation image corresponding to a non-occlusion of the device provided in an embodiment of the present application; FIG. 8(b) is a region segmentation image corresponding to a full-occlusion of the device provided in an embodiment of the present application; FIG. 8(c) is a region segmentation image corresponding to a partial-occlusion of the device provided in an embodiment of the present application; Figure 9 FIG. 1 is a block diagram of an occlusion detection apparatus provided in an embodiment of the present application; Figure 10 FIG. 3 is a structural schematic diagram of a computer system of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The above embodiments of the present application are described with reference to the drawings, and other advantages and effects of the present application will be more clearly understood from the following detailed description when read in conjunction with the appended drawings. The present application can be applied to other embodiments that are not described below, and the embodiments described below can be combined with each other to the extent that they do not conflict each other. The embodiments described below are provided to make the present application more comprehensible to those skilled in the art.

[0019] It should be noted that the drawings provided in the following embodiments only schematically illustrate the basic concept of the present application, and the drawings only show the components related to the present application, not the number, shape and size of the components when actually implemented. The actual implementation of each component can be randomly changed in terms of shape, number and proportion, and the layout pattern of the components can also be more complex.

[0020] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details, and in other embodiments, the known structures and devices are shown in the form of block diagrams rather than in the form of details, so as not to make the embodiments of the present application difficult to understand.

[0021] In the related art, there is a complex problem of occlusion detection of an image acquisition device. However, the present application can quickly determine the occlusion detection result of the image acquisition device by presetting a gray scale threshold and an initial seed point obtained based on saliency detection to perform region growing to obtain a region segmentation image.

[0022] Referring to Figure 1 , Figure 1 a schematic diagram of an exemplary system architecture provided in an embodiment of the present application. As shown in the figure, the system architecture can include an image acquisition device 110 and a computer device 120. The image acquisition device 110 can include an infrared camera and a color camera. The computer device 120 can be at least one of a microcomputer, an embedded computer, a general-purpose computer, a desktop graphic processing unit (GPU) computer, a GPU computing cluster, etc. The original image is acquired by the image acquisition device 110 and transmitted to the computer device 120 for occlusion detection of the image acquisition device 110. Figure 1

[0023] To solve the above technical problems, the present application provides an occlusion detection method and device, electronic equipment and storage medium. The implementation details of the technical solutions of the embodiments of the present application are described in detail below.

[0024] Referring to Figure 2 , Figure 2 a flowchart of an occlusion detection method provided in an embodiment of the present application. As shown in the figure, in an exemplary embodiment, the occlusion detection method includes at least steps S210 to S240, which are described in detail as follows: Figure 2 Step S210, acquiring an original image and pre-processing the original image to obtain a to-be-detected image.

[0025] The preprocessing includes image enhancement and / or image graying, and the original image contains image information acquired by the image acquisition device.

[0026] Step S220, performing saliency detection on the to-be-detected image to obtain growth information.

[0027] The growth information includes at least one growth region and at least one initial seed point in the growth region.

[0028] Step S230, performing region growing according to a preset gray threshold and the difference between each seed pixel value and a plurality of neighborhood pixel values in the corresponding growth neighborhood to obtain a region segmentation image.

[0029] The seed pixel value and the neighborhood pixel value are obtained based on the initial seed point and the to-be-detected image.

[0030] Step S240, determining an occlusion detection result of the image acquisition device according to the segmentation features of the target region in the region segmentation image.

[0031] ​​In some embodiments of the present application, the original image may involve privacy data such as a human face, and the image information corresponding to the original image is obtained under the authorization of the user or is necessarily uploaded by the user when using the auxiliary function, which includes a driver monitoring system (DMS) function.

[0032] In some embodiments of the present application, if the image information corresponding to the original image is collected by an infrared camera, the original image is in grayscale format, and at this time, image grayscaling is not required.

[0033] In some embodiments of the present application, if the image information corresponding to the original image is collected by a color camera, the original image is in color format, for example, red-green-blue format, and at this time, image grayscaling is required.

[0034] In some embodiments of the present application, during driving, the image acquisition device may be affected by factors such as device quality, changes in ambient light, and vehicle vibration, resulting in problems such as increased noise in the original image and poor image display consistency, so image enhancement is required at this time.

[0035] In some embodiments of the present application, the seed pixel value is the pixel value of the initial seed point in the image to be detected, and the neighborhood pixel value is the pixel value of a neighborhood pixel point in the growing neighborhood in the image to be detected.

[0036] In some embodiments of the present application, the preset gray scale threshold can be set to 5.

[0037] In some embodiments of the present application, please refer to FIGS. 3(a) to 3(c), FIG. 3(a) is an original image corresponding to a device without occlusion provided in an embodiment of the present application, FIG. 3(b) is an original image corresponding to a device with full occlusion provided in an embodiment of the present application, and FIG. 3(c) is an original image corresponding to a device with partial occlusion provided in an embodiment of the present application. As shown in FIGS. 3(a) to 3(c), to protect privacy, in the original image corresponding to FIG. 3(a), the face region is centered on the nose, the lower face region is occluded by a mask, and the upper face region covers a privacy block with a pixel value similar to the skin. The actual original image can display eyebrow and eye features and skin features in the upper face region. In the image processing process of the present application, the original image does not include the covered privacy block, i.e., FIG. 3(a) is only used for display; FIG. 3(b) and FIG. 3(c) are original images without any privacy processing.

[0038] In some embodiments of the present application, the occlusion detection result includes at least one of an occlusion state, an occlusion region, and an occlusion ratio.

[0039] The application performs region growing on initial seed points of random growth determined by the result of saliency detection, without introducing artificial prior knowledge, and is more adaptable to scenes, so as to accurately and quickly determine the occlusion detection result of the image acquisition device in the image segmentation idea, and improve the accuracy and fine granularity of occlusion detection by the way of saliency detection combined with region growing.

[0040] In some embodiments of the application, the original image is preprocessed to obtain a to-be-detected image, including: obtaining a to-be-enhanced image, the to-be-enhanced image being a gray-scale original image; performing gamma transformation on the to-be-enhanced image to obtain a transformed image, and performing bilateral filtering on the transformed image to obtain the to-be-detected image.

[0041] In some embodiments of the application, in order to improve the robustness and adaptability of occlusion detection, it is necessary to improve the image quality and image display consistency, which is realized by performing gamma transformation on the to-be-enhanced image. The principle of gamma transformation is as follows: Formula (1) Wherein, is an output pixel value of an output image, is a preset constant, is an input pixel value of an input image, is a preset transformation control parameter.

[0042] In some embodiments of the application, the output image is the transformed image, and the input image is the to-be-enhanced image.

[0043] In some embodiments of the application, the output range is controlled by the preset constant, which is usually set to 1.

[0044] In some embodiments of the application, if is equal to 1, the output image is the same as the input image and does not change. If is less than 1, the dynamic range of the dark area in the input image will be stretched and the bright area will be compressed, so that the output image looks brighter. If is greater than 1, the opposite is true, the dynamic range of the bright area will be expanded and the dark area will be compressed, so that the output image looks darker.

[0045] In some embodiments of the application, the brightness distribution of the to-be-enhanced image corresponding to the infrared camera can be improved by gamma transformation to adapt to the human visual characteristics and improve the image quality and display consistency.

[0046] In some embodiments of the present application, please refer to FIG. 4(a) to FIG. 4(c), FIG. 4(a) is a corresponding transformed image provided by the device in an embodiment of the present application without occlusion, FIG. 4(b) is a corresponding transformed image provided by the device in an embodiment of the present application with full occlusion, and FIG. 4(c) is a corresponding transformed image provided by the device in an embodiment of the present application with partial occlusion. As shown in FIG. 4(a) to FIG. 4(c), in the embodiments of the present application, the dynamic range of the bright area of the transformed image is expanded after the gamma transformation with the value of 0.8. As can be seen from the un-occluded image, compared with the original image of FIG. 3(a), the transformed image of FIG. 4(a) is brighter and has higher contrast. FIG. 4(a) is still displayed by covering the face area with the privacy block. Compared with FIG. 3(b), FIG. 4(b) is brighter. Compared with FIG. 3(c), FIG. 4(c) is brighter. As can be seen from the un-occluded image, compared with the original image of FIG. 3(a), the transformed image of FIG. 4(a) is brighter and has higher contrast. FIG. 4(a) is still displayed by covering the face area with the privacy block. Compared with FIG. 3(b), FIG. 4(b) is brighter. Compared with FIG. 3(c), FIG. 4(c) is brighter.

[0047] In some embodiments of the present application, the gamma transformation can improve the consistency of image display, but cannot suppress image noise, so the transformed image needs to be subjected to bilateral filtering. The bilateral filtering is different from the traditional linear filter (such as Gaussian filtering). The bilateral filtering not only considers the spatial position information of the pixel, but also considers the difference of the pixel value, so that the noise can be removed while the image edge is kept clear, which is very suitable for the scene requiring accurate edge information.

[0048] In some embodiments of the present application, the bilateral filtering is as follows: Formula (2) Wherein, is the pixel value to be detected of the point in the image to be detected, the point is any pixel point in the transformed image, the point is the pixel point in the transformed image corresponding to the point is any pixel point in the filtering neighborhood of the point is a preset normalization factor, is the original pixel value of the point in the transformed image, is the spatial neighborhood kernel between the point and the point , is the value range kernel between and , is the original pixel value of the point in the transformed image.

[0049] In some embodiments of the present application, each original pixel value in the transformed image is subjected to the bilateral filtering calculation. The preset normalization factor is used to ensure that the output pixel value to be detected is within a reasonable range.

[0050] In some embodiments of the present application, the spatial neighborhood kernel is defined by a Gaussian kernel as follows: Equation (3) wherein, is the spatial neighborhood kernel between point and point is the Euclidean distance between point and point is the spatial standard deviation, point is any pixel point in the transformed image, point is the original pixel value of point in the transformed image.

[0051] In some embodiments of the present application, the spatial standard deviation is used to control the degree of influence of the spatial neighborhood, which can be preset to 75.

[0052] In some embodiments of the present application, the value domain kernel is also defined by a Gaussian kernel as follows: Equation (4) wherein, is the value domain kernel between point and point is the original pixel value of point in the transformed image, is the original pixel value of point in the transformed image, is the color space standard deviation, is the difference between the two original pixel values, point is any pixel point in the transformed image, point is the original pixel value of point in the corresponding filter neighborhood in the transformed image.

[0053] In some embodiments of the present application, the filter kernel size of the bilateral filter is preset, such as the preset first kernel size of the spatial neighborhood kernel and the preset second kernel size of the value domain kernel are both set to 5, thereby obtaining the filter neighborhood .

[0054] In some embodiments of the present application, the color space standard deviation can also be preset.

[0055] ​​​In some embodiments of the present application, please refer to FIG. 5(a) to FIG. 5(c), FIG. 5(a) is a corresponding to-be-detected image provided by the device without occlusion in an embodiment of the present application, FIG. 5(b) is a corresponding to-be-detected image provided by the device with full occlusion in an embodiment of the present application, and FIG. 5(c) is a corresponding to-be-detected image provided by the device with partial occlusion in an embodiment of the present application. As shown in FIG. 5(a) to FIG. 5(c), the corresponding to-be-detected image is obtained by performing bilateral filtering on different transformed images, and FIG. 5(ab) is still displayed by covering the face region with a privacy block.

[0056] In some embodiments of the present application, the growth information is obtained by performing saliency detection on the to-be-detected image, including: performing local contrast saliency detection on the to-be-detected image to obtain a saliency image, and binarizing the saliency image to obtain a binary image; dividing the binary image into at least one growth region, and determining at least one initial pixel point with an initial pixel value of zero in each growth region as an initial seed point of the corresponding growth region.

[0057] In some embodiments of the present application, the saliency image is obtained by performing local contrast saliency detection on the to-be-detected image, and the binary image is obtained by binarizing the saliency image, including: performing mean filtering on any to-be-detected pixel value in the to-be-detected image according to a preset second neighborhood size to obtain a mean filtering image; determining the saliency image based on the difference between the mean filtering image and the to-be-detected image; and performing maximum inter-quantile threshold binarization on the saliency image to obtain the binary image.

[0058] In some embodiments of the present application, after the image enhancement stage, the image noise is suppressed, and the image quality and image display consistency are improved. The image characteristics of the image acquisition device after occlusion are that the local large-area gray scale is concentrated and the edge texture is less, and the image characteristics of the unoccluded image are that the image gray scale span is large and the edge texture is more. Through image segmentation by region growing, it is analyzed from the image characteristic performance that the occlusion area needs to be segmented to distinguish whether it is occluded and the degree of occlusion, so the initial seed point needs to avoid the edge and texture concentrated area when the image is segmented by region growing. Compared with the conventional edge detection method, the saliency detection method of the present application can provide more comprehensive information, not only limited to the boundary of the object, but also including the part that is most likely to attract attention; and because the saliency image generated by saliency detection has a corresponding saliency value for each pixel point, which represents its relative importance in the entire scene, more context information can be retained to help understand the overall structure of the scene, while the edge detection output is a binary edge image, which only emphasizes the image boundary information and loses a large amount of internal details and background information.

[0059] In some embodiments of this application, considering the limited computing power of the cockpit-mounted chip and the high frame rate requirements of the DMS function, a local contrast saliency detection method is adopted. The core idea is to calculate the difference between the image to be detected and the blurred background, thereby highlighting local salient areas. The principle is as follows: Equation (5) in, For saliency image at location The significance value of local contrast at that location. For the image to be detected at the location The pixel value to be detected at that location. For mean-filtered image at position The blurred pixel value at that location.

[0060] In some embodiments of this application, The larger the value, the more obvious the difference between the location and the background, that is, the more significant the area.

[0061] In some embodiments of this application, the absolute value is used to ensure that the local contrast saliency value is non-negative.

[0062] In some embodiments of this application, the mean-filtered image is determined as follows: Equation (6) in, For mean-filtered image at position Blur pixel values ​​at that location To preset the neighborhood radius corresponding to the second neighborhood size, This represents the total number of pixels in the neighborhood of the corresponding mean. For the image to be detected at the location The pixel value to be detected at that location, and In respectively The range varies to traverse all pixels within the entire mean neighborhood.

[0063] In some embodiments of this application, this application pre-sets A value of 5 indicates a pixel within an 11×11 mean neighborhood.

[0064] In some embodiments of this application, by Normalization is performed to ensure that the calculated local average value is within a reasonable range.

[0065] In some embodiments of this application, the most attractive region in the image to be detected can be obtained by using the local contrast saliency detection method. The region with a larger saliency value in the saliency image is often the edge or foreground, while the region with a low saliency value is the gray-scale smooth region or background.

[0066] In some embodiments of the present application, referring to FIG. 6(a) to FIG. 6(c), FIG. 6(a) is a corresponding saliency image provided by the device in an embodiment of the present application without occlusion, FIG. 6(b) is a corresponding saliency image provided by the device in an embodiment of the present application with full occlusion, and FIG. 6(c) is a corresponding saliency image provided by the device in an embodiment of the present application with partial occlusion. As shown in FIG. 6(a) to FIG. 6(c), the corresponding saliency image is obtained by performing saliency detection on different to-be-detected images, and FIG. 6(a) is still displayed by covering the face region with a privacy color block.

[0067] In some embodiments of the present application, the saliency image is subjected to maximum intra-inter variance threshold binarization to obtain a binary image; and the binary image is divided into 9 growth regions, and at least one initial pixel point with an initial pixel value of 0 is randomly selected in each growth region as an initial seed point for region growth. Referring to FIG. 7(a) to FIG. 7(c), FIG. 7(a) is a corresponding initial seed point provided by the device in an embodiment of the present application without occlusion, FIG. 7(b) is a corresponding initial seed point provided by the device in an embodiment of the present application with full occlusion, and FIG. 7(c) is a corresponding initial seed point provided by the device in an embodiment of the present application with partial occlusion. As shown in FIG. 7(a) to FIG. 7(c), the red pixel points in the figures are the randomly selected initial seed points, and 30 initial seed points are displayed on the corresponding to-be-detected images, and FIG. 7(a) is still displayed by covering the face region with a privacy color block.

[0068] In some embodiments of the present application, the image characteristics of the occlusion region corresponding to the image acquisition device are low contrast, so the random selection points in the non-saliency value region are used as the initial seed points for region growth.

[0069] Compared with edge detection, the present application simulates the human visual attention mechanism through saliency detection, can automatically identify the region in the to-be-detected image that attracts the most attention, not only pays attention to low-level visual features such as edges or textures, but also can combine high-level semantic information to determine which region is the most salient, so as to more comprehensively describe the image content, is less affected by noise and background, and has stronger robustness.

[0070] In some embodiments of the present application, the region growing is performed according to a preset gray scale threshold and a difference between each sub-pixel value and a plurality of neighborhood pixel values in a corresponding growing neighborhood, including: determining a growing neighborhood corresponding to each initial seed point according to a preset first neighborhood size, the growing neighborhood including a plurality of neighborhood pixel points in the image to be detected; if an absolute value of a difference between a seed pixel value of an initial seed point and a neighborhood pixel value of a neighborhood pixel point is less than a preset gray scale threshold, the neighborhood pixel point is determined as a same kind seed point of the initial seed point, so as to determine a new same kind seed point according to a growing neighborhood corresponding to the same kind seed point; and determining at least one target region according to each initial seed point and the corresponding same kind seed point.

[0071] In some embodiments of the present application, the region growing starts from the initial seed point, and the image segmentation is performed on the original image with a gray scale image as a reference image to segment each region.

[0072] In some embodiments of the present application, the growing rule is determined according to the preset gray scale threshold, the region growing is performed from the initial seed point to the growing neighborhood, the neighborhood pixel points satisfying the growing rule are merged into the same target region, and the growing is continued until no new neighborhood pixel point satisfying the growing rule can be added, so that the image segmentation is performed on the image to be detected in the gray scale format with the initial seed point as a starting point and a reference image, and a region segmentation image is obtained.

[0073] In some embodiments of the present application, considering the difference between the occluded image and the non-occluded image of the image acquisition device, the initial seed point is obtained after the saliency detection, the preset first neighborhood size is set to 3, the preset gray scale threshold is set to 5, the neighborhood pixel value and the seed pixel value are defined as the same target region if an absolute value of a difference between the neighborhood pixel value and the seed pixel value is less than 5, and the growing is continued based on the neighborhood pixel point. Finally, the region growing segmentation image is obtained. Referring to FIGS. 8(a) to 8(c), FIG. 8(a) is a region segmentation image corresponding to a non-occluded device according to an embodiment of the present application, FIG. 8(b) is a region segmentation image corresponding to a fully occluded device according to an embodiment of the present application, and FIG. 8(c) is a region segmentation image corresponding to a partially occluded device according to an embodiment of the present application. As shown in FIGS. 8(a) to 8(c), the region segmentation image in FIG. 8(a) is covered with a face region by the region segmentation color block, and the actual region segmentation image can display eyebrow and eye features, and different color blocks represent different target regions.

[0074] The present application realizes step-by-step merging of adjacent pixels between pixels by region growing, has strong anti-noise capability compared with threshold segmentation or simple morphological operation, and can adapt to irregular, curved or hole-containing regions. The segmentation process can be controlled by an initial seed point, has strong controllability, does not need complex mathematical modeling, and is easy to understand and implement. Multiple initial seed points can be set, each growing region grows independently and does not interfere with each other, multiple target regions in the region segmentation image can be segmented at the same time, the requirement for computing resources is low, and the method is suitable for embedded devices or real-time system deployment.

[0075] In some embodiments of the present application, the occlusion detection result of the image acquisition device is determined according to the segmentation features of the target regions in the region segmentation image, including: if the total number of regions in the region segmentation image is less than a preset first threshold, the occlusion state in the occlusion detection result is determined as device occlusion; if the total number of regions in the region segmentation image is greater than or equal to the preset first threshold, the occlusion state in the occlusion detection result is determined as device non-occlusion.

[0076] In some embodiments of the present application, the occlusion detection result of the image acquisition device is determined according to the segmentation features of the target regions in the region segmentation image, and further includes: if the occlusion state is device occlusion, the shape features of at least one target region are determined, the shape features including circularity and / or rectangularity; if at least one of the circularity of a target region is greater than or equal to a preset second threshold, and the rectangularity of the target region is greater than or equal to a preset third threshold, the target region is determined as an occlusion region; if there is an occlusion region, the occlusion proportion is determined according to the total area of the image to be detected and the occlusion area of at least one occlusion region; wherein the occlusion detection result further includes the occlusion proportion and / or at least one occlusion region.

[0077] In some embodiments of the present application, the target regions with concentrated gray scale and small texture difference in the image to be detected are segmented out by region growing, and the occlusion detection result of the image acquisition device is determined by counting the total number of regions and the shape features of each target region in the region segmentation image. Since the image acquisition device is in a close-range occlusion state, the original image will show a virtual focus performance, and the occlusion region often shows a rectangular or elliptical shape, so the circularity and / or rectangularity in the shape features can be used as an occlusion evaluation index.

[0078] In some embodiments of the present application, the preset first threshold can be set to 10. If the total number of regions is greater than or equal to 10, the occlusion state is device non-occlusion; if the total number of regions is greater than 10, the occlusion state is device occlusion.

[0079] In some embodiments of the present application, the circularity is determined as follows: Formula (7) wherein, a roundness of the target region, an area of the target region, a perimeter of the target region.

[0080] In some embodiments of the present application, the determination of the rectangularity is as follows: Formula (8) wherein, a rectangularity of the target region, an area of the target region, a width of the target region, a height of the target region.

[0081] In some embodiments of the present application, a target region can be determined as an occlusion region through the roundness and / or the rectangularity, so that an occlusion ratio is obtained by dividing the total area of the occlusion region corresponding to the occlusion region by the total area of the image to be detected.

[0082] Compared with the black box unexplainability of the deep learning model and the need for a large amount of image data for model training. The occlusion detection scheme provided by the present application can achieve strong robustness in the occlusion application scenario, especially for scenes where the original image corresponding to the occlusion does not have diversity, and each step has a scientific explanation, the occlusion detection speed is fast, the input cost and time cost of data acquisition, labeling, acceptance, checking, training and iteration can be saved, the external dependence in the occlusion detection development process is saved, and the risk in the development process is reduced. In addition, compared with the traditional image processing method which mainly obtains the occlusion state through edge detection, the present application can also quickly determine the occlusion degree, such as the occlusion region and the occlusion ratio.

[0083] Please refer to Figure 9 , Figure 9 is a block diagram of an occlusion detection device provided in an embodiment of the present application. The device can be applied to the implementation environment shown in Figure 1 , and is specifically configured in the computer device 120. The device can also be applied to other exemplary implementation environments, and is specifically configured in other devices, and the implementation environment to which the device is applied is not limited in the present embodiment.

[0084] As shown in Figure 9 , the occlusion detection device 900 according to one embodiment of the present application includes an image acquisition module 910, a growth information determination module 920, a region growth module 930 and an occlusion detection module 940.

[0085] The image acquisition module 910 is configured to acquire an original image and pre-process the original image to obtain a to-be-detected image, the pre-processing including image enhancement and / or image graying, and the original image containing image information collected by an image collection device. The growth information determination module 920 is configured to perform saliency detection on the to-be-detected image to obtain growth information, the growth information including at least one growth region and at least one initial seed point in the growth region. The region growing module 930 is configured to perform region growing according to a preset gray scale threshold and a difference between each sub-pixel value and a plurality of neighborhood pixel values in a corresponding growth neighborhood to obtain a region segmentation image, the seed pixel value and the neighborhood pixel value being obtained based on the initial seed point and the to-be-detected image. The occlusion detection module 940 is configured to determine an occlusion detection result of the image acquisition device according to a segmentation feature of the target region in the region segmentation image.

[0086] It should be noted that the occlusion detection apparatus provided in the above embodiments and the occlusion detection method provided in the above embodiments belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiments, which will not be described here. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above, and this is not limited herein.

[0087] Embodiments of the present application also provide an electronic device, including: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the electronic device implements the occlusion detection method provided in each of the above embodiments.

[0088] Please refer to Figure 10 , Figure 10 is a structural schematic diagram of a computer system of an electronic device provided in an embodiment of the present application. Figure 10 The computer system 1000 of the electronic device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0089] As Figure 10 shown, the computer system 1000 includes a central processing unit 1001, which can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or programs loaded from a storage part 1008 to a random access memory 1003, such as performing the method in the above embodiments. In the random access memory 1003, various programs and data required for system operation are also stored. The central processing unit 1001, the read-only memory 1002, and the random access memory 1003 are connected to each other through a bus 1004. An input / output interface 1005 is also connected to the bus 1004.

[0090] The following components are connected to the input / output interface 1005: an input section 1006 including input devices such as a keyboard and mouse; an output section 1007 including output devices such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the input / output interface 1005 as necessary. A removable media 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 1010 as necessary, so that a computer program read therefrom is installed in the storage section 1008 as necessary.

[0091] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing a computer program for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 1009, and / or installed from the removable media 1011. When the computer program is executed by the central processing unit 1001, various functions defined in the system of the present application are executed.

[0092] The computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, in which the computer readable computer program is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit the program for use by or in connection with an instruction execution system, apparatus or device. The computer program contained on the computer readable medium can be transmitted in any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination of the above.

[0093] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0094] The units described in the embodiments of the present application can be implemented in the form of software, or can be implemented in the form of hardware, and the described units can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, and the software product can be stored in a nonvolatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

[0095] Another aspect of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to perform the occlusion detection method provided in each of the above embodiments. The computer-readable storage medium can be included in the electronic device described in the above embodiments, or can exist separately and not be assembled into the electronic device.

[0096] In the above embodiments, unless otherwise specified, the common objects are described by using the ordinal numbers such as "first" and "second", which only represent different instances of the same object, and do not mean that the described objects must be in a given order, whether in time, space, sequence or any other manner.

[0097] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed in the present application should still be covered by the claims of the present application.

Claims

1. A method of occlusion detection, the method comprising: The method comprises: obtaining an original image, and pre-processing the original image to obtain a to-be-detected image, the pre-processing comprising image enhancement and / or image grayscale, the original image containing image information collected by an image collection device; performing saliency detection on the to-be-detected image to obtain growth information, the growth information comprising at least one growth region and at least one initial seed point in the growth region; performing region growth according to a preset gray threshold and a difference between each sub-pixel value and a plurality of neighborhood pixel values in a corresponding growth neighborhood to obtain a region segmentation image, the seed pixel value and the neighborhood pixel value being obtained based on the initial seed point and the to-be-detected image; determining an occlusion detection result of the image collection device according to a segmentation feature of a target region in the region segmentation image.

2. The occlusion detection method of claim 1, wherein The method comprises: if a total number of regions in the region segmentation image is less than a preset first threshold, determining an occlusion state in the occlusion detection result as device occlusion; if the total number of regions in the region segmentation image is greater than or equal to the preset first threshold, determining the occlusion state in the occlusion detection result as device non-occlusion.

3. The occlusion detection method of claim 2, wherein The method further comprises: if the occlusion state is device occlusion, determining a shape feature of at least one target region, the shape feature comprising circularity and / or rectangularity; if at least one of the circularity of a target region is greater than or equal to a preset second threshold and the rectangularity of the target region is greater than or equal to a preset third threshold, determining the target region as an occlusion region; if there is an occlusion region, determining an occlusion proportion according to a total area of the to-be-detected image and an occlusion area of at least one occlusion region; wherein the occlusion detection result further comprises the occlusion proportion and / or at least one occlusion region.

4. The occlusion detection method of claim 1, wherein The method comprises: determining a growth neighborhood corresponding to each initial seed point according to a preset first neighborhood size, the growth neighborhood comprising a plurality of neighborhood pixel points in the to-be-detected image; if an absolute value of a difference between a seed pixel value of an initial seed point and a neighborhood pixel value of a neighborhood pixel point is less than the preset gray threshold, determining the neighborhood pixel point as a same-class seed point of the initial seed point, so as to determine a new same-class seed point according to a growth neighborhood corresponding to the same-class seed point; determining at least one target region according to each initial seed point and a corresponding same-class seed point.

5. The occlusion detection method according to any one of claims 1 to 4, characterized in that, The method comprises: performing local contrast saliency detection on the to-be-detected image to obtain a saliency image, and binarizing the saliency image to obtain a binary image; The binary image is divided into at least one growth region, and at least one initial pixel point with an initial pixel value of zero in each growth region is determined as an initial seed point of the corresponding growth region.

6. The occlusion detection method of claim 5, wherein The local contrast saliency detection is performed on the image to be detected to obtain a saliency image, and the saliency image is binarized to obtain a binary image, including: The mean value filtering is performed on any to-be-detected pixel value in the image to be detected according to a preset second neighborhood size to obtain a mean value filtering image; The saliency image is determined based on a difference between the mean value filtering image and the image to be detected; The saliency image is subjected to maximum inter-quantile variance threshold binarization to obtain a binary image.

7. The occlusion detection method of claim 5, wherein The original image is preprocessed to obtain an image to be detected, including: An image to be enhanced is obtained, and the image to be enhanced is an original image in a gray scale format; Gamma transformation is performed on the image to be enhanced to obtain a transformed image, and bilateral filtering is performed on the transformed image to obtain an image to be detected.

8. An occlusion detection device, characterized by The device includes: An image acquisition module is configured to acquire an original image, and pre-process the original image to obtain an image to be detected, wherein the pre-processing includes image enhancement and / or image gray scale conversion, and the original image contains image information collected by an image collection device; A growth information determination module is configured to perform saliency detection on the image to be detected to obtain growth information, wherein the growth information includes at least one growth region and at least one initial seed point in the growth region; A region growing module is configured to perform region growing according to a preset gray scale threshold and a difference between each seed pixel value and a plurality of neighborhood pixel values in a corresponding growth neighborhood to obtain a region segmentation image, wherein the seed pixel value and the neighborhood pixel value are obtained based on the initial seed point and the image to be detected; An occlusion detection module is configured to determine an occlusion detection result of the image collection device according to a segmentation feature of a target region in the region segmentation image.

9. An electronic device, comprising: The electronic device includes: One or more processors; A storage device configured to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the occlusion detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, which, when executed by a processor of a computer, causes the computer to perform the occlusion detection method according to any one of claims 1 to 7.

Images