Method, device, electronic device, storage medium, and computer program product for detecting occlusion in a camera device

The method and device improve occlusion detection in camera devices by encoding image frames based on pixel values and comparing feature encoding information, addressing the issue of inaccurate occlusion detection and enhancing driving safety.

JP2024544344AActive Publication Date: 2024-11-28SHANGHAI SENSETIME INTELLIGENT TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024538727
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-31
Filing Date
2022-10-12
Publication Date
2024-11-28
Estimated Expiration
2042-10-12

AI Technical Summary

Technical Problem

Existing camera devices in vehicles are prone to occlusion, leading to inaccurate detection of driver actions, which compromises driving safety due to the inability to accurately determine if the camera is shielded.

Method used

A method and device for detecting occlusion in camera devices by encoding image frames based on pixel values and comparing feature encoding information with predetermined encoding information to improve detection accuracy, utilizing face detection and encoding techniques.

Benefits of technology

Enhances the accuracy of occlusion detection in camera devices by ensuring accurate identification of driver presence, thereby improving driving safety by preventing misjudgments and enhancing the reliability of camera operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024544344000001_ABST
    Figure 2024544344000001_ABST
Patent Text Reader

Abstract

The present application provides a camera device occlusion detection method, device, electronic device, storage medium, and computer program product, the camera device occlusion detection method includes: acquiring video data of a driving area of ​​a vehicle through a camera device; performing face detection on a current image frame in the video data; if a face is not detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature coding information of the current image frame; and determining whether the camera device is occluded based on the feature coding information of the current image frame and predetermined feature coding information, where the predetermined feature coding information includes feature coding information of an image frame including a face in the video data. The present application can improve the occlusion detection accuracy of the camera device.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is filed based on a Chinese patent application bearing application number 202111668662.2 and entitled "Method, apparatus, electronic device, storage medium, and computer program product for detecting occlusion in a camera device" filed with the China Patent Office on December 31, 2021, and claims priority to the Chinese patent application, the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD The present application relates to the field of image processing technology, and in particular to a method, device, electronic device, storage medium, and computer program product for detecting occlusion in a camera device. [Background technology]

[0003] As people's living standards improve, vehicles have become an indispensable means of transportation in people's lives. Camera devices inside the vehicle can restrict the driver's driving behavior, thereby reducing the possibility of traffic accidents and helping to improve driving safety.

[0004] However, in the actual use process, the camera device may be blocked, and when the camera device is blocked, the driver's behavior cannot be accurately detected. Therefore, it is particularly important to detect whether the camera device is blocked and to improve the detection accuracy of the blocking of the camera device. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments of the present application provide at least a method, device, electronic device, storage medium, and computer program product for detecting occlusion in a camera device, which can not only realize occlusion detection for a camera device, but also improve the accuracy of the detection. [Means for solving the problem]

[0006] An embodiment of the present application provides a method for detecting occlusion in a camera device, the method comprising: Obtaining video data of a driving area of ​​a vehicle via a camera device; performing face detection on a current image frame in the video data, and if no face is detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; determining whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, the predetermined feature coding information including feature coding information of an image frame that includes a face in the video data.

[0007] An embodiment of the present application provides an apparatus for detecting occlusion of a camera apparatus, the apparatus comprising: a video capture module configured to capture video data of a driving area of ​​the vehicle via a camera device; a face detection module configured to perform face detection on a current image frame in the video data, and if no face is detected, to encode the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; and an occlusion determination module configured to determine whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, the predetermined feature coding information including feature coding information of an image frame including a face in the video data.

[0008] An embodiment of the present application provides an electronic device comprising a processor, a memory, and a bus, the memory storing machine-readable instructions executable by the processor, and when the electronic device is operating, the processor and the memory communicate via the bus, and the machine-readable instructions cause the processor to execute the camera device occlusion detection method described in any of the above embodiments.

[0009] An embodiment of the present application provides a computer-readable storage medium having a computer program stored therein, the computer program causing a processor to execute the camera apparatus occlusion detection method described in any of the above embodiments.

[0010] An embodiment of the present application further provides a computer program product including a computer readable storage medium having program code stored thereon, the instructions contained in the program code causing a processor of a computing device to perform the above method. Effect of the Invention

[0011] According to the camera device occlusion detection method, device, electronic device, storage medium, and computer program product of the present application, if a face is not identified, the current image frame is encoded based on pixel values ​​in the current image frame, feature encoding information of the current image frame is obtained, and based on the feature encoding information of the current image frame and predetermined feature encoding information, it is determined whether the camera device is occluded or not; in this way, if a face is not identified, further judgments can be made on the image, thereby improving the accuracy of the judgment.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. [Brief description of the drawings]

[0013] [Figure 1] 1 illustrates an exemplary flowchart of a method for occlusion detection in a camera device according to an embodiment of the present application. [Diagram 2] 2 illustrates an exemplary flowchart of a method for determining feature encoding information of a current image frame according to an embodiment of the present application; [Diagram 3] 1 illustrates another exemplary flowchart of a method for occlusion detection in a camera apparatus, according to an embodiment of the present application. [Figure 4]1 illustrates an exemplary flowchart of a method for outputting presentation information according to an embodiment of the present application. [Diagram 5] 1 illustrates an exemplary flowchart of yet another camera apparatus occlusion detection method according to an embodiment of the present application. [Figure 6] 1 illustrates an exemplary structural diagram of an occlusion detection device of a camera device according to an embodiment of the present application; [Figure 7] FIG. 2 illustrates another exemplary structural diagram of an occlusion detection device of a camera device according to an embodiment of the present application; [Figure 8] 1 shows a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] In order to more clearly explain the technical solutions of the embodiments of the present application, the drawings necessary in the embodiments are briefly introduced above. The drawings herein are incorporated into and form a part of this specification, and these drawings are for illustrating the embodiments of the present application, and are used together with this specification to explain the technical solutions of the embodiments of the present application. The drawings only show a part of the embodiments of the present application, and therefore should not be considered as limiting the scope of the present application, and it should be understood that those skilled in the art can also obtain other related drawings according to these drawings without creative efforts.

[0015] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings of the embodiments of the present application; obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Usually, the components of the embodiments of the present application described and shown in the drawings of the present application can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided in the drawings below is not intended to limit the scope of the present application, but to illustrate optional embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts shall be included in the protection scope of the present application.

[0016] It should be noted that like symbols and letters represent like items in subsequent drawings, and thus, once any item is defined in one drawing, there is no need to further define and explain it in subsequent drawings.

[0017] The term "and / or" in this specification describes an association relationship and indicates that a three-way relationship may exist, for example, A and / or B may indicate three cases: A is present independently, A and B are present, and B is present independently. In addition, the term "at least one" in this specification indicates any one of a plurality, or any combination of at least two of a plurality, for example, including at least one of A, B, and C indicates including any one or more elements selected from the set consisting of A, B, and C.

[0018] With the improvement of people's living standards, vehicles have become an indispensable means of transportation in people's lives, and a camera device in a vehicle can restrict the driving behavior of a driver, thereby reducing the possibility of traffic accidents and improving driving safety. However, in the actual use process, the camera device may be blocked, and when the camera device is blocked, the driver's behavior cannot be accurately detected.

[0019] According to research, the prior art has a method for detecting whether a camera device is occluded, such as a method for determining whether a camera device is occluded by using a face detection method, but this method is prone to misjudgment or missed judgment when the brightness of the entire image is high or low.

[0020] In view of the above research, the present application provides a method for detecting occlusion of a camera device, the method including: obtaining video data of a driving area of ​​a vehicle via a camera device; performing face detection on a current image frame in the video data; if a face is not detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; and determining whether the camera device is occluded based on the feature encoding information of the current image frame and predetermined feature encoding information, where the predetermined feature encoding information includes feature encoding information of an image frame including a face in the video data.

[0021] In an embodiment of the present application, if a face is not identified, the current image frame is encoded based on pixel values ​​in the current image frame, feature encoding information of the current image frame is obtained, and based on the feature encoding information of the current image frame and predetermined feature encoding information, it is determined whether the camera device is occluded or not; in this way, if a face is not identified, further judgments can be made on the image, thereby improving the accuracy of the judgment.

[0022] Hereinafter, the method for detecting occlusion of a camera device according to an embodiment of the present application will be described in detail with reference to the drawings. Referring to Fig. 1, an exemplary flowchart of the method for detecting occlusion of a camera device according to an embodiment of the present application is shown, and the method for detecting occlusion of a camera device includes the following steps S101 to S103.

[0023] In step S101, video data of the driving area of ​​the vehicle is obtained via a camera device.

[0024] Here, the driving area refers to an area in the vehicle where the driver controls the driving of the vehicle. When the vehicle is traveling, the driver is usually located in the driving area of ​​the vehicle, and the terminal device can obtain video data of the driving area of ​​the vehicle.

[0025] Video data refers to a continuous image sequence, which essentially consists of a series of consecutive images, where an image frame is the smallest visual unit that constitutes a video, and is one static image. A dynamic video is obtained by synthesizing a time-continuous image frame sequence. In this embodiment, it is necessary to extract image frames in the video data to facilitate subsequent detection and identification.

[0026] For example, video data typically includes a large number of frames of images per second (e.g., 24 frames of images per second), so frame extraction can be performed in the process of extracting image frames from the video data. Here, frame extraction refers to performing frame extraction at intervals of a predetermined number of frames, for example, one frame of image can be extracted every 20 frames. Frame extraction can also be performed at a predetermined time interval, and images can be extracted every 10 milliseconds (ms).

[0027] It should be mentioned that the predetermined frame number interval and the predetermined time interval can be set according to actual requirements, and the present application is not limited thereto.

[0028] Optionally, the video data of the driving area is captured by a camera device installed in the vehicle, and the terminal device obtains the video data captured by the camera device. That is, in the embodiment of the present application, the execution subject of the camera device occlusion detection method may be a terminal device, where the terminal device includes but is not limited to an in-vehicle device, a wearable device, a user terminal, a handheld device, etc.

[0029] In other embodiments, the entity performing the occlusion detection method of the camera device may be a server, where the server may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms.

[0030] In some possible embodiments, the camera apparatus occlusion detection method may also be implemented by a processor invoking computer readable instructions stored in a memory.

[0031] In step S102, face detection is performed on a current image frame in the video data, and if no face is detected, the current image frame is encoded based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame.

[0032] As can be seen, after performing the image frame extraction process on the video data, a number of image frames are obtained, where the current image frame refers to the image frame that currently needs to perform the detection and identification process, and the image frame that is earlier in timing than the current image frame in the video data is called a pre-order image frame, and the image frame that is later in timing than the current image frame is called a post-order image frame.

[0033] Illustratively, after extracting the above multi-frame image, face detection can be performed on the extracted image to determine whether there is a face in the current image frame.In some embodiments, if a face is detected in the current image frame, it means that the driver is in the driving area and the camera device is not blocked; if no face is detected in the current image frame, further judgment is required for the current image frame to improve the accuracy of judgment.

[0034] In this embodiment, if no face is identified in the current image frame, further encode the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame. In some embodiments, referring to FIG. 2, encoding the current image frame based on pixel values ​​in the current image frame may include the following steps S1021 and S1022.

[0035] In step S1021, a reference pixel threshold value of the current image frame is determined.

[0036] In this embodiment, the reference pixel threshold is the average pixel value of the current image frame, so that the feature coding information obtained based on the reference pixel threshold has a unique correspondence with the current image frame, which can further improve the accuracy of the judgment. Of course, in other embodiments, the reference pixel threshold can be set according to the actual situation, for example, the corresponding reference pixel threshold can be determined according to the overall brightness of the current image, rather than the average pixel value of the current image frame.

[0037] For example, in order to improve the accuracy of determining the reference pixel threshold, a noise reduction process can be performed on the current image frame before determining the reference pixel threshold for the current image frame, thereby reducing the effects of imaging noise.

[0038] In some embodiments, a Gaussian smoothing algorithm may be employed to perform the noise reduction process on the current image frame. In other embodiments, a median filtering algorithm or a mean filtering algorithm may be used to perform the noise reduction process, and the present application is not limited thereto.

[0039] In step S1022, the pixel value of each pixel point of the current image frame is sequentially compared with the reference pixel threshold, and pixel points greater than the reference pixel threshold are coded as 1, and pixel points less than or equal to the reference pixel threshold are coded as 0, thereby obtaining feature coding information of the current image frame.

[0040] Exemplarily, after determining the reference pixel threshold, the current image frame can be encoded according to the reference pixel threshold, and the encoding process may obtain the feature encoding information by encoding pixel points larger than the reference pixel threshold to 1 and encoding pixel points equal to or smaller than the reference pixel threshold to 0. The feature encoding information may be a two-dimensional array or a two-dimensional matrix having the same size as the current image frame to be encoded. For example, when the average pixel of the current image frame is 100, if the pixel value of a pixel point in the current image frame is 123, the pixel point is encoded to 1, and if the pixel value of a pixel point in the current image frame is 80, the pixel point is encoded to 0.

[0041] Optionally, in order to reduce the dimension of the feature encoding information and reduce the amount of calculation, after encoding each pixel point in the current image frame, the codes of each pixel point are serially concatenated in sequence according to a certain scanning order (e.g., scanning rows and then scanning columns) to form a one-dimensional array or vector, which can be used as the feature encoding information of the image frame.

[0042] In step S103, it is determined whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, where the predetermined feature coding information includes feature coding information of an image frame that includes a face in the video data.

[0043] In this embodiment, the predetermined feature coding information includes feature coding information of an image frame including a face in the video data, i.e. the predetermined feature coding information is feature coding information of a previous image frame of the current image frame, and the previous image frame includes a face, i.e. an image including a face captured after a camera device is activated is used as a basis for determining whether an occlusion of the camera device occurs in a subsequent image frame.

[0044] As can be understood, the camera device is powered on and operates after the vehicle is started, and begins to capture the driving area of ​​the vehicle to obtain video data; if the first frame image obtained based on the video data does not include a face, the pre-stored feature encoding information including a face can be used as the predetermined feature encoding information; if the first frame image includes a face, the feature encoding information of the first frame image is used as the predetermined feature encoding information.

[0045] It should be noted that, when it is determined based on the video data that a face is included in a consecutive multi-frame image, the predetermined feature coding information can be updated, i.e., the subsequent feature coding information including the face is used to replace the feature coding information of the first frame image, for example, it can be updated once at a predetermined time interval, the predetermined time interval is not limited, and can be 1 minute, 2 minutes, etc.

[0046] Exemplarily, the feature coding information of the current image frame may be compared with the predefined feature coding information to determine whether the camera device is occluded. In some embodiments, a Hamming distance between the feature coding information of the current image frame and the predefined feature coding information may be calculated, and if the Hamming distance between the feature coding information of the current image frame and the predefined feature coding information is greater than a predefined threshold, it may be determined that the camera device is occluded.

[0047] In an embodiment of the present application, if a face is not identified, the current image frame is encoded based on pixel values ​​of the current image frame to obtain feature encoding information of the current image frame, and based on the feature encoding information of the current image frame and predetermined feature encoding information, it is determined whether the camera device is occluded or not. In this way, if a face is not identified, further judgments can be made on the image, which can improve the accuracy of the judgment, in particular, making it more accurate to judge whether the driver has left the seat.

[0048] Referring to FIG. 3, another exemplary flowchart of an occlusion detection method for a camera device according to an embodiment of the present application includes the following steps S201 to S205.

[0049] In step S201, video data of the driving area of ​​the vehicle is obtained via a camera device.

[0050] Here, this step is similar to step S101 above.

[0051] In step S202, face detection is performed on a current image frame in the video data, and if no face is detected, the current image frame is encoded based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame.

[0052] Here, this step is similar to step S102 above.

[0053] In step S203, it is determined whether the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold. If it is greater than the predetermined threshold, execute step S205; if not, execute step S204.

[0054] Here, this step is similar to step S103 above.

[0055] In step S204, it is determined whether the camera device is occluded based on the pixel distribution histogram of the current image frame.

[0056] For example, if the Hamming distance between the feature encoding information of the current image frame and the specified feature encoding information is below a specified threshold, it means that a face may exist in the current image frame, but may not be identified in the process of face detection due to some specific causes (such as dark ambient light), therefore, in order to further improve the accuracy of the judgment, the current image frame needs to be judged in another way to further determine whether the camera device is occluded.

[0057] In an embodiment of the present application, if the Hamming distance between the feature coding information of the current image frame and the specified feature coding information is less than or equal to a specified threshold, a pixel distribution histogram of the current image frame is determined, and if the pixel distribution proportion of a specified interval in the pixel distribution histogram is greater than a specified proportion threshold, it is determined that the camera device is occluded.

[0058] In some embodiments, the distribution of pixel values ​​of each pixel point in the current image frame can be statistically obtained to obtain a pixel distribution histogram of the current image frame. For example, the distribution of pixel values ​​of each pixel point in the current image frame can be statistically obtained according to a plurality of pixel intervals previously divided, and the plurality of pixel intervals may be [0-19], [20-80], [81-126], and [127-255], etc. It can be understood that the plurality of pixel intervals shown in this embodiment are merely examples, and in other embodiments, the plurality of pixel intervals may be divided according to other requirements.

[0059] In some embodiments, the predetermined interval can be obtained by conducting a large amount of testing based on the actual usage environment of the camera device, for example, the histogram of an image of a vehicle driving area scene can be statistically obtained, and it is determined that there is a relatively clear boundary between the proportion of pixels in the pixel interval [20~80] in a normal image captured when the camera device is not obstructed and the proportion of pixels in the pixel interval [20~80] in an image captured when the camera device is obstructed, and therefore the predetermined interval can be set to the pixel interval [20~80].

[0060] Furthermore, the greater the percentage of the total pixels that have low pixel values ​​(e.g., less than 20) or greater than the high brightness threshold, the less likely it is that there is valid content in the image, and thus, in other embodiments, the specified interval may be a low pixel value interval or a high pixel value interval, for example, a pixel interval [0 to 20] or a pixel interval [130 to 255], and the present application is not limited thereto.

[0061] In this embodiment, the specified interval is one of the multiple pixel intervals, for example, the specified interval is [20~80], and if the number of pixel points within the specified interval [20~80] is large and greater than a specified percentage threshold, it can be determined that the camera device is occluded.

[0062] In step S205, it is determined that the camera device is occluded.

[0063] In step S206, the presentation information is output.

[0064] For example, after determining that the camera device is occluded, presentation information may be output to inform the driver to process the occluded camera device so that the camera device can normally capture an image of the driver, thereby helping to improve driving safety.

[0065] Here, the presentation information includes, but is not limited to, audio presentation information, graphic presentation information, lighting presentation information, etc. For example, audio presentation information such as "The camera device is blocked. Please check" can be output.

[0066] In some embodiments, in order to reduce the impact on people inside the vehicle of frequent output of presentation information due to momentary occlusion, as shown in FIG. 4, step S206 of outputting the above-mentioned presentation information can be performed in the manner of the following steps S2061 and S2062.

[0067] In step S2061, a duration of occlusion of the camera device is determined based on a detection result of the camera device for each frame image in the video data.

[0068] In step S2062, when the sustained occlusion time reaches a predetermined time, the presentation information is output.

[0069] In this embodiment, if it is determined based on the current image frame that the camera device is occluded, it is necessary to determine whether the judgment result of the image of the subsequent frame is also occluded, and if it is determined based on the image of at least one frame after that that the camera device is not occluded, it means that the current occlusion is merely a phenomenon that appears and disappears instantaneously, and in this case, the presentation information is not output. If the judgment result of the image of at least one frame after that is also that the camera device is occluded, that is, if it is determined by each of the consecutive multi-frame images that the camera device is occluded, it means that the camera device continues to be occluded at this time, which is an intentional occlusion and not an accidental occlusion, and when the continuous occlusion time reaches a predetermined time (for example, 5 seconds), the presentation information is output and presented.

[0070] Referring to FIG. 5, another exemplary flowchart of an occlusion detection method for a camera apparatus according to an embodiment of the present application is shown, which includes the following steps S301 to S310.

[0071] In step S301, video data of the driving area of ​​the vehicle is obtained via a camera device.

[0072] Here, this step is similar to step S101 above.

[0073] In step S302, face detection is performed on a current image frame in the video data, and if no face is detected, the current image frame is encoded based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame.

[0074] Here, this step is similar to step S102 above.

[0075] In step S303, it is determined whether the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold, and if it is greater than the predetermined threshold, execute step S308; if not, execute step S304.

[0076] For example, if the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold, it means that the current image frame does not contain a face, in which case, execute step S308 and determine that the camera device is occluded. If the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is less than or equal to a predetermined threshold, it means that the current image frame may contain a face, and further judgment is required, so execute step S304.

[0077] In step S304, a pixel distribution histogram of the current image frame is determined.

[0078] Here, this step is similar to step S204 above.

[0079] In step S305, it is determined whether the pixel distribution ratio in a predetermined section in the pixel distribution histogram is greater than a predetermined ratio threshold. If it is greater than the predetermined ratio threshold, step S308 is executed; if not, step S306 is executed.

[0080] For example, if the pixel distribution ratio of a predetermined section in the pixel distribution histogram is greater than the predetermined ratio threshold, it means that the invalid content ratio of the current image frame is large, in this case, execute step S308 to determine that the camera device is occluded; if the pixel distribution ratio of a predetermined section in the pixel distribution histogram is equal to or less than the predetermined ratio threshold, it means that the valid content ratio of the current image frame is normal, in this case, execute step S306 to make further judgment.

[0081] In step S306, a maximum connected region of the current image frame is determined.

[0082] In step S307, it is determined whether the area of ​​the maximum connected region is greater than a predetermined area threshold. If it is greater than the predetermined area threshold, step S308 is executed; if not, step S310 is executed.

[0083] Here, the maximum connected region refers to an image region composed of adjacent pixel points that have the same pixel value or all pixel values ​​within a certain error, and the maximum connected region is a closed region. For example, the predetermined area threshold can be set to an area that occupies 60% of the entire image, and if the maximum connected region is greater than the predetermined area threshold, execute step S308 to determine that the camera device is occluded, and if the maximum connected region is equal to or smaller than the predetermined area threshold, execute step S310 to determine that the camera device is not occluded.

[0084] As can be appreciated, in order to further improve the accuracy of the determination and reduce misjudgments due to sudden changes (such as misjudgments due to momentary occlusion), in some embodiments, if it is determined that the area of ​​the maximum connected region of the current image frame is greater than the predetermined area threshold, the following steps (1) to (3) can be further performed.

[0085] (1) determining a maximum connected region of images in at least one of the image frames preceding and succeeding the current image frame;

[0086] (2) determining an average area of ​​the largest connected region of the current image frame and the at least one image frame;

[0087] (3) determining that the camera device is occluded if an average value of an area of ​​a maximum connected region of the current image frame and the at least one frame of images is greater than the predetermined area threshold;

[0088] In this manner, when it is determined that the area of ​​the maximum connected region of the current image frame is greater than a predetermined area threshold, the average value of the areas of the maximum connected regions of the images in the multiframes before and after the current image frame is further determined, thereby reducing the incidence of erroneous judgments due to an object appearing and disappearing in an instant (e.g., passing in front of a camera device in an instant) and further improving the accuracy of judgment.

[0089] In step S308, it is determined that the camera device is occluded.

[0090] In step S309, the presentation information is output.

[0091] Here, this step is similar to step S206 above.

[0092] In step S310, it is determined that the camera device is not occluded.

[0093] As can be understood by those skilled in the art, in the methods in the above specific embodiments, the described order of each step does not imply a strict execution order, and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined by its function and internal logic.

[0094] Based on the same technical concept, the embodiments of the present application further provide an occlusion detection device for a camera device corresponding to the occlusion detection method for a camera device, and the principle by which the device in the embodiments of the present application solves the problem is similar to the occlusion detection method for a camera device in the above embodiments of the present application, so the implementation of the device can refer to the implementation of the method.

[0095] Referring to FIG. 6, a schematic diagram of an occlusion detection device 500 of a camera device according to an embodiment of the present application is shown, which includes a video capture module 501, a face detection module 502, The video acquisition module 501 is configured to acquire video data of a driving area of ​​the vehicle through a camera device; the face detection module 502 is configured to perform face detection on a current image frame in the video data, and if no face is detected, encode the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; The occlusion determination module 503 is configured to determine whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, where the predetermined feature coding information includes feature coding information of an image frame that includes a face in the video data.

[0096] In one possible embodiment, the face detection module 502 further comprises: determining a reference pixel threshold for the current image frame; The method is configured to sequentially compare the pixel value of each pixel point of the current image frame with the reference pixel threshold, encode the pixel points that are greater than the reference pixel threshold as 1, and encode the pixel points that are less than or equal to the reference pixel threshold as 0, thereby obtaining feature encoding information of the current image frame.

[0097] In one possible embodiment, the reference pixel threshold is the average pixel value of the current image frame.

[0098] In one possible embodiment, the occlusion determination module 503 further comprises: The camera arrangement is configured to determine that the camera arrangement is occluded if a Hamming distance between the feature coding of the current image frame and the predetermined feature coding is greater than a predetermined threshold.

[0099] In one possible embodiment, the occlusion determination module 503 further comprises: if the Hamming distance is less than or equal to the predetermined threshold, determining a pixel distribution histogram for the current image frame; The camera arrangement is configured to determine whether or not the camera arrangement is occluded based on a pixel distribution histogram of the current image frame.

[0100] In one possible embodiment, the occlusion determination module 503 further comprises: The camera device is configured to determine that the camera device is occluded if a ratio of pixel distribution in a predetermined interval in the pixel distribution histogram is greater than a predetermined ratio threshold.

[0101] In one possible embodiment, the occlusion determination module 503 further comprises: determining a maximum connected region of the current image frame when a pixel distribution ratio of a predetermined section in the pixel distribution histogram is equal to or less than the predetermined ratio threshold; If the area of ​​the largest connected region is greater than a predefined area threshold, then it is configured to determine that the camera arrangement is occluded.

[0102] Referring to FIG. 7, in one possible embodiment, the device further comprises: The camera device includes an information output module 504 configured to output presentation information if it is determined that the camera device is occluded.

[0103] In one possible embodiment, the information output module 504 further comprises: determining a duration of occlusion of the camera device based on a detection result of the camera device for each frame image in the video data; When the sustained occlusion time reaches a predetermined time, the presentation information is output.

[0104] The description of the processing process of each module in the apparatus and the interaction process between each module can be referred to the relevant description in the above method embodiment, and will not be described in detail here.

[0105] Based on the same technical concept, an embodiment of the present application further provides an electronic device. Referring to FIG. 8, an exemplary structural diagram of an electronic device 700 according to an embodiment of the present application is shown, and the electronic device 700 includes a processor 701, a memory 702, and a bus 703. Here, the memory 702 is configured to store execution instructions, and includes an internal memory 7021 and an external memory 7022, where the internal memory 7021 is also called internal storage, and is configured to temporarily store operation data of the processor 701 and data to be exchanged with an external memory 7022 such as a hard disk, and the processor 701 exchanges data with the external memory 7022 through the internal memory 7021.

[0106] In an embodiment of the present application, the memory 702 is configured to store application program code for executing the technical solution of the present application, and the execution thereof is controlled by the processor 701. That is, when the electronic device 700 operates, the processor 701 and the memory 702 communicate with each other via the bus 703, so that the processor 701 executes the application program code stored in the memory 702 to perform the method according to any of the above-mentioned embodiments.

[0107] Here, the memory 702 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EPROM), or the like.

[0108] The processor 701 may be an integrated circuit chip with signal processing capabilities. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and may implement or execute each method, step, and logic block diagram disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0109] As can be appreciated, the structure shown in the embodiments of the present application does not constitute a limitation on the electronic device 700. In some other embodiments of the present application, the electronic device 700 may include more or fewer components than those shown, may combine certain components, may separate certain components, or may have a different component arrangement. The components shown may be implemented by hardware, software, or a combination of software and hardware.

[0110] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, the computer program causing a processor to execute the steps of the camera device occlusion detection method in the above method embodiment, where the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0111] An embodiment of the present application further provides a computer program product, the computer program product including a program code, and instructions included in the program code are used to perform steps of the occlusion detection method of the camera device in the above method embodiment, and reference can be made to the above method embodiment.

[0112] Here, the above-mentioned computer program product can be realized by hardware, software, or a combination thereof. In one exemplary embodiment, the computer program product is embodied as a computer storage medium, and in another exemplary embodiment, the computer program product is embodied as a software product such as a Software Development Kit (SDK).

[0113] As can be understood by those skilled in the art, for convenience and conciseness of description, the operation processes of the above systems and devices can refer to the corresponding processes in the above method embodiments. It should be understood that in some embodiments provided in the present application, the disclosed systems, devices and methods can be realized in other manners. The above-described device embodiments are illustrative, for example, the division of the units is a division of logical functions, and in actual implementation, other division manners can be adopted, for example, multiple units or components may be combined or integrated into another system, and some features thereof may be ignored or not implemented. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections can be realized using some communication interfaces, and the indirect couplings or communication connections between the devices or units may be electrical, mechanical, or other forms.

[0114] The units described as separate parts may or may not be physically separated, and the parts shown as units may or may not be physical units, located in one place or distributed across multiple network units. According to actual needs, some or all of the units therein may be selected to achieve the objective of the technical solution in this embodiment.

[0115] Furthermore, each functional unit in each embodiment of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0116] When the function is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the essential part of the technical solution of the present application, i.e., the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in one storage medium and includes several instructions for causing one computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0117] When the technical solution of the present application involves personal information, the product to which the technical solution of the present application is applied will clearly inform the individual of the rules for processing the personal information before processing the personal information, and the individual's own consent will be obtained. When the technical solution of the present application involves sensitive personal information, the product to which the technical solution of the present application is applied will clearly inform the individual of the rules for processing the personal information before processing the sensitive personal information, and the requirement of "explicit consent" will also be met. For example, a personal information collection device such as a camera will have a clear and vivid sign installed to inform the individual that the personal information collection range has been entered and that the personal information will be collected, and if the individual voluntarily enters the collection range, it will be deemed that the individual has consented to the collection of personal information. Alternatively, when the personal information processing device uses an explicit sign / information to inform the individual of the rules for processing the personal information, the individual's permission will be obtained by a method such as pop-up information or voluntarily uploading personal information, and the rules for processing the personal information may include information such as the personal information controller, the purpose of personal information processing, the processing method, and the type of personal information to be processed.

[0118] Finally, it should be explained that the above examples are only specific embodiments of the present application to explain the technical solutions of the present application, and are not intended to limit the present application, and the protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the above examples, it should be understood by those skilled in the art that any technical person familiar with the present technical field can easily think of modifying or altering the technical solutions described in the above examples within the technical scope disclosed in the present application, or equivalently replace some of the technical features thereof, and these modifications, alterations or substitutions do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the examples of the present application, and should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the claims.

Claims

1. A method for detecting occlusion in a camera device, comprising: Obtaining video data of a driving area of ​​a vehicle via a camera device; performing face detection on a current image frame in the video data, and if no face is detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; A method for detecting occlusion of a camera device, comprising: determining whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, the predetermined feature coding information including feature coding information of an image frame including a face in the video data.

2. encoding the current image frame based on pixel values ​​in the current image frame, determining a reference pixel threshold for the current image frame; Sequentially comparing pixel values ​​of each pixel point of the current image frame with the reference pixel threshold, encoding pixel points that are greater than the reference pixel threshold as 1, and encoding pixel points that are equal to or less than the reference pixel threshold as 0, to obtain feature encoding information of the current image frame. The method for detecting occlusion in a camera device according to claim 1 .

3. the reference pixel threshold is a mean pixel value of the current image frame; The method for detecting occlusion of a camera device according to claim 2.

4. determining whether the camera arrangement is occluded based on feature encoding information of the current image frame and the predetermined feature encoding information; determining that the camera arrangement is occluded if a Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold. The method for detecting occlusion of a camera device according to any one of claims 1 to 3.

5. The method for detecting occlusion of a camera device includes the steps of: if the Hamming distance is less than or equal to the predetermined threshold, determining a pixel distribution histogram for the current image frame; determining whether the camera device is occluded based on a pixel distribution histogram of the current image frame. The method for detecting occlusion of a camera device according to claim 4.

6. Determining whether the camera device is occluded based on a pixel distribution histogram of the current image frame includes: determining that the camera device is occluded when a ratio of pixel distribution in a predetermined section in the pixel distribution histogram is greater than a predetermined ratio threshold; The method for detecting occlusion of a camera device according to claim 5.

7. The method for detecting occlusion of a camera device includes the steps of: determining a maximum connected region of the current image frame when a pixel distribution ratio of a predetermined section in the pixel distribution histogram is equal to or less than the predetermined ratio threshold; determining that the camera device is occluded if an area of ​​the largest connected region is greater than a predetermined area threshold. The method for detecting occlusion of a camera device according to claim 5 or 6.

8. The method for detecting occlusion of a camera device includes the steps of: and outputting presentation information when the camera device is determined to be occluded. The method for detecting occlusion of a camera device according to any one of claims 1 to 7.

9. outputting presentation information when it is determined that the camera device is occluded, determining a duration of occlusion of the camera device based on a detection result of the camera device for each frame image in the video data; outputting the presentation information when the sustained occlusion time reaches a predetermined time. The method for detecting occlusion of a camera device according to claim 8.

10. An occlusion detection device for a camera device, comprising: a video capture module configured to capture video data of a driving area of ​​the vehicle via a camera device; a face detection module configured to perform face detection on a current image frame in the video data, and if no face is detected, to encode the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; an occlusion detection device for the camera device, comprising: an occlusion determination module configured to determine whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, the predetermined feature coding information including feature coding information of an image frame including a face in the video data;

11. The face detection module is further configured to determine a reference pixel threshold of the current image frame, sequentially compare a pixel value of each pixel point of the current image frame with the reference pixel threshold, and encode pixel points that are greater than the reference pixel threshold as 1 and encode pixel points that are less than or equal to the reference pixel threshold as 0 to obtain feature encoding information of the current image frame. The occlusion detection device for a camera device according to claim 10.

12. the reference pixel threshold is a mean pixel value of the current image frame; The occlusion detection device for a camera device according to claim 11.

13. the occlusion determination module is further configured to determine that the camera arrangement is occluded if a Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold. The occlusion detection device for a camera device according to any one of claims 10 to 12.

14. The occlusion determination module further comprises: if the Hamming distance is less than or equal to the predetermined threshold, determining a pixel distribution histogram of the current image frame; and determining whether the camera arrangement is occluded based on the pixel distribution histogram of the current image frame. The occlusion detection device for a camera device according to claim 13.

15. The occlusion determination module is further configured to determine that the camera device is occluded when a ratio of a pixel distribution in a predetermined interval in the pixel distribution histogram is greater than a predetermined ratio threshold. The occlusion detection device for a camera device according to claim 14.

16. The occlusion determination module further comprises: if a ratio of pixel distribution in a predetermined section in the pixel distribution histogram is equal to or less than the predetermined ratio threshold, determining a maximum connected region of the current image frame, and if an area of ​​the maximum connected region is greater than a predetermined area threshold, determining that the camera device is occluded. The occlusion detection device for a camera device according to claim 13 or 14.

17. the occlusion detection device of the camera device further comprises an information output module configured to output presentation information when it is determined that the camera device is occluded; An occlusion detection device for a camera apparatus according to any one of claims 10 to 16.

18. The information output module is further configured to determine a duration of occlusion of the camera device based on a detection result of the camera device for each frame image in the video data, and output the presentation information when the duration of occlusion reaches a predetermined time. The occlusion detection device for a camera device according to claim 17.

19. An electronic device comprising a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor, and when the electronic device is operated, the processor and the memory communicate via the bus, and the machine-readable instructions cause the processor to execute the camera device occlusion detection method described in any one of claims 1 to 9.

20. A computer-readable storage medium having stored thereon a computer program for causing a processor to execute the method for detecting occlusion in a camera apparatus according to any one of claims 1 to 9.

21. 10. A computer program product comprising a computer readable storage medium having program code stored thereon, the program code including instructions causing a processor of a computing device to execute the camera apparatus occlusion detection method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Method and device for detecting shielding of camera and electronic equipment

    CN110557628A

  • Image detection method and system and camera shielding monitoring method and system

    CN112261402A

  • Imaging apparatus

    JP2019159346A