Method, device, electronic device, storage medium, and computer program for detecting occlusion of a camera device

Abstract

The present application provides a camera device occlusion detection method, device, electronic device, storage medium, and computer program product, the camera device occlusion detection method includes: acquiring video data of a driving area of ​​a vehicle through a camera device; performing face detection on a current image frame in the video data; if a face is not detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature coding information of the current image frame; and determining whether the camera device is occluded based on the feature coding information of the current image frame and predetermined feature coding information, where the predetermined feature coding information includes feature coding information of an image frame including a face in the video data. The present application can improve the occlusion detection accuracy of the camera device.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is based on a Chinese patent application filed with the China Patent Office on December 31, 2021, bearing application number 202111668662.2 and entitled "Method, device, electronic device, storage medium, and computer program product for detecting occlusion in a camera device," and claims priority to that Chinese patent application, the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD Embodiments of the present application relate to the field of image processing technology, and in particular to a method, device, electronic equipment, storage medium, and computer program product for detecting occlusion in a camera device. [Background technology]

[0003] As people's living standards improve, vehicles have become an essential means of transportation in people's lives. Camera devices inside the vehicle can restrict the driver's driving behavior, thereby reducing the possibility of traffic accidents and helping to improve driving safety.

[0004] However, in actual use, the camera device may be blocked, and when the camera device is blocked, the driver's behavior cannot be accurately detected. Therefore, it is particularly important to detect whether the camera device is blocked and to improve the detection accuracy of the camera device being blocked. Summary of the Invention [Problem to be solved by the invention]

[0005] The embodiments of the present application provide at least a method, device, electronic device, storage medium, and computer program product for detecting occlusion in a camera device, which can not only realize occlusion detection for a camera device but also improve the accuracy of the detection. [Means for solving the problem]

[0006] An embodiment of the present application provides a method for detecting occlusion in a camera device, the method comprising: acquiring video data of a driving area of ​​a vehicle via a camera device; performing face detection on a current image frame in the video data, and if no face is detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; determining whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, wherein the predetermined feature coding information includes feature coding information of an image frame that includes a face in the video data.

[0007] An embodiment of the present application provides an occlusion detection device for a camera device, the device comprising: a video acquisition module configured to acquire video data of a driving area of ​​the vehicle via a camera device; a face detection module configured to perform face detection on a current image frame in the video data, and if no face is detected, encode the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; an occlusion determination module configured to determine whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, wherein the predetermined feature coding information includes feature coding information of an image frame in the video data that includes a face;

[0008] An embodiment of the present application provides an electronic device comprising a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is operating, the processor and the memory communicate via the bus, and the machine-readable instructions cause the processor to perform the camera device occlusion detection method described in any of the above embodiments.

[0009] An embodiment of the present application provides a computer-readable storage medium having a computer program stored therein, the computer program causing a processor to execute the camera device occlusion detection method described in any of the above embodiments.

[0010] An embodiment of the present application further provides a computer program product including a computer-readable storage medium having program code stored thereon, the instructions contained in the program code causing a processor of a computing device to perform the above-described method. [Effects of the Invention]

[0011] According to the camera device occlusion detection method, device, electronic device, storage medium, and computer program product of the present application, if a face is not identified, the current image frame is encoded based on pixel values ​​in the current image frame, feature encoding information of the current image frame is obtained, and based on the feature encoding information of the current image frame and predetermined feature encoding information, it is determined whether the camera device is occluded. In this way, if a face is not identified, further determination can be made on the image, thereby improving the accuracy of the determination.

[0012] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. [Brief explanation of the drawings]

[0013] [Figure 1] 1 illustrates an exemplary flowchart of a method for occlusion detection in a camera device according to an embodiment of the present application. [Figure 2] 1 illustrates an exemplary flowchart of a method for determining feature coding information of a current image frame according to an embodiment of the present application; [Figure 3] 10 illustrates another exemplary flowchart of a method for occlusion detection in a camera device, according to an embodiment of the present application. [Figure 4]1 illustrates an exemplary flowchart of a method for outputting presentation information according to an embodiment of the present application. [Figure 5] 10 illustrates an exemplary flowchart of yet another camera device occlusion detection method, according to an embodiment of the present application. [Figure 6] 1 illustrates an exemplary structural diagram of an occlusion detection device of a camera device according to an embodiment of the present application; [Figure 7] 1 illustrates another exemplary structural diagram of an occlusion detection device of a camera device according to an embodiment of the present application; [Figure 8] 1 shows a schematic diagram of an electronic device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0014] In order to more clearly explain the technical solutions of the embodiments of the present application, the drawings necessary for the embodiments have been briefly introduced above. The drawings herein are incorporated into and constitute a part of this specification, and these drawings are intended to illustrate the embodiments of the present application and are used together with this specification to explain the technical solutions of the embodiments of the present application. The drawings only illustrate a part of the embodiments of the present application, and therefore should not be considered as limiting the scope of the present application. It should be understood that those skilled in the art can also obtain other related drawings according to these drawings without creative efforts.

[0015] In order to make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided in the drawings below is not intended to limit the scope of the present application, but to illustrate optional embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0016] It should be noted that like symbols and letters represent like items in subsequent drawings, and therefore, once any item is defined in one drawing, it need not be further defined or explained therefor in subsequent drawings.

[0017] The term "and / or" herein describes an association relationship and indicates that three relationships may exist; for example, A and / or B can represent three cases: A exists independently, A and B both exist, and B exists independently. Also, the term "at least one" herein indicates any one of a plurality, or any combination of at least two of a plurality; for example, including at least one of A, B, and C indicates including any one or more elements selected from the set consisting of A, B, and C.

[0018] With the improvement of people's living standards, vehicles have become an indispensable means of transportation in people's lives, and a camera device inside the vehicle can restrict the driver's driving behavior, thereby reducing the possibility of traffic accidents and improving driving safety. However, in actual use, the camera device may be blocked, and when the camera device is blocked, the driver's behavior cannot be accurately detected.

[0019] According to research, the prior art has a method for detecting whether a camera device is occluded, such as a method for determining whether a camera device is occluded by using a face detection method, but this method is prone to misjudgment or missed judgment when the overall brightness of the image is high or low.

[0020] In view of the above research, the present application provides a method for detecting occlusion of a camera device, the method including: obtaining video data of a driving area of ​​a vehicle through a camera device; performing face detection on a current image frame in the video data; if a face is not detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature coding information of the current image frame; and determining whether the camera device is occluded based on the feature coding information of the current image frame and predetermined feature coding information, wherein the predetermined feature coding information includes feature coding information of an image frame in the video data that includes a face.

[0021] In an embodiment of the present application, if a face is not identified, the current image frame is encoded based on pixel values ​​in the current image frame, feature encoding information of the current image frame is obtained, and based on the feature encoding information of the current image frame and predetermined feature encoding information, it is determined whether the camera device is occluded. In this way, if a face is not identified, further judgment can be made on the image, thereby improving the accuracy of the judgment.

[0022] The method for detecting occlusion in a camera device according to an embodiment of the present application will be described in detail below with reference to the drawings. Referring to Fig. 1, an exemplary flowchart of the method for detecting occlusion in a camera device according to an embodiment of the present application is shown, and the method for detecting occlusion in a camera device includes the following steps S101 to S103.

[0023] In step S101, video data of the driving area of ​​the vehicle is obtained via a camera device.

[0024] Here, the driving area refers to an area within the vehicle where the driver controls the driving of the vehicle. When the vehicle is traveling, the driver is usually located in the driving area of ​​the vehicle, and the terminal device can acquire video data of the driving area of ​​the vehicle.

[0025] Video data refers to a continuous image sequence, which essentially consists of a series of consecutive images, where an image frame is the smallest visual unit that constitutes a video, and is a single static image. A dynamic video is obtained by synthesizing a time-sequential sequence of image frames. In this embodiment, it is necessary to extract image frames from the video data to facilitate subsequent detection and identification.

[0026] For example, video data typically includes a large number of frames per second (e.g., 24 frames per second), so frame extraction can be performed in the process of extracting image frames from the video data. Here, frame extraction refers to performing frame extraction at intervals of a predetermined number of frames, for example, one frame of image can be extracted every 20 frames. Frame extraction can also be performed at predetermined time intervals, for example, an image can be extracted every 10 milliseconds (ms).

[0027] It should be mentioned that the predetermined frame number interval and the predetermined time interval can be set according to actual requirements, and the present application is not limited thereto.

[0028] Optionally, the video data of the driving area is captured by a camera device installed in the vehicle, and the terminal device acquires the video data captured by the camera device, that is, in the embodiment of the present application, the execution entity of the camera device occlusion detection method may be the terminal device, where the terminal device includes but is not limited to an in-vehicle device, a wearable device, a user terminal, a handheld device, etc.

[0029] In other embodiments, the execution entity of the camera device occlusion detection method may be a server, where the server may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data and artificial intelligence platforms.

[0030] In some possible embodiments, the camera device occlusion detection method may also be implemented by a processor invoking computer-readable instructions stored in a memory.

[0031] In step S102, face detection is performed on the current image frame in the video data, and if no face is detected, the current image frame is encoded based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame.

[0032] As can be seen, after performing the image frame extraction process on the video data, a number of image frames are obtained, where the current image frame refers to the image frame on which the current detection and identification process needs to be performed, and the image frame that is earlier in timing than the current image frame in the video data is called a pre-order image frame, and the image frame that is later in timing than the current image frame is called a post-order image frame.

[0033] For example, after extracting the above multi-frame images, face detection can be performed on the extracted images to determine whether a face exists in the current image frame. In some embodiments, if a face is detected in the current image frame, it means that the driver is in the driving area and the camera device is not occluded; if a face is not detected in the current image frame, further judgment is required for the current image frame to improve the accuracy of the judgment.

[0034] In this embodiment, if no face is identified in the current image frame, the current image frame is further encoded based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame. In some embodiments, referring to Figure 2, the step of encoding the current image frame based on pixel values ​​in the current image frame may include the following steps S1021 and S1022.

[0035] In step S1021, a reference pixel threshold value of the current image frame is determined.

[0036] In this embodiment, the reference pixel threshold is the average pixel value of the current image frame, so that the feature coding information obtained based on the reference pixel threshold has a unique correspondence with the current image frame, thereby further improving the accuracy of the judgment. Of course, in other embodiments, the reference pixel threshold may be set according to actual conditions, for example, the corresponding reference pixel threshold may be determined according to the overall brightness of the current image, rather than the average pixel value of the current image frame.

[0037] For example, to improve the accuracy of determining the reference pixel threshold, a noise reduction process can be performed on the current image frame before determining the reference pixel threshold for the current image frame, thereby reducing the effect of imaging noise.

[0038] In some embodiments, a Gaussian smoothing algorithm may be employed to perform noise reduction on the current image frame. In other embodiments, a median filtering algorithm or an average filtering algorithm may be used to perform noise reduction, and the present application is not limited thereto.

[0039] In step S1022, the pixel value of each pixel point of the current image frame is sequentially compared with the reference pixel threshold, and pixel points greater than the reference pixel threshold are coded as 1, and pixel points less than or equal to the reference pixel threshold are coded as 0, thereby obtaining feature coding information of the current image frame.

[0040] For example, after determining the reference pixel threshold, the current image frame can be encoded based on the reference pixel threshold, and the encoding process may obtain the feature encoding information by encoding pixel points greater than the reference pixel threshold as 1 and encoding pixel points equal to or less than the reference pixel threshold as 0. The feature encoding information may be a two-dimensional array or a two-dimensional matrix having the same size as the current image frame to be encoded. For example, when the average pixel of the current image frame is 100, if a pixel point in the current image frame has a pixel value of 123, the pixel point is encoded as 1, and if a pixel point in the current image frame has a pixel value of 80, the pixel point is encoded as 0.

[0041] Optionally, in order to reduce the dimension of the feature encoding information and reduce the amount of calculation, after encoding each pixel point in the current image frame, the codes of each pixel point are serially concatenated in sequence according to a certain scanning order (e.g., scanning rows and then scanning columns) to form a one-dimensional array or vector, which can be used as the feature encoding information of the image frame.

[0042] In step S103, it is determined whether the camera device is occluded based on the feature coding information of the current image frame and predetermined feature coding information, where the predetermined feature coding information includes the feature coding information of an image frame containing a face in the video data.

[0043] In this embodiment, the predetermined feature coding information includes feature coding information of an image frame including a face in the video data, i.e., the predetermined feature coding information is feature coding information of a previous image frame of the current image frame, and the previous image frame includes a face, that is, an image including a face captured after the camera device is turned on is used as a basis for determining whether the camera device is occluded in a subsequent image frame.

[0044] As can be understood, the camera device is powered on and operates after the vehicle is started, and begins to capture the driving area of ​​the vehicle, and can obtain video data; if the first frame image obtained based on the video data does not contain a face, the pre-stored feature encoding information containing a face can be used as the predetermined feature encoding information; if the first frame image contains a face, the feature encoding information of the first frame image is used as the predetermined feature encoding information.

[0045] It should be noted that, if it is determined based on the video data that a face is included in a successive multi-frame image, the predetermined feature coding information can be updated, i.e., the subsequent feature coding information including the face is used to replace the feature coding information of the first frame image, for example, the predetermined time interval can be updated once, and the predetermined time interval is not limited, and can be one minute or two minutes.

[0046] Illustratively, the feature coding information of the current image frame may be compared with predetermined feature coding information to determine whether the camera device is occluded. In some embodiments, a Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information may be calculated, and if the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold, it may be determined that the camera device is occluded.

[0047] In an embodiment of the present application, if a face is not identified, the current image frame is encoded based on pixel values ​​of the current image frame to obtain feature encoding information of the current image frame, and based on the feature encoding information of the current image frame and predetermined feature encoding information, it is determined whether the camera device is occluded. In this way, if a face is not identified, further judgment can be made on the image, which can improve the accuracy of the judgment, and in particular, the judgment of whether the driver has left the seat becomes more accurate.

[0048] Referring to FIG. 3, there is shown another exemplary flowchart of an occlusion detection method for a camera device according to an embodiment of the present application, which includes the following steps S201 to S205.

[0049] In step S201, video data of the driving area of ​​the vehicle is obtained via a camera device.

[0050] Here, this step is the same as step S101 above.

[0051] In step S202, face detection is performed on the current image frame in the video data, and if no face is detected, the current image frame is encoded based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame.

[0052] Here, this step is the same as step S102 above.

[0053] In step S203, it is determined whether the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold. If it is greater than the predetermined threshold, execute step S205; if not, execute step S204.

[0054] Here, this step is the same as step S103 above.

[0055] In step S204, it is determined whether the camera device is occluded based on the pixel distribution histogram of the current image frame.

[0056] For example, if the Hamming distance between the feature encoding information of the current image frame and the predetermined feature encoding information is less than or equal to a predetermined threshold, it means that there may be a face in the current image frame, but due to some specific causes (such as dark ambient light), it may not be identified in the process of face detection; therefore, in order to further improve the accuracy of the judgment, it is necessary to judge the current image frame in another way to further determine whether the camera device is occluded.

[0057] In an embodiment of the present application, if the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is less than or equal to a predetermined threshold, a pixel distribution histogram of the current image frame is determined, and if the pixel distribution ratio of a predetermined section in the pixel distribution histogram is greater than a predetermined ratio threshold, it is determined that the camera device is occluded.

[0058] In some embodiments, the distribution of pixel values ​​of each pixel point in the current image frame can be statistically calculated to obtain a pixel distribution histogram of the current image frame. For example, the distribution of pixel values ​​of each pixel point in the current image frame can be statistically calculated according to a plurality of pre-divided pixel intervals, and the plurality of pixel intervals may be [0-19], [20-80], [81-126], [127-255], etc. It can be understood that the plurality of pixel intervals shown in this embodiment are merely examples, and in other embodiments, the plurality of pixel intervals may be divided according to other requirements.

[0059] In some embodiments, the predetermined interval can be obtained by conducting a large amount of testing based on the actual usage environment of the camera device. For example, the histogram of an image of a vehicle driving area scene can be statistically analyzed, and it can be determined that there is a relatively clear boundary between the proportion of the number of pixels in the pixel interval [20~80] in a normal image captured when the camera device is not obstructed and the proportion of the number of pixels in the pixel interval [20~80] in an image captured when the camera device is obstructed. Therefore, the predetermined interval can be set to the pixel interval [20~80].

[0060] Furthermore, the greater the percentage of total pixels with low pixel values ​​(e.g., less than 20) or greater than the high brightness threshold, the less likely there is valid content in the image; therefore, in other embodiments, the predetermined interval may be a low pixel value interval or a high pixel value interval, for example, a pixel interval of [0-20] or a pixel interval of [130-255], and the present application is not limited thereto.

[0061] In this embodiment, the predetermined interval is one of the multiple pixel intervals, for example, [20~80], and if the number of pixel points within the predetermined interval [20~80] is large and greater than a predetermined percentage threshold, it can be determined that the camera device is occluded.

[0062] In step S205, it is determined that the camera device is occluded.

[0063] In step S206, the presentation information is output.

[0064] Illustratively, after determining that the camera device is occluded, presentation information may be output to notify the driver to process the occluded camera device so that the camera device can successfully capture an image of the driver, thereby assisting in improving driving safety.

[0065] Here, the presentation information includes, but is not limited to, audio presentation information, graphic presentation information, lighting presentation information, etc. For example, audio presentation information such as "The camera device is blocked. Please check." can be output.

[0066] In some embodiments, in order to reduce the impact on people inside the vehicle of frequent output of presentation information due to momentary occlusion, as shown in FIG. 4, step S206 of outputting the above presentation information can be performed in the manner of the following steps S2061 and S2062.

[0067] In step S2061, the camera device's occlusion duration is determined based on the camera device's detection result for each frame image in the video data.

[0068] In step S2062, when the continuous occlusion time reaches a predetermined time, the presentation information is output.

[0069] In this embodiment, if it is determined that the camera device is occluded based on the current image frame, it is necessary to determine whether the judgment result of the image of the subsequent frame also indicates occlusion. If it is determined that the camera device is not occluded based on the image of at least one subsequent frame, it means that the current occlusion is merely a phenomenon that appears and disappears momentarily, and in this case, no presentation information is output. If the judgment result of the image of at least one subsequent frame also indicates that the camera device is occluded, that is, if it is determined that the camera device is occluded based on each of the consecutive multi-frame images, it means that the camera device continues to be occluded at this point, which is intentional occlusion and not accidental occlusion, and if the continuous occlusion time reaches a predetermined time (e.g., 5 seconds), presentation information is output and presented.

[0070] Referring to FIG. 5, another exemplary flowchart of an occlusion detection method for a camera device according to an embodiment of the present application is shown, which includes the following steps S301 to S310.

[0071] In step S301, video data of the driving area of ​​the vehicle is obtained via a camera device.

[0072] Here, this step is the same as step S101 above.

[0073] In step S302, face detection is performed on the current image frame in the video data, and if no face is detected, the current image frame is encoded based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame.

[0074] Here, this step is the same as step S102 above.

[0075] In step S303, it is determined whether the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold. If it is greater than the predetermined threshold, execute step S308; if not, execute step S304.

[0076] For example, if the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold, it means that the current image frame does not contain a face, and in this case, step S308 is executed to determine that the camera device is occluded. If the Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is equal to or less than a predetermined threshold, it means that the current image frame may contain a face, and further determination is required, and step S304 is executed.

[0077] In step S304, a pixel distribution histogram of the current image frame is determined.

[0078] Here, this step is the same as step S204 above.

[0079] In step S305, it is determined whether the pixel distribution ratio in a predetermined section in the pixel distribution histogram is greater than a predetermined ratio threshold. If it is greater than the predetermined ratio threshold, step S308 is executed; if not, step S306 is executed.

[0080] For example, if the pixel distribution ratio of a predetermined section in the pixel distribution histogram is greater than a predetermined ratio threshold, it means that the proportion of invalid content in the current image frame is large, in this case, step S308 is performed to determine that the camera device is occluded.If the pixel distribution ratio of a predetermined section in the pixel distribution histogram is equal to or less than a predetermined ratio threshold, it means that the proportion of valid content in the current image frame is normal, in this case, step S306 needs to be performed for further determination.

[0081] In step S306, the maximum connected region of the current image frame is determined.

[0082] In step S307, it is determined whether the area of ​​the maximum connected region is greater than a predetermined area threshold. If it is greater than the predetermined area threshold, step S308 is executed; if not, step S310 is executed.

[0083] Here, the maximum connected area refers to an image area consisting of adjacent pixel points that all have the same pixel value or pixel values ​​within a certain error, and the maximum connected area is a closed area. For example, the predetermined area threshold can be set to an area that occupies 60% of the entire image. If the maximum connected area is greater than the predetermined area threshold, step S308 is performed to determine that the camera device is occluded. If the maximum connected area is equal to or less than the predetermined area threshold, step S310 is performed to determine that the camera device is not occluded.

[0084] As can be appreciated, in order to further improve the accuracy of the determination and reduce erroneous determinations due to sudden changes (such as erroneous determinations due to momentary occlusions), in some embodiments, if it is determined that the area of ​​the maximum connected region of the current image frame is greater than the predetermined area threshold, the following steps (1) to (3) can be further performed.

[0085] (1) determining a maximum connected area of ​​an image in at least one of the image frames preceding and following the current image frame;

[0086] (2) determining an average area of ​​the largest connected region of the current image frame and the at least one image frame;

[0087] (3) determining that the camera device is occluded if the average area of ​​the largest connected region of the current image frame and the at least one frame of images is greater than the predetermined area threshold;

[0088] In this way, if it is determined that the area of ​​the maximum connected area of ​​the current image frame is larger than a predetermined area threshold, the average value of the areas of the maximum connected areas of the images in the multi-frames before and after the current image frame is further determined, thereby reducing the incidence of erroneous judgments due to an object appearing and disappearing in an instant (for example, passing in front of the camera device in an instant), and further improving the accuracy of the judgment.

[0089] In step S308, it is determined that the camera device is occluded.

[0090] In step S309, the presentation information is output.

[0091] Here, this step is the same as step S206 above.

[0092] In step S310, it is determined that the camera device is not occluded.

[0093] As can be understood by those skilled in the art, in the methods of the above specific embodiments, the described order of each step does not imply a strict execution order, and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined by its function and internal logic.

[0094] Based on the same technical concept, the embodiments of the present application further provide an occlusion detection device for a camera device corresponding to the occlusion detection method for a camera device, and since the principle by which the device in the embodiments of the present application solves the problem is similar to the occlusion detection method for a camera device in the above embodiments of the present application, the implementation of the device can refer to the implementation of the method.

[0095] Referring to FIG. 6, there is shown a schematic diagram of an occlusion detection device 500 of a camera device according to an embodiment of the present application, which includes a video capture module 501, a face detection module 502, and The video acquisition module 501 is configured to acquire video data of the driving area of ​​the vehicle through a camera device; the face detection module 502 is configured to perform face detection on a current image frame in the video data, and if no face is detected, encode the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; The occlusion determination module 503 is configured to determine whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, where the predetermined feature coding information includes feature coding information of an image frame that includes a face in the video data.

[0096] In one possible embodiment, the face detection module 502 further comprises: determining a reference pixel threshold for the current image frame; The pixel value of each pixel point of the current image frame is sequentially compared with the reference pixel threshold, and pixel points greater than the reference pixel threshold are coded as 1, and pixel points less than or equal to the reference pixel threshold are coded as 0, thereby obtaining feature coding information of the current image frame.

[0097] In one possible embodiment, the reference pixel threshold is the average pixel value of the current image frame.

[0098] In one possible embodiment, the occlusion determination module 503 further comprises: The camera device is configured to determine that it is occluded if a Hamming distance between the feature coding of the current image frame and the predetermined feature coding is greater than a predetermined threshold.

[0099] In one possible embodiment, the occlusion determination module 503 further comprises: If the Hamming distance is less than or equal to the predetermined threshold, determining a pixel distribution histogram for the current image frame; The camera device is configured to determine whether the camera device is occluded based on a pixel distribution histogram of the current image frame.

[0100] In one possible embodiment, the occlusion determination module 503 further comprises: The camera device is configured to determine that the camera device is occluded when the proportion of pixel distribution in a predetermined section in the pixel distribution histogram is greater than a predetermined proportion threshold.

[0101] In one possible embodiment, the occlusion determination module 503 further comprises: determining a maximum connected region of the current image frame when the pixel distribution ratio of a predetermined section in the pixel distribution histogram is equal to or less than the predetermined ratio threshold; The camera device is configured to determine that it is occluded if the area of ​​the largest connected region is greater than a predetermined area threshold.

[0102] Referring to FIG. 7, in one possible embodiment, the device further comprises: An information output module 504 is configured to output presentation information if it is determined that the camera device is occluded.

[0103] In one possible embodiment, the information output module 504 further comprises: determining a duration of occlusion of the camera device based on a detection result of the camera device for each frame image in the video data; When the continuous occlusion time reaches a predetermined time, the presentation information is output.

[0104] The processing process of each module in the device and the interaction process between each module can be described in the relevant description in the above method embodiment, and will not be described in detail here.

[0105] Based on the same technical concept, an embodiment of the present application further provides an electronic device. Referring to Figure 8, an exemplary structural diagram of an electronic device 700 according to an embodiment of the present application is shown, which includes a processor 701, a memory 702, and a bus 703. Here, the memory 702 is configured to store execution instructions and includes an internal memory 7021 and an external memory 7022. Here, the internal memory 7021, also referred to as internal storage, is configured to temporarily store calculation data of the processor 701 and data to be exchanged with an external memory 7022 such as a hard disk, and the processor 701 exchanges data with the external memory 7022 via the internal memory 7021.

[0106] In the embodiment of the present application, the memory 702 is configured to store application program code for executing the technical solution of the present application, and the execution thereof is controlled by the processor 701. That is, when the electronic device 700 operates, the processor 701 and the memory 702 communicate with each other via the bus 703, so that the processor 701 executes the application program code stored in the memory 702 to perform the method according to any of the above-described embodiments.

[0107] Here, the memory 702 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EPROM), etc.

[0108] The processor 701 may be an integrated circuit chip with signal processing capabilities. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor, or any conventional processor.

[0109] As can be appreciated, the structure shown in the embodiments of the present application does not constitute a limitation on the electronic device 700. In other embodiments of the present application, the electronic device 700 may include more or fewer components than those shown, may combine certain components, may separate certain components, or may have a different component arrangement. The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0110] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, the computer program causing a processor to perform the steps of the camera device occlusion detection method in the above-described method embodiment, where the storage medium may be a volatile or non-volatile computer-readable storage medium.

[0111] An embodiment of the present application further provides a computer program product, the computer program product including a program code, and instructions included in the program code are used to perform steps of the occlusion detection method for a camera device in the above method embodiment, and reference can be made to the above method embodiment.

[0112] The computer program product may be implemented in hardware, software, or a combination thereof. In one exemplary embodiment, the computer program product is embodied as a computer storage medium, and in another exemplary embodiment, the computer program product is embodied as a software product such as a software development kit (SDK).

[0113] As will be understood by those skilled in the art, for convenience and brevity of description, the operation processes of the above-described systems and devices may refer to the corresponding processes in the above-described method embodiments. It should be understood that in some embodiments provided herein, the disclosed systems, devices, and methods may be realized in other manners. The above-described device embodiments are merely illustrative. For example, the division of the units is a division of logical functions. In actual implementation, other division methods may be adopted. For example, multiple units or components may be combined or integrated into another system, and some features may be ignored or not implemented. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be realized using several communication interfaces, and the indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0114] The units described as separate parts may or may not be physically separated, and the parts shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Depending on actual needs, some or all of the units may be selected to achieve the objectives of the technical solutions in this embodiment.

[0115] Furthermore, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0116] When the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, an essential part of the technical solution of the present application, i.e., a part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media capable of storing program code, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0117] If the technical solution of the present application involves personal information, the product to which the technical solution of the present application is applied will clearly inform the individual of the personal information processing rules and obtain the individual's consent before processing the personal information. If the technical solution of the present application involves sensitive personal information, the product to which the technical solution of the present application is applied will clearly inform the individual of the personal information before processing the sensitive personal information, thereby meeting the requirement of "explicit consent." For example, a personal information collection device such as a camera may display clear and visible signs to notify the individual that they are within the personal information collection range and that their personal information will be collected, and if the individual voluntarily enters the collection range, they are deemed to have consented to the collection of their personal information. Alternatively, a personal information processing device may use explicit signs / information to inform the individual of the personal information processing rules and obtain the individual's permission by means of pop-up information, voluntarily uploading their personal information, etc., where the personal information processing rules may include information such as the personal information controller, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0118] Finally, it should be noted that the above examples are merely specific embodiments of the present application to illustrate the technical solutions of the present application, and are not intended to limit the present application, and the scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above examples, it should be understood by those skilled in the art that any technician familiar with the present technical field can easily modify or devise variations of the technical solutions described in the above examples, or equivalently replace some technical features, within the technical scope disclosed in the present application. These modifications, variations, or substitutions do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the examples of the present application, and should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be governed by the claims.

Claims

1. A method for detecting occlusion in a camera device, comprising: acquiring video data of a driving area of ​​a vehicle via a camera device; performing face detection on a current image frame in the video data, and if no face is detected, encoding the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; determining whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, wherein the predetermined feature coding information includes feature coding information of an image frame in the video data that includes a face.

2. encoding the current image frame based on pixel values ​​in the current image frame, determining a reference pixel threshold for the current image frame; Sequentially comparing the pixel value of each pixel point of the current image frame with the reference pixel threshold, encoding pixel points that are greater than the reference pixel threshold as 1, and encoding pixel points that are equal to or less than the reference pixel threshold as 0, thereby obtaining feature encoding information of the current image frame. The method for detecting occlusion of a camera device according to claim 1 .

3. the reference pixel threshold is the average pixel value of the current image frame; The method for detecting occlusion of a camera device according to claim 2 .

4. determining whether the camera device is occluded based on feature encoding information of the current image frame and the predetermined feature encoding information; determining that the camera device is occluded if a Hamming distance between the feature coding information of the current image frame and the predetermined feature coding information is greater than a predetermined threshold. The method for detecting occlusion of a camera device according to claim 1 .

5. The occlusion detection method for the camera device includes: if the Hamming distance is less than or equal to the predetermined threshold, determining a pixel distribution histogram for the current image frame; determining whether the camera device is occluded based on a pixel distribution histogram of the current image frame. The method for detecting occlusion of a camera device according to claim 4.

6. determining whether the camera device is occluded based on a pixel distribution histogram of the current image frame; determining that the camera device is occluded when a pixel distribution ratio of a predetermined section in the pixel distribution histogram is greater than a predetermined ratio threshold; and determining a maximum connected region of the current image frame when a pixel distribution ratio of a predetermined section in the pixel distribution histogram is equal to or less than the predetermined ratio threshold, and determining that the camera device is occluded when an area of ​​the maximum connected region is greater than a predetermined area threshold. The method for detecting occlusion of a camera device according to claim 5.

7. The occlusion detection method for the camera device includes: and outputting presentation information when the camera device is determined to be occluded. The method for detecting occlusion of a camera device according to any one of claims 1 to 6.

8. outputting presentation information when it is determined that the camera device is occluded, determining a duration of occlusion of the camera device based on a detection result of the camera device for each frame image in the video data; outputting the presentation information when the continuous occlusion time reaches a predetermined time. The method for detecting occlusion of a camera device according to claim 7.

9. An occlusion detection device for a camera device, comprising: a video acquisition module configured to acquire video data of a driving area of ​​the vehicle via a camera device; a face detection module configured to perform face detection on a current image frame in the video data, and if no face is detected, encode the current image frame based on pixel values ​​in the current image frame to obtain feature encoding information of the current image frame; an occlusion determination module configured to determine whether the camera device is occluded based on feature coding information of the current image frame and predetermined feature coding information, wherein the predetermined feature coding information includes feature coding information of an image frame in the video data that includes a face.

10. An electronic device comprising a processor, a memory, and a bus; The memory stores machine-readable instructions executable by the processor, and when the electronic device is operating, the processor and the memory communicate via the bus, and the machine-readable instructions cause the processor to execute the camera device occlusion detection method described in any one of claims 1 to 6.

11. A computer-readable storage medium storing a computer program for causing a processor to execute the method for detecting occlusion in a camera device according to any one of claims 1 to 6.

12. A computer program for causing a processor to execute the occlusion detection method for a camera device described in any one of claims 1 to 6.

Images