Human shape detection method and human shape detection equipment
By screening the humanoid detection area in the security surveillance camera, using disturbance and mobile information to reduce false detection, improving the accuracy of humanoid detection, and solving the problem of high false detection rate in the prior art.
Patent Information
- Application Number
- CN202310871975.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-14
- Publication Date
- 2025-01-14
AI Technical Summary
The error detection rate of humanoid detection of existing security surveillance cameras is relatively high, and due to computing power limitations and external interference such as lens disturbances and humanoid movements, the detection accuracy is insufficient.
By obtaining a single frame image from the video stream, identifying the humanoid detection area and its confidence, and filtering out the target detection area based on the disturbance information and humanoid movement information, eliminating lens disturbances and humanoid movement interference, and improving detection accuracy.
It effectively reduces the error detection rate of humanoid detection, improves detection accuracy, and ensures the security monitoring effect of the shooting area.
Smart Images
Figure CN119314073A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image detection, and in particular to a human shape detection method and a human shape detection device. Background Art
[0002] In the field of security monitoring, human detection is an important basic function. When a security monitoring camera detects a human figure, it can perform operations such as reporting human events to the cloud, issuing human figure alarms, and recording human figure videos to achieve security in the shooting area.
[0003] At present, human detection on cameras is generally based on deep neural network methods to detect human figures with a certain degree of confidence. Due to the computing power limitations of the camera, the deep neural networks generally deployed on camera devices are relatively lightweight, resulting in a high risk of false detection by the camera. In actual security monitoring scenarios, the camera may be disturbed, the detected human figure may move, and the camera may sometimes capture an occasional human figure, which further increases the false detection rate of human detection.
[0004] Therefore, how to reduce the false detection rate of human figure detection is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention
[0005] The main purpose of the embodiments of the present application is to provide a human figure detection method and a human figure detection device, which aim to eliminate the false detection effects on human figure detection caused by external interference such as lens disturbance and human figure movement, so as to improve the accuracy of human figure detection.
[0006] In a first aspect, an embodiment of the present application provides a human figure detection method, comprising:
[0007] Get a single frame image from the video stream to be analyzed;
[0008] Identify and obtain a human figure detection area in a single frame image and a confidence level corresponding to the human figure detection area, wherein the confidence level is used to characterize the probability of a human figure being present in the human figure detection area;
[0009] Determine disturbance information of a shooting mechanism shooting the video stream and movement information of a human figure in a human figure detection area according to the video stream;
[0010] According to the confidence, disturbance information and human movement information, the target detection area corresponding to the single frame image is screened out from the human detection area of the single frame image.
[0011] In some implementations, selecting a target detection area from a human detection area of a single frame image according to confidence, disturbance information, and human movement information includes:
[0012] When it is determined according to the disturbance information that the shooting mechanism is in a preset static state when shooting a single frame image, each human figure detection area is determined to be a human figure static area or a human figure moving area according to the human figure movement information;
[0013] Compare and match the human-shaped static area with a preset human-shaped sample set, wherein the human-shaped sample set includes at least one sample area;
[0014] If there is at least one sample area that successfully matches the human-shaped static area, and the confidence of the human-shaped static area exceeds a first confidence threshold, the human-shaped static area is determined to be a target detection area.
[0015] In some implementations, filtering out a target detection area from a human detection area of a single frame image according to the confidence level, disturbance information, and human movement information further includes:
[0016] When it is determined according to the disturbance information that the shooting mechanism is in a preset disturbance state when shooting a single-frame image, the human figure detection area is determined to be a disturbance area;
[0017] If the confidence level corresponding to the disturbance area exceeds the second confidence level threshold, the disturbance area is determined to be a target detection area.
[0018] In some embodiments, the method further comprises:
[0019] When it is determined according to the disturbance information that the shooting mechanism is in a preset disturbance state when shooting a single-frame image, the human shape sample set is cleared.
[0020] In some implementations, after determining whether a human figure in each human figure detection area moves according to the human figure movement information, the method further includes:
[0021] Traverse all humanoid static areas and compare them with the preset humanoid sample set;
[0022] The sample areas in the human sample set that fail to match all human static areas are deleted.
[0023] In some implementations, filtering out a target detection area from a human detection area of a single frame image according to the confidence level, disturbance information, and human movement information further includes:
[0024] If the confidence of the human figure moving region exceeds the third confidence threshold, the human figure moving region is determined to be a target detection region, and the target detection region is recorded as a sample region in the human figure sample set.
[0025] In some implementations, after determining that the human-shaped static area is a target detection area, the method further includes:
[0026] If the first confidence threshold does not reach the preset confidence upper limit, the first confidence threshold is increased by a preset step size to update the first confidence threshold;
[0027] If the first confidence threshold has reached the preset upper confidence limit, the first confidence threshold is not increased.
[0028] In some implementations, comparing and matching the human-shaped static region with a preset human-shaped sample set includes:
[0029] Compare the human-shaped static area with each sample area to obtain the overlap between the human-shaped static area and the sample area;
[0030] If the overlap exceeds the first overlap threshold, it is determined that the corresponding sample area matches the human detection area successfully;
[0031] If the overlap does not exceed the first overlap threshold, it is determined that the corresponding sample area fails to match the human figure detection area.
[0032] In some implementations, determining disturbance information of a shooting mechanism that shoots the video stream according to the video stream includes:
[0033] Sampling two consecutive frames of disturbance detection images from the video stream, and obtaining a difference image of the two frames of detection images;
[0034] Binarizing the difference image according to a preset binarization threshold to obtain a binarized image;
[0035] The disturbance information is determined according to the proportion of moving pixels in the binary image.
[0036] In some implementations, before determining disturbance information of a shooting mechanism that shoots the video stream according to the video stream, the method further includes:
[0037] Sample time-detected images from a video stream;
[0038] Determine whether the shooting time corresponding to the video stream is daytime or nighttime according to the time detection image;
[0039] If the shooting time is daytime, the binarization threshold is set to a first preset value;
[0040] If the shooting time is at night, the binarization threshold is set to a second preset value, wherein the first preset value is smaller than the second preset value.
[0041] In some embodiments, the method further comprises:
[0042] If a single-frame image includes only one target detection area and a moving human figure appears in the target detection area, the target detection area is determined to be a single moving human figure area;
[0043] If the confidence of the single humanoid moving region exceeds the fourth confidence threshold, the single humanoid moving region is retained;
[0044] If the confidence of the single human-shaped moving region does not exceed the fourth confidence threshold, the previous frame image of the single frame image is obtained; if a human-shaped region with a human figure appears in the previous frame image, and the overlap between the human-shaped region and the single human-shaped moving region exceeds the second overlap threshold, the single human-shaped moving region is retained;
[0045] The retained human shape moving area is output as the human shape detection result corresponding to the single frame image.
[0046] In a second aspect, an embodiment of the present application further provides a human shape detection device, the human shape detection device comprising:
[0047] Memory for storing computer programs;
[0048] A processor is used to execute a computer program and implement any one of the human figure detection methods provided in the specification of this application when executing the computer program.
[0049] In summary, the embodiment of the present application provides a human figure detection method and a human figure detection device. The human figure detection method provided by the embodiment of the present application includes: obtaining a single frame image from a video stream to be analyzed; identifying and obtaining a human figure detection area in the single frame image and the confidence corresponding to the human figure detection area, wherein the confidence is used to characterize the probability of having a human figure in the human figure detection area; determining the disturbance information of the shooting mechanism that shoots the video stream and the human figure movement information in the human figure detection area according to the video stream; filtering out the target detection area corresponding to the single frame image from the human figure detection area of the single frame image according to the confidence, disturbance information and human figure movement information. It should be noted that the generated target detection area is the human figure detection result. By filtering out the target detection area corresponding to the single frame image from the human figure detection area of the single frame image according to the confidence, disturbance information and human figure movement information, the false detection effect of external interference such as lens disturbance and human figure movement on human figure detection is effectively eliminated, thereby improving the accuracy of human figure detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0051] Figure 1 A flowchart of a method for detecting a human figure provided in an embodiment of the present application;
[0052] Figure 2A schematic diagram of the relationship between a video stream and each single frame image in a human figure detection method provided in an embodiment of the present application;
[0053] Figure 3 A schematic diagram of identifying a single frame image in a human figure detection method provided in an embodiment of the present application;
[0054] Figure 4 A schematic diagram of a process for processing a single frame image in one implementation of a human figure detection method provided in an embodiment of the present application;
[0055] Figure 5 A logical judgment diagram for screening a target detection area in a human figure detection method provided in an embodiment of the present application;
[0056] Figure 6 A schematic block diagram of the structure of a human shape detection device provided in an embodiment of the present application;
[0057] Figure 7 A schematic block diagram of the structure of a security surveillance camera provided in an embodiment of the present application. DETAILED DESCRIPTION
[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0059] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0060] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0061] The embodiments of the present application provide a human shape detection method and a human shape detection device. The following is a detailed description of some embodiments of the present application in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0062] Please refer to Figure 1 and Figure 2 , Figure 1A flowchart of a method for detecting a human figure provided in an embodiment of the present application; Figure 2 A schematic diagram of the relationship between a video stream and each single frame image in a human figure detection method provided in an embodiment of the present application.
[0063] like Figure 1 to Figure 2 As shown, an embodiment of the present application provides a human figure detection method for performing detection processing on a single frame image in a video stream to detect a target detection area in the single frame image as a human figure detection result, thereby accurately detecting a human figure that appears in the video stream and is worth detecting.
[0064] For example, the human figure detection method provided in this application can be applied to human figure detection equipment. In particular, it can be applied in the field of security monitoring, such as cameras: when the security monitoring camera shoots a preset shooting area and detects a human figure, it can perform operations such as reporting a human figure event to the cloud, issuing a human figure alarm, and recording a human figure video to achieve security in the shooting area.
[0065] Specifically, the process of the human figure detection method includes steps 1 to 4:
[0066] Step 1: Get a single frame image from the video stream to be analyzed.
[0067] like Figure 2 As shown, the video stream includes multiple continuous single-frame images. The single-frame images are extracted and analyzed to detect the target detection area in the single-frame images as the human detection result, thereby accurately detecting the human figures that appear in the video stream and are worth detecting.
[0068] Specifically, the video stream is obtained by shooting a preset shooting area by a shooting device. For example, the device executing this method can extract the latest single frame image from the video stream to timely perform real-time detection and analysis on the captured video stream.
[0069] Step 2: Identify and obtain the human shape detection area in the single frame image and the confidence level corresponding to the human shape detection area, wherein the confidence level is used to characterize the probability of having a human shape in the human shape detection area.
[0070] It should be noted that the human figure detection area obtained by identifying a single frame image does not necessarily indicate that there must be a valid human figure element in the human figure detection area. Each human figure detection area also has a confidence level used to indicate the probability of having a human figure element in the human figure detection area. It should be understood that the higher the confidence level of the human figure detection area, the higher the probability of having a human figure element in the human figure detection area, and the lower the confidence level of the human figure detection area, the lower the probability of having a human figure element in the human figure detection area.
[0071] Please refer to Figure 3, Figure 3 A schematic diagram of recognizing a single frame image in a human figure detection method provided in an embodiment of the present application.
[0072] like Figure 3 As shown, in some embodiments, identifying and obtaining a human shape detection area in a single-frame image and a confidence level corresponding to the human shape detection area includes: inputting the single-frame image into a preset human shape detection algorithm, so that the human shape detection algorithm outputs the human shape detection area in the single-frame image and the corresponding confidence level according to the single-frame image.
[0073] For example Figure 3 In the figure, the human shape detection algorithm recognizes the single frame image and obtains the human shape detection area A, the human shape detection area B and the human shape detection area C, wherein the confidence of the human shape detection area A is 65%, the confidence of the human shape detection area B is 25%, and the confidence of the human shape detection area C is 50%.
[0074] It should be noted that the human detection algorithm used in the embodiment of the present application can be an algorithm based on a deep neural network or a non-neural network algorithm. For example, the algorithm based on a deep neural network includes but is not limited to Yolov1 / 2 / 3 / 4 / 5 / 6 / 7, SSD (Single Shot MultiBox Detector) algorithm, CenterNet, Faster RCNN and other neural networks. If a non-neural network algorithm is used for processing, the HOG (Histogram of Oriented Gradient) algorithm combined with the SVM (support vector machines) model can be used to obtain the human detection area and the corresponding confidence.
[0075] Step 3: Determine disturbance information of a shooting mechanism that shoots the video stream and movement information of a human figure in a human figure detection area according to the video stream.
[0076] After identifying and acquiring the human figure detection area in a single frame image and the confidence level corresponding to the human figure detection area, the device executing the method determines the disturbance information of the shooting mechanism for shooting the video stream and the human figure movement information in the human figure detection area according to the video stream.
[0077] It should be noted that the disturbance information specifically refers to the disturbance of the position and posture of the shooting mechanism, and the disturbance information includes that the shooting device is in a preset static state or a preset disturbance state. It should be understood that due to the influence of the external environment on the shooting mechanism, such as the effect of wind on the shooting mechanism, the shooting mechanism is prone to disturbance, thereby entering a preset disturbance state.
[0078] It should also be noted that the human figure movement information specifically refers to the movement information of the human figure appearing in the human figure detection area of a single frame image in a video stream, and the human figure movement information includes when the human figure is still or moving, wherein, when the human figure is still, the human figure detection area is a human figure still area, and when the human figure is moving, the human figure detection area is a human figure moving area.
[0079] Please refer to Figure 4 , Figure 4 A schematic diagram of the process of processing a single frame image in one implementation of the human figure detection method provided in an embodiment of the present application.
[0080] like Figure 4 As shown, the device executing the method may first determine the disturbance information of the shooting mechanism for shooting the video stream, and then determine the movement information of the human figure in the human figure detection area.
[0081] In some implementations, determining disturbance information of a shooting mechanism that shoots the video stream according to the video stream includes:
[0082] Sampling two consecutive frames of disturbance detection images from the video stream, and obtaining a difference image of the two frames of detection images;
[0083] Binarizing the difference image according to a preset binarization threshold to obtain a binarized image;
[0084] The disturbance information is determined according to the proportion of moving pixels in the binary image.
[0085] like Figure 2 As shown, it should be noted that the two consecutive disturbance detection images can be randomly sampled from the video stream based on the time axis of the video stream. For example, in order to accurately determine the disturbance information of the shooting mechanism when shooting a single frame image, an image that is relatively close to the time of the single frame image can be collected as the disturbance detection image, for example, Figure 2 The “previous frame image + single frame image” or “previous two frame image + previous frame image” is taken as two consecutive frames of disturbance detection images.
[0086] Specifically, after sampling two consecutive frames of disturbance detection images, the pixel values of each pixel of the two frames of detection images are subtracted to obtain a differential image corresponding to the two frames of detection images, and the differential image is binarized according to a preset binarization threshold to obtain a binarized image, and then the disturbance information is determined according to the proportion of moving pixels in the binarized image.
[0087] Specifically, the differential image is binarized according to a preset binarization threshold to obtain a binarized image. Specifically, pixels whose pixel values are greater than or equal to the binarization threshold are marked as 1, and pixels whose pixel values are less than the binarization threshold are marked as 0, thereby obtaining a binarized image in which pixels are marked as 0 or 1, respectively.
[0088] Specifically, determining the disturbance information according to the mobile pixel ratio in the binary image specifically includes determining the pixels marked as 1 in the binary image as mobile pixels, and then calculating the ratio of the number of mobile pixels to the total number of pixels in the binary image as the mobile pixel ratio in the binary image, so as to determine the disturbance information of the single frame image according to the mobile pixel ratio. For example, when the mobile pixel ratio exceeds a first preset ratio threshold, it is determined that the camera is in a preset disturbance state, and when the mobile pixel ratio does not exceed the first preset ratio threshold, the camera is in a preset static state.
[0089] In some implementations, before determining disturbance information of a shooting mechanism that shoots the video stream according to the video stream, the method further includes:
[0090] Sample time-detected images from a video stream;
[0091] Determine whether the shooting time corresponding to the video stream is daytime or nighttime according to the time detection image;
[0092] If the shooting time is daytime, the binarization threshold is set to a first preset value;
[0093] If the shooting time is at night, the binarization threshold is set to a second preset value, wherein the first preset value is smaller than the second preset value.
[0094] It should be understood that the camera mechanism is, for example, a security surveillance camera that captures color images during the day and infrared night vision images at night. Since the images captured in the infrared night vision scene have more noise points, the first preset value is less than the second preset value. For example, the first preset value is 20 and the second preset value is 35.
[0095] In some embodiments, determining the human figure movement information in the human figure detection area based on the video stream includes: traversing all human figure detection areas in a single-frame image, determining the pixels in the human figure detection area that move between two frames of images based on the previous frame of the single-frame image, calculating the human figure movement ratio of the pixels that move in the human figure detection area, and then determining the human figure movement information based on the human figure movement ratio.
[0096] Exemplarily, when the proportion of human figure movement exceeds a second preset proportion threshold, it is determined that the human figure is stationary, and the human figure detection area is a human figure stationary area; when the proportion of human figure movement does not exceed the second preset proportion threshold, it is determined that the human figure is moving, and the human figure detection area is a human figure moving area.
[0097] Step 4: Filter out the target detection area corresponding to the single-frame image from the human detection area of the single-frame image based on the confidence, disturbance information and human movement information.
[0098] It should be noted that the target detection area corresponding to the single-frame image is selected from the human detection area by combining the confidence, disturbance information and human movement information, in order to eliminate the false detection effects of external interferences such as lens disturbance and human movement on human detection, so as to improve the accuracy of human detection. It should also be noted that after obtaining the target detection area, the target detection area can be directly output as the human detection result corresponding to the single-frame image.
[0099] See also Figure 5 , Figure 5 A logical judgment diagram for screening a target detection area in a human figure detection method provided in an embodiment of the present application.
[0100] like Figure 5 As shown, in some embodiments, selecting a target detection area from a human detection area of a single frame image according to confidence, disturbance information, and human movement information includes:
[0101] When it is determined according to the disturbance information that the shooting mechanism is in a preset static state when shooting a single frame image, each human figure detection area is determined to be a human figure static area or a human figure moving area according to the human figure movement information;
[0102] Compare and match the human-shaped static area with a preset human-shaped sample set, wherein the human-shaped sample set includes at least one sample area;
[0103] If there is at least one sample area that successfully matches the human-shaped static area, and the confidence of the human-shaped static area exceeds a first confidence threshold, the human-shaped static area is determined to be a target detection area.
[0104] Specifically, first determine whether the shooting mechanism is in a preset static state or a preset disturbance state when shooting a single-frame image based on the disturbance information; under the condition that the shooting mechanism is in a preset static state when shooting a single-frame image based on the disturbance information, determine whether each human figure detection area is a human figure static area or a human figure moving area based on the human figure movement information, and adopt corresponding strategies to process the human figure static area or the human figure moving area.
[0105] Specifically, the strategy for processing the human figure static area by the device executing the present method includes: comparing and matching the human figure static area with a preset human figure sample set, wherein the human figure sample set includes at least one sample area; if there is at least one sample area in the human figure sample set that successfully matches the human figure detection area, and the confidence of the human figure detection area exceeds a first confidence threshold, then the human figure detection area is judged to be a target detection area.
[0106] On the other hand, if no sample area in the human shape sample set successfully matches the human shape detection area, that is, all sample areas in the current human shape sample set fail to match the human shape detection area, the human shape detection area is deleted and not output.
[0107] It should be understood that the human sample set specifically includes the sample area recorded by the device in the image of the previous frame, and the sample area represents the human figure that appeared in the image of the previous frame. For the human static area that successfully matches the sample area, under the condition that the confidence exceeds the first confidence threshold, it can be considered that the human static area is the position of the human figure that appeared in the image of the previous frame and has moved to a certain extent, so the human static area is judged as the target detection area. For the human static area that fails to match all sample areas, it can be considered that the human static area is misdetected in the previous recognition process, and the human detection area is deleted and not output to improve the accuracy of human recognition.
[0108] like Figure 5 As shown, in some embodiments, filtering out a target detection area from a human detection area of a single frame image according to confidence, disturbance information, and human movement information also includes:
[0109] When it is determined according to the disturbance information that the shooting mechanism is in a preset disturbance state when shooting a single-frame image, the human figure detection area is determined to be a disturbance area;
[0110] If the confidence level corresponding to the disturbance area exceeds the second confidence level threshold, the disturbance area is determined to be a target detection area.
[0111] Specifically, under the condition that it is determined that the shooting mechanism is in a preset disturbance state when shooting a single-frame image, the human detection area is first determined to be a disturbance area, and then the confidence corresponding to the disturbance area is compared with the second confidence threshold; if the confidence corresponding to the disturbance area exceeds the second confidence threshold, the disturbance area is determined to be a target detection area.
[0112] like Figure 5 As shown, in some embodiments, the method further comprises:
[0113] When it is determined according to the disturbance information that the shooting mechanism is in a preset disturbance state when shooting a single-frame image, the human shape sample set is cleared.
[0114] It should be understood that the human figure sample set specifically includes sample areas recorded by the device in images of previous frames, and the sample areas represent human figures that have appeared in images of previous frames. If the shooting mechanism is in a preset disturbance state when shooting a single frame image, it means that the shooting mechanism has a certain degree of lens jitter due to external interference and / or active adjustment of the shooting angle. In this case, the sample area recorded in the image of the previous frame has a reduced reference value for human figure recognition in subsequent frame images, and therefore all sample areas in the human figure sample set are cleared.
[0115] like Figure 5 As shown, in some implementations, after determining whether a human figure in each human figure detection area moves according to the human figure movement information, the method further includes:
[0116] Traverse all humanoid static areas and compare them with the preset humanoid sample set;
[0117] The sample areas in the human sample set that fail to match all human static areas are deleted.
[0118] It should be understood that the human figure sample set includes multiple sample areas for representing human figures that have appeared in the image of the previous frame. If there is zero or at least one human figure static area in a single frame image, after overlapping and comparing all human figure static areas in the single frame image with the preset human figure sample set, if there is a sample area in the human figure sample set that fails to match all human figure static areas, it can be considered that the human figure that appeared in the image of the previous frame has left the corresponding shooting area, and therefore the sample area is deleted.
[0119] like Figure 5 As shown, in some embodiments, filtering out a target detection area from a human detection area of a single frame image according to confidence, disturbance information, and human movement information also includes:
[0120] If the confidence of the human figure moving region exceeds the third confidence threshold, the human figure moving region is determined to be a target detection region, and the target detection region is recorded as a sample region in the human figure sample set.
[0121] It should be understood that, under the condition that it is judged based on the disturbance information that the shooting mechanism is in a preset static state when shooting a single-frame image, each human figure detection area is judged as a human figure static area or a human figure moving area according to the human figure movement information, and the human figure static area or human figure moving area is processed according to the corresponding strategy.
[0122] Specifically, the strategy for processing the humanoid moving area by the device executing this method includes: if the confidence of the humanoid moving area exceeds the third confidence threshold, determining the humanoid moving area as a target detection area, and recording the target detection area as a sample area in the humanoid sample set.
[0123] It should be understood that in addition to determining the human figure moving area as the target detection area, the target detection area is also added as a sample area to the human figure sample set to update the human figure sample set in real time, thereby, in the subsequent process.
[0124] In some embodiments, the third confidence threshold is greater than the first confidence threshold. It should be understood that the human figure in the human figure moving area moves, so a third confidence threshold higher than the first confidence threshold is set, and only when the confidence of the human figure moving area exceeds the third confidence threshold, it can be confirmed that the human figure moving area is a target detection area.
[0125] In some implementations, after determining that the human-shaped static area is a target detection area, the method further includes:
[0126] If the first confidence threshold does not reach the preset confidence upper limit, the first confidence threshold is increased by a preset step size to update the first confidence threshold;
[0127] If the first confidence threshold has reached the preset upper confidence limit, the first confidence threshold is not increased.
[0128] It should be noted that, in the process of analyzing multiple single-frame images or multiple human-shaped static regions in a single-frame image, the first confidence threshold can be dynamically adjusted. Specifically, the strategy for dynamically adjusting the first confidence threshold includes, after each time a human-shaped static region is determined to be a target detection region, if the first confidence threshold does not reach a preset confidence upper limit, the first confidence threshold is increased by a preset step size to update the first confidence threshold; and if a confidence threshold has reached a preset confidence upper limit, the first confidence threshold is not increased.
[0129] Exemplarily, the initial first confidence threshold can be set to 0.6, the upper confidence limit can be set to 0.8, and the step amplitude can be set to 0.01. That is, each time a human static area is determined to be a target detection area, the first confidence threshold is increased by 0.01 until the first confidence threshold rises to 0.8, and then the first confidence threshold is no longer increased.
[0130] In some implementations, comparing and matching the human-shaped static region with a preset human-shaped sample set includes:
[0131] Compare the human-shaped static area with each sample area to obtain the overlap between the human-shaped static area and the sample area;
[0132] If the overlap exceeds the first overlap threshold, it is determined that the corresponding sample area matches the human detection area successfully;
[0133] If the overlap does not exceed the first overlap threshold, it is determined that the corresponding sample area fails to match the human figure detection area.
[0134] The result of comparing and matching the human-shaped static region with the preset human-shaped sample set includes a successful match and a failed match, and the comparison matching specifically compares the human-shaped static region with each sample region to obtain the overlap between the human-shaped static region and the sample region, and then determines the successful match or failed match based on the magnitude relationship between the overlap and the preset first overlap threshold. Specifically, if the overlap exceeds the first overlap threshold, it is determined that the corresponding sample region successfully matches the human-shaped detection region; if the overlap does not exceed the first overlap threshold, it is determined that the corresponding sample region fails to match the human-shaped detection region.
[0135] For example, the overlap may be represented by an IOU (Intersection over Union) value between the human-shaped static region and the sample region.
[0136] After selecting the target detection area corresponding to the single-frame image from the human figure detection area in the single-frame image according to the confidence, disturbance information and human figure movement information, the target detection area can be directly output as the human figure detection result, or after obtaining the target detection area, the target detection area can be filtered for occasional human figures, and the selected human figure movement area can be used as the human figure detection result corresponding to the single-frame image.
[0137] It should be noted that after the above series of screening strategies, there may still be accidental human figures or falsely detected human figures in the target detection area, so further screening is required to filter out the idol human figures.
[0138] like Figure 4 As shown, in some embodiments, after selecting the target detection area corresponding to the single-frame image from the human shape detection area of the single-frame image according to the confidence, disturbance information and human shape movement information, the method further includes: filtering the target detection area for occasional human shapes; wherein filtering the target detection area for occasional human shapes includes:
[0139] If a single-frame image includes only one target detection area and a moving human figure appears in the target detection area, the target detection area is determined to be a single moving human figure area;
[0140] If the confidence of the single humanoid moving region exceeds the fourth confidence threshold, the single humanoid moving region is retained;
[0141] If the confidence of the single human-shaped moving region does not exceed the fourth confidence threshold, the previous frame image of the single frame image is obtained; if a human-shaped region with a human figure appears in the previous frame image, and the overlap between the human-shaped region and the single human-shaped moving region exceeds the second overlap threshold, the single human-shaped moving region is retained;
[0142] The retained human shape moving area is output as the human shape detection result corresponding to the single frame image.
[0143] It should be noted that, under the condition that a single-frame image includes only one target detection area and a moving human figure appears in the target detection area, the target detection area is determined to be a single human figure moving area, and this single human figure moving area may be an occasional human figure appearing in the single-frame image, so further judgment is required on the single human figure moving area. Specifically, if the confidence of the single human figure moving area exceeds the fourth confidence threshold, the single human figure moving area is retained; if the confidence of the single human figure moving area does not exceed the fourth confidence threshold, the previous frame image of the single frame image is obtained; if a human figure area with a human figure appears in the previous frame image, and the overlap between the human figure area and the single human figure moving area exceeds the second overlap threshold, the single human figure moving area is retained; and the retained human figure moving area is output as the human figure detection result corresponding to the single-frame image.
[0144] See also Figure 6 and Figure 7 , Figure 6 is a schematic block diagram of a human shape detection device provided in an embodiment of the present application. Figure 7 A schematic block diagram of the structure of a security surveillance camera provided in an embodiment of the present application.
[0145] like Figure 6 As shown, an embodiment of the present application also provides a human shape detection device 5, which includes at least a memory 521 and a processor 522, wherein the memory 521 is used to store computer programs, and the processor 522 is used to execute computer programs and implement any one of the human shape detection methods provided in the specification of this application when executing the computer programs.
[0146] like Figure 7 As shown, for example, the human detection device can be a security monitoring camera used to shoot a preset shooting area. Specifically, the security monitoring camera includes a shooting mechanism and a video stream processing mechanism, and the video stream processing mechanism includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program and implement any one of the human detection methods provided in the specification of this application when executing the computer program, thereby filtering out the target detection area.
[0147] Those skilled in the art will understand that Figure 6 and Figure 7The structure shown is merely a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the human shape detection method and human shape detection device to which the embodiment of the present application is applied. The specific human shape detection device and security monitoring camera may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0148] In summary, the embodiment of the present application provides a human figure detection method and a human figure detection device. The human figure detection method provided by the embodiment of the present application includes: obtaining a single frame image from a video stream to be analyzed; identifying and obtaining a human figure detection area in the single frame image and the confidence corresponding to the human figure detection area, wherein the confidence is used to characterize the probability of having a human figure in the human figure detection area; determining the disturbance information of the shooting mechanism that shoots the video stream and the human figure movement information in the human figure detection area according to the video stream; filtering out the target detection area corresponding to the single frame image from the human figure detection area of the single frame image according to the confidence, disturbance information and human figure movement information. It should be noted that the generated target detection area is the human figure detection result. By filtering out the target detection area corresponding to the single frame image from the human figure detection area of the single frame image according to the confidence, disturbance information and human figure movement information, the false detection effect of external interference such as lens disturbance and human figure movement on human figure detection is effectively eliminated, thereby improving the accuracy of human figure detection.
[0149] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0150] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.
[0151] The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A human figure detection method, characterized in that: The method comprises: Get a single frame image from the video stream to be analyzed; Identify and obtain a human figure detection area in the single frame image and a confidence level corresponding to the human figure detection area, wherein the confidence level is used to characterize the probability that there is a human figure in the human figure detection area; Determine disturbance information of a shooting mechanism shooting the video stream and movement information of a human figure in the human figure detection area according to the video stream; A target detection area corresponding to the single-frame image is screened out from the human detection area of the single-frame image according to the confidence level, disturbance information and human movement information.
2. The method according to claim 1, characterized in that The step of selecting a target detection area from the human shape detection area of the single frame image according to the confidence level, the disturbance information and the human shape movement information includes: When it is determined according to the disturbance information that the shooting mechanism is in a preset static state when shooting the single-frame image, it is determined according to the human figure movement information that each of the human figure detection areas is a human figure static area or a human figure moving area; Comparing and matching the human-shaped static area with a preset human-shaped sample set, wherein the human-shaped sample set includes at least one sample area; If there is at least one of the sample areas that successfully matches the human-shaped static area, and the confidence of the human-shaped static area exceeds a first confidence threshold, the human-shaped static area is determined to be the target detection area.
3. The method according to claim 2, characterized in that The step of filtering out a target detection area from the human shape detection area of the single frame image according to the confidence level, disturbance information and human shape movement information further includes: When it is determined according to the disturbance information that the shooting mechanism is in a preset disturbance state when shooting the single-frame image, determining the human figure detection area as a disturbance area; If the confidence level corresponding to the disturbance area exceeds a second confidence level threshold, the disturbance area is determined to be the target detection area.
4. The method according to claim 3, characterized in that The method further comprises: When it is determined according to the disturbance information that the shooting mechanism is in a preset disturbance state when shooting the single-frame image, the human shape sample set is cleared.
5. The method according to claim 2, characterized in that After judging whether the human figures in each of the human figure detection areas move according to the human figure movement information, the method further includes: Traversing all the human-shaped static areas and performing overlapping comparison with a preset human-shaped sample set; The sample areas in the human shape sample set that fail to match all the human shape static areas are deleted.
6. The method according to claim 2, characterized in that The step of filtering out a target detection area from the human shape detection area of the single frame image according to the confidence level, disturbance information and human shape movement information further includes: If the confidence of the human figure moving area exceeds a third confidence threshold, the human figure moving area is determined to be the target detection area, and the target detection area is recorded as the sample area in the human figure sample set.
7. The method according to claim 2, characterized in that After determining that the human-shaped static area is the target detection area, the method further includes: If the first confidence threshold does not reach the preset confidence upper limit, the first confidence threshold is increased by a preset step size to update the first confidence threshold; If the first confidence threshold has reached the preset upper confidence limit, the first confidence threshold is not increased.
8. The method according to claim 2, characterized in that The comparing and matching the human-shaped static area with a preset human-shaped sample set includes: Comparing the human-shaped static area with each of the sample areas to obtain the degree of overlap between the human-shaped static area and the sample area; If the overlap exceeds a first overlap threshold, it is determined that the corresponding sample area successfully matches the human figure detection area; If the overlap does not exceed the first overlap threshold, it is determined that the corresponding sample area fails to match the human figure detection area.
9. The method according to any one of claims 1 to 8, characterized in that The determining, according to the video stream, disturbance information of a shooting mechanism that shoots the video stream comprises: Sampling two consecutive frames of disturbance detection images from the video stream, and obtaining a difference image of the two frames of detection images; Binarizing the difference image according to a preset binarization threshold to obtain a binarized image; The disturbance information is determined according to the proportion of moving pixels in the binary image.
10. The method according to claim 9, characterized in that Before determining disturbance information of a shooting mechanism for shooting the video stream according to the video stream, the method further includes: sampling time detection images from the video stream; Determining, according to the time detection image, whether the shooting time corresponding to the video stream is daytime or nighttime; If the shooting time is daytime, the binarization threshold is set to a first preset value; If the shooting time is at night, the binarization threshold is set to a second preset value, wherein the first preset value is smaller than the second preset value.
11. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: If the single-frame image includes only one target detection area and a moving human figure appears in the target detection area, determining that the target detection area is a single moving human figure area; If the confidence level of the single humanoid moving region exceeds a fourth confidence level threshold, retaining the single humanoid moving region; If the confidence of the single human-shaped moving region does not exceed the fourth confidence threshold, acquiring a previous frame image of the single frame image; if a human-shaped region having a human figure appears in the previous frame image, and the degree of overlap between the human-shaped region and the single human-shaped moving region exceeds the second overlap threshold, retaining the single human-shaped moving region; The retained human figure moving area is output as a human figure detection result corresponding to the single frame image.
12. A human figure detection device, characterized in that: The human shape detection device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the human figure detection method according to any one of claims 1 to 11 when executing the computer program.