Humanoid target detection method, electronic device, and storage medium

Abstract

The application relates to the technical field of image detection, and provides a human target detection method, an electronic device and a storage medium. The human target detection method comprises the following steps: acquiring an image frame from a video stream; extracting a foreground image of the image frame by using a mixed Gaussian background model; extracting an outline of the foreground image; calculating the area of a minimum circumscribed rectangle of the outline; judging whether the area of the minimum circumscribed rectangle of the outline is greater than or equal to a preset area threshold; and if the area of the minimum circumscribed rectangle of the outline is greater than or equal to the preset area threshold, performing human target detection on the image frame by using a single-shot multi-box detection (SSD) model. The application can effectively reduce the calculation of the image frame without a human target in the video stream detection task, and improve the human target detection speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular to a human target detection method, an electronic device and a storage medium. BACKGROUND

[0002] Human target detection is one of the core technologies of intelligent security. Using a deep learning model for human target detection has high recognition accuracy, but has the problems of large amount of calculation and long time consumption. SUMMARY

[0003] In view of the above problems, it is necessary to propose a human target detection method and related equipment to improve the human target detection speed.

[0004] The first aspect of the present application provides a human target detection method, the method comprising:

[0005] obtaining an image frame from a video stream;

[0006] extracting a foreground image of the image frame using a Gaussian mixture background model;

[0007] extracting the contour of the foreground image;

[0008] calculating the area of the minimum circumscribed rectangle of the contour;

[0009] determining whether the area of the minimum circumscribed rectangle of the contour is greater than or equal to a preset area threshold;

[0010] if the area of the minimum circumscribed rectangle of the contour is greater than or equal to the preset area threshold, performing human target detection on the image frame using a single shot multi-box detector (SSD) model.

[0011] The SSD model has good effect on human target detection, but also has the problems of complex model design and large amount of calculation. In the process of human target detection in actual scenes, the image frames obtained from the video stream may not have human targets for a long time. If the SSD model is always used to calculate the image frames, a large amount of CPU resources will be occupied, affecting the computer to process other tasks. The human target detection method provided by the present application first extracts the foreground image of the image frame using the Gaussian mixture background model, extracts the contour of the foreground image, determines whether a human target appears according to the size of the contour of the foreground image (i.e. the contour whose minimum circumscribed rectangle has an area greater than or equal to the preset area threshold), and if a human target appears, the SSD model is used for human target detection. The present application can effectively reduce the calculation of image frames without human targets in the video stream detection task, and improve the human target detection speed.

[0012] In another possible implementation, before the foreground image of the image frame is extracted by using the Gaussian mixture background model, the method further includes:

[0013] The Gaussian mixture background model is established.

[0014] In another possible implementation, the establishment of the Gaussian mixture background model includes:

[0015] A plurality of image frames are obtained from the video stream;

[0016] One of the plurality of image frames is taken as a first sample, a Gaussian mixture model is established for each pixel point of the first sample, the Gaussian mixture model is initialized by using the first sample, and the Gaussian mixture background model is obtained, where the Gaussian mixture model includes a plurality of Gaussian distributions;

[0017] Other image frames in the plurality of image frames are taken as second samples, and the Gaussian mixture background model is updated in parameters by using the second samples.

[0018] In another possible implementation, before the contour of the foreground image is extracted, the method further includes:

[0019] The foreground image is subjected to a morphological filtering operation.

[0020] In another possible implementation, the morphological filtering operation on the foreground image includes:

[0021] The foreground image is subjected to erosion and expansion processing.

[0022] In another possible implementation, the method further includes:

[0023] If an area of a minimum circumscribed rectangle of the contour is less than a preset area threshold, prompt information that a human body target is not detected is output.

[0024] In another possible implementation, before the single-shot multi-box detection (SSD) model is used to detect the human body target in the image frame, the method further includes:

[0025] The SSD model is established;

[0026] Human body picture data sets are input to the SSD model for training.

[0027] A second aspect of the present application provides a computer-readable storage medium including computer instructions, when the computer instructions are run on an electronic device, the electronic device executes the human body target detection method as described in the first aspect.

[0028] The third aspect of the present application provides an electronic device, the electronic device comprising a processor and a memory, the memory being configured to store instructions, and the processor being configured to invoke the instructions in the memory, so that the electronic device performs the human target detection method according to the first aspect.

[0029] The fourth aspect of the present application provides a computer program product, when the computer program product is run on a computer, so that the computer performs the human target detection method according to the first aspect.

[0030] It should be understood that the computer readable storage medium of the second aspect, the electronic device of the third aspect, and the computer program product of the fourth aspect provided above all correspond to the method of the first aspect, and thus the beneficial effects achieved thereby can refer to the beneficial effects of the corresponding method provided above, which will not be described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart of the method for establishing a mixed Gaussian background model provided by the embodiments of the present application.

[0032] Figure 2 is a flowchart of human target detection on a single image frame according to the human target detection method provided by the embodiments of the present application.

[0033] Figure 3 is a flowchart of human target detection on a real-time video stream according to the human target detection method provided by the embodiments of the present application.

[0034] Figure 4 is an example of an image frame without a human target in the embodiments of the present application.

[0035] Figure 5 is an example of an image frame containing a human target in the embodiments of the present application.

[0036] Figure 6 is an example of a foreground image extracted from an image frame in the embodiments of the present application.

[0037] Figure 7 is an architecture diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0038] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0039] Many particular details are set forth in the following description in order to provide a thorough understanding of the application. The embodiments described are merely part of the embodiments of the application, and do not limit the application in any way. Based on the embodiments of the application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of the application.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0041] The human target detection method provided by the embodiments of the application can be applied in one or more electronic devices. The electronic device is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and the hardware thereof includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0042] The electronic device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The electronic device can interact with a user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device and the like.

[0043] Figure 1 FIG. 1 is a flowchart of a method for establishing a mixture Gaussian background model according to an embodiment of the application.

[0044] The mixture Gaussian background model is a background representation method based on pixel sample statistical information, which uses the statistical information (such as the number of modes, the mean and variance of each mode, etc.) of the probability density of a large number of sample values of a pixel point in a long time to represent the background, and uses statistical difference (such as the 3σ principle) to judge the target pixel. The background with slight changes can be modeled to be applied in a scene with slight repetitive motion, such as a scene with shaking leaves, rotating fans and light reflection, etc.

[0045] Specifically, as shown in FIG. 1, the method for establishing a mixture Gaussian background model includes the following steps. Figure 1

[0046] 101, obtaining a plurality of image frames from a video stream.

[0047] ​In embodiments of the present application, the electronic device can acquire the video stream in different ways. For example, it can be acquired by a video capture device (such as a mobile phone, a tablet computer, a video camera, etc.), it can be acquired by user uploading, and it can also be acquired from the storage device of the electronic device. The video stream is composed of a plurality of image frames. The video stream acquired by the electronic device has a certain resolution, such as 1080p.

[0048] 102, one image frame in the plurality of image frames is taken as a first sample, a mixture Gaussian model is established for each pixel point of the first sample, and the mixture Gaussian model includes a plurality of Gaussian distributions.

[0049] In the mixture Gaussian model, the processing of each pixel point is independent of each other. For each pixel point in the video image, the change of the pixel point in the sequence image can be regarded as a random process of continuously generating pixel values, that is, a single Gaussian distribution model or a plurality of Gaussian distribution models are used to describe the color presentation rule of each pixel point. For the plurality of Gaussian distribution models, each pixel point of the image is modeled by superposition of a plurality of Gaussian distributions with different weights. Any one image frame can be acquired from the video stream as a first sample, and a mixture Gaussian model is established according to the resolution and the number of channels of the first sample. For example, if a mixture Gaussian model is established for each pixel point of the selected single-channel first sample, 3 to 5 Gaussian distributions need to be established for each mixture Gaussian model, and the initial values of the mean, variance and weight of each Gaussian distribution are all set to 0. If a mixture Gaussian model is established for each pixel point of a three-channel (R, G, B) first sample, a mixture Gaussian model needs to be established for each color channel of each pixel point. For example, a three-channel image frame with a resolution of 20 rows and 15 columns is established to establish a mixture Gaussian model, and 300 mixture Gaussian models are needed for each color channel. Since there are three color channels, 900 mixture Gaussian models are needed in total.

[0050] 103, initializing the mixture Gaussian model using the first sample to obtain a mixture Gaussian background model.

[0051] When initializing the mixture Gaussian model, a Gaussian distribution is first created, the mean of the pixel value of each pixel point of the first sample is determined as the mean of the created Gaussian distribution, the weight of the created Gaussian distribution is set to a preset weight (for example, the preset weight can be 0.1), and the variance of the created Gaussian distribution is set to a preset constant (for example, the preset constant can be 1). The first Gaussian distribution of the mixture Gaussian model is replaced by the created Gaussian distribution, and the first Gaussian distribution of the mixture Gaussian model is taken as an initial mixture Gaussian background model (for the sake of brevity, hereinafter referred to as "background model").

[0052] Figure 4is an image frame example for initializing the mixture Gaussian model, and the image frame does not contain a human-shaped target.

[0053] After the mixture Gaussian model is established for each pixel point of the first sample, and the mixture Gaussian model is initialized by the first sample to obtain the initial background model, other image frames in the plurality of image frames are taken as second samples, and the background model is iteratively updated by the second samples (see 104-108 below).

[0054] In a real scene, the images in the video stream are not fixed pictures, and often have various slight changes. For example, the curtains may be blown due to the wind blowing indoors. The above-mentioned background model taking the first Gaussian distribution as the background model is prone to considering the repeatedly swinging moving objects as human-shaped targets, and thus in this micro-change scene, a single Gaussian distribution cannot be used as the background model. A single Gaussian distribution is only suitable for a scene with relatively single changes, and in order to more clearly express the current changing background, other Gaussian distributions in the mixture Gaussian model also need to be used for the background model. Therefore, the background model needs to be updated in parameters to adapt to the current micro-change scene. For example, the background model is updated in parameters by the second samples to learn the dynamic background, and 100 image frames of the dynamic background can be selected. In the image frames of the changing dynamic background, there can be local or weak fluctuations in the background region, such as the swinging of curtains and the swaying of plants.

[0055] 104. Other image frames in the plurality of image frames are taken as second samples, and for each second sample, it is judged whether the pixel value of the pixel point of the second sample matches the Gaussian distribution of the background model.

[0056] In an embodiment of the present application, whether the pixel value of the pixel point matches the i-th Gaussian distribution of the background model can be judged according to the observation formula (1):

[0057] |X t -μ i，t-1 |<2.5σ i，t-1 (1)

[0058] wherein μ i，t-1 in the observation formula (1) is the mean of the Gaussian distribution of the background model, σ i，t-1 is the standard deviation of the Gaussian distribution, the subscript i indicates that the i-th Gaussian distribution is taken for comparison, and the subscript t indicates that the t-th image frame of the video stream is taken, and the subscript t-1 indicates the previous frame of the t-th image frame.

[0059] If the observation formula (1) is established, it indicates that the pixel value of the pixel point matches the i-th Gaussian distribution; if the observation formula (1) is not established, it indicates that the pixel value of the pixel point does not match the i-th Gaussian distribution.

[0060] 105, if the pixel value of the pixel point of the second sample matches all or part of the Gaussian distribution of the background model, then for the matched Gaussian distribution, the mean, variance and weight of the Gaussian distribution are updated, and for the unmatched Gaussian distribution, the mean and variance of the Gaussian distribution are kept unchanged, and the weight of the Gaussian distribution is updated.

[0061] For example, if the pixel value of the pixel point of the second sample matches the i-th Gaussian distribution of the background model, the mean, variance and weight of the i-th Gaussian distribution are updated. If the pixel value of the pixel point of the second sample does not match the j-th Gaussian distribution of the background model, the mean and variance of the j-th Gaussian distribution are kept unchanged, and the weight of the j-th Gaussian distribution is updated.

[0062] In an embodiment of the present application, the parameters of the background model can be updated according to formulas (2)-(7):

[0063] w i，t =(1-α)w i，t-1 +α(M i，t ) (2)

[0064] μ i，t =(1-ρ)μ i，t-1 +ρX t (3)

[0065]

[0066]

[0067]

[0068]

[0069] If the observation formula (1) is established, M i，t takes 1, and if the observation formula (1) is not established, M i，t takes 0. Wherein, M i，t obtained by the observation formula (1) is substituted into formula (2) for calculation, in formulas (2)-(7), w i，t is the weight, μ i，t is the mean, is the variance, η is the Gaussian probability density, ∑ i，t is the covariance matrix, ρ is the update rate, α is the learning rate, and the subscript K represents the number of Gaussian distributions. Formula (2) is used to update the weight of the Gaussian distribution, formula (3) is used to update the mean of the Gaussian distribution, and formula (4) is used to update the variance of the Gaussian distribution. The learning rate α can be taken between 0 and 1, when α is 0, the parameters are not updated, when α is 1, the parameters are updated frame by frame, if N frames of image frames are taken for parameter update, the learning rate can be set to

[0070] 106, if the pixel point of the second sample does not match any of the Gaussian distributions of the background model, the Gaussian distribution with the minimum weight of the mixture Gaussian model is determined, and the Gaussian distribution with the minimum weight is replaced.

[0071] Specifically, a Gaussian distribution is first created, and the mean of the Gaussian distribution is set according to the pixel value of the pixel point, and the initial values of the variance and weight of the Gaussian distribution are set, and then the created Gaussian distribution is used to replace the Gaussian distribution with the minimum weight in the mixture Gaussian model.

[0072] 107, the weights of the Gaussian distributions of the mixture Gaussian model are normalized, and the Gaussian distributions of the mixture Gaussian model are sorted according to the normalized weights and standard deviations.

[0073] The weights of the Gaussian distributions of the mixture Gaussian model can be normalized according to formula (8):

[0074]

[0075] wherein, is the normalized weight.

[0076] The Gaussian distributions of the mixture Gaussian model can be sorted in descending order according to and the standard deviation σ i，t is the standard deviation of the Gaussian distribution.

[0077] 108, the target Gaussian distribution is selected from the sorted Gaussian distributions to update the background model, and the target Gaussian distribution satisfies:

[0078]

[0079] wherein, T is a threshold for selecting the background model, B is the number of target Gaussian distributions, and the first B Gaussian distributions are calculated from the mixture Gaussian model according to formula (9) as the background model.

[0080] For each second sample, 104 judges whether the pixel value of the pixel point of the second sample matches each Gaussian distribution of the background model, and 108 selects the target Gaussian distribution from the sorted Gaussian distributions as the background model to obtain a background model. The second sample (for example, 100 image frames) is used to update the parameters of the background model, and if it is not the last image frame, the process goes to 104 to obtain the next image frame from the second sample in order to calculate.

[0081] If the second sample has updated the parameters of the background model, the final background model is obtained.

[0082] In actual application, the foreground image can be obtained according to the matching result of each pixel point of the image frame and the corresponding background model. For example, if the pixel point of the image frame matches the corresponding background model, the pixel value of the corresponding position of the foreground image can be set to 0 to represent the background region and display as black; if the pixel value of the pixel point of the image frame cannot match the corresponding background model, the pixel value of the corresponding position of the foreground image can be set to 255 or the pixel value of the image frame to represent the foreground, and if set to 255, display as white. If a black image is finally output after parameter updating of multiple image frames, it is indicated that the background model of the Gaussian mixture model is established.

[0083] Figure 2 The single image frame is a main method for human target detection.

[0084] Specifically, as shown in Figure 2 , the human target detection method comprises:

[0085] 201. An image frame is obtained from a video stream.

[0086] In the embodiment of the present application, the image frame with human target in the video stream can be detected, for example, it can be the image frame with personnel walking in the office, and the image frame has both the person with the occluded lower body and the person walking. Figure 5

[0087] 202. A foreground image of the image frame is extracted by using the Gaussian mixture background model.

[0088] In the embodiment of the present application, the foreground image can be extracted by using the background model established by Figure 1

[0089] In order to enable the background model to learn the changing scene in real time, the parameter of each image frame can be updated according to the formula (2)-(7). If the human target stays in the picture for a long time without moving, the pixel point corresponding to the human target is easy to be regarded as the background after parameter updating of multiple image frames. Therefore, in the scene with fixed camera or little background change, if the human target staying for a long time is not wanted to be regarded as the background, the background model parameter can also be fixed, and real-time parameter updating is not needed, or the parameter updating is only performed periodically according to the scene light change.

[0090] Figure 6 is a schematic view of the extracted foreground image.

[0091] In an embodiment of the present application, the foreground image is obtained by using the background model established by Figure 5 ​​The three-channel image extracts the foreground image. Since each pixel point has three color channels, the foreground image obtained is also three channels. The average value of the pixel values of the three channels of the extracted foreground image can be taken to obtain a single-channel foreground image, and the effect is as shown in the single-channel gray image of Figure 6

[0092] 203, the contour of the foreground image is extracted.

[0093] Contour extraction is to connect the detected edges to form a closed boundary, which can reflect the boundary information, area range and geometric shape of the target. Contour extraction can be performed through the Open Source Computer Vision Library (OpenCv).

[0094] In an embodiment of the present application, the contour extraction of the input foreground image can be performed through the image contour acquisition function findContours of OpenCv, wherein the findContours function can define the contour retrieval mode as CV_RETR_EXTERNAL mode, indicating that only the outermost contour of the human-shaped target in the foreground image is detected. The findContours function can also select various contour fitting modes, for example, CHAIN_APPROX_SIMPLE mode can be selected, indicating that the positions of straight lines and diagonal lines of the obtained contour are optimized, and then the contour is represented in an approximate manner; or CHAIN_APPROX_NONE mode can be selected, indicating that the selected contour is not processed. In order to ensure the continuity, integrity and authenticity of the obtained contour, the present application selects CHAIN_APPROX_NONE mode. Through the contour extraction using the findContours function, the coordinate point set of each contour is obtained, wherein each coordinate point set is composed of multiple coordinates.

[0095] In another embodiment of the present application, in order to improve the accuracy of contour extraction of the foreground image, morphological filtering operation can be performed on the foreground image to remove noise, and then the contour of the denoised foreground image is extracted. Specifically, the foreground image extracted in 202 may contain certain noise. For example, the extracted foreground image may be affected by changing light, resulting in isolated noise points.

[0096] The morphological filtering operation can include: performing an erosion operation on the foreground image to remove isolated noise points; and performing an inflation operation on the foreground image after the erosion operation to restore the original size of the white region and remove the black holes inside the white region.

[0097] After the morphological filtering operation, the denoised foreground image is obtained, and then the contour extraction is performed according to the above-mentioned manner.​

[0098] 204, calculate the area of the minimum bounding rectangle of the contour.

[0099] If the foreground image includes multiple contours, calculating the area of the minimum bounding rectangle of the contour is to calculate the area of the minimum bounding rectangle of each contour in the foreground image.

[0100] In an embodiment of the present application, the area of the minimum bounding rectangle of the contour can be calculated using the boundingRect function of OpenCv. For example, first, the top-left corner coordinates and length and width values of the minimum bounding rectangle of the above-mentioned contour point set are calculated by the boundingRect function, and then the area of the minimum bounding rectangle is calculated according to the length and width values.

[0101] 205, determine whether the area of the minimum bounding rectangle of the contour is greater than or equal to a preset area threshold.

[0102] Determining whether the area of the minimum bounding rectangle of the contour is greater than or equal to a preset area threshold is to determine whether a human target is detected. If the area of the minimum bounding rectangle of the contour is greater than or equal to the preset area threshold, a human target is detected. If the area of the minimum bounding rectangle of the contour is less than the preset area threshold, a human target is not detected. The preset area threshold can be adjusted according to the actual application scenario to select an appropriate preset area threshold. For example, when the distance between the human and the camera is relatively far, the proportion of the human in the entire image frame will be relatively small, and a smaller preset area threshold can be selected so that a human standing far away can be detected.

[0103] In an embodiment of the present application, if the areas of the minimum bounding rectangles of all the contours obtained are less than the preset area threshold, the number of contours extracted is 0, and a prompt information that a human target is not detected will be output.

[0104] 206, if the area of the minimum bounding rectangle of the contour is greater than or equal to the preset area threshold, a Single Shot MultiBox Detector (SSD) model is used to perform human target detection on the image frame.

[0105] The SSD model is an end-to-end target detection model. The model uses the Oxford University Computer Vision Group (Visual Geometry Group Network, VGG) VGG-16 as a base model, adds auxiliary convolution and pooling layers on the basis, obtains features from different layers and makes predictions, generates a fixed-size bounding box set and a score of the object class in the box, and then performs Non-maximum suppression (NMS) to generate the final detection.

[0106] In an embodiment of the present application, before the image frame is detected for human target by using the SSD model, the SSD model needs to be established. The VGG-16 model of the established SSD model is followed by two fully connected layers, and the extracted deep features of the target image are input into the prediction layer and the bounding box generation layer. The establishment of the SSD model includes the following steps: inputting a training set containing multiple human picture samples into the SSD model to train the SSD model. For example, the human picture data set that has been labeled and preprocessed is input into the SSD model for training. The calculated prediction value and the true value are input into the loss function. After continuously training and updating the weight of the SSD model, the loss function converges, and the SSD model can be output. The SSD model is used for human target detection.

[0107] The SSD model has good effect on human target detection, but there are problems of complex model design and large amount of calculation. In the process of human target detection in the actual scene, the image frame obtained from the video stream may not have a human target for a long time. If the SSD model is used for calculation on the image frame all the time, a large amount of CPU resources will be occupied, which affects the computer to process other tasks. The human target detection method provided in the present application first extracts the foreground image of the image frame by using the mixed Gaussian background model, extracts the contour of the foreground image, and judges whether a human target appears according to the size of the contour of the foreground image (i.e. the contour whose minimum bounding rectangle area is greater than or equal to the preset area threshold). If a human target appears, the SSD model is used for human target detection. The present application can effectively reduce the calculation of the image frame without human target in the video stream detection task, and improve the human target detection speed.

[0108] Figure 3 The mixed Gaussian model and the SSD model are combined to detect whether a human target exists in the current image frame. If the mixed Gaussian model detects that a human target exists in the current image frame, the next frame uses the SSD model to detect whether a human target exists in the subsequent frame of the current image frame. If the SSD model fails to detect a human target from the subsequent image frame of the current image frame within a time threshold T, the step of using the mixed Gaussian model to detect whether a human target exists in the current image frame is returned.

[0109] Specifically, as shown in Figure 3 The human target detection method for real-time video stream includes:

[0110] 301, obtaining a first image frame from a video stream.

[0111] 302, detecting a human target in the first image frame by using a mixed Gaussian background model.

[0112] In the human target detection on the first image frame using the mixed Gaussian background model, the mixed Gaussian background model is used to extract a foreground image from the obtained image frame, a contour is extracted from the foreground image, and an area of a minimum circumscribed rectangle of the contour is calculated. Whether the human target exists in the first image frame is determined according to the area of the minimum circumscribed rectangle of the contour. Before the contour is extracted from the foreground image, the foreground image can be de-noised.

[0113] 303, whether the human target is detected from the first image frame is determined.

[0114] If the area of the minimum circumscribed rectangle of the contour is less than the preset area threshold, it indicates that the human target is not detected, and the flow is transferred to 301. If the area of the minimum circumscribed rectangle of the contour is greater than or equal to the preset area threshold, it indicates that the human target is detected, and 304 is executed.

[0115] 304, the first time when the human target is detected is recorded.

[0116] 305, the second image frame is obtained from the video stream, the second time when the second image frame is obtained is recorded, and the second time is after the first time.

[0117] 306, the human target detection is performed on the second image frame using the SSD model.

[0118] 307, whether the human target is detected from the second image frame is determined.

[0119] If the human target is detected from the second image frame, the flow is transferred to 304.

[0120] 308, if the human target is not detected from the second image frame, whether the time difference between the second time and the first time is less than the time threshold is determined.

[0121] If the human target is not detected by the SSD model all the time, the first time remains unchanged, the second time will become larger and larger, and the time difference between the second time and the first time will also become larger and larger.

[0122] If the time difference between the second time and the first time is less than the time threshold, the flow is transferred to 305, and the human target detection is continued using the SSD model. If the time difference between the second time and the first time is greater than or equal to the time threshold, it indicates that the human target is not detected by the SSD model beyond the time threshold, and the flow is ended, and the human target detection is not performed using the SSD model.

[0123] In the embodiment, whether the human target is detected by the SSD model within the time threshold is determined, according to the determination result, the mixed Gaussian background model with small operation amount or the SSD model with large operation amount is selected to perform the human target detection, which can effectively reduce the calculation of the image frame without human target in the real-time video stream detection task, and the human detection speed is improved.

[0124] Referring to Figure 7 Fig. 1 shows a structural schematic diagram of an electronic device provided by the preferred embodiment of the present application. In the preferred embodiment of the present application, the electronic device 70 comprises a memory 701 and at least one processor 702. Those skilled in the art should understand that, Figure 7 The structure of the electronic device shown is not a limitation of the embodiments of the present application, and can be a bus structure or a star structure. The electronic device 70 can further comprise more or less other hardware or software, or different component arrangements.

[0125] In some embodiments, the electronic device 70 comprises a terminal capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions. The hardware of the terminal comprises, but is not limited to, a microprocessor, an application specific integrated circuit, a programmable gate array, a digital processor, and an embedded device.

[0126] It should be noted that the electronic device 70 is only an example, and other existing or future electronic products, such as those adaptable to the present application, should also be included in the protection scope of the present application and are hereby incorporated by reference.

[0127] In some embodiments, the memory 701 is used to store program codes and various data. For example, the memory 701 can be used to store image frames, and can also store a human target detection system 703 installed in the electronic device 70, and to realize high-speed and automatic program or data access during the operation of the electronic device 70. The memory 701 comprises a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically-erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk memories, a magnetic disk memory, a magnetic tape memory, or any other computer-readable storage medium capable of carrying or storing data.

[0128] In some embodiments, the at least one processor 702 can be composed of integrated circuits, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits of the same function or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The at least one processor 702 is the control unit of the electronic device 70, and connects various components of the electronic device 70 through various interfaces and lines, and executes programs or modules stored in the memory 701 and calls data stored in the memory 701 to perform various functions and process data of the electronic device 70.

[0129] In some embodiments, the human target detection system 703 runs in the electronic device 70. The human target detection system 703 can include a plurality of functional modules composed of program code segments. The program code of each program segment in the human target detection system 703 can be stored in the memory 701 of the electronic device 70 and executed by the at least one processor 702 to implement Figure 2 the functions of the human target detection shown.

[0130] In this embodiment, the human target detection system 703 can be divided into a plurality of functional modules according to the functions it performs. The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which are stored in the memory.

[0131] Although not shown, the electronic device 70 can also include a power supply (such as a battery) for powering various components. Preferably, the power supply can be logically connected to the at least one processor 702 through a power management device, so as to realize the functions of managing charging, discharging, and power consumption management, etc. through the power management device. The power supply can also include one or more direct or alternating power supplies, recharging devices, power supply fault detection circuits, power converters or inverters, power supply status indicators, etc. Any components. The electronic device 70 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described here.

[0132] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.

[0133] The integrated unit in the form of the software function module can be stored in a computer readable storage medium. The software function module is stored in a storage medium, and includes instructions for enabling an electronic device (which can be a server, a personal computer, etc.) or a processor to execute part of the method described in each embodiment of the present application.

[0134] In a further embodiment, in combination with Figure 7 The at least one processor 702 can execute the operating system of the electronic device 70 and various application programs (such as the human target detection system 703), program codes, etc. installed, for example, the various modules described above.

[0135] The memory 701 stores program codes, and the at least one processor 702 can call the program codes stored in the memory 701 to execute related functions. The program codes stored in the memory 701 can be executed by the at least one processor 702, thereby realizing the functions of the various modules.

[0136] In an embodiment of the present application, the memory 701 stores one or more instructions (i.e., at least one instruction) executed by the at least one processor 702 to realize Figure 2 The purpose of the human target detection method shown.

[0137] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiment described above is only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner.

[0138] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0139] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or hardware plus software function module.

[0140] It is apparent for a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments, but that the application can be implemented in other concrete forms without departing from the spirit or the basic characteristics of the application. Therefore, the embodiments should be considered as exemplary and not limiting in any way, the scope of the application being defined by the claims appended hereto and not by the above description, and all the changes which fall within the meaning and the scope of the equivalent elements of the claims are therefore intended to be embraced therein. Any reference signs in the claims should not be construed as limiting the claims to the figures in which the reference signs are used. Further, it is explicitly stated that the word comprising does not exclude other elements not listed and the singular does not exclude the plural. Multiple units or apparatuses stated in an apparatus claim can also be implemented by one unit or apparatus by means of software or hardware. The words first, second, etc. are used to indicate names and not a particular order.

[0141] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the present application. Although the present application has been described in detail with reference to the above preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the present application.

Claims

1. A method for detecting humanoid targets, characterized in that, The method includes: Obtain image frames from the video stream; Establish a Gaussian mixture background model, including: Multiple image frames are acquired from the video stream; One of the multiple image frames is taken as the first sample. A Gaussian mixture model is established for each pixel of the first sample. The Gaussian mixture model is initialized using the first sample to obtain the Gaussian mixture background model, wherein the Gaussian mixture model contains multiple Gaussian distributions. Other image frames among the plurality of image frames are used as second samples, and the parameters of the Gaussian mixture background model are updated using the second samples. The foreground image of the image frame is extracted using the Gaussian mixture background model. Extract the contour of the foreground image; Calculate the area of ​​the smallest bounding rectangle of the given contour; Determine whether the area of ​​the minimum bounding rectangle of the contour is greater than or equal to a preset area threshold; If the area of ​​the minimum bounding rectangle of the contour is greater than or equal to a preset area threshold, record the first time. A second image frame is obtained from the video stream, the second time of obtaining the second image frame is recorded, and a human target is detected in the second image frame using a single-shot multi-frame detection (SSD) model. If the humanoid target is not detected from the second image frame, determine whether the time difference between the second time and the first time is less than a time threshold. If so, use the aforementioned Single-Shot Multi-Frame Detection (SSD) model for human target detection; If not, use the Gaussian mixture background model for human target detection.

2. The humanoid target detection method as described in claim 1, characterized in that, Before extracting the contour of the foreground image, the method further includes: Morphological filtering is performed on the foreground image.

3. The humanoid target detection method as described in claim 2, characterized in that, The morphological filtering operation on the foreground image includes: The foreground image is subjected to erosion and dilation processing.

4. The humanoid target detection method as described in claim 1, characterized in that, The method further includes: If the area of ​​the minimum bounding rectangle of the outline is less than a preset area threshold, a prompt message indicating that no human-shaped target was detected is output.

5. The humanoid target detection method as described in claim 1, characterized in that, Before performing human target detection on the image frame using the Single-Shot Multi-Frame Detection (SSD) model, the method further includes: Establish an SSD model; The dataset of human-shaped images is input into the SSD model for training.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the human target detection method as described in any one of claims 1 to 5.

7. An electronic device, characterized in that, The electronic device includes a memory and at least one processor, wherein the memory stores at least one instruction, which, when executed by the at least one processor, implements the human target detection method as described in any one of claims 1 to 5.

Images