Object detection method and apparatus, storage medium, and electronic device

Abstract

Embodiments of the present application provide an object detection method and device, a storage medium and an electronic device, wherein the method comprises: in the case that a target sensor detects that an object of a target type appears in a target region, acquiring a real-time video stream obtained by a target shooting device shooting the target region; determining whether a moving target object appears in the target region in the real-time video stream, wherein the type of the target object is the target type; in the case that it is determined that the moving target object appears in the target region, sending a target signal, wherein the target signal is used to prompt that the moving target object appears in the target region. Through the embodiments of the present application, the problem of low accuracy of object detection in the related art is solved.

Description

Technical Field

[0001] The present invention relates to the field of human detection technology, and more specifically, to an object detection method, apparatus, storage medium, and electronic device. Background Technology

[0002] In recent years, human detection has been used to detect and report intrusions by human targets within the detection range of equipment, and is commonly used in unattended surveillance video recording and automatic alarms. Currently, most human detection relies on infrared sensors for motion detection. However, infrared sensors are susceptible to interference from various heat sources and light sources, and their sensitivity decreases when the ambient temperature is close to human body temperature, leading to false alarms. Therefore, related technologies suffer from low accuracy in object detection.

[0003] There is currently no effective solution to the problem of low accuracy in object detection in related technologies. Summary of the Invention

[0004] This invention provides an object detection method, apparatus, storage medium, and electronic device to at least address the problem of low accuracy in object detection in related technologies.

[0005] According to an embodiment of the present invention, an object detection method is provided, comprising: when a target sensor detects an object of a target type in a target area, acquiring a real-time video stream captured by a target imaging device of the target area; determining in the real-time video stream whether a moving target object appears in the target area, wherein the type of the target object is the target type; and when the moving target object is determined to appear in the target area, sending a target signal, wherein the target signal is used to indicate that a moving target object has appeared in the target area.

[0006] According to another embodiment of the present invention, an object detection device is also provided, comprising: an acquisition module, configured to acquire a real-time video stream captured by a target imaging device of the target area when a target sensor detects an object of the target type in the target area;

[0007] A determination module is used to determine whether a moving target object appears in the target area in the real-time video stream, wherein the type of the target object is the target type;

[0008] A sending module is configured to send a target signal when it is determined that the moving target object has appeared in the target area, wherein the target signal is used to indicate that the moving target object has appeared in the target area.

[0009] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0010] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0011] This invention addresses the issue of low object detection accuracy in related technologies by filtering the sensor's judgment results based on real-time video streams. Instead of directly alerting staff or users when the target sensor detects an object in the target area, it further analyzes the real-time video stream to determine if a moving target object is present. This reduces false alarms caused by interference with the target sensor and improves the accuracy of object detection. Attached Figure Description

[0012] Figure 1 This is a block diagram of the mobile terminal hardware structure of the object detection method according to an embodiment of the present invention;

[0013] Figure 2 This is a flowchart of an object detection method according to an embodiment of the present invention;

[0014] Figure 3 This is a schematic diagram of a binary image according to an embodiment of the present invention;

[0015] Figure 4 This is a schematic diagram of a target bounding box mapped onto a binary image according to an embodiment of the present invention;

[0016] Figure 5 This is a schematic diagram of the RTOS system architecture according to a specific embodiment of the present invention;

[0017] Figure 6 This is a schematic diagram of the overall process of object detection according to a specific embodiment of the present invention;

[0018] Figure 7 This is a structural block diagram of an object detection device according to an embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0021] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a block diagram of the mobile terminal hardware structure of the object detection method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0022] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the object detection method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0023] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0024] This embodiment provides an object detection method. Figure 2 This is a flowchart of an object detection method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0025] Step S202: When the target sensor detects an object of the target type in the target area, acquire the real-time video stream captured by the target shooting device on the target area;

[0026] Step S204: Determine whether a moving target object appears in the target area in the real-time video stream, wherein the type of the target object is the target type;

[0027] Step S206: If it is determined that the moving target object has appeared in the target area, a target signal is sent, wherein the target signal is used to indicate that the moving target object has appeared in the target area.

[0028] In this implementation, the target sensor is an infrared sensor. Human body temperature is typically 36-37 degrees Celsius, emitting infrared radiation of a specific wavelength of about 10µm. The infrared sensor captures this specific wavelength range in real time, detecting the infrared radiation emitted by a person and converting it into an electrical signal output. When someone enters the sensing range (i.e., the target area), the infrared sensor detects the human infrared radiation and issues a warning signal. Detecting a target-type object within the target area means detecting that someone has entered the target area.

[0029] The target sensor performs an initial detection to determine whether a human-shaped object is present in the target area. If the target sensor detects a target-type object in the target area, the video stream captured by the target shooting device further confirms whether a target-type object is present in the target area. Only if the target object is moving will a target signal be emitted to alert staff or users that a moving target object has appeared in the target area.

[0030] Through the above steps, when the target sensor detects a target-type object in the target area, instead of directly issuing a prompt signal to the staff or user, it further judges whether there is a moving target object in the target area based on the real-time video stream, and filters the judgment results of the target sensor, effectively filtering out false alarms caused by interference with the target sensor. This solves the problem of low object detection accuracy in related technologies and achieves the effect of improving the accuracy of object detection.

[0031] The execution entity for steps S202 to S206 can be an RTOS (Real-Time Operating System). This system can quickly respond to and process data, schedule all available resources, and control all real-time tasks to run in a coordinated manner while completing real-time tasks. When the target sensor detects a target object in the target area, it issues a prompt signal. Upon receiving this prompt, the RTOS system quickly starts, begins acquiring real-time data streams, and determines whether a moving target object exists in the target area. If a target object is present, it sends a target signal.

[0032] Because the target sensor has low power consumption and can run continuously for a long time, and the RTOS system has the advantages of fast startup, high efficiency and low power consumption, the combination of the target sensor and the RTOS can improve the accuracy of object detection while reducing the power consumption of the device.

[0033] In an optional embodiment, determining whether a moving target object appears in the target area in the real-time video stream includes: acquiring multiple frames of images in the real-time video stream and identifying objects of the target type in the multiple frames of images; if the same object of the target type is identified in K consecutive frames of the multiple frames of images, determining that the identified same object is the target object; and if the target object is in a moving state, determining that the moving target object appears in the target area.

[0034] In this embodiment, the real-time video stream captured by the target shooting device is acquired. The real-time video stream is the video stream that the target shooting device sends the acquired video data to the RTOS system while shooting. That is, when acquiring the real-time video stream, it is either acquired frame by frame, or a preset number of frames are acquired each time, or the real-time video stream is acquired once every predetermined time interval. In order to ensure real-time performance, the predetermined time interval should be as short as possible.

[0035] In the multi-frame images acquired from the real-time video stream, the target type of object is identified frame by frame in the image, and a target bounding box is added to the image for the target object that appears in the image. The target bounding box is used to identify the object.

[0036] To filter out false alarms caused by false detections in a single frame and improve the accuracy of object detection reporting, only objects that are identified in K consecutive frames are considered to have appeared in the target area, i.e., a human-shaped target appears in the target area. Once the target area is determined to have appeared, the motion state of the target object is judged. If the target object is moving, a target signal is sent.

[0037] Optionally, while acquiring the real-time video stream, the acquired image frames are cached. The cached image frames are used to generate and save the recording when the target signal is sent. The caching starts from the first acquired image frame. When a moving target object is determined to appear in the target area, all cached image frames are saved. New video frames acquired after the moving target object is determined to appear in the target area are also saved until the recording duration reaches a preset duration threshold or the number of saved image frames reaches a preset frame number threshold.

[0038] In an optional embodiment, when the same object of the target type is identified in K consecutive frames of the multi-frame images, determining that the identified same object is the target object includes: if the first object of the target type is first identified in the i-th frame of the multi-frame images, repeating the following steps until the first object is identified in K-1 consecutive frames after the i-th frame, or the first object is not identified in the frame after the i-th frame, where the initial value of j is 1, and i is a positive integer greater than or equal to 1: determining whether there is an object matching the first object in the j-th object set, wherein the j-th object set includes objects of the target type identified in the (i+j)-th frame; if there is an object matching the first object in the j-th object set, determining that the first object is identified in the (i+j)-th frame; updating j to j+1; if there is no object matching the first object in the j-th object set, determining that the first object is not identified in the (i+j)-th frame.

[0039] In this embodiment, when determining whether the same object is identified in K consecutive frames, firstly, when the first object of the target type is first identified in the i-th frame, it is determined whether the first object is identified in all K-1 consecutive frames after the i-th frame. If the first object is identified in all frames, it is determined that the same object is identified in K consecutive frames.

[0040] Determine whether the first object has been identified in all K-1 consecutive frames after the i-th frame. Repeat the following operations for the K-1 consecutive frames after the i-th frame: identify all target types in the i+j-th frame to obtain the j-th object set. Search the j-th object set for an object that matches the first object. If an object that matches the first object exists in the j-th object set, then the first object has been identified in the i+j-th frame. Record the position information of the first object in the i+j-th frame, including the position and size of the bounding box.

[0041] When determining whether the same object appears in multiple consecutive frames, the object identified in the current frame is matched with the object identified in the previous frame. If the object matches successfully, it means that it was identified in both frames. If the object identified in the current frame does not match successfully, it means that the object appears for the first time in the current frame. If the object identified in the previous frame does not match successfully, it is determined that the object does not appear in the current frame.

[0042] In an optional embodiment, determining whether there is an object matching the first object in the j-th object set includes: obtaining a first target box and a first target box set in the (i+j-1)-th frame image for identifying the first object, wherein the first target box set includes target boxes in the (i+j)-th frame image for identifying each object in the first object set; determining the intersection-union ratio (IU / U) of the first target box and each target box in the first target box set; and determining that there is an object matching the first object in the j-th object set if the IU / U is greater than a preset threshold.

[0043] In this embodiment, when determining whether there is an object matching the first object in the j-th object set, the target box is used to determine whether there is a match with the first object. The target box of the first object in the (i+j-1)-th frame image is the first target box. The target boxes of all target types of objects identified in the (i+j)-th frame image form a target box set, which contains one or more target boxes. The intersection-union ratio (IUR) of the first target box is calculated with the target boxes in the target box set. If there is an IUR greater than a preset threshold, it is determined that there is an object matching the first object in the j-th object set, and the object corresponding to the IUR is determined as the first object. The corresponding target box is determined as the target box used to identify the first object in the (i+j)-th frame image.

[0044] It should be noted that the intersection-union ratio (IUR) reflects the degree of overlap between two target boxes. The larger the IUR, the larger the overlapping area between the two target boxes.

[0045] In an optional embodiment, when the target object is in a moving state, determining that the moving target object appears in the target region includes: acquiring a target image set, wherein the target object is identified in each image in the target image set, the target image set includes the i-th frame image in the multi-frame image set, the i-th frame image is the image in the multi-frame image set where the target object is first identified, and i is a positive integer greater than or equal to 1; determining the displacement of the target object in each image in the target image set, and determining the movement direction of the target object in each image in the target image set; determining that the moving target object appears in the target region when there are images in the target image set where the displacement of the target object is greater than a preset displacement, and the number of images in the target image set with the same movement direction of the target object is greater than a preset number threshold.

[0046] In this embodiment, if it is determined that there is a target object in the target area, the motion state of the target object is further determined. If the motion state is moving, the target signal is sent; otherwise, if the motion state is stationary, the target signal is not sent.

[0047] In determining the motion state of the target object, it is necessary to determine whether the displacement of the target object is greater than a threshold and whether the target object has a main motion direction.

[0048] Obtain the target image set, which includes all or part of the images that identify the target object. The first frame of the target image set is image 1, which is the i-th frame mentioned above. The displacement of the target object in each image in the target set refers to the distance between the target point on the bounding box corresponding to the target object in each image and the target point on the bounding box corresponding to the target object in the first image. The target point can be a point in the same bounding box, such as the center point or the top left corner of the bounding box.

[0049] To determine whether a target object has a main direction of motion, first determine the direction of motion of the target object in each image. The direction of motion of the target object in each image is determined by the relative position and direction between the target bounding box of the target object in each image and the target bounding box of the target object in the previous image.

[0050] Determine whether the target object in each image moves in the same direction. If the number of images with the same direction of movement is greater than a preset threshold, determine that the target object has a main direction of movement. For example, if it is determined that the target object moves to the right in the images that have exceeded the preset threshold, then the target object has a main direction of movement, and the main direction of movement is to the right.

[0051] In an optional embodiment, determining the displacement of the target object in each image of the target image set includes: obtaining the m-th position of the target object in the m-th image of the target image set, and the first position of the target object in the 1-th image of the target image set, wherein the 1-th image is the i-th frame image, and m is a positive integer greater than or equal to 1; determining the displacement of the target object in the m-th image by the distance between the m-th position and the 1-th position.

[0052] In this embodiment, the m-th position is the position of the target point on the target bounding box corresponding to the target object in the m-th image. For example, the center point of the target bounding box is selected as the m-th position. The displacement of the target object in the m-th image is determined by the distance between the m-th position and the 1st position.

[0053] In an optional embodiment, determining the motion direction of the target object in each image of the target image set includes: obtaining the m-th target box in the m-th image of the target image set and the (m+1)-th target box in the (m+1)-th image of the target image set, where m is a positive integer greater than or equal to 1; and determining the motion direction of the target object in the (m+1)-th image based on the m-th target box and the (m+1)-th target box.

[0054] In this embodiment, the direction of motion can be along the directions decomposed from the horizontal and vertical directions, namely the four directions of up, down, left, and right. By examining the m-th and (m+1)-th bounding boxes of the target object, the direction of motion of the target object in the (m+1)-th image can be obtained.

[0055] Optionally, the direction of the target object's movement—whether it moves to the right or left—can be determined by the differences in the coordinates of the top-left corner, bottom-right corner, and center point of the m-th target box and the (m+1)-th target box. This can be achieved through the following method:

[0056] △X_m+1_left_up=X_m+1_left_up–X_m_left_up;

[0057] △X_m+1_center=X_m+1_center–X_m_center;

[0058] △X_m+1_right_low=X_m+1_right_low–X_m_right_low;

[0059] Where X represents the x-coordinate, left_up represents the top left corner, right_low represents the bottom right corner, center represents the center point, X_m+1_left_up represents the x-coordinate of the top left corner of the (m+1)th bounding box of the target object in the (m+1)th image, X_m+1_left_up represents the x-coordinate of the top left corner of the (m+1)th bounding box of the target object in the (m+1)th image, X_m+1_center represents the x-coordinate of the center point of the (m+1)th bounding box of the target object in the (m+1)th image, X_m_center represents the x-coordinate of the center point of the (m)th bounding box of the target object in the (m)th image, X_m+1_right_low represents the x-coordinate of the center point of the (m+1)th bounding box of the target object in the (m+1)th image, and X_m_right_low represents the x-coordinate of the center point of the (m)th bounding box of the target object in the (m)th image.

[0060] If two or more of the three variables △X_m+1_left_up, △X_m+1_center, and △X_m+1_right_low are greater than 0, the target object is judged to be moving to the right; if two or more of the three variables are less than 0, the target object is judged to be moving to the left; otherwise, the direction of movement cannot be determined.

[0061] Similarly, by examining the vertical coordinate Y, we can determine whether the target object is moving upwards or downwards.

[0062] Optionally, the motion state of the target object can also be determined by calculating the proportion of foreground pixels in the target box. Specifically, each pixel is classified into two categories. By analyzing the changes in pixel values ​​of pixels at the same position between consecutive frames, each pixel is divided into foreground and background pixels. The foreground pixel is the pixel on the target object.

[0063] The process involves comparing each pixel in the m-th image with its corresponding pixel in the (m+1)-th image to determine the difference in pixel values ​​between the two images. If the difference is greater than a preset pixel threshold, the pixel in the (m+1)-th image is designated as a foreground pixel; otherwise, it is designated as a background pixel. Figure 3 This is a schematic diagram of a binary image according to an embodiment of the present invention, such as... Figure 3 As shown, the final output is a binary image of the same size as the original image.

[0064] Figure 4 This is a schematic diagram of the target bounding box being mapped onto a binary image according to an embodiment of the present invention, such as... Figure 4The m-th bounding box of the target object in the m-th image is mapped to the binary image above. The foreground pixel ratio of the m-th bounding box is obtained by dividing the number of foreground pixels in the m-th bounding box by the total number of pixels enclosed by the bounding box. If the foreground pixel ratio of the m-th bounding box is greater than a preset ratio threshold, the target object is determined to be moving.

[0065] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments.

[0066] The present invention will be specifically described below with reference to embodiments:

[0067] Figure 5 This is a schematic diagram of the RTOS system architecture according to a specific embodiment of the present invention, such as... Figure 5 As shown, the RTOS system consists of a target detection module, a background modeling module, a target tracking module, and a target signal determination module.

[0068] The target detection module primarily identifies and locates objects of target types within the target area, obtaining their location information. This embodiment employs the YOLO target detection algorithm based on neural networks to detect common target categories in monitoring scenarios, such as pedestrians, animals, and vehicles. Other target detection algorithms, such as the RCNN series and SSD, can achieve similar results. Thanks to data-driven deep learning algorithms, targets that may cause false PIR alarms, such as animals, light sources, and heat sources, can be effectively filtered out.

[0069] The background modeling module's main function is to detect moving targets. This proposal uses the Vibe algorithm for background modeling, but it is not limited to Vibe; other background modeling algorithms such as Gaussian Mixture Model (GMM) and Vibeplus can achieve similar results. Background modeling can be considered a process of binary classification of pixels. By analyzing the changes in pixel values ​​at the same location across consecutive frames, each pixel is divided into foreground and background points; the foreground point represents the moving target. The output of the background modeling module is a foreground image, a binary image of the same size as the original image.

[0070] The target tracking module primarily performs inter-frame correlation on target bounding boxes detected in different images, identifies the same target in adjacent frames, and maintains target status and historical trajectory information.

[0071] The target state of a target object can be divided into five types: init, create, update, lost, and delete. The first target to appear is in the init state. If a target in the init state successfully matches for k consecutive frames, it enters the create state. If a target in the create state still matches successfully in subsequent frames, it changes to the update state; otherwise, it enters the lost state. Targets in the lost state can still participate in matching in subsequent frames until they fail to match for n consecutive frames, reaching the delete state. In this embodiment, only targets in the update state can trigger an alarm, effectively filtering out false alarms caused by misdetection or missed detection in a single frame. For the same target object, if a match is successful, its position information is recorded, thus obtaining the target's motion trajectory for subsequent module analysis.

[0072] The target signal determination module determines whether to emit a target signal based on the output results of target tracking and background modeling.

[0073] To send a target signal, three conditions must be met simultaneously: Condition 1: The target object is a target type (filtering out false alarms caused by animals, heat sources, light sources, etc.). Condition 2: The target object is in an update state (filtering out false alarms caused by false detections or missed detections in a single frame image). Condition 3: The target object is in motion (filtering out false alarms caused by stationary targets).

[0074] The conditions for determining that an object is in motion are as follows: Condition 1: The target displacement is greater than a set threshold; Condition 2: The target has a main direction of motion; Condition 3: The percentage of pixels in front of the target bounding box is greater than a set threshold. If two of these three conditions are met, the target is determined to be in motion.

[0075] Figure 6 This is a schematic diagram of the overall process of object detection according to a specific embodiment of the present invention, such as... Figure 6 As shown, it includes:

[0076] Step 601: Use an infrared sensor to capture wavelengths within a specific range in real time to perform preliminary detection of the target type of object;

[0077] Step 602: Determine whether the infrared sensor has detected an object of the target type. If yes, proceed to step 603; otherwise, proceed to step 601.

[0078] Step 603: Wake up the RTOS system;

[0079] Step 604: Enable video stream push, acquire the captured real-time video stream and cache image frames;

[0080] Step 605: Intelligent video analysis to determine whether a moving target object is present in the target area;

[0081] Step 605: Determine whether a moving target object appears in the target area. If yes, proceed to step 606; otherwise, proceed to step 607.

[0082] Step 606: Send the target signal;

[0083] Step 607: Hibernate the RTOS system. After sending the target signal, end the video algorithm operation; or after the recording is saved, or if the target signal is not triggered, end the video stream push, then hibernate the RTOS system, and then execute step 601 to wait for the next time the infrared sensor detects an object of the target type.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0085] This embodiment also provides an object detection device. Figure 7 This is a structural block diagram of an object detection device according to an embodiment of the present invention, such as... Figure 7 As shown, the device includes:

[0086] The acquisition module 702 is used to acquire a real-time video stream captured by the target imaging device of the target area when the target sensor detects an object of the target type in the target area;

[0087] The determining module 704 is used to determine whether a moving target object appears in the target area in the real-time video stream, wherein the type of the target object is the target type;

[0088] The sending module 706 is configured to send a target signal when it is determined that the moving target object has appeared in the target area, wherein the target signal is used to indicate that the moving target object has appeared in the target area.

[0089] In an optional embodiment, the apparatus is further configured to: acquire multiple frames of images in the real-time video stream and identify objects of the target type in the multiple frames of images; if the same object of the target type is identified in K consecutive frames of the multiple frames of images, determine that the identified same object is the target object; and if the target object is in a moving state, determine that the moving target object appears in the target region.

[0090] In an optional embodiment, the apparatus is further configured to, if a first object of the target type is first identified in the i-th frame of the multi-frame images, repeat the following steps until the first object is identified in all K-1 consecutive frames after the i-th frame, or the first object is not identified in a frame after the i-th frame, wherein the initial value of j is 1, and i is a positive integer greater than or equal to 1: determining whether there is an object matching the first object in the j-th object set, wherein the j-th object set includes objects of the target type identified in the (i+j)-th frame; if there is an object matching the first object in the j-th object set, determining that the first object is identified in the (i+j)-th frame; updating j to j+1; if there is no object matching the first object in the j-th object set, determining that the first object is not identified in the (i+j)-th frame.

[0091] In an optional embodiment, the apparatus is further configured to: acquire a first target bounding box and a first target bounding box set in the (i+j-1)th frame image for identifying the first object, wherein the first target bounding box set includes target bounding boxes in the (i+j)th frame image for identifying each object in the first object set; determine the intersection-union ratio (IU / U) of the first target bounding box and each target bounding box in the first target bounding box set; and, if there is a target bounding box in the first target bounding box set whose IU / U is greater than a preset threshold, determine that there is an object matching the first object in the jth object set.

[0092] In an optional embodiment, the apparatus is further configured to: acquire a target image set, wherein the target object is identified in each image of the target image set, the target image set includes an i-th frame image of the multi-frame images, the i-th frame image being the image in the multi-frame images where the target object is first identified, and i being a positive integer greater than or equal to 1; determine the displacement of the target object in each image of the target image set, and determine the direction of movement of the target object in each image of the target image set; and, if there are images in the target image set where the displacement of the target object is greater than a preset displacement, and the number of images in the target image set where the target object has the same direction of movement is greater than a preset number threshold, determine that a moving target object has appeared in the target region.

[0093] In an optional embodiment, the device is further configured to: obtain the m-th position of the target object in the m-th image of the target image set, and the first position of the target object in the 1-th image of the target image set, wherein the 1-th image is the i-th frame image, and m is a positive integer greater than or equal to 1; and determine the displacement of the target object in the m-th image by the distance between the m-th position and the 1-th position.

[0094] In an optional embodiment, the device is further configured to: acquire the m-th target bounding box in the m-th image of the target image set, and the (m+1)-th target bounding box in the (m+1)-th image, wherein m is a positive integer greater than or equal to 1; and determine the motion direction of the target object in the (m+1)-th image based on the m-th target bounding box and the (m+1)-th target bounding box.

[0095] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0096] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0097] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0098] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0099] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0100] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0101] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An object detection method, characterized in that, include: When the target sensor detects an object of the target type in the target area, a real-time video stream captured by the target imaging device of the target area is obtained; In the real-time video stream, it is determined whether a moving target object appears in the target area, wherein the type of the target object is the target type; If it is determined that the moving target object has appeared in the target area, a target signal is sent, wherein the target signal is used to indicate that the moving target object has appeared in the target area; The step of determining whether a moving target object appears in the target area in the real-time video stream includes: acquiring multiple frames of images in the real-time video stream and identifying objects of the target type in the multiple frames of images; if the same object of the target type is identified in K consecutive frames of the multiple frames of images, determining that the identified same object is the target object; if the target object's motion state is moving, determining that the moving target object appears in the target area; determining that the target object's motion state is moving, further includes: acquiring a target image set, wherein the target object is identified in each image in the target image set, and the target image set includes the first image in the multiple frames of images. The i-th frame image is the image in which the target object is first identified among the multiple frames, where i is a positive integer greater than or equal to 1. The displacement of the target object in each image of the target image set is determined, and the direction of movement of the target object in each image of the target image set is determined. The displacement is the distance between the target point on the target frame corresponding to the target object in each image and the target point on the target frame corresponding to the target object in the first frame image of the target image set. If there are images in the target image set where the displacement of the target object is greater than a preset displacement, and the number of images in the target image set with the same direction of movement of the target object is greater than a preset threshold, then a moving target object is determined to exist in the target region.

2. The method according to claim 1, characterized in that, In the case where the same object of the target type is identified in K consecutive frames of the multi-frame images, determining that the identified same object is the target object includes: If the first object of the target type is first identified in the i-th frame of the multi-frame image, the following steps are repeated until the first object is identified in all K-1 consecutive frames after the i-th frame, or the first object is not identified in the next frame after the i-th frame, where the initial value of j is 1, and i is a positive integer greater than or equal to 1: Determine whether there is an object matching the first object in the j-th object set, wherein the j-th object set includes objects of the target type identified in the (i+j)-th frame image; If an object matching the first object exists in the j-object set, it is determined that the first object is identified in the (i+j)-th frame image; j is updated to j+1. If no object matching the first object exists in the j-th object set, it is determined that the first object was not identified in the (i+j)-th frame image.

3. The method according to claim 2, characterized in that, Determining whether there exists an object matching the first object in the j-th object set includes: Obtain the first target bounding box and the first target bounding box set in the (i+j-1)th frame image for identifying the first object, wherein the first target bounding box set includes the target bounding boxes in the (i+j)th frame image for identifying each object in the first object set; Determine the intersection-union ratio of the first target box with each target box in the first target box set; If there is a target box in the first target box set whose intersection-union ratio with the first target box is greater than a preset threshold, it is determined that there is an object in the j-th object set that matches the first object.

4. The method according to claim 1, characterized in that, Determining the displacement of the target object in each image of the target image set includes: Obtain the m-th position of the target object in the m-th image of the target image set, and the 1-th position of the target object in the 1-th image of the target image set, wherein the 1-th image is the i-th frame image, and m is a positive integer greater than or equal to 1; The displacement of the target object in the m-th image is determined by the distance between the m-th position and the 1-th position.

5. The method according to claim 1, characterized in that, Determining the motion direction of the target object in each image of the target image set includes: Obtain the m-th bounding box used to identify the target object in the m-th image of the target image set, and the (m+1)-th bounding box used to identify the target object in the (m+1)-th image, where m is a positive integer greater than or equal to 1; Based on the m-th target bounding box and the (m+1)-th target bounding box, determine the motion direction of the target object in the (m+1)-th image.

6. An object detection device, characterized in that, include: The acquisition module is used to acquire a real-time video stream captured by the target imaging device on the target area when the target sensor detects an object of the target type in the target area; A determination module is used to determine whether a moving target object appears in the target area in the real-time video stream, wherein the type of the target object is the target type; A sending module is configured to send a target signal when it is determined that the moving target object has appeared in the target area, wherein the target signal is used to indicate that the moving target object has appeared in the target area; The determining module determines whether a moving target object appears in the target area in the real-time video stream in the following manner: acquiring multiple frames of images in the real-time video stream and identifying objects of the target type in the multiple frames of images; if the same object of the target type is identified in K consecutive frames of the multiple frames of images, determining that the identified same object is the target object; if the target object's motion state is moving, determining that the moving target object appears in the target area; if the target object's motion state is moving, the determining module determines that the moving target object appears in the target area in the following manner: acquiring a target image set, wherein the target object is identified in each image in the target image set, and the target image set... The set includes the i-th frame image from the multi-frame images, where the i-th frame image is the first image in the multi-frame images in which the target object is identified, and i is a positive integer greater than or equal to 1; the displacement of the target object in each image in the target image set is determined, and the direction of movement of the target object in each image in the target image set is determined, wherein the displacement is the distance between the target point on the target frame corresponding to the target object in each image and the target point on the target frame corresponding to the target object in the first frame image in the target image set; if there are images in the target image set where the displacement of the target object is greater than a preset displacement, and the number of images in the target image set where the target object has the same direction of movement is greater than a preset number threshold, then it is determined that the moving target object appears in the target region.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 5.

Images