An access door behavior detection method and device, a storage medium and an electronic device

Abstract

Embodiments of the present application provide a door entering and exiting behavior detection method and device, a storage medium and an electronic device. The method comprises: obtaining a single frame target image containing a target door body, and then completing the door entering and exiting behavior detection. After obtaining the target image, the cross-door corner point at the bottom of the target door body is calibrated to determine the cross-door baseline. Based on the length of the cross-door baseline and the preset initial detection width, a behavior detection region is constructed. Then, in combination with the cross-door trigger region determined based on the target object detection box, the space intersection condition of the behavior detection region and the cross-door trigger region is determined to realize the detection of the door entering and exiting behavior of the target object. The method is suitable for the condition of a second-level sampling interval and limited network bandwidth, and improves the detection accuracy of the door entering and exiting behavior.

Description

Technical Field

[0001] This application belongs to the field of computer vision, and in particular relates to a method, device, storage medium and electronic device for detecting entry and exit behavior. Background Technology

[0002] In home security scenarios, entry and exit behavior refers to the passage events of target objects (people, pets, vehicles, etc.) at the entrances and exits of living spaces (such as the front door and yard gate). The time and frequency of these events constitute a residential log of interactions between the living space and the outside world, serving as data support for home security management and enabling the detection of abnormal behaviors such as strangers entering the living space. Therefore, entry and exit behavior detection is a crucial aspect of home security.

[0003] Currently, cloud-based methods for detecting entry and exit behavior primarily target urban home scenarios. These methods typically rely on cameras to capture image data of the entrance and exit areas and upload it to the cloud. Cloud-based algorithms then identify target objects within the image data and analyze their motion trajectories using consecutive image frames to determine if entry or exit behavior has occurred. However, in rural home scenarios, limited network bandwidth leads to delays or data loss during cloud uploads. Furthermore, to reduce network transmission costs, image sampling intervals in rural environments are often extended to the second level, making it difficult to capture the continuous motion trajectories of target objects in the entrance and exit areas, thus hindering accurate detection of entry and exit behavior. Summary of the Invention

[0004] This application provides a method, apparatus, storage medium, and electronic device for detecting entry and exit behavior, which can effectively improve the accuracy of entry and exit behavior detection.

[0005] In a first aspect, embodiments of this application provide a method for detecting entry and exit behavior, the method comprising: Acquire a target image including the target gate; The corner points at the bottom of the target gate in the target image are marked, and the baseline of the gate is determined based on the corner points; Based on the length of the cross-gate baseline and the preset initial detection width, a behavior detection region is constructed; The target image is detected by a preset target detection algorithm, and when a target object is detected, the door-crossing trigger area where the target object enters or exits is determined based on the target detection box of the target object. Based on the spatial intersection of the behavior detection area and the door-crossing trigger area, the target object is detected for its door-entry behavior.

[0006] In one feasible implementation, the corner points of the target door in the target image and the ground are calibrated, and a baseline for the door is determined based on the corner points, specifically including: Send a corner point calibration request for the target image to the user; Receive calibration information returned by the user in response to the corner calibration request, the calibration information including the two cross-door corners located at the bottom edge of the target door body; The line segment connecting the two corner points of the gate is used as the gate baseline.

[0007] In one feasible implementation, a behavior detection region is constructed based on the length of the cross-gate baseline and a preset initial detection width, specifically including: The initial detection width is determined based on the height of the target gate in the target image and a preset scaling factor; Based on the length of the cross-gate baseline and the initial detection width, a behavior detection region is constructed with the cross-gate baseline as the center line.

[0008] In one feasible implementation, a behavior detection region is constructed based on the length of the cross-gate baseline and a preset initial detection width, specifically including: In the target image, the central vertical line of the cross-door baseline is determined, and the target line segment whose central vertical line is intercepted by the edge of the target image is determined; Determine the ratio between the length of the target line segment and the height of the target image, and use the ratio as the tilt correction ratio; The target detection width is determined based on the tilt correction ratio and the initial detection width; The behavior detection region is constructed based on the length of the cross-gate baseline and the target detection width.

[0009] In one feasible implementation, based on the target object's target detection bounding box, the door-crossing trigger area where the target object engages in door-entry behavior is determined, specifically including: Identify the target object category of the target object; From the preset door crossing ratios corresponding to multiple object categories entering and exiting, the target door crossing ratio for the target object category is retrieved; The door-crossing triggering area is determined based on the target door-crossing ratio and the target detection box of the target object.

[0010] In one feasible implementation, the entry / exit behavior of the target object is detected based on the spatial intersection of the behavior detection area and the door-crossing trigger area, specifically including: If there is no spatial intersection between the behavior detection area and the door-crossing trigger area, it is determined that the target object does not have the door-entry behavior; If there is a spatial intersection between the behavior detection area and the door-crossing trigger area, it is determined that the target object has the door-entry behavior; The detection results of the entry and exit behavior and the target image are stored in a preset detection database.

[0011] In one feasible implementation, after determining that the target object has engaged in the entry / exit behavior, the method further includes: Determine the target time node of the target image; Historical images within a preset time range prior to the target time node are selected from the detection database. Based on the spatial location difference of the target object in the historical image and the target image, the behavior type of the target object is determined, and the behavior type includes entering behavior and leaving behavior.

[0012] In one feasible implementation, after determining the behavior type of the target object based on the spatial location difference between the target object in the historical image and the target object, the method further includes: Based on the target object, the behavior type of the target object, and the target time node, alarm prompt information is determined through preset alarm rules.

[0013] Secondly, embodiments of this application provide an entry / exit behavior detection device, the device comprising: The network initialization module is used for...

[0014] Fourthly, embodiments of this application provide an entry / exit behavior detection device, the device comprising: a processor, and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement any of the above-described entry / exit behavior detection methods.

[0015] Fifthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement any of the above-described methods for detecting entry and exit behaviors.

[0016] Sixthly, the embodiments of this application provide a computer program product, wherein when the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs any of the above-described entry / exit behavior detection methods.

[0017] This application discloses a method, apparatus, medium, and device for detecting entry and exit behavior. It can detect entry and exit behavior simply by acquiring a single frame image containing the target door. After acquiring the target image, the corner points at the bottom of the target door can be marked to determine the door-crossing baseline. Based on the length of this baseline and a preset initial detection width, a behavior detection region is constructed. Then, combined with the door-crossing trigger region determined by the target object's detection bounding box, the entry and exit behavior of the target object can be detected based on the spatial intersection of the behavior detection region and the door-crossing trigger region. This method is suitable for conditions with second-level sampling intervals and limited network bandwidth, and improves the accuracy of entry and exit behavior detection.

[0018] Furthermore, when the target door is tilted relative to the target image, the central vertical line of the door baseline can represent the actual vertical direction of the target door. By extracting the target line segment between the central vertical line and the image edge, and using the ratio between the length of the target line segment and the height of the target image as the tilt correction ratio, the behavior detection area can be adaptively corrected, providing a more accurate spatial reference for door entry and exit behavior detection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an entry / exit behavior detection method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the target object detection coverage area in the target image provided in the embodiments of this application; Figure 3 This is a schematic diagram of the adaptive adjustment behavior detection area provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an entry / exit behavior detection device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an entry / exit behavior detection device provided in an embodiment of this application. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0023] In home security scenarios, entry and exit behavior refers to the passage events of target objects (people, pets, vehicles, etc.) at the entrances and exits of living spaces (such as the front door and yard gate). The time and frequency of these events constitute a residential log of interactions between the living space and the outside world, serving as data support for home security management and enabling the detection of abnormal behaviors such as strangers entering the living space. Therefore, entry and exit behavior detection is a crucial aspect of home security.

[0024] Currently, entry and exit behavior detection methods that rely on cloud computing power are mainly aimed at urban home scenarios. They typically rely on camera devices to capture image data of the entrance and exit areas and upload it to the cloud. Based on cloud algorithms, target objects in the image data are identified, and the motion trajectory of the target objects is analyzed in combination with continuous image frames to determine whether entry and exit behavior has occurred.

[0025] However, in rural home settings, there are limitations in network bandwidth, which can lead to delays or data loss when uploading to the cloud. In order to reduce network transmission costs, the image sampling interval in rural settings is increased to the second level, making it difficult to capture the continuous movement trajectory of the target object in the entrance and exit area and to accurately detect the behavior of entering and leaving the house.

[0026] To address the problems in the prior art, embodiments of this application provide an entry / exit behavior detection method, apparatus, device, and computer storage medium.

[0027] In practical applications, the execution subject of the entry / exit behavior detection method of this application embodiment can be a terminal device, such as a desktop computer or laptop computer, or a remote device similar to a server. Of course, the execution subject of this application embodiment can also be a software entity, such as a client or software program installed on a terminal device. The specific type of execution subject corresponding to the technical solution provided in this application embodiment is not strictly limited here, and can be flexibly selected according to the actual application scenario and actual needs.

[0028] The following describes specific embodiments of an entry / exit behavior detection method, apparatus, medium, and device provided in this application. First, an entry / exit behavior detection method is introduced.

[0029] Figure 1 A flowchart illustrating an embodiment of an entry / exit behavior detection method provided in this application is shown. Figure 1 As shown, the method includes steps S100 to S104: S100: Obtain the target image including the target door.

[0030] In one or more embodiments of this application, in order to determine the cross-door baseline of the target door in subsequent steps, this application needs to acquire a target image containing the target door in this step.

[0031] It should be noted that this application does not limit the method of acquiring the target image, which can be set according to actual needs. For example, the target image can be acquired by image acquisition devices (such as surveillance cameras) deployed around the entrance and exit of the user's living space. The acquisition range of the image acquisition device must cover the entire area of ​​the target door (including the bottom of the target door) and the surrounding preset range (for example, by determining the four corner points of the target door, obtaining the smallest bounding rectangle containing the target door, and then expanding outward by twice the range of the smallest bounding rectangle as the center) to achieve the detection of target objects near the target door. At the same time, this application does not limit the acquisition period of the target image, which can be set according to actual needs. For example, in a rural family scene, the sampling interval is on the order of seconds (such as 1-3 seconds per frame). In order to ensure that the target image can be used for the detection of entry and exit behavior in subsequent steps, the bottom of the target door and the ground in the target image must be unobstructed or the obstruction range must be small to avoid affecting subsequent steps. If the acquisition environment of the image acquisition device has problems such as backlighting or dim lighting, this application can perform image preprocessing such as image enhancement and optimization on the target image after acquisition to improve the quality of the target image. In other words, this application has no special requirements for image acquisition equipment, and can support the low-cost security needs of rural families as well as be compatible with smart camera devices in urban family scenarios. Moreover, this application can run on local edge devices (such as home gateways and camera built-in chips) without relying on the high computing power of the cloud, thus reducing hardware and network costs.

[0032] like Figure 2 The diagram shown is a schematic of the target object detection coverage area in a target image provided in an embodiment of this application. The slanted rectangle is the target door 200; the dashed box is the minimum bounding rectangle 201 of the target door, with a size of W*H. The target object detection coverage area is the area expanded around 201, such as having a size of 2W*2H. The outermost rectangle is the target image 203.

[0033] S101: Calibrate the door-crossing corner point at the bottom of the target door in the target image, and determine the door-crossing baseline based on the door-crossing corner point.

[0034] In one or more embodiments of this application, in order to provide a spatial reference and geometric reference for subsequent steps of detecting entry and exit behavior, this step requires marking the baseline across the bottom of the target door in the target image.

[0035] Specifically, this application can mark the door-crossing corner points at the bottom of the target door in the target image, and then determine the door-crossing baseline based on the door-crossing corner points.

[0036] It should be noted that the door-crossing corner point is a feature point where the bottom of the target door meets the ground in the target image. The door-crossing baseline, as a line segment connecting two door-crossing corner points, is essentially a graphical representation of the boundary of the target door, serving as the core decision axis for entry and exit behavior. It directly reflects the position, tilt angle, and length of the target door in the target image, thus providing a spatial reference for subsequent steps of entry and exit behavior detection. Of course, this application does not limit the calibration method of the door-crossing corner points; it can be set according to actual needs, such as target detection, feature point recognition, etc. In one or more embodiments of this application, the calibration of the door-crossing corner points can be achieved through user interaction, as follows: First, this application can send a corner point calibration request for a target image to the user. Second, it receives calibration information returned by the user in response to the corner point calibration request, which includes two corner points spanning the bottom edge of the target door. Finally, the line segment connecting the two corner points is used as the door-crossing baseline. The corner point calibration request can be displayed to the user through an interactive interface (such as home security software), prompting the user to calibrate the two corner points at the bottom of the target door that contact the ground. The user can manually select the two corner points in the target image according to the actual shape of the target door; for example, for regular doors such as rectangular entrance doors, the two right-angled endpoints at the bottom can be selected, or for fence doors or curved doors, the boundary points on both sides of the bottom can be selected. After receiving the user's calibration information, the line segment connecting the two corner points can be determined as the door-crossing baseline based on a straight-line fitting algorithm.

[0037] Furthermore, upon receiving calibration information, the system can detect this information and, if it detects that the user-calibrated corner points do not meet the requirements (the distance between corner points is much smaller than the door length, or the line connecting two corner points is not adjacent to the ground), send a prompt to the user to reconfirm or recalibrate the corner points. When sending the prompt, the system can also display candidate locations for automatically identified corner points for the user's reference, reducing the complexity of the calibration process. Simultaneously, if the target door's shape changes due to modification or replacement, a manual or automatic prompt can be triggered to initiate the recalibration process for the corner points.

[0038] S102: Construct a behavior detection region based on the length of the cross-gate baseline and the preset initial detection width.

[0039] In one or more embodiments of this application, in order to prepare for subsequent steps in determining whether a target object crosses a target door, and to avoid false alarms or missed alarms caused by a fixed behavior detection area, this application needs to construct a behavior detection area adapted to the shape of the target door based on the actual characteristics of the door-crossing baseline and a preset initial detection width.

[0040] Specifically, this application uses the door-crossing baseline as a reference axis and, in conjunction with the length of the door-crossing baseline and a preset initial detection width, constructs a behavior detection area. This behavior detection area is the core spatial basis for determining whether a target object triggers entry / exit behavior and matches the actual passage range of the target door.

[0041] It should be noted that the length of the door-crossing baseline is the lateral detection range of the door-entry / door-exit behavior, while the initial detection width is the longitudinal detection range of the door-entry / door-exit behavior. That is, the detection area covers the door-crossing activity range of the target object when it engages in door-entry / door-exit behavior. The length of the detection area is equal to the length of the door-crossing baseline. Setting the detection area width too large can easily lead to false alarms, while setting it too small can easily lead to missed alarms. This application does not limit the specific size of the initial detection width; it can be set according to actual needs, such as by interacting with the user to determine the initial detection width. In one or more embodiments of this application, the initial detection width can be determined based on the height of the target door in the target image, as follows: This application can determine the initial detection width based on the height of the target door in the target image and a preset scaling factor, and then construct a behavior detection region centered on the door baseline based on the length of the door baseline and the initial detection width.

[0042] In this embodiment, the behavior detection area is not constructed with a fixed size, but rather dynamically adapted to the entry and exit behavior detection requirements of different types of target doors based on the length of the door-crossing baseline. Using a uniform horizontal detection length would result in false alarms due to exceeding the actual door's range in narrow door scenarios, and false alarms due to insufficient coverage of the entire door in wide door scenarios. Simultaneously, the initial detection width can be set according to the door-crossing range of the target object in a home security scenario to cover the target object's activity range when crossing the door. For example, in rural security scenarios, the size of the target door in the image acquisition device varies due to factors such as device parameters and distance from the target door. The farther the image acquisition device is from the target door, the smaller the target door appears in the image, and the smaller the corresponding behavior detection area should be; conversely, the closer the image acquisition device is to the target door, the larger the behavior detection area should be. Therefore, incorporating the height of the target door in the target image into the calculation of the vertical detection range of the behavior detection area allows for adaptation to different application scenarios.

[0043] Furthermore, in urban and rural home security scenarios, the image acquisition angle of the image acquisition device is at a certain tilt angle to the target door. Therefore, the target door in the target image is usually tilted, meaning that the baseline across the door and the width and height of the target image are usually at a certain angle. To further improve the adaptability of the behavior detection region, in one or more embodiments of this application, during the construction of the behavior detection region, the tilt of the target image and the target door can also be corrected, as follows: First, this application determines the central vertical line of the cross-gate baseline in the target image and identifies the target line segment whose central vertical line is truncated by the edge of the target image. Second, it determines the ratio between the length of the target line segment and the height of the target image, and uses this ratio as a tilt correction ratio. Finally, based on the tilt correction ratio and the initial detection width, it determines the target detection width and constructs the behavior detection region according to the length of the cross-gate baseline and the target detection width.

[0044] The formula for calculating the target detection width (vertical coverage area) is as follows:

[0045] In formula (1), For the target detection width, The length of the target line segment. DH represents the height of the target image; DH represents the height of the target gate in the target image. This is a preset scaling factor, which can be 0.1 in practical applications. When there is no tilt between the target gate and the target image, the target detection width is the initial detection width determined based on the height of the target gate and the scaling factor.

[0046] like Figure 3 The diagram shown is a schematic of an adaptive adjustment of the behavior detection area according to an embodiment of this application. In this diagram, the quadrilateral with the filled pattern represents the behavior detection area 300, the dashed quadrilateral represents the target door 301, the solid rectangle represents the target image 302, and the target line segment 303 has a length of [missing information]. .

[0047] In this embodiment, a straight line perpendicular to the midpoint of the doorway baseline is taken as the central perpendicular line. This central perpendicular line intersects the edge of the target image at two points, forming the target line segment. Due to differences in the focal length of the image acquisition device and its distance from the target door, the size of the target door in the captured image varies. Therefore, the behavior detection area needs to be adaptively adjusted according to the size of the target door. Based on the length of the target line segment intercepted by the image edge from the central perpendicular line of the doorway baseline, a tilt correction ratio is calculated. This ratio is then used to correct the initial detection width, preventing the area from deviating longitudinally from the actual doorway movement range due to the tilt of the target door. If the user subsequently reports false alarms in the behavior detection area, the user can be prompted to manually adjust the horizontal length or vertical width by dragging the edge of the detection area in the interactive interface, further reducing scene adaptation errors.

[0048] S103: Perform target detection on the target image using a preset target detection algorithm, and if a target object is detected, determine the door-crossing trigger area where the target object enters or exits the door based on the target object's target detection bounding box.

[0049] In one or more embodiments of this application, in order to accurately locate key areas / structures / parts in the target object that are strongly related to the behavior of entering and exiting, and to avoid misjudgments caused by some areas / structures / parts of the overall area of ​​the target object being close to the target door (such as a pedestrian's body part appearing near the target door being misjudged as having entered or exited), this application can identify the target object in the target image, and then extract the cross-door triggering area that triggers the behavior of entering and exiting the target object from the target detection box of the target object.

[0050] Specifically, this application can use a preset target detection algorithm to detect targets within the coverage area of ​​the target image, and when a target object is detected, determine the door-crossing trigger area where the target object enters or exits the door based on the target detection box of the target object.

[0051] It should be noted that this application does not limit the category of the target object, such as pedestrians, pets, vehicles, etc. After detecting the target object, a target detection box containing the target object is generated, and then a local region directly related to the door-crossing action is extracted from the target object's target detection box as the door-crossing trigger region. However, the door-crossing characteristics of different categories of target objects are significantly different, that is, the extraction logic of the door-crossing trigger region is strongly correlated with the category of the target object. Therefore, in order to improve the accuracy of determining the door-crossing trigger region, in one or more embodiments of this application, this application can identify the target object category of the target object, and then call the target door-crossing ratio of the target object category from a preset set of door-crossing ratios corresponding to multiple object categories entering and exiting door behaviors. Then, this application can determine the door-crossing trigger region based on the target door-crossing ratio and the target detection box.

[0052] For example, for pedestrians, the door-crossing trigger area can be the area below the knees (this area inevitably passes through the behavior detection area when entering or leaving a door). Since the foot area accounts for 25% of the human body structure, the door-crossing ratio for pedestrians is 25%, thus 25% of the target detection box can be used as the door-crossing trigger area. For vehicles, the wheel area can be used as the door-crossing trigger area. In other words, this application can cover multiple categories of target objects, including people, pets, and vehicles. Furthermore, considering the morphological differences of different target objects, the door-crossing trigger area can be extracted from the target detection box according to the door-crossing ratio of different categories, improving detection accuracy.

[0053] Furthermore, since image acquisition devices typically capture images from a top-down perspective, to improve the accuracy of determining the door-crossing trigger region, a vertical correction ratio for the top-down angle can be determined based on the device parameters. This ratio can then be used to vertically correct the door-crossing trigger region. The extraction logic for the door-crossing trigger region is strongly correlated with the category of the target object. This is because different categories of target objects exhibit significant differences in their door-crossing regions / structures / locations. If a uniform door-crossing ratio or a uniform door-crossing trigger region (such as the overall range of the target detection box) is used to determine the door-crossing trigger region, it can lead to misjudgments of entry and exit behaviors (e.g., a pedestrian standing near the target door but not crossing it may have spatial overlap with the behavior detection region). Therefore, the pre-designed target detection algorithm must simultaneously possess the ability to identify the target object category and generate target detection boxes, such as the YOLOX-based pedestrian detection network or the Faster R-CNN algorithm.

[0054] S104: Based on the spatial intersection of the behavior detection area and the door-crossing trigger area, the target object is detected for the door-entry behavior.

[0055] In one or more embodiments of this application, in order to determine whether a target object has entered or exited a door, it is necessary to compare the spatial positional relationship between the behavior detection area (the core determination area of ​​the target door) and the door-crossing trigger area (the key door-crossing area / structure / part of the target object) and detect whether the target object has entered or exited a door from the intersection state of the two.

[0056] It should be noted that, in order to facilitate the tracking of the passage status of the target door, the detection results obtained in this step can be stored in a preset detection database. In one or more embodiments of this application, this application can determine that the target object does not have the behavior of entering or leaving the door when there is no spatial intersection between the behavior detection area and the door-crossing trigger area; and determine that the target object has the behavior of entering or leaving the door when there is a spatial intersection between the behavior detection area and the door-crossing trigger area; and store the detection results of the behavior of entering or leaving the door and the target image in the detection database.

[0057] After identifying that a target object is entering or leaving a door, it is necessary to further distinguish the type of behavior to achieve more refined security management. In one or more embodiments of this application, the application may first determine the target time node of the target image, and then filter historical images within a preset time range before the target time node from the detection database. Subsequently, by comparing the spatial position difference of the target object in the historical images and the target image, the behavior type is determined.

[0058] If the target object is located outside the behavior detection area in the historical image (the side furthest from the user's living space, with the door baseline as the boundary) and inside the behavior detection area in the target image (the side closer to the user's living space), it is determined as an entry behavior; if the target object is located inside the behavior detection area in the historical image and outside the behavior detection area in the target image, it is determined as an exit behavior.

[0059] The target time point can be the image acquisition time of the target image or the reception time of the target image. This preset time range can be set according to actual needs, such as 30 seconds or 5 minutes, and can be adjusted according to the actual sampling interval. This application can also filter historical images based on the target object; that is, historical images must contain the same target object, and filtering can be performed through feature matching, such as the clothing characteristics of pedestrians or the appearance characteristics of vehicles.

[0060] Furthermore, in one or more embodiments of this application, after determining the behavior type, this application can generate targeted alarm notification information by combining target object information, behavior type, and target time node through preset alarm rules. The preset alarm rules can be set according to the security needs of urban and rural families, and the alarm notification information can also be divided into multiple alarm levels (different alarm levels can be set with different alarm notification methods) to achieve timely response and intervention to abnormal entry and exit behaviors. For example: an alarm is triggered when a stranger enters the house late at night (e.g., by sending an emergency push to family members' mobile phones and simultaneously activating local sound and light alarms); a level two alarm is triggered when authorized personnel (such as the elderly) do not leave the house within 24 hours (e.g., by sending a reminder message to authorized members to check); a level three alarm is triggered when pets frequently enter and exit the house (more than 5 times in 1 hour) (e.g., by saving a log for later review). Of course, this application can also push alarm notification information containing key information such as the category of the target object and the time of the behavior to the receiving terminal of authorized members through home security apps, SMS, etc.

[0061] The aforementioned door entry / exit detection method is applicable to various home security scenarios in urban and rural areas. It does not rely on continuous frame trajectory analysis and high computing power in the cloud; accurate detection of door entry / exit behavior can be achieved using only a single frame of the target image. First, a target image containing the target door is acquired. The corner points where the bottom of the target door meets the ground are calibrated through user interaction. These corner points are then connected to form a baseline reflecting the position and tilt angle of the target door. Second, by combining the length of the baseline with a dynamically adjusted detection width, the door tilt and image visual proportions are corrected to construct a behavior detection area adapted to the target door's shape. Finally, a preset target detection algorithm is used to identify target objects (such as people, pets, and vehicles) in the image. Triggering areas (such as pedestrian feet or vehicle wheels) are extracted from the target detection box according to the object category. The spatial intersection of the triggering area and the behavior detection area is then used to determine whether the target object is engaging in door entry / exit behavior. In other words, this application can adaptively adjust the tilt of the target door based on the ratio of the target line segment's length to the target image's height, reducing false alarms and improving the accuracy of door entry / exit behavior detection.

[0062] Furthermore, the detection results and target images are stored in the detection database. If entry / exit behavior is determined, this application can also combine the target image's target time point with the historical images in the detection database and the target image to compare the positional differences of the target object within the target image, distinguishing between entry / exit types. Then, based on preset alarm rules (such as a stranger entering late at night or an authorized person not leaving for an extended period), corresponding alarm notifications are generated and promptly pushed to the user's terminal, providing a low-cost, highly adaptable monitoring solution for home security.

[0063] Based on the aforementioned method for detecting entry and exit behavior, this application also provides a specific embodiment of an entry and exit behavior detection device.

[0064] like Figure 4 As shown, Figure 4 This is a schematic diagram of an entry / exit behavior detection device provided in an embodiment of this application. The device 400 includes an acquisition module 401, a calibration module 402, a detection area module 403, a trigger area module 404, and a behavior detection module 405.

[0065] Acquisition module 401 is used to acquire a target image including the target door body; The calibration module 402 is used to calibrate the door-crossing corner points at the bottom of the target door in the target image, and determine the door-crossing baseline based on the door-crossing corner points; The detection area module 403 is used to construct a behavior detection area based on the length of the cross-gate baseline and the preset initial detection width; The trigger area module 404 is used to perform target detection on the target image using a preset target detection algorithm, and when a target object is detected, to determine the door-crossing trigger area where the target object enters or exits the door based on the target detection box of the target object; The behavior detection module 405 is used to detect the entry and exit behavior of the target object based on the spatial intersection of the behavior detection area and the door triggering area.

[0066] In one feasible implementation, the calibration module is specifically used to send a corner calibration request of the target image to the user; receive calibration information returned by the user in response to the corner calibration request, the calibration information including two cross-door corners located at the bottom edge of the target door; and use the line segment connecting the two cross-door corners as the cross-door baseline.

[0067] In one feasible implementation, the detection region module is specifically used to determine an initial detection width based on the height of the target gate in the target image and a preset scaling factor; and to construct a behavior detection region with the gate-crossing baseline as the center line based on the length of the gate-crossing baseline and the initial detection width.

[0068] In one feasible implementation, the detection region module can also be used to: determine the central vertical line of the cross-gate baseline in the target image, and determine the target line segment intercepted by the edge of the target image; determine the ratio between the length of the target line segment and the height of the target image, and use the ratio as a tilt correction ratio; determine the target detection width based on the tilt correction ratio and the initial detection width; and construct the behavior detection region according to the length of the cross-gate baseline and the target detection width.

[0069] In one feasible implementation, the triggering area module is specifically used to identify the target object category of the target object; from a preset set of multiple object categories corresponding to the door crossing ratios for entering and exiting door behaviors, call the target door crossing ratio of the target object category; and determine the door crossing triggering area based on the target door crossing ratio and the target detection box of the target object.

[0070] In one feasible implementation, the behavior detection module is specifically used to determine that the target object does not have the door entry / exit behavior when there is no spatial intersection between the behavior detection area and the door crossing trigger area; and to determine that the target object has the door entry / exit behavior when there is a spatial intersection between the behavior detection area and the door crossing trigger area; and to store the detection result of the door entry / exit behavior and the target image into a preset detection database.

[0071] In one feasible implementation, the behavior detection module can also be used to determine the target time node of the target image; filter historical images within a preset time range before the target time node in the detection database; and determine the behavior type of the target object based on the spatial position difference between the target object in the historical images and the target image, wherein the behavior type includes entering behavior and leaving behavior.

[0072] In one feasible implementation, the behavior detection module can also be used to determine alarm prompt information based on the target object, the behavior type of the target object, and the target time node, through preset alarm rules.

[0073] Figure 5 A schematic diagram of the hardware structure of an entry / exit behavior detection device provided in an embodiment of this application is shown.

[0074] An entry / exit behavior detection device may include a processor 501 and a memory 502 storing computer program instructions.

[0075] Specifically, the processor 501 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0076] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one embodiment, memory 502 may include removable or non-removable (or fixed) media, or memory 502 may be non-volatile solid-state memory. Memory 502 may be internal or external to the integrated gateway disaster recovery device.

[0077] In one instance, memory 502 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0078] Memory 502 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0079] The processor 501 reads and executes computer program instructions stored in the memory 502 to achieve... Figure 1 An entry / exit behavior detection method is shown in the embodiment.

[0080] In one example, an entry / exit behavior detection device may further include a communication interface 503 and a bus 504. Wherein, as Figure 5 As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 504 and complete communication with each other.

[0081] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0082] Bus 504 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not as a limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 504 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0083] In addition, in conjunction with the entry / exit behavior detection method in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the entry / exit behavior detection methods in the above embodiments.

[0084] This application also provides a computer program product, including a computer program, which, when executed, implements any of the entry / exit behavior detection methods described in the above embodiments.

[0085] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0086] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0087] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0088] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0089] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for detecting entry and exit behavior, characterized in that, The method includes: Acquire a target image including the target gate; The corner points at the bottom of the target gate in the target image are marked, and the baseline of the gate is determined based on the corner points; Based on the length of the cross-gate baseline and the preset initial detection width, a behavior detection region is constructed; The target image is detected by a preset target detection algorithm, and when a target object is detected, the door-crossing trigger area where the target object enters or exits is determined based on the target detection box of the target object. Based on the spatial intersection of the behavior detection area and the door-crossing trigger area, the target object is detected for its door-entry behavior.

2. The method according to claim 1, characterized in that, The process involves calibrating the corner points of the target door in the target image that cross the ground, and determining the baseline of the door crossing based on these corner points. Specifically, this includes: Send a corner point calibration request for the target image to the user; Receive calibration information returned by the user in response to the corner calibration request, the calibration information including the two cross-door corners located at the bottom edge of the target door body; The line segment connecting the two corner points of the gate is used as the gate baseline.

3. The method according to claim 1, characterized in that, Based on the length of the cross-gate baseline and the preset initial detection width, a behavior detection region is constructed, specifically including: The initial detection width is determined based on the height of the target gate in the target image and a preset scaling factor; Based on the length of the cross-gate baseline and the initial detection width, a behavior detection region is constructed with the cross-gate baseline as the center line.

4. The method according to claim 1, characterized in that, Based on the length of the cross-gate baseline and the preset initial detection width, a behavior detection region is constructed, specifically including: In the target image, the central vertical line of the cross-door baseline is determined, and the target line segment whose central vertical line is intercepted by the edge of the target image is determined; Determine the ratio between the length of the target line segment and the height of the target image, and use the ratio as the tilt correction ratio; The target detection width is determined based on the tilt correction ratio and the initial detection width; The behavior detection region is constructed based on the length of the cross-gate baseline and the target detection width.

5. The method according to claim 1, characterized in that, Based on the target object's bounding box, the door-crossing trigger area where the target object engages in door entry / exit behavior is determined, specifically including: Identify the target object category of the target object; From the preset door crossing ratios corresponding to multiple object categories entering and exiting, the target door crossing ratio for the target object category is retrieved; The door-crossing triggering area is determined based on the target door-crossing ratio and the target detection box of the target object.

6. The method according to claim 1, characterized in that, Based on the spatial intersection of the behavior detection area and the door-crossing trigger area, the target object is detected for its door-entry behavior, specifically including: If there is no spatial intersection between the behavior detection area and the door-crossing trigger area, it is determined that the target object does not have the door-entry behavior; If there is a spatial intersection between the behavior detection area and the door-crossing trigger area, it is determined that the target object has the door-entry behavior; The detection results of the entry and exit behavior and the target image are stored in a preset detection database.

7. The method according to claim 6, characterized in that, After determining that the target object has engaged in the entry / exit behavior, the method further includes: Determine the target time node of the target image; Historical images within a preset time range prior to the target time node are selected from the detection database. Based on the spatial location difference of the target object in the historical image and the target image, the behavior type of the target object is determined, and the behavior type includes entering behavior and leaving behavior.

8. The method according to claim 7, characterized in that, After determining the behavior type of the target object based on the spatial location difference between the target object and the historical image, the method further includes: Based on the target object, the behavior type of the target object, and the target time node, alarm prompt information is determined through preset alarm rules.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the entry / exit behavior detection method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the entry / exit behavior detection method as described in any one of claims 1-8.

Images