A method and system for monitoring video distribution

Abstract

This specification provides a method and system for distributing surveillance video. The method includes acquiring a first video of a first preset area; determining the activity trajectory of a preset object based on the first video, the activity trajectory including at least one of entering a sensitive monitoring area, leaving a sensitive monitoring area, and a regular trajectory; determining the activity scene of the preset object based on the activity trajectory; determining the identity of the preset object; determining the push level of the surveillance video, including the first video, based on the activity scene and identity; and distributing the surveillance video to one or more users based on the push level.

Description

[0001] Case Analysis

[0002] This case is a divisional application of Chinese patent application No. 202210137781.3, entitled "A method and system for distributing surveillance video", filed on February 15, 2022. Technical Field

[0003] This specification relates to the field of image communication, and in particular to a method and system for distributing surveillance video. Background Technology

[0004] With the rapid pace of industrialization and urbanization, many cities have become societies of strangers. Driven by distrust and anxiety towards strangers, more and more users are choosing to install cameras, use smart doorbells, and smart locks with integrated cameras. However, many people may pass by the door, potentially leading to the monitoring of a large number of individuals, some of whom pose no security risk, such as casual passersby. If all monitored information were pushed to the user—for example, by recording videos and sending them to the user's mobile app—the user would be bombarded with a large amount of useless information and easily miss crucial details.

[0005] Therefore, there is an urgent need for a method to distribute surveillance videos in order to better realize the push of surveillance videos. Summary of the Invention

[0006] One embodiment of this specification provides a method for distributing surveillance video. The method includes: acquiring a first video of a first preset area; determining the activity trajectory of a preset object based on the first video, the activity trajectory including at least one of an entry trajectory into a sensitive monitoring area, an exit trajectory from the sensitive monitoring area, and a regular trajectory; determining the activity scene of the preset object based on the activity trajectory; determining the identity of the preset object; determining a push level for the surveillance video based on the activity scene and the identity, the surveillance video including the first video; and distributing the surveillance video to one or more users based on the push level.

[0007] One embodiment of this specification provides a surveillance video distribution system, comprising: a video acquisition module for acquiring a first video of a first preset area; a trajectory determination module for determining the activity trajectory of a preset object based on the first video, the activity trajectory including at least one of an entry trajectory into a sensitive monitoring area, an exit trajectory from the sensitive monitoring area, and a regular trajectory; a scene determination module for determining the activity scene of the preset object based on the activity trajectory; an identity determination module for determining the identity of the preset object; a push level determination module for determining the push level of the surveillance video based on the activity scene and the identity, the surveillance video including the first video; and a distribution module for distributing the surveillance video to one or more users based on the push level.

[0008] One embodiment of this specification provides a surveillance video distribution device, including a processor, which is used to execute a surveillance video distribution method.

[0009] One embodiment of this specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes a surveillance video distribution method. Attached Figure Description

[0010] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0011] Figure 1 These are schematic diagrams illustrating application scenarios of a surveillance video distribution system based on some embodiments of this specification;

[0012] Figure 2 This is an exemplary flowchart of a surveillance video distribution method according to some embodiments of this specification;

[0013] Figure 3 This is an exemplary flowchart of a method for determining an activity scenario according to some embodiments of this specification;

[0014] Figure 4 This is an exemplary flowchart of an identity identification method according to some embodiments of this specification;

[0015] Figure 5 This is another exemplary flowchart of an identity identification method shown in some embodiments of this specification;

[0016] Figure 6 This is a schematic diagram illustrating a surveillance video distribution method according to some embodiments of this specification. Detailed Implementation

[0017] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0018] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0019] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0020] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0021] The surveillance video distribution method and system described in one or more embodiments of this specification can be applied to various scenarios in the security field, such as banks, hotels, hospitals, computer rooms, warehouses, confidential rooms, office buildings, office areas, schools, kindergartens, residential areas, factories, elevators, etc. In some embodiments, the surveillance video distribution method and system can be used with various types of doors, such as swing doors, double doors, sliding doors, folding doors, roller shutters, revolving doors, automatic doors, etc. In some embodiments, the surveillance video distribution method and system can be used with various windows, balconies, rooftops, etc. In some embodiments, the surveillance video distribution method and system can be used with bank counters, ATMs, etc.

[0022] This surveillance video distribution method and system can achieve one or more functions, including: identifying personnel based on features such as face, gait, and attachments; adaptive learning; determining activity scenarios based on activity trajectories; and combining personnel identification and activity scenarios to achieve hierarchical information push from intelligent security devices. This hierarchical security information method and system can achieve one or more beneficial effects, such as accurate personnel identification, reducing or avoiding blind spots, and providing reasonable and effective information push.

[0023] It should be understood that the application scenarios of the surveillance video distribution method and system of this application are merely some examples or embodiments of this application. For those skilled in the art, without creative effort, this application can be applied to other similar scenarios based on these figures.

[0024] Figure 1 This is a schematic diagram illustrating an application scenario of a surveillance video distribution system according to some embodiments of this specification.

[0025] like Figure 1 As shown, the surveillance video distribution system 100 may include a server 110, a processor 112, a terminal 120, cameras 130 (hereinafter referred to as cameras 130) installed in security areas (e.g., in front of doors, on door frames, on window frames, around ATMs, etc.), a storage device 140, and a network 150.

[0026] In some embodiments, server 110 can be used to process information and / or data related to the surveillance video distribution system 100, such as acquiring video and identifying personnel. In some embodiments, server 110 can be a single server or a group of servers. The server group can be centralized or distributed (e.g., server 110 can be a distributed system). In some embodiments, server 110 can be local or remote. For example, server 110 can access information and / or data stored in terminal 120, camera 130, and storage device 140 via network 150. As another example, server 110 can directly connect to terminal 120, camera 130, and / or storage device 140 to access stored information and / or data. In some embodiments, server 110 can be implemented on a cloud platform or provided virtually. By way of example only, the cloud platform can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-tiered cloud, etc., or any combination thereof.

[0027] In some embodiments, server 110 may include processor 112. Processor 112 may process information and / or data related to the surveillance video distribution system 100 to perform one or more functions described in this application. For example, processor 112 may acquire a second video, a first video, and determine a push level.

[0028] In some embodiments, processor 112 may include one or more processing engines (e.g., a single-chip processing engine or a multi-chip processing engine).

[0029] Terminal 120 refers to one or more terminal devices or software used by a user. In some embodiments, the user of terminal 120 may be one or more users, including homeowners, family members, security personnel, property management personnel, etc. In some embodiments, the user's identity can be identified through terminal 120.

[0030] In some embodiments, push information can be presented to the user via terminal 120, for example, via the user's mobile phone or via a display of a building visualization system. In some embodiments, terminal 120 can be one or any combination of other devices with input and / or output functions, such as mobile device 120-1, tablet computer 120-2, laptop computer 120-3, desktop computer 120-4, etc.

[0031] In some embodiments, the mobile device 120-1 may include a mobile phone, smartphone, personal digital assistant (PDA), navigation device, handheld terminal (POS), or any combination thereof. In some embodiments, the desktop computer 120-4 may be an in-vehicle computer, in-vehicle television, or the like.

[0032] The security zone may include the area around a door, which may include doors in various locations, such as entrance doors, unit doors, building doors, villa doors, courtyard gates, garage doors, etc. In some embodiments, the door may open inwards and / or outwards. The security zone may also include the area around windows, balconies, rooftops, bank counters, ATMs, etc. In some embodiments, one or more cameras may be installed within the security zone, for example, on the door (e.g., the door frame), around the door (e.g., the walls or other objects where cameras can be installed), on windows, around windows, around balconies, around rooftops, around bank counters, or around ATMs. Camera 130 may include ordinary cameras, high-definition cameras, visible light cameras, infrared cameras, optical flow cameras, night vision cameras, etc. In some embodiments, camera 130 may be installed inside the door, outside the door, behind the door, on the door frame, etc., or any combination thereof. Camera 130 can be used to capture video of a security area (e.g., inside a door, outside a door, behind a door, door frame, outside a window, outside a bank counter, inside a bank counter, etc.). In some embodiments, one or more cameras can transmit the captured video to server 110 via network 150.

[0033] Storage device 140 can be used to store data and / or instructions related to the surveillance video distribution system 100. In some embodiments, storage device 140 can store data obtained / acquired from terminal 120 and / or camera 130. In some embodiments, storage device 140 can store historical data, video data, training samples, etc. In some embodiments, storage device 140 can store data and / or instructions used by server 110 to perform or use in order to complete the exemplary methods described in this application. In some embodiments, storage device 140 may include one or a combination of mass storage, removable storage, volatile read-write storage, read-only storage (ROM), etc. In some embodiments, storage device 140 can be implemented using the cloud platform described in this specification. For example, the cloud platform may include one or a combination of private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, cross-cloud, multi-cloud, etc.

[0034] Network 150 can facilitate the exchange of information and / or data. In some embodiments, one or more components of the surveillance video distribution system 100 (e.g., server 110, camera 130, storage device 140) can transmit information and / or data to other components of the surveillance video distribution system 100 via network 150. For example, camera 130 can transmit video around a door (e.g., first video, second video, image behind the door, etc.) to server 110 via network 150. In some embodiments, the surveillance video distribution system 100 may include one or more network access points. For example, base stations and / or wireless access points 150-1, 150-2, ..., and one or more components of the surveillance video distribution system 100 can connect to network 150 to exchange data and / or information.

[0035] It should be noted that the surveillance video distribution system 100 is provided for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can make various modifications or variations based on the description in this application. For example, the surveillance video distribution system 100 may also include a database. Furthermore, the surveillance video distribution system 100 may implement similar or different functions on other devices. However, these variations and modifications will not depart from the scope of this application.

[0036] In some embodiments, the surveillance video distribution system 100 may include a video acquisition module, a trajectory determination module, a scene determination module, an identity determination module, a push level determination module, and a distribution module.

[0037] In some embodiments, the video acquisition module can be used to acquire a first video in a first preset area and a second video in a second preset area, wherein the security level of the first preset area is higher than that of the second preset area.

[0038] In some embodiments, the trajectory determination module can be used to determine the activity trajectory of a preset object based on a first video.

[0039] In some embodiments, the scene determination module can be used to determine the activity scene of a preset object based on the activity trajectory.

[0040] In some embodiments, the scene determination module can further be used to determine at least one type of preset activity scene and the activity trajectory corresponding to each preset activity scene. The at least one type of preset activity scene includes at least a first type of activity scene and a second type of activity scene. The first type of activity scene includes scenes passing through sensitive monitoring areas; the second type of activity scene includes scenes not passing through sensitive monitoring areas. The module determines a preset activity scene that matches the activity trajectory of a preset object within the at least one type of preset activity scene, and determines the activity scene of the preset object based on the matching preset activity scene. In some embodiments, the activity trajectory corresponding to the first type of activity scene includes at least one of an entry trajectory and an exit trajectory; the activity trajectory corresponding to the second type of activity scene includes a regular trajectory.

[0041] In some embodiments, the scene determination module can also be used to determine the movement state of a preset object based on the first video, the preset object movement state including whether the preset object is moving or not; determine the movement state of the preset object corresponding to each preset activity scene; determine the preset activity scene that matches the activity trajectory and movement state of the preset object in at least one type of preset activity scene, and determine the activity scene of the preset object based on the matching preset activity scene; wherein, the movement of the preset object corresponds to the first type of activity scene, and the absence of the preset object movement corresponds to the second type of activity scene.

[0042] In some embodiments, the scene determination module can also be used to determine the door lock state, which includes an open state or a closed state; determine the door lock state corresponding to each preset activity scene; determine the preset activity scene that matches the activity trajectory and door lock state of the preset object in at least one type of preset activity scene, and determine the activity scene of the preset object based on the matched preset activity scene; wherein the open state corresponds to the first type of activity scene, and the closed state corresponds to the second type of activity scene.

[0043] In some embodiments, the identity determination module can be used to identify the identity of a preset object based on a second video.

[0044] In some embodiments, the identity determination module can also be used to obtain at least one preset object feature from the second video, including facial features, preset object gait features, and preset object accessory features, wherein the preset object accessory features include at least one of clothing features, hairstyle features, and accessories features; and determine the identity of the preset object based on at least one preset object feature.

[0045] In some embodiments, the identity determination module may also be used to determine at least one preset identity and at least one preset object feature corresponding to each preset identity; determine a preset identity that matches at least one preset object feature of the preset object among the at least one preset identity, and determine the identity of the preset object based on the matching preset identity.

[0046] In some embodiments, the identity determination module can also be used to identify the identity of a preset object based on a second video using a machine learning model.

[0047] In some embodiments, the push level determination module can be used to determine the push level of a preset object based on the activity scenario and identity.

[0048] In some embodiments, different identities corresponding to different activity scenarios can correspond to different preset object push levels.

[0049] In some embodiments, different push levels may correspond to different push methods.

[0050] In some embodiments, the distribution module may be used to distribute a first video and / or a second video to one or more users based on push levels.

[0051] It should be noted that the above description of the candidate display, preset object information hierarchical system, and its modules is for ease of description only and should not be construed as limiting this specification to the scope of the embodiments described. It is understood that those skilled in the art, after understanding the principle of this system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from this principle. In some embodiments, Figure 1 The video acquisition module, trajectory determination module, scene determination module, identity determination module, push level determination module, and distribution module disclosed herein can be different modules within a single system, or a single module can implement the functions of two or more of the aforementioned modules. For example, the modules can share a single storage module, or each module can have its own separate storage module. Such variations are all within the scope of protection of this specification.

[0052] Figure 2 This is an exemplary flowchart illustrating a surveillance video distribution method according to some embodiments of this specification. Figure 2 As shown, process 200 may include the following steps. In some embodiments, process 200 may be executed by processor 112.

[0053] Step 210: Acquire a first video from a first preset region and a second video from a second preset region, wherein the security level of the first preset region is higher than that of the second preset region. In some embodiments, step 210 may be performed by a video acquisition module.

[0054] In some embodiments, when a preset object exists in the security area, the video acquisition module can acquire a first video of the first preset area and a second video of the second preset area. The first preset area includes at least a sensitive monitoring area, and the second preset area includes at least a non-sensitive monitoring area. The security level of the first preset area is higher than that of the second preset area.

[0055] Pre-defined objects refer to those that may actively or passively enter the protected area. Pre-defined objects can include people, animals, and movable objects. Protected areas can include residential buildings, factories, office areas, and the interior of bank counters, etc.

[0056] The presence of preset objects in a security zone refers to activities involving preset objects within a certain threshold distance from doors, windows, etc. Examples include: cleaning staff sweeping the hallway or stairs in front of the door, cleaning windows, neighbors passing by, visitors (friends, neighborhood committee members, delivery personnel) knocking or ringing the doorbell, property management staff repairing facilities in front of the door or moving items, family members opening the door (using keys, fingerprints, facial recognition, etc.), customers conducting business at bank counters or ATMs, unauthorized personnel damaging windows or bank counter glass, unauthorized personnel damaging ATMs, cats or birds on balconies, and delivery personnel leaving items at the door. The threshold can be determined based on needs or experience, such as 1 meter, 1.5 meters, 5 meters, 10 meters, etc.

[0057] In some embodiments, the video acquisition module can determine whether a preset object exists in the security area using sensors, such as infrared sensors, vibration sensors, and sound sensors. If the sensors detect people, animals, vibrations, or sounds (footsteps, breathing, rustling of clothing, meows, flapping of wings, etc.), the acquisition of a first video and / or a second video is triggered. In some embodiments, the video acquisition module can automatically trigger the acquisition of the first video and / or the second video, for example, acquiring it every 30 seconds. In some embodiments, security personnel can manually trigger the acquisition of the first video and / or the second video when they see a preset object in the surveillance video.

[0058] For more details about security zones, please refer to [link / reference]. Figure 1 This will not be elaborated upon here.

[0059] In some embodiments, one or more cameras (e.g., cameras installed in the security area) Figure 1The camera 130 described herein can remain on. In some embodiments, some or all cameras can be activated when a preset object is present in the security area. For example, a regular camera (e.g., a low-resolution, low-power camera) can remain on, while other cameras such as a high-definition camera and an optical flow camera can be activated when a preset object is present in the security area. As another example, an infrared camera can remain on at night, while a light source and other cameras such as a high-definition camera and an optical flow camera can be activated when a preset object is present in the security area.

[0060] In some embodiments, the video acquisition module can determine whether a preset object exists in the security area using various methods. For example, by analyzing videos captured by ordinary cameras or infrared cameras in real time, and determining that a face or human figure is present in the video, it can be determined that a preset object exists around the door. Another example is that when footsteps are detected by a sensor, it can be determined that a preset object exists in the security area. Yet another example is that when movement is detected by a sensor, it can be determined that a preset object exists in the security area. The second preset area includes at least a non-sensitive monitoring area. The second video refers to the video captured in the second preset area, such as video outside an entrance door, video outside a factory door, video outside a window, video outside a bank counter, etc. The second preset area can be set according to actual needs.

[0061] In some embodiments, the second video can be captured by various cameras (e.g., ordinary cameras, high-definition cameras, infrared cameras, etc.).

[0062] In some embodiments, the video acquisition module can acquire second video from various cameras via a network. Alternatively, the video acquisition module can be integrated with each camera. In some embodiments, the video acquisition module can acquire second video from various cameras via a bus. In some embodiments, the second video can be acquired through an interface, which includes, but is not limited to, a program interface, a data interface, and a transmission interface. For example, when the information classification system of an intelligent security device is working, it can automatically extract the second video from the interface.

[0063] The first pre-defined area must include at least a sensitive monitoring area. A sensitive monitoring area refers to the area that must be accessed to enter the protected area or to perform a specific operation, such as the area directly below a door frame, the area directly below a window frame, the glass partition of a bank counter and its immediate area, the area connecting a balcony or rooftop to the interior, and the operating area of ​​an ATM. The protected area can include residential buildings, factories, office areas, and the interior of bank counters.

[0064] It is understood that entering and / or leaving the protected area requires passing through a sensitive monitoring area. By recording video of the sensitive monitoring area, all instances of people, animals, or other entities entering and / or leaving the protected area can be captured. The first video refers to the video taken within a first preset area, such as the video directly below the door frame of an entrance door, the video directly below the door frame of an office area, or the video of the glass partition of a bank counter and its immediate area. The first preset area can be set according to actual needs and may also include the area near the area directly below the door frame (e.g., the area within a preset distance of 50cm around the area directly below the door frame).

[0065] The first video can be obtained by capturing images using various cameras. Specifically, in some embodiments, the first video can be obtained by capturing images using a camera capable of recording motion trajectories, such as an optical flow camera. An optical flow camera refers to a camera that can reflect an optical flow field. The video image captured by the optical flow camera can reflect the speed and direction of motion of pixels in the image.

[0066] In some embodiments, the video acquisition module can acquire the first video from the aforementioned camera via a network. Alternatively, the video acquisition module can be integrated with the aforementioned camera. In some embodiments, the video acquisition module can acquire the first video from the aforementioned camera via a bus. In some embodiments, the first video can be acquired through an interface, which includes, but is not limited to, a program interface, a data interface, and a transmission interface. For example, when the information classification system of an intelligent security device is working, it can automatically extract the first video from the interface.

[0067] Step 220: Determine the activity trajectory of the preset object based on the first video.

[0068] An activity trajectory can refer to a pre-defined object, such as a person, and the route they take. For example, an activity trajectory could include elevator entrance >> front door >> doorway >> inside door.

[0069] The activity trajectory determination module can determine the activity trajectory of a preset object based on a first video using various methods. For example, optical flow analysis can be performed on the first video. Another example is analyzing the sequence features of the image sequence composed of each frame of the first video, inputting these sequence features into a trained machine learning model to determine the activity trajectory of the preset object.

[0070] Step 230: Determine the activity scenario of the preset object based on the activity trajectory.

[0071] An activity scenario refers to a pre-defined context in which an object is active, such as a person leaving home, entering a room, passing by a door, lingering in front of a door, a cat entering a balcony, or birds feeding on a rooftop.

[0072] In some embodiments, the scene determination module can determine the activity scene of a preset object based on the activity trajectory using various methods. For example, by training a machine learning model, the input to the machine learning model can be a curve connecting the positions of the preset object at various time points, and the output can be the corresponding activity scene type.

[0073] In some embodiments, the scene determination module can determine at least one type of preset activity scene and the activity trajectory corresponding to each preset activity scene, and can determine a preset activity scene that matches the activity trajectory of a preset object within the at least one type of preset activity scene, and determine the activity scene of the preset object based on the matching preset activity scene. For more information on determining the activity scene of a preset object based on its activity trajectory, please refer to [link to relevant documentation]. Figure 3 And related explanations.

[0074] Step 240: Identify the identity of the preset object based on the second video.

[0075] The identity of a preset object refers to its relationship or role with the user. For example, the identity of a preset object could include a homeowner's family member, a stranger, a homeowner's neighbor or friend, a delivery person, or a food delivery person. Another example is that the identity of a preset object could include a company's employee, a company visitor, or a company support staff member.

[0076] The identity verification module can identify the identity of a preset object using various methods. In some embodiments, the identity verification module can acquire at least one feature of the preset object based on a second video. These features may include facial features, gait features, accessory features, fingerprints, voiceprints, irises, etc., and the identity of the preset object can be determined based on at least one of these features. The accessory features may include at least one of clothing features, hairstyle features, accessories, and other features found on the person. For more information on identifying the identity of a preset object, please refer to this specification. Figure 4 And its explanation.

[0077] In some embodiments, the identity determination module may identify the identity of a preset object based on a second video using a machine learning model. For details on identifying the identity of a preset object using a machine learning model, please refer to this specification. Figure 5 And its explanation.

[0078] Step 250: Based on the activity scenario and the identity, determine the push level of the first video and / or the second video.

[0079] The push notification level can reflect the importance and / or urgency of the first and / or second videos. In some embodiments, the push notification level may include level 1, level 2, level 3, etc., wherein the higher the level, the higher the importance and / or urgency of the security area information. For example, level 1 is the highest level, corresponding to the highest importance and / or urgency, level 2 is next, and level 3 is the lowest level, corresponding to the lowest importance and / or urgency.

[0080] In some embodiments, the push level determination module can determine the push level of a first video and / or a second video related to a preset object based on the preset object's activity scenario and identity. In some embodiments, the push level determination module can further determine the push level of the first video and / or the second video related to the preset object based on duration, specific sounds, specific actions (e.g., picking a lock, unlocking, knocking, ringing a doorbell, breaking a window), etc.

[0081] This instruction manual Figure 6 An example is provided for determining the push level of the first video and / or the second video.

[0082] Step 260: Based on the push level, distribute the first video and / or the second video to one or more users.

[0083] In some embodiments, different push levels correspond to different video (first video and / or second video) distribution methods. For example, a level 1 push level could correspond to sending video to the user and enabling the connection between the user's terminal and each camera, allowing the user to choose to view the video or directly view live footage. Another example is a level 2 push level, which could correspond to sending video to the user (but without enabling the connection to each camera). Yet another example is a level 3 push level, which could correspond to periodically sending video to the user's terminal, user account, or user email. In some embodiments, the distribution module can distribute videos of different push levels to users with different permissions. For example, a level 1 push level video could be sent to homeowners and their family members, security personnel, monitoring personnel, etc. Yet another example is a level 2 or 3 push level video, which could be sent only to homeowners.

[0084] Figure 3 This is an exemplary flowchart of a method for determining an activity scenario according to some embodiments of this specification.

[0085] Step 310: Determine at least one type of preset activity scenario and the activity trajectory corresponding to each preset activity scenario.

[0086] At least one type of preset activity scenario may include a first type of activity scenario and a second type of activity scenario; wherein, the first type of activity scenario may include scenarios that pass through sensitive monitoring areas, such as visiting scenarios, entering scenarios, and leaving scenarios; the second type of activity scenario may include scenarios that do not pass through sensitive monitoring areas, such as passing by scenarios, lingering in front of the door scenarios, and moving items in front of the door scenarios.

[0087] In some embodiments, the scene determination module can determine the activity trajectory corresponding to each preset activity scenario. For example, the activity trajectory corresponding to the first type of activity scenario may include an entry trajectory and a exit trajectory from the sensitive monitoring area, while the activity trajectory corresponding to the second type of activity scenario may include a regular trajectory. A regular trajectory refers to the movement trajectory of the preset object without any actions requiring special user attention, such as entering or leaving a door (e.g., downstairs >> stairs >> front door >> stairs >> upstairs). An entry trajectory from the sensitive monitoring area refers to the activity trajectory corresponding to the preset object entering the sensitive monitoring area, such as an entry trajectory. An exit trajectory from the sensitive monitoring area refers to the activity trajectory corresponding to the preset object leaving the sensitive monitoring area, such as an exit trajectory.

[0088] Step 320: Determine the preset activity scenario that matches the activity trajectory of the preset object in the at least one type of preset activity scenario, and determine the activity scenario of the preset object based on the matching preset activity scenario.

[0089] In some embodiments, the activity scenario corresponding to the activity trajectory of the preset object can be determined by whether the activity trajectory of the preset object matches one or more activity trajectories included in a certain activity scenario of at least one type of preset activity scenario. For example, if the activity trajectory of the preset object is determined to be downstairs >> stairs >> front door >> stairs >> upstairs, and this activity trajectory is a regular trajectory, then the activity scenario of the preset object can be one or a combination of several of the following: passing by scenario, lingering in front of the door scenario, and moving items in front of the door scenario. As another example, if the activity trajectory of the preset object is determined to be an entry trajectory, then the activity scenario of the preset object can be an entry scenario. As yet another example, if the activity trajectory of the preset objects around the door is determined to be an exit trajectory, then the activity scenario of the preset object can be an exit scenario.

[0090] In some embodiments, the scene determination module may also determine the preset object movement state based on the first video, and the preset object movement state may include preset object movement or no preset object movement.

[0091] In some embodiments, the scene determination module can determine the movement state of preset objects corresponding to each preset activity scene. For example, the presence of preset object movement corresponds to a first type of activity scene, while the absence of preset object movement corresponds to a second type of activity scene.

[0092] In some embodiments, the scene determination module can determine the activity trajectory and movement state of preset objects corresponding to each preset activity scene. For example:

[0093] If the preset object activity trajectory includes an entry trajectory and the preset object movement status is "preset object moving", then it corresponds to an entry scene; if the preset object activity trajectory includes an exit trajectory and the preset object movement status is "preset object moving", then it corresponds to an exit scene; if the preset object activity trajectory does not include entry or exit trajectories but the preset object movement status is "preset object moving", then it corresponds to a passing scene; if the preset object activity trajectory does not include entry or exit trajectories but the preset object movement status is "preset object moving", then it corresponds to a lingering scene; if the preset object activity trajectory does not include entry or exit trajectories, but the preset object activity trajectory appears in a specific area behind the door (e.g., a package placement area set by the user or system), and the preset object movement status is "preset object moving", then it corresponds to a moving item in front of the door scene.

[0094] In some embodiments, the scene determination module can determine a preset activity scene that matches the activity trajectory and movement state of the preset object in at least one type of preset activity scene, and determine the activity scene of the preset object based on the matching preset activity scene.

[0095] In some embodiments, the scene determination module can also determine the door lock status, which may include an open or closed state. The scene determination module can determine the door lock status using various methods such as image recognition and door lock detection.

[0096] In some embodiments, the scenario determination module can determine the door lock status corresponding to each preset activity scenario. For example, the open state corresponds to the first type of activity scenario, and the closed state corresponds to the second type of activity scenario.

[0097] In some embodiments, the scene determination module can determine the activity trajectory of preset objects and the door lock status corresponding to each preset activity scene. For example:

[0098] If the preset object activity trajectory includes an entry trajectory and the door lock is open during this process, it corresponds to an entry scenario; if the preset object activity trajectory includes an exit trajectory and the door lock is open during this process, it corresponds to an exit scenario; if the preset object activity trajectory does not include an exit or entry trajectory, the time of the activity trajectory is below a threshold, and the door lock is closed during this process, it corresponds to a passing scenario; if the preset object activity trajectory does not include an exit or entry trajectory, the time of the activity trajectory is above a threshold, and the door lock is closed during this process, it corresponds to a loitering scenario; if the preset object activity trajectory does not include an exit or entry trajectory, but the preset object activity trajectory appears in a specific area behind the door (e.g., a package drop-off area set by the user or system), and the door lock is closed during this process, it corresponds to a moving item scenario.

[0099] In some embodiments, the scene determination module can determine a preset activity scene that matches the activity trajectory and door lock status of a preset object in at least one type of preset activity scene, and determine the activity scene of the preset object based on the matched preset activity scene.

[0100] In some embodiments, the scene determination module can determine preset activity scenes that match the activity trajectory, behind-the-door image, and door lock status of a preset object from at least one type of preset activity scenes, and determine the activity scene of the preset object based on the matched preset activity scenes. For example:

[0101] If the preset object activity trajectory includes an entry trajectory, the image behind the door shows the preset object moving, and the door lock is open during this process, then it is judged as an entry scenario; if the preset object activity trajectory includes an exit trajectory, the image behind the door shows the preset object moving, and the door lock is open during this process, then it is judged as an exit scenario; if the preset object activity trajectory does not include an exit trajectory or an entry trajectory, the time of the activity trajectory is below a threshold, and the door lock is closed during this process, then it is judged as a passing scenario; if the preset object activity trajectory does not include an exit trajectory or an entry trajectory, the time of the activity trajectory is above a threshold, and the door lock is closed during this process, then it is judged as a loitering scenario; if the preset object activity trajectory does not include an exit trajectory or an entry trajectory, but the preset object activity trajectory appears in a specific area behind the door (e.g., a package placement area set by the user or system), the image behind the door shows the preset object moving and the item moving, and the door lock is closed during this process, then it is judged as a moving item in front of the door scenario.

[0102] It should be noted that the above description of process 300 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to process 300 under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0103] Figure 4 This is an exemplary flowchart of an identity identification method according to some embodiments of this specification.

[0104] Step 410: Based on the second video, acquire at least one preset object feature selected from facial features, preset object gait features, and preset object accessory features. The preset object accessory features include at least one of clothing features, hairstyle features, and accessories features. In some embodiments, this step 410 can be performed by an identity recognition module.

[0105] Facial features refer to a person's facial characteristics. Facial features can include skin color, skin texture, facial features, makeup features, etc.

[0106] Gait characteristics refer to the features of a person's gait that reflect the magnitude, direction, and point of force application during walking. They reflect a person's walking habits during the foot landing, foot take-off, and support swing phases. Gait characteristics can include stride length, stride width, stride frequency, gait speed, walking cycle, and equal stride length, stride frequency, and gait speed during walking.

[0107] Auxiliary features refer to the characteristics of clothing or carried items. Examples include work badges, helmets, food delivery boxes, water bottles, and trolleys carried by a person, as well as clothing, headwear, and hats worn by a person. In some embodiments, auxiliary features include clothing characteristics, hairstyle characteristics, accessories characteristics, and carried item characteristics.

[0108] The identity recognition module can obtain at least one of the aforementioned preset object features through various methods. For example, it can obtain facial features and accessory features through image recognition, or obtain gait features through gait analysis of video images.

[0109] By combining gait features and auxiliary features, we can avoid the inability to identify individuals when their faces are obscured, and improve the accuracy of identity recognition.

[0110] Step 420: Determine the identity of the preset object based on the at least one preset object feature.

[0111] In some embodiments, the identity recognition module may determine the identity of the preset object based on at least one preset object feature.

[0112] In some embodiments, firstly, the identity recognition module can determine at least one preset identity and at least one preset object feature corresponding to each preset identity. The at least one preset identity may include a homeowner's family member, a stranger, a homeowner's neighbor or friend, a delivery person, a company employee, a company visitor, or a company logistics personnel, etc. The at least one preset object feature corresponding to each preset identity can be user-defined or obtained through methods such as feature extraction from historical data.

[0113] Secondly, the identity recognition module can determine a preset identity that matches the features of the preset object from at least one preset identity, and determine the identity of the preset object based on the matched preset identity. For example, the identity of a delivery person is matched with features such as wearing a work badge, wearing a uniform, and carrying a delivery box. When the preset object has features such as wearing a work badge, wearing a uniform, and carrying a delivery box, the identity recognition module can set the identity of the preset object as a delivery person.

[0114] In some embodiments, the identity recognition module can utilize sensors capable of detecting preset objects, such as point infrared (PIR) sensors or laser distance sensors, to detect the activity of preset objects around the door. When the sensor detects a person approaching (e.g., the person's distance from the door is less than a preset threshold), the identity recognition module can perform human detection based on the image and estimate the height of the preset object based on the human detection. Then, it can calculate the optimal face recognition position based on the preset object's height (e.g., 30cm from the door), allowing for more accurate face recognition. Human detection refers to the detection of the preset object's shape, outline, etc. In some embodiments, the identity recognition module can also guide the preset object in front of the door to move to the optimal face recognition position by displaying the optimal face recognition position on a feedback screen. In some embodiments, the identity recognition module can also indicate the optimal standing position on the ground using laser light; the optimal standing position can be the position corresponding to the optimal face recognition position.

[0115] In some embodiments, prompting for the optimal face position and the optimal standing position can improve the accuracy of identity recognition.

[0116] Figure 5 This is another exemplary flowchart of an identity identification method shown in some embodiments of this specification.

[0117] In some embodiments, the identity recognition module can identify the identity of a preset object based on a second video using a machine learning model. The machine learning model can be, but is not limited to, a combination of one or more of the following: neural network model, support vector machine model, k-nearest neighbor model, decision tree model, etc. The input to the machine learning model can include images related to the preset object, and the output of the machine learning model can include the identity of the preset object, such as a family member, a delivery person, etc.

[0118] like Figure 5 As shown, the machine learning model can initially label the identified preset objects with their identity and preset object characteristics. On the one hand, the push level determination module can base its algorithm on the initially labeled identity and... Figure 3The identified activity scenario determines the preset target's push notification level. On the other hand, the identity recognition module can remind users to verify the initial tagging, for example, by reminding them to verify in push notifications or by periodically reminding them to verify (every night at 8 PM). If the user believes the initial tagging is incorrect, they can re-tag it.

[0119] In some embodiments, the face, gait, ancillary features, and corresponding identity categories of the relabeled preset objects can be stored in a sample library for training machine learning models.

[0120] Figure 6 This is a schematic diagram illustrating a surveillance video distribution method according to some embodiments of this specification.

[0121] In some embodiments, different activity scenarios and different identities correspond to different push notification levels.

[0122] like Figure 6 As shown, in the first type of activity scenario (e.g., visiting, entering, leaving, etc.), if the preset object identity is a family member, neighbor, or delivery person, then the corresponding preset object push level is lower (e.g., level 2 or level 3); if the preset object identity is a stranger, then the corresponding push level is higher (e.g., level 1).

[0123] In some embodiments, the trajectory determination module can record the duration of action trajectories belonging to the same identity. For example, if the duration in a second type of activity scenario (e.g., loitering, etc.) is greater than or equal to a threshold (e.g., 5 minutes), it corresponds to a higher push level (e.g., level 1); if the duration in a second type of scenario is less than the threshold, it corresponds to a lower push level (e.g., level 3).

[0124] In some embodiments, the push level determination module can determine the push level based on the scenario type, duration, and preset object identity. For example, when the duration of a second type of activity scenario exceeds a threshold, if the preset object identity is property management personnel or cleaning staff, a lower push level (e.g., level 3) is assigned; if the preset object identity is a stranger, a higher push level (e.g., level 1) is assigned. Figure 6 As shown, if the duration in the first type of scenario is greater than or equal to the threshold (e.g., 5 minutes) and the preset object identity is a delivery person or food delivery person, then the corresponding push level is higher (e.g., level 1).

[0125] In some embodiments, the push level determination module can further determine the push level based on the sound and / or action detected by each camera. For example, in the second type of activity scenario, if actions or sounds such as ringing a doorbell or knocking are detected, the push level is set to level 2; if actions or sounds such as picking a lock are detected, the push level is set to level 1.

[0126] In some embodiments, users can set up pre-announced event information via terminal 120. The pre-announced event information may include a preset identity, a preset occurrence time, etc. For example, a friend will visit at 10:00, or takeout will arrive at 11:50.

[0127] In some embodiments, the push level determination module can receive user-set pre-announcement event information, extract preset identity, preset occurrence time, etc., and set a preset push level for the pre-announcement event information. In some embodiments, when the identity determination module identifies a person with a preset identity at the preset occurrence time, it pushes the information according to the preset push level. For example, if the push level determination module sets the push level of the event "takeout will be delivered at 11:50" to level 1, and the identity determination module identifies that a takeout delivery person has appeared before leaving home at 11:50, it will push the information to the user using the push method corresponding to level 1 (e.g., calling the user).

[0128] The beneficial effects that some embodiments of this application may bring include, but are not limited to: (1) being able to identify an individual based on features such as gait and accessories when the face is occluded or the face image is incomplete, thereby improving the accuracy and efficiency of identity recognition; (2) being able to automatically collect personnel samples, achieve adaptive learning, avoid manual sample collection, and save manpower costs; (3) being able to avoid monitoring blind spots by collecting videos from multiple perspectives such as second videos and first videos; (4) being able to determine the activity scene by activity trajectory, comprehensively consider the activity scene and identity to determine the push level, achieve reasonable and effective distribution of monitoring videos, and avoid information bombardment and the submersion of important information. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced can be any one or a combination of the above, or any other possible beneficial effects.

[0129] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0130] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0131] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0132] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0133] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0134] For each patent, patent application, patent application publication, and other material, such as articles, books, specifications, publications, and documents, referenced in this specification, the entire contents of which are incorporated herein by reference. This excludes historical application documents that are inconsistent with or conflict with the content of this specification, as well as documents that limit the broadest scope of the claims in this specification (currently or subsequently appended to this specification). It should be noted that in the event of any inconsistency or conflict between the descriptions, definitions, and / or terminology used in the supplementary materials to this specification and the content of this specification, the descriptions, definitions, and / or terminology used in this specification shall prevail.

[0135] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for distributing surveillance video, characterized in that, include: Obtain the first video from the first preset region; Based on the first video, the activity trajectory of the preset object is determined, and the activity trajectory includes at least one of the following: a trajectory of entering the sensitive monitoring area, a trajectory of leaving the sensitive monitoring area, and a regular trajectory. The activity scenario of the preset object is determined based on the activity trajectory; Determine the identity of the preset object; Based on the activity scenario and the identity, the push level of the surveillance video is determined. The surveillance video includes the first video, and the push level includes level 1, level 2, and level 3, with level 1 being the highest level and level 3 being the lowest level. Based on the aforementioned push level, the surveillance video is distributed to one or more users, wherein: The distribution method corresponding to Level 1 includes sending the surveillance video to the one or more users and enabling the connection between the user terminals of the one or more users and each camera, so as to allow the one or more users to view the first video or directly view the real-time image; The distribution method corresponding to Level 2 includes sending the surveillance video to the one or more users without activating the connection between the user terminals of the one or more users and each camera. The distribution method corresponding to Level 3 includes periodically sending the surveillance video to the user terminals, user accounts, or user email addresses of one or more users.

2. The method as described in claim 1, characterized in that, The method further includes: Acquire a second video from a second preset area, wherein the security level of the first preset area is higher than that of the second preset area, and the surveillance video includes the second video; determining the identity of the preset object includes: identifying the identity of the preset object based on the second video.

3. The method as described in claim 2, characterized in that, The step of identifying the preset object based on the second video includes: Based on the second video, the identity of the preset object is identified through a machine learning model.

4. The method as described in claim 3, characterized in that, The method further includes: The machine learning model identifies the identity and features of the preset object. The initial marker is verified by the user; In response to the incorrect initial labeling, the identity and characteristics of the preset object are relabeled. The identity of the relabeled preset object and the features of the preset object are added to the sample library used for training the machine learning model.

5. The method as described in claim 2, characterized in that, The method further includes: Human detection is performed based on the second video; The height of the preset object is determined based on the human figure detection. The optimal face recognition position is determined based on the height of the preset object. The identity of the preset object is identified based on the optimal face recognition location.

6. The method as described in claim 5, characterized in that, The method further includes: The optimal face recognition location is displayed on the feedback screen to guide the preset object to move to the optimal face recognition location.

7. The method as described in claim 6, characterized in that, The method further includes: The optimal standing position on the ground is indicated by a laser light, which is the position corresponding to the optimal face recognition position.

8. The method as described in claim 1, characterized in that, The method further includes: Receive user-defined pre-event information; Extract the preset personnel identity and preset occurrence time from the predicted event information; Set the preset push level corresponding to the aforementioned event information; In response to the fact that the identity of the preset object identified at the preset occurrence time is the preset personnel identity, the preset push level is used as the push level of the surveillance video.

9. A surveillance video distribution system, characterized in that, The system includes: The video acquisition module is used to acquire the first video of the first preset area; The trajectory determination module is used to determine the activity trajectory of a preset object based on the first video, wherein the activity trajectory includes at least one of the following: a trajectory of entering a sensitive monitoring area, a trajectory of leaving the sensitive monitoring area, and a regular trajectory. The scene determination module is used to determine the activity scene of the preset object based on the activity trajectory; An identity determination module is used to determine the identity of the preset object; The push level determination module is used to determine the push level of the surveillance video based on the activity scenario and the identity. The surveillance video includes the first video, and the push level includes level 1, level 2, and level 3, where level 1 is the highest level and level 3 is the lowest level. The distribution module is used to distribute the surveillance video to one or more users based on the push level, wherein: The distribution method corresponding to Level 1 includes sending the surveillance video to the one or more users and enabling the connection between the user terminals of the one or more users and each camera, so as to allow the one or more users to view the first video or directly view the real-time image; The distribution method corresponding to Level 2 includes sending the surveillance video to the one or more users without activating the connection between the user terminals of the one or more users and each camera. The distribution method corresponding to Level 3 includes periodically sending the surveillance video to the user terminals, user accounts, or user email addresses of one or more users.

10. A surveillance video distribution device, the device comprising a processor and a memory; the memory being used to store instructions, characterized in that, When the instruction is executed by the processor, it causes the device to implement the method as described in any one of claims 1 to 8.

Images