A video analysis system and community management application

Abstract

The present invention relates to the field of video analysis technology, specifically a video analysis system and community management application, the system collects video data in the community in real time through a video acquisition module, then detects, identifies and tracks entities in the video through an identification module and a trajectory tracking analysis module, then analyzes the relationship between entities through a relationship analysis module, and judges the activity state of animals according to the relationship between entities and the position relationship of entities in the video through a state analysis module, and finally issues a warning message through an alarm module when the animal is not towed or is in a lost or wandering state. Thus, the automatic monitoring of dangerous pets in the community is realized, which not only reduces labor costs, but also enables community managers to quickly respond to various dangerous events from animals in the community, effectively improving the safety management level of the community.

Description

Technical Field

[0001] The present invention relates to the technical field of video analysis, and in particular to a video analysis system and community management application. Background Art

[0002] With the continuous expansion of community scale and the increase of population density, the management problems of pets and stray dogs in the community are becoming increasingly prominent: such as leash management problems, stray dog ​​identification and treatment problems, and pet loss problems. When stray dogs or pet dogs are not on leashes and move around in the community, they may bring safety hazards to community residents and even cause chaos in community order, posing safety risks; even finding lost pets is time-consuming and laborious. At present, there is a lack of scientific management methods for community pets and stray dogs. Traditional human publicity can only serve as information notification. Although the video surveillance system can provide real-time monitoring functions, it cannot conduct in-depth analysis of video content, and cannot automatically identify abnormal situations and respond in time. Therefore, it is necessary to develop an intelligent video analysis system to improve the efficiency and accuracy of community safety management. Summary of the invention

[0003] The technical problem solved by the present invention is to provide a video analysis system to realize the automated analysis of community pets and stray animals, and provide information support for scientific management of the community.

[0004] The basic solution provided by the present invention: a video analysis system, including a video acquisition module, a recognition module, a relationship analysis module, a state analysis module and an alarm module;

[0005] The video acquisition module is used to collect video data within the community;

[0006] The recognition module is used to identify entities including people, animals, and animal leashes in the video;

[0007] The relationship analysis module obtains the entity information of the recognition module and analyzes the relationship between entities. The relationship between entities includes owner-pet relationship and acquaintance relationship. The acquaintance relationship includes intimacy, familiarity and unfamiliarity.

[0008] The state analysis module has preset missing range thresholds and missing duration thresholds. Combined with the recognition results of the recognition module and the relationship analysis module, it determines whether the activity state of the target animal is normal based on the video. The activity state includes traction state, lost state and wandering state.

[0009] After the state analysis module obtains the target animal information identified by the identification module and the entity relationship information of the corresponding target animal output by the relationship analysis module, the activity state of the target animal is determined; when the target animal has an owner-pet relationship, if the owner of the target animal is outside the lost range of the animal for a time that reaches a lost time threshold, the animal is determined to be in a lost state, otherwise it is in a traction state; when the target animal does not have an owner-pet relationship, the animal is determined to be in a wandering state; when the activity state is in a lost state and a wandering state, an alarm message is output to the alarm module;

[0010] The alarm module is used to obtain alarm information and send an alarm to a terminal; the terminal includes at least one of a notice screen, a broadcast speaker, a mobile phone or a computer.

[0011] Furthermore, it also includes a trajectory tracking analysis module; the trajectory tracking analysis module forms the location information of the corresponding entity based on the location information and time information of the video collected by the video acquisition module, combined with each entity identified by the recognition module, and generates the corresponding movement trajectory in combination with the time information.

[0012] Furthermore, it also includes a feature storage module; the feature storage module is connected to the identification module and stores feature information of community residents and their registered pets.

[0013] Furthermore, the relationship analysis module includes a character relationship determination module and a familiarity determination module. The character relationship determination module is provided with a behavior recognition module, which analyzes whether a person has a pulling behavior on an animal. If there is a pulling behavior, it is determined to be an owner-pet relationship; the familiarity determination module is used to determine the familiarity relationship according to the familiarity. :

[0014]

[0015]

[0016] In the formula, is the quantitative value of the number of peers, is the quantitative value of the joint stay duration, is the quantitative value of the temporal overlap, is the quantitative value of feeding frequency, , , and are the weights of the number of trips, the duration of stay together, the time overlap and the feeding frequency, among which, , and This is not 0 only if the target animal is being led by its owner.

[0017] Furthermore, the behavior recognition module is used to judge the pulling behavior and the walking behavior between the person and the animal, and send the non-pulling behavior as an alarm message to the alarm module;

[0018] The behavior recognition module is provided with a traction duration threshold and a traction distance threshold. The acquired entity includes a traction rope. At the same time, when the target person, the target animal, and the traction rope exist at the same time, and the relative distance between the target person and the target animal in the continuous video frames is not greater than the traction distance, and the duration reaches the traction duration threshold, it is determined to be a traction behavior;

[0019] The behavior recognition module is also provided with a walking time threshold and a distance threshold; when the target person and the target animal are within the distance threshold in the continuous video frames that reach the walking time threshold, it is judged as walking together; when the target animal lacks a traction rope, or has a traction rope but no walking behavior, it is judged as a non-traction state; the behavior recognition module sends an alarm message to the alarm module.

[0020] Further, the recognition module includes a preprocessing module, a feature extraction module, an entity detection module, a feature matching module and an entity tracking module;

[0021] The preprocessing module is used to perform preprocessing operations such as denoising, resizing and cropping on the collected video data; the feature extraction module is used to extract entity features from video frames through convolutional neural networks; the entity detection module is used to identify entities in the video through target detection algorithms based on the extracted features; the feature matching module is used to identify the identity information of the target entity based on the extracted entity features; the entity tracking module is used to identify the same entity in consecutive video frames and track the detected entity

[0022] Further, the trajectory tracking and analysis module includes a trajectory generation module, a trajectory correction module and a trajectory analysis module;

[0023] The trajectory generation module is used to generate the moving trajectory of the entity according to the tracking result of the entity tracking module; the trajectory correction module is used to eliminate the random fluctuation in the center of the trajectory data through a filtering algorithm; and the trajectory analysis module is used to extract the moving speed, residence time and path frequency of the target entity according to the trajectory data.

[0024] A community management application includes a terminal APP, a client, a server and a video analysis system as described in any of the above items.

[0025] Furthermore, the terminal APP includes an input module, and the client or server includes a management module; the input module is used to provide community personnel with the information to be warned, and pass it to the management module. The management module obtains the information to be warned transmitted by the input module, generates warning information, and passes it to the warning module.

[0026] The principle and advantage of the present invention are: the present invention proposes a video analysis system and community management application, the system includes a video acquisition module, an identification module, a trajectory tracking analysis module, a relationship analysis module, a state analysis module, a feature storage module and an alarm module. This solution collects video data in the community in real time through the video acquisition module, and then detects, identifies and tracks the entities in the video through the identification module and the trajectory tracking analysis module, and then analyzes the relationship between the entities through the relationship analysis module, and judges the activity state of the animal according to the relationship between the entities and the position relationship of the entities in the video through the state analysis module, and finally issues a warning message through the alarm module when the animal is not towed or is in a lost or wandering state. Thereby, the automated monitoring of dangerous pets in the community is realized, which not only reduces the labor cost, but also enables the community managers to quickly respond to various dangerous events from animals in the community, effectively improving the safety management level of the community. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a logic block diagram of a first embodiment of a video analysis system of the present invention. DETAILED DESCRIPTION

[0028] The following is further described in detail through specific implementation methods:

[0029] The specific implementation process is as follows:

[0030] Embodiment 1

[0031] Embodiment 1 is basically as shown in the attached Figure 1 As shown, a video analysis system includes a video acquisition module, an identification module, a trajectory tracking analysis module, a relationship analysis module, a state analysis module, a feature storage module and an alarm module. This solution collects video data in the community in real time through the video acquisition module, and then detects, identifies and tracks the entities in the video through the identification module and the trajectory tracking analysis module. The relationship between the entities is analyzed by the relationship analysis module, and the activity state of the animal is judged by the state analysis module according to the relationship between the entities and the position relationship of the entities in the video. Finally, the alarm module sends a warning message when the animal is not towed or is in a lost or wandering state. This realizes the automated monitoring of dangerous pets in the community, which not only reduces labor costs, but also enables community managers to quickly respond to various dangerous incidents from animals in the community, effectively improving the safety management level of the community.

[0032] Specifically, the video acquisition module in this embodiment collects video data in the community in real time by installing high-definition cameras at various locations in the community.

[0033] The recognition module is used to recognize entities including people, animals and animal leashes in the video, and includes a preprocessing module, a feature extraction module, an entity detection module, a feature matching module and an entity tracking module. The feature storage module in this embodiment is connected to the recognition module and stores feature information of community residents and their registered pets.

[0034] The preprocessing module is used to perform denoising on the video data obtained from the video acquisition module to reduce the impact of environmental noise on image quality. After that, the size of the video frame is adjusted and necessary cropping operations are performed to remove irrelevant areas in the video frame and retain areas of interest, such as cropping out the sky or the ground to retain only areas where pedestrians and animals are active.

[0035] The feature extraction module uses a convolutional neural network (CNN) model, which has been pre-trained on large-scale image datasets and can extract complex features in images. Each frame of video image is input into the CNN, and the network will output the corresponding feature maps, which encode information such as objects, textures, shapes, etc. in the image.

[0036] The entity detection module applies the object detection algorithm YOLO (You Only Look Once) to detect and identify the target entity on the feature map. The object detection algorithm outputs the category (such as people, animals, animal leashes, etc.) and location information (usually bounding box coordinates) of the detected entity.

[0037] The feature matching module is used to perform feature matching based on the extracted entity features and the feature information of the community residents and their pets registered and stored in the feature storage module, so as to identify the identity information of the target entity. When the match fails, it is stored as temporary identity information based on the characteristics of the target entity.

[0038] The entity tracking module uses the SORT (Simple Online and Realtime Tracking) tracking algorithm to link the appearances of the same entity in different frames based on the entity's position in consecutive video frames. The tracking algorithm uses clues such as the entity's motion pattern, appearance characteristics, and position proximity to perform data association and generate entity trajectory information.

[0039] The trajectory tracking analysis module forms the location information of the corresponding entity based on the location information and time information of the video collected by the video acquisition module, combined with each entity identified by the recognition module, and generates the corresponding movement trajectory in combination with the time information. Specifically, the trajectory tracking analysis module includes a trajectory generation module, a trajectory correction module and a trajectory analysis module.

[0040] In this embodiment, when the entity detection module detects an entity for the first time, the trajectory generation module creates a new trajectory point for the entity. During initialization, the entity's position, timestamp, speed and other information are recorded, and a unique trajectory identifier is assigned to it. As the entity moves in the video, the entity's trajectory points are updated in real time. Each new trajectory point contains information such as the new position, timestamp and speed of the entity. Based on the entity's trajectory point sequence, the trajectory generation module generates the entity's movement trajectory. This can be done by connecting the positions of adjacent trajectory points to obtain the entity's complete motion path in the video. The trajectory can be represented as a series of line segments or curves, depending on the required accuracy and visualization effect.

[0041] The trajectory correction module is used to eliminate random fluctuations in the center of the trajectory data through a filtering algorithm. Specifically, the trajectory correction module applies a trajectory filtering algorithm, such as Kalman filtering or particle filtering, to optimize the trajectory data, reduce random fluctuations and errors, and use the entity's dynamic model and observation model to predict and correct the entity's position, thereby improving the accuracy and stability of the trajectory.

[0042] The trajectory analysis module is used to extract key features from the trajectory, such as the entity's movement speed, dwell time, path frequency, etc. The movement pattern of the entity is analyzed to be applied to the processing of subsequent modules.

[0043] The relationship analysis module obtains the entity information of the recognition module and analyzes the relationship between entities. The relationship between entities includes owner-pet relationship and acquaintance relationship. The acquaintance relationship includes intimacy, familiarity and strangeness.

[0044] Specifically, the relationship analysis module includes a character relationship determination module and a familiarity determination module. The character relationship determination module is provided with a behavior recognition module, which analyzes whether a person has a pulling behavior on an animal. If there is a pulling behavior, it is determined to be an owner-pet relationship; the familiarity determination module is used to determine the familiarity relationship based on the familiarity. :

[0045]

[0046]

[0047] In the formula, is the quantitative value of the number of peers, is the quantitative value of the joint stay duration, is the quantitative value of the temporal overlap, is the quantitative value of feeding frequency, , , and are the weights of the number of trips, the duration of stay together, the time overlap and the feeding frequency, among which, , and This is not 0 only if the target animal is being led by its owner.

[0048] Specifically, the behavior recognition module is used to judge the pulling behavior and the walking behavior between the person and the animal, and send the non-pulling behavior as alarm information to the alarm module.

[0049] The behavior recognition module is provided with a traction duration threshold and a traction distance threshold. The acquired entities include the traction rope. At the same time, when the target person, target animal, and traction rope exist at the same time, and the relative distance between the target person and the target animal in continuous video frames is not greater than the traction distance, and the duration reaches the traction duration threshold, it is judged as a traction behavior.

[0050] The behavior recognition module is also provided with a walking time threshold and a distance threshold; when the target person and the target animal are within the distance threshold in the continuous video frames that reach the walking time threshold, it is judged as walking together; when the target animal lacks a traction rope, or has a traction rope but no walking behavior, it is judged as a non-traction state; the behavior recognition module sends an alarm message to the alarm module.

[0051] After the state analysis module obtains the target animal information identified by the identification module and the entity relationship information of the corresponding target animal output by the relationship analysis module, it determines the activity state of the target animal; when the target animal has an owner-pet relationship, if the target animal's owner is outside the animal's lost range for a time that reaches the lost time threshold, the animal is determined to be in a lost state, otherwise it is in a traction state; when the target animal does not have an owner-pet relationship, the animal is determined to be in a wandering state; when the activity state is in a lost state and a wandering state, an alarm message is output to the alarm module. By analyzing the state of the animal, timely information is provided so that problems can be quickly intervened and handled when they occur.

[0052] The alarm module in this embodiment is used to obtain alarm information and send an alarm to a terminal; the terminal includes at least one of a notice screen, a broadcast speaker, a mobile phone or a computer. For example, when the target animal is in a stray state, the alarm module sends the stray animal information to the mobile phone or computer held by the property management personnel, prompting them to handle it, and publishes a pet adoption notice on several notice screens where the stray animal passes most frequently. The alarm module can collect, integrate and intelligently analyze entity status and relationship data, generate and optimize targeted notice content, achieve fast and accurate information transmission and community interaction, and ensure information security and privacy protection.

[0053] In summary, this solution uses the video acquisition module to collect video data in the community in real time, and then uses the recognition module and trajectory tracking analysis module to detect, identify and track entities in the video. The relationship analysis module then analyzes the relationship between entities, and the state analysis module determines the activity status of animals based on the relationship between entities and the position relationship of entities in the video. Finally, the alarm module issues a warning message when the animal is not towed or is lost or wandering. This realizes the automated monitoring of dangerous pets in the community, which not only reduces labor costs, but also enables community managers to quickly respond to various dangerous incidents from animals in the community, effectively improving the safety management level of the community.

[0054] Embodiment 2

[0055] The only difference between the second embodiment and the first embodiment is that the second embodiment is a community management application platform including a terminal APP, a client, a server and a video analysis system as described above.

[0056] The terminal APP in this embodiment includes an input module, and the client or server includes a management module; the input module is used to provide community personnel with information to be alerted, and transmit it to the management module, the management module obtains the information to be alerted transmitted by the input module, generates alert information, and transmits it to the alert module. It is convenient for community personnel to send alert information to property management personnel or other community personnel after encountering stray animals or untethered animals, which is beneficial for others to avoid risky animals in the community, and also convenient for property management personnel to promptly discover the location of such risky animals and respond and manage them, thereby improving the safety of the community.

[0057] The server in this embodiment also includes a regional division module and a regional activity frequency calculation module; the regional division module is used to divide the community into regions according to square grids, and a single grid area in this embodiment is 30m×30m. The regional activity frequency calculation module is used to calculate the average number of animals appearing in each region in each time interval of a day within a unit time (a week in this embodiment) based on the animal movement trajectory in the community in historical data. In this embodiment, the calculation is performed with half an hour as a time interval.

[0058] The terminal APP also includes an animal appearance visualization module, which includes a regional animal real-time monitoring module and a regional animal appearance frequency visualization module. The regional animal real-time monitoring module is used to display the video image of the target area through the video data obtained by the video acquisition module, so that community personnel can view the animal activities in the area in real time. The regional animal appearance frequency visualization module is used to display the community electronic map, and display the average number of animals appearing in each time interval within a week in each area on the map, so that community personnel can refer to it for subsequent activities and travel in the community, so that community personnel can choose areas with fewer animals at a predetermined time to avoid them, reduce the risk of being attacked by animals, and provide safety protection for the travel of community personnel.

[0059] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A video analysis system, characterized in that: It includes video acquisition module, recognition module, relationship analysis module, status analysis module and alarm module; The video acquisition module is used to collect video data within the community; The recognition module is used to identify entities including people, animals, and animal leashes in the video; The relationship analysis module obtains the entity information of the recognition module and analyzes the relationship between entities. The relationship between entities includes owner-pet relationship and acquaintance relationship. The acquaintance relationship includes intimacy, familiarity and unfamiliarity. The state analysis module has preset missing range thresholds and missing duration thresholds. Combined with the recognition results of the recognition module and the relationship analysis module, it determines whether the activity state of the target animal is normal based on the video. The activity state includes traction state, lost state and wandering state. After the state analysis module obtains the target animal information identified by the identification module and the entity relationship information of the corresponding target animal output by the relationship analysis module, the activity state of the target animal is determined; when the target animal has an owner-pet relationship, if the owner of the target animal is outside the lost range of the animal for a time that reaches a lost time threshold, the animal is determined to be in a lost state, otherwise it is in a traction state; when the target animal does not have an owner-pet relationship, the animal is determined to be in a wandering state; when the activity state is in a lost state and a wandering state, an alarm message is output to the alarm module; The alarm module is used to obtain alarm information and send an alarm to a terminal; the terminal includes at least one of a notice screen, a broadcast speaker, a mobile phone or a computer; The relationship analysis module includes a character relationship determination module and a familiarity determination module. The character relationship determination module is provided with a behavior recognition module, which analyzes whether a person has a pulling behavior on an animal. If there is a pulling behavior, it is determined to be an owner-pet relationship; the familiarity determination module is used to determine the familiarity relationship according to the familiarity. : In the formula, is the quantitative value of the number of peers, is the quantitative value of the joint stay duration, is the quantitative value of the temporal overlap, is the quantitative value of feeding frequency, , , and are the weights of the number of trips, the duration of stay together, the time overlap and the feeding frequency, among which, , and It is not 0 only when the target animal is being pulled by its owner; The behavior recognition module is used to judge the pulling behavior and the walking behavior between the person and the animal, and send the non-pulling behavior as alarm information to the alarm module; The behavior recognition module is provided with a traction duration threshold and a traction distance threshold. The acquired entity includes a traction rope. At the same time, when the target person, the target animal, and the traction rope exist at the same time, and the relative distance between the target person and the target animal in the continuous video frames is not greater than the traction distance, and the duration reaches the traction duration threshold, it is determined to be a traction behavior; The behavior recognition module is also provided with a walking time threshold and a distance threshold; when the target person and the target animal are within the distance threshold in the continuous video frames that reach the walking time threshold, it is judged as walking together; when the target animal lacks a traction rope, or has a traction rope but no walking behavior, it is judged as a non-traction state; the behavior recognition module sends an alarm message to the alarm module.

2. A video analysis system according to claim 1, characterized in that: It also includes a trajectory tracking analysis module; the trajectory tracking analysis module forms the location information of the corresponding entity based on the location information and time information of the video collected by the video acquisition module, combined with each entity identified by the recognition module, and generates the corresponding movement trajectory in combination with the time information.

3. A video analysis system according to claim 1, characterized in that: It also includes a feature storage module; the feature storage module is connected to the identification module and stores feature information of community residents and their registered pets.

4. A video analysis system according to claim 1, characterized in that: The recognition module includes a preprocessing module, a feature extraction module, an entity detection module, a feature matching module and an entity tracking module; The preprocessing module is used to perform denoising, resizing and cropping on the collected video data; the feature extraction module is used to extract entity features from video frames through convolutional neural networks; the entity detection module is used to identify entities in the video through target detection algorithms based on the extracted features; the feature matching module is used to identify the identity information of the target entity based on the extracted entity features; the entity tracking module is used to identify the same entity in consecutive video frames and track the detected entity.

5. A video analysis system according to claim 2, characterized in that: The trajectory tracking and analysis module includes a trajectory generation module, a trajectory correction module and a trajectory analysis module; The trajectory generation module is used to generate the moving trajectory of the entity according to the tracking result of the entity tracking module; the trajectory correction module is used to eliminate the random fluctuation in the center of the trajectory data through a filtering algorithm; and the trajectory analysis module is used to extract the moving speed, residence time and path frequency of the target entity according to the trajectory data.

6. A community management application system, characterized by: It comprises a terminal APP, a client, and a server, wherein the terminal APP, the client, and the server are integrated with a video analysis system as described in any one of claims 1 to 5.

7. A community management application system according to claim 6, characterized in that: The terminal APP includes an input module, and the client or server includes a management module; the input module is used to provide community personnel with the information to be warned and pass it to the management module. The management module obtains the information to be warned passed by the input module, generates warning information, and passes it to the warning module.

Images