An intelligent video security monitoring system and method based on deep learning

Abstract

The present invention relates to the field of computer vision technology, and in particular to an intelligent video security monitoring system and method based on deep learning, which collects and monitors the behavior trajectory semantic records of monitoring behaviors with the existence of behavior initiation actions and monitoring behavior termination actions, obtains a behavior trajectory semantic record list, and monitors the existence of behavioral characters interacting alone or with multiple people; when the interactive behavior begins, the monitoring behavior record is determined from the behavior trajectory semantic record list; the behavioral character marking is performed in the order in which each behavioral character exists, and in each behavioral character, action prediction is performed according to the action prediction record, and the action prediction record is determined based on the real-time interactive action snapshot and the monitoring behavior record. The present invention can timely discover sudden physical contact events in video monitoring, analyze and formulate countermeasures for interactive behaviors, and thus formulate relevant countermeasures to evacuate personnel.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an intelligent video security monitoring system and method based on deep learning. Background Art

[0002] Deep learning is one of the most important breakthroughs in the field of artificial intelligence in the past decade, and has achieved great success in many fields such as speech recognition, computer vision, image and video analysis, and multimedia. In the past two years, as the understanding of deep learning algorithms has gradually deepened, more and more researchers have begun to use deep learning models from input to output and conduct end-to-end training, making relatively large breakthroughs in many tasks.

[0003] Using surveillance cameras to observe crowded places is a common public safety prevention and control method. It can detect sudden emergencies in a timely manner and formulate countermeasures to deal with related emergencies. The existing method of identifying interactive behaviors is often for the monitoring professionals to take turns to watch the monitoring screen to ensure real-time monitoring, and then manually analyze the interactive behaviors and formulate countermeasures. Summary of the invention

[0004] To achieve the above objectives, this application provides the following technical solutions:

[0005] According to a first aspect of the present invention, the present invention claims protection for an intelligent video security monitoring method based on deep learning, characterized in that the method comprises:

[0006] Collect and monitor the behavior trajectory semantic records of the monitoring behavior that has the behavior initiation action and the monitoring behavior termination action, and obtain a behavior trajectory semantic record list; the monitoring behavior has a behavior character that interacts alone or with multiple people;

[0007] When the interactive behavior begins, determining the monitoring behavior record from the behavior trajectory semantic record list;

[0008] The actor marking is performed in the order in which each actor exists, and in each actor, action prediction is performed according to the action prediction record; the action prediction record is determined based on the real-time interactive action snapshot and the monitoring behavior record.

[0009] Further, the step of performing actor marking in the order in which the actors exist, and performing action prediction in each actor according to the action prediction record includes:

[0010] When the real-time interactive action snapshot does not reach the interactive termination behavior where the real-time behavior character exists, action prediction is performed according to the action prediction record;

[0011] When the real-time interactive action snapshot reaches the interactive termination behavior of the real-time actor and does not reach the monitoring behavior termination action, it is marked to the next actor.

[0012] Furthermore, the monitoring behavior record includes a semantic record of sub-behavior trajectories of each behavior character; when the real-time interactive action snapshot does not reach the interactive termination behavior of the real-time behavior character, before performing action prediction according to the action prediction record, the method further includes:

[0013] According to the semantic records of sub-behavior trajectories of real-time behavior characters, a corresponding deep learning model of target interactive behavior is created;

[0014] The target interactive behavior deep learning model is used to collect the action prediction record existing in the real-time interactive action snapshot from the sub-behavior trajectory semantic record when the real-time interactive action snapshot has not reached the interactive termination behavior of the real-time behavior character, and report the action prediction instruction carrying the action prediction record;

[0015] When the real-time interactive action snapshot does not reach the interactive termination behavior of the real-time behavior character, performing action prediction according to the action prediction record includes:

[0016] When the real-time interactive action snapshot does not reach the interactive termination behavior of the real-time behavior character, in response to the action prediction instruction, action prediction is performed according to the action prediction record.

[0017] Furthermore, the step of creating a corresponding target interactive behavior deep learning model based on the semantic record of the sub-behavior trajectory of the real-time behavior character includes:

[0018] Collecting an interactive behavior feature list; the interactive behavior feature list includes a feature sublist of each actor; the feature sublist of each actor includes a sub-interactive behavior state of each actor;

[0019] The sub-interactive behavior state of the real-time behavior character is determined as a real-time control feature to start creating an interactive behavior deep learning model of the real-time behavior character, and the created interactive behavior deep learning model is initialized according to the semantic record of the sub-behavior trajectory to obtain a target interactive behavior deep learning model of the real-time behavior character.

[0020] Furthermore, the interactive behavior feature list also includes a termination feature; and the method further includes:

[0021] When the real-time interactive action snapshot reaches the monitoring behavior termination action, determining the termination feature as a real-time control feature to start terminating all interactive behaviors;

[0022] The interactive behavior feature list also includes a termination feature; and the method further includes:

[0023] When an interactive behavior termination message is obtained, the termination feature is determined as a real-time control feature to start terminating the interactive behavior;

[0024] The feature sublists of each actor include the sub-termination features of each actor; before marking to the next actor, the method further includes:

[0025] When the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavior character and does not reach the monitoring behavior termination action, the sub-termination feature of the real-time behavior character is determined as the real-time control feature to start terminating the real-time interactive behavior.

[0026] Furthermore, the marking to the next actor includes:

[0027] The sub-interaction behavior state of the next actor is determined as a real-time control feature to start creating a deep learning model of the interaction behavior of the next actor, and the created deep learning model of the interaction behavior is initialized according to the semantic record of the sub-behavior trajectory of the next actor to obtain the target deep learning model of the interaction behavior of the next actor.

[0028] Furthermore, the feature sublist of each actor includes the abnormal features of each actor; the method further includes:

[0029] When the real-time interactive action snapshot deviates from the action of the real-time behavior character, the abnormal feature of the real-time behavior character is determined as a real-time control feature to start collecting the action re-monitoring record starting from the real-time interactive action snapshot and ending at the interaction termination action and the action of the real-time behavior character;

[0030] After collecting the action re-monitoring record, the sub-interaction behavior state of the real-time behavior character is re-determined as a real-time control feature to start updating the target interaction behavior deep learning model according to the action re-monitoring record.

[0031] Furthermore, the target interactive behavior deep learning model is also used to report an abnormal instruction carrying the real-time interactive action snapshot when the real-time interactive action snapshot deviates from the action of the real-time behavior character;

[0032] When the real-time interactive action snapshot deviates from the action of the real-time action character, determining the abnormal feature of the real-time action character as the real-time control feature includes:

[0033] When the real-time interactive action snapshot deviates from the action of the real-time actor, in response to the abnormal instruction, the abnormal feature of the real-time actor is determined as the real-time control feature.

[0034] Furthermore, after determining the monitoring behavior record from the behavior trajectory semantic record list, the method further includes:

[0035] Control the real-time monitoring display screen to display the existing monitoring behavior according to the monitoring behavior record;

[0036] The performing action prediction according to the action prediction record comprises:

[0037] Control the real-time monitoring display screen to display the action prediction record.

[0038] According to a second aspect of the present invention, the present invention claims protection for an intelligent video security monitoring system based on deep learning, characterized in that the system comprises:

[0039] A behavior trajectory semantic record collection unit collects behavior trajectory semantic records of monitoring behaviors including monitoring behavior initiation actions and monitoring behavior termination actions, and obtains a behavior trajectory semantic record list; the monitoring behaviors include behavior characters interacting alone or with multiple people;

[0040] A monitoring behavior record determination unit, configured to determine a monitoring behavior record from the behavior trajectory semantic record list after the interactive behavior begins;

[0041] The marking unit is used to perform actor marking in the order in which each actor exists, and to perform action prediction in each actor according to the action prediction record; the action prediction record is determined based on the real-time interactive action snapshot and the monitored behavior record.

[0042] The present invention relates to the field of computer vision technology, and in particular to an intelligent video security monitoring system and method based on deep learning, which collects and monitors the behavior trajectory semantic records of monitoring behaviors with the existence of behavior initiation actions and monitoring behavior termination actions, obtains a behavior trajectory semantic record list, and monitors the existence of behavioral characters interacting alone or with multiple people; when the interactive behavior begins, the monitoring behavior record is determined from the behavior trajectory semantic record list; the behavioral character marking is performed in the order in which each behavioral character exists, and in each behavioral character, action prediction is performed according to the action prediction record, and the action prediction record is determined based on the real-time interactive action snapshot and the monitoring behavior record. The present invention can timely discover sudden physical contact events in video monitoring, analyze and formulate countermeasures for interactive behaviors, and thus formulate relevant countermeasures to evacuate personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A workflow diagram of an intelligent video security monitoring method based on deep learning involved in an embodiment of the present application;

[0044] Figure 2 A second working flow chart of an intelligent video security monitoring method based on deep learning involved in an embodiment of the present application;

[0045] Figure 3 This is a structural module diagram of a deep learning-based intelligent video security monitoring system involved in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] For safety, one person alone or multiple people in one embodiment, such as Figure 1 As shown, a deep learning-based intelligent video security monitoring method is provided, comprising the following steps:

[0048] S202, collecting and monitoring behavior initiation actions and monitoring behavior termination actions to obtain a behavior trajectory semantic record list, the monitoring behavior has a behavior character interacting alone or with multiple people.

[0049] Among them, the monitoring behavior initiation action refers to the interactive behavior starting point specified by the user. The monitoring behavior initiation action is usually determined based on the user's real-time position, so the monitoring behavior initiation action can be a behavior type obtained by performing joint activities on the user's real-time position; the monitoring behavior initiation action can also be a specific behavior type input by the user; or a specific behavior type selected by the user on the database. The monitoring behavior termination action refers to the interactive termination behavior specified by the user. The monitoring behavior termination action can be a specific behavior type input by the user, or a specific behavior type selected by the user on the database, or a default behavior type pre-specified and saved by the user.

[0050] The actions that exist with the monitoring behavior initiation action and the monitoring behavior termination action refer to the actions obtained by executing the action plan with the monitoring behavior initiation action as the starting point and the monitoring behavior termination action as the end point. There may be multiple actions that exist with the monitoring behavior initiation action and the monitoring behavior termination action. Among these actions, there may be a single actor in some actions, and there may be actors who interact alone or with multiple people. The monitoring behavior refers to the actions that involve one person alone or with multiple actors in these actions.

[0051] Specifically, after the user specifies the monitoring behavior initiation action and the monitoring behavior termination action on the terminal, the control related to the action plan is started, and the terminal collects the action plan message. According to the action plan message, the terminal can collect the behavior trajectory semantic record of the monitoring behavior obtained by executing the action plan with the monitoring behavior initiation action and the monitoring behavior termination action. The behavior trajectory semantic records of multiple monitoring behaviors constitute a behavior trajectory semantic record list.

[0052] It is understandable that in actual applications, when multiple behavioral characters are combined, users can often be provided with a more accurate travel method. However, considering that different users have different interactive behavior needs, when the terminal collects the semantic records of the behavioral trajectories, in addition to collecting the semantic records of the behavioral trajectories of the monitored behaviors, it can also simultaneously collect the semantic records of the behavioral trajectories of the actions in the single-scene interactive behaviors to meet the needs of different users for interactive behaviors.

[0053] In another embodiment, after collecting the action planning message, the terminal can send the action planning message to the server. After the server parses the action planning message, it collects the monitoring behavior initiation action and the monitoring behavior termination action carried therein, executes the action plan according to the monitoring behavior initiation action and the target terminal, and obtains the behavior trajectory semantic record of the monitoring behavior that exists with the monitoring behavior initiation action and the monitoring behavior termination action, forming a behavior trajectory semantic record list.

[0054] S204: When the interactive behavior starts, the monitoring behavior record is determined from the behavior track semantic record list.

[0055] Specifically, different behavior trajectory semantic records in the behavior trajectory semantic record list contain different monitoring behaviors. After obtaining the behavior trajectory semantic record list, the terminal can display the behavior trajectory semantic record of each monitoring behavior and start the interactive behavior after collecting the interactive behavior start instruction. After starting the interactive behavior, the terminal determines the behavior trajectory semantic record where the interactive behavior start instruction exists from the behavior trajectory semantic record list as the monitoring behavior record based on the collected interactive behavior start instruction.

[0056] In one embodiment, the terminal interface may display an interactive behavior start mark, and the interactive behavior start instruction is a start instruction for the interactive behavior start mark. The interactive behavior start mark may be a prompt record or a start control that can start the terminal to start the interactive behavior. After obtaining the start instruction for the interactive behavior start mark, the terminal starts the interactive behavior. Among them, the start instruction for the interactive behavior start mark refers to a preset instruction that acts on the interactive behavior start mark. The start instruction may specifically be a touch instruction, a cursor instruction, a key instruction, or a voice instruction. Among them, the touch instruction may be a touch click instruction, a touch press instruction, or a touch slide instruction, and the touch instruction may also be a single-point touch instruction or a multi-point touch instruction; the cursor instruction may be an instruction to control the cursor to click or an instruction to control the cursor to press; the key instruction may be a virtual key instruction or a physical key instruction, etc.

[0057] In another embodiment, the terminal may display the behavior trajectory semantic record in the form of voice broadcast.

[0058] S206, performing actor marking in the order in which the actors exist, and performing action prediction in each actor according to the action prediction record, where the action prediction record is determined based on the real-time interactive action snapshot and the monitoring action record.

[0059] Specifically, since the monitoring behavior is planned in advance, the order of the actors is fixed. Therefore, after the interactive behavior begins, the terminal can mark the actors in the order in which they exist, and in each actor, the terminal collects the user's real-time interactive action snapshot, determines the action prediction record based on the real-time interactive action snapshot and the monitoring behavior record, and then performs action prediction based on the action prediction record.

[0060] In one embodiment, when the terminal performs action prediction, it can display the action prediction record on the real-time interactive behavior interface, where the action prediction record is the interactive behavior record used to perform the interactive behavior, including at least one of a text prediction record and an icon indication record. The text prediction record refers to a record of performing interactive behavior prediction through text.

[0061] In the above-mentioned interactive behavior method, a list of behavior trajectory semantic records is obtained by collecting the behavior trajectory semantic records of the monitoring behavior initiating action and the monitoring behavior existing in the target terminal carried in the collection and action planning message. The monitoring behavior exists in the behavioral characters interacting alone or with multiple people. When the interactive behavior starts, the monitoring behavior record is determined from the list of behavior trajectory semantic records, and the behavioral characters are marked in the order in which they exist. In each behavioral character, action prediction is performed according to the action prediction record. In the present application, since the behavior trajectory semantic records of multiple behavioral characters can be collected at one time, and after the interactive behavior starts, the behavioral characters can be automatically marked in order, the time waste caused by users initiating interactive behaviors multiple times and manually marking interactive behaviors is avoided, and the efficiency of interactive behaviors is improved.

[0062] Furthermore, since the complete semantic records of the behavior trajectories of multiple actors are obtained before the start of the interactive behavior, there is no need to collect the semantic records of the behavior trajectories multiple times when performing actor tagging during the interactive behavior, thus avoiding the failure of collecting the semantic records of the behavior trajectories due to interference from network signals during the actor tagging process, thereby improving the reliability of the interactive behavior.

[0063] In one embodiment, actor marking is performed in the order in which each actor exists, and in each actor, action prediction is performed according to the action prediction record, including: when the real-time interactive action snapshot has not reached the interaction termination behavior of the real-time actor, action prediction is performed according to the action prediction record; when the real-time interactive action snapshot reaches the interaction termination behavior of the real-time actor and has not reached the monitoring behavior termination action, marking to the next actor.

[0064] It is understandable that since the monitoring behavior involves a single actor or multiple actors interacting, each actor will have a sub-action segment in the monitoring behavior. The monitoring behavior can be seen as being composed of the sub-action segments of each actor connected in sequence.

[0065] Specifically, in each behavioral character, the terminal collects the user's real-time interactive action snapshot, and determines whether the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavioral character. If the real-time interactive action snapshot does not reach the interactive termination behavior of the real-time behavioral character, the interactive behavior continues to be executed in the real-time scenario. When executing the interactive behavior, the terminal performs action prediction on the user according to the action prediction record; if the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavioral character, the terminal continues to determine whether the real-time interactive action snapshot reaches the monitoring behavior termination action. If the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavioral character and does not reach the monitoring behavior termination action, the terminal executes the marking of the behavioral character, that is, determines the next behavioral character of the real-time behavioral character as the real-time behavioral character.

[0066] It is understandable that when the real-time interactive action snapshot reaches the interactive termination action of the real-time behavior character and reaches the monitoring behavior termination action, it means that the user has reached the monitoring behavior termination action at this time, and the terminal will terminate all interactive behaviors.

[0067] In the above embodiment, by determining whether the real-time interactive action snapshot reaches the interactive termination behavior and monitoring the behavior termination action, the terminal can accurately perform the marking of the person who performs the behavior.

[0068] In one embodiment, when the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavioral character and has not reached the monitored behavior termination action, before marking to the next behavioral character, the interactive behavior method also includes: determining the real-time contact point of the real-time interactive action snapshot on the action where the monitored behavior record exists; when the distance between the real-time interactive action snapshot and the interactive termination behavior does not exceed a first preset distance threshold and the distance between the real-time contact point and the interactive termination behavior does not exceed a second preset distance threshold, determining that the real-time interactive action snapshot has reached the interactive termination behavior of the real-time behavioral character.

[0069] It can be understood that during interactive behavior, the real-time interactive action snapshot of the user collected will usually not be exactly on the action where the monitoring behavior record exists. Therefore, the terminal needs to determine the real-time contact point of the real-time interactive action snapshot based on the action where the monitoring behavior record exists. The real-time contact point of the real-time interactive action snapshot is the contact point on the action where the monitoring behavior record exists that is closest to the real-time interactive action snapshot.

[0070] The distance between the real-time interactive action snapshot and the interactive termination behavior is usually a straight-line distance. However, in some cases, the user's real-time interactive action snapshot cannot reach the interactive termination behavior in a straight line, and the user may need to take a detour to reach the interactive termination behavior. In this case, even if the distance between the user's real-time interactive action snapshot and the interactive termination behavior does not exceed the preset threshold, the user still has not reached the interactive termination behavior. To avoid premature termination of the interactive behavior, after determining the real-time contact point where the real-time interactive action snapshot exists, the terminal can simultaneously determine whether the distance between the real-time interactive action snapshot and the interactive termination behavior exceeds the first preset distance threshold and whether the distance between the real-time contact point and the interactive termination behavior exceeds the second preset distance threshold. Only when the distance between the real-time interactive action snapshot and the interactive termination behavior does not exceed the first preset distance threshold and the distance between the real-time contact point and the interactive termination behavior does not exceed the second preset distance threshold, the real-time interactive action snapshot is determined to have reached the interactive termination behavior where the real-time behavior character exists. Among them, the first preset distance threshold and the second preset distance threshold can be set in advance according to actual needs.

[0071] In the above embodiment, by simultaneously performing judgment on the real-time interaction action snapshot and the real-time contact point to determine whether the real-time interaction action snapshot has reached the interaction termination behavior where the actor exists, it is possible to accurately determine whether the user has reached the interaction termination behavior at this time, thereby avoiding premature termination of the interaction behavior caused by judging only based on the real-time interaction action snapshot.

[0072] In one embodiment, Figure 2 As shown, a deep learning-based intelligent video security monitoring method is provided, comprising the following steps:

[0073] S302, collecting behavior trajectory semantic records of monitoring behaviors that have monitoring behavior initiation actions and monitoring behavior termination actions, obtaining a behavior trajectory semantic record list, and monitoring behaviors that have behavioral characters interacting alone or with multiple people.

[0074] S304, when the interactive behavior starts, a monitoring behavior record is determined from the behavior track semantic record list, and the monitoring behavior record includes the sub-behavior track semantic records of each behavior character.

[0075] The semantic record of the sub-behavior trajectory of the behavior character refers to the semantic record of the behavior trajectory of the sub-action fragment of the behavior character.

[0076] S306, creating a corresponding target interactive behavior deep learning model based on the semantic records of the sub-behavior trajectories of the real-time behavior characters.

[0077] S308, using the target interactive behavior deep learning model to detect whether the real-time interactive action snapshot reaches the interactive termination behavior where the real-time behavior character exists; if not, enter S310; if so, terminate.

[0078] The interactive behavior deep learning model refers to a function library used to drive interactive behaviors. Different interactive behavior deep learning models exist for different behavioral characters. The target interactive behavior deep learning model for real-time behavioral characters refers to an interactive behavior deep learning model that exists with the real-time behavioral characters and with the monitored behavior records.

[0079] S310, collecting action prediction records existing in the real-time interaction action snapshot from the sub-behavior trajectory semantic records through the target interaction behavior deep learning model, and reporting action prediction instructions carrying the action prediction records.

[0080] Specifically, when the real-time interactive action snapshot does not reach the interactive termination behavior where the real-time behavior character exists, it means that the user is still in the real-time behavior character, then the interactive behavior can be executed through the target interactive behavior deep learning model where the real-time behavior character exists. Since the behavior trajectory semantic record is saved in the target interactive behavior deep learning model, and the behavior trajectory semantic record includes the action prediction record at each point on the action, the terminal can first determine the real-time contact point with the real-time interactive action snapshot on the action where the monitoring behavior record exists through the target interactive behavior deep learning model, determine the action prediction record at the real-time contact point as the action prediction record in the real-time interactive action snapshot, and report the action prediction instruction through the target interactive behavior deep learning model, and the action prediction instruction carries the determined action prediction record.

[0081] S312 , in response to the action prediction instruction, performing action prediction according to the action prediction record.

[0082] Specifically, the terminal responds to the action prediction instruction, obtains the action prediction record therein by parsing the action prediction instruction, and performs action prediction on the user according to the action prediction record.

[0083] In the above embodiment, a deep learning model of the target interactive behavior of the real-time behavior character is collected and used to detect whether the real-time interactive action snapshot reaches the interactive termination behavior. When the real-time interactive action snapshot does not reach the interactive termination behavior of the real-time behavior character, an action prediction instruction is reported to realize action prediction, thereby improving the accuracy and efficiency of the interactive behavior.

[0084] In one embodiment, creating a corresponding target interactive behavior deep learning model based on the semantic records of sub-behavior trajectories of real-time behavioral characters includes: collecting an interactive behavior feature list; the interactive behavior feature list includes a feature sub-list of each behavioral character; the feature sub-list of each behavioral character includes a sub-interactive behavior state of each behavioral character; determining the sub-interactive behavior state of the real-time behavioral character as a real-time control feature to start creating an interactive behavior deep learning model of the real-time behavioral character, and initializing the created interactive behavior deep learning model based on the semantic records of the sub-behavior trajectories to obtain a target interactive behavior deep learning model of the real-time behavioral character.

[0085] Among them, the interactive behavior feature list refers to a list composed of multiple different interactive behavior features. The interactive behavior feature list includes feature sublists for each behavioral character. Each feature sublist for each behavioral character includes multiple interactive behavior features under the behavioral character, and each interactive behavior feature is used to control the call of different interactive behavior functions under the behavioral character. Real-time control features refer to interactive behavior features that play a real-time control role. Each time, the terminal selects an interactive behavior feature from the interactive behavior feature list as a real-time control feature to achieve marking between different interactive behavior features, so that different interactive behavior functions can be controlled.

[0086] In this embodiment, the terminal determines the sub-interactive behavior state of the real-time behavior character as a real-time control feature. At this time, the sub-interactive behavior state controls the call of the existing interactive behavior function to create a corresponding target interactive behavior deep learning model. Since after the interactive behavior begins, the terminal needs to collect action prediction records and report action prediction instructions through the target interactive behavior deep learning model, then in this sub-interactive behavior state, the terminal will be started to create an interactive behavior deep learning model for the real-time behavior character, and the created interactive behavior deep learning model will be initialized according to the semantic record of the sub-behavior trajectory of the real-time behavior character, so as to obtain the target interactive behavior deep learning model of the real-time behavior character.

[0087] In one embodiment, the terminal can implement the selection and marking of interactive behavior features through an interactive behavior feature machine. The interactive behavior feature machine is responsible for managing all interactive behavior features in the interactive behavior feature list. The interactive behavior feature machine includes multiple sub-feature machines, each of which is responsible for managing the feature sub-lists of different behavioral characters, and marking different interactive behavior features according to the situation. After each marking, the marked feature is a real-time control feature. After the interactive behavior begins, the interactive behavior feature machine will mark the interactive behavior feature to the sub-feature machine where the behavioral character exists, and the sub-feature machine will control the marking of the interactive behavior feature under the behavioral character. When the user reaches the interactive termination behavior of a certain behavioral character, the interactive behavior feature machine will mark the interactive behavior feature to the sub-feature machine of the next behavioral character until the user finally reaches the monitoring behavior termination action.

[0088] In this embodiment, by determining the sub-interaction behavior state of the real-time actor as the real-time control feature to obtain the target interaction behavior deep learning model of the real-time actor, it is possible to achieve action prediction in each actor through the interaction behavior deep learning model of each actor, thereby ensuring the accuracy of the interaction behavior.

[0089] In one embodiment, the feature sublist of each actor includes the sub-termination feature of each actor; when the real-time interactive action snapshot reaches the interactive termination behavior of the real-time actor and has not reached the monitoring behavior termination action, marking to the next actor includes: when the real-time interactive action snapshot reaches the interactive termination behavior of the real-time actor and has not reached the monitoring behavior termination action, determining the sub-termination feature of the real-time actor as a real-time control feature to start terminating the real-time interactive behavior; determining the sub-interaction behavior state of the next actor as a real-time control feature to start creating an interactive behavior deep learning model for the next actor, and initializing the created interactive behavior deep learning model according to the semantic record of the sub-behavior trajectory of the next actor to obtain the target interactive behavior deep learning model of the next actor.

[0090] In this embodiment, when the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavioral character and has not reached the monitoring behavior termination action, it means that the action of the real-time behavioral character has been terminated, and the user has not reached the monitoring behavior termination action at this time, then the terminal can determine the sub-termination feature of the real-time behavioral character as the real-time control feature, under this sub-termination feature, the target interactive behavior deep learning model of the real-time behavioral character will be destroyed. At this time, the real-time behavioral character is terminated, and the terminal determines the sub-interactive behavior state of the next behavioral character as the real-time control feature. Under this sub-interactive behavior state, the terminal will initialize the interactive behavior deep learning model of the next behavioral character created according to the semantic record of the sub-behavior trajectory of the next behavioral character, and obtain the target interactive behavior deep learning model of the next behavioral character. After the target interactive behavior deep learning model of the next behavioral character is created, it means that the terminal can enter the next behavioral character to continue the interactive behavior, thereby realizing the marking of the behavioral character.

[0091] In this embodiment, automatic labeling of different actors is achieved by determining the sub-interaction behavior states of different actors as real-time control features, and when labeling the actors, the interactive behavior deep learning model of the real-time actors is destroyed, so that only one interactive behavior deep learning model is working in one actor. This not only allows the terminal to accurately perform the labeling of the actors, but also avoids memory waste caused by unnecessary interactive behavior deep learning models.

[0092] In one embodiment, the interactive behavior feature list also includes a termination feature; the interactive behavior method also includes: when the real-time interactive action snapshot reaches the monitoring behavior termination action, determining the termination feature as a real-time control feature to start terminating all interactive behaviors.

[0093] It can be understood that when the user reaches the monitoring behavior termination action, the interactive behavior will automatically terminate. Then the interactive behavior feature list can include a termination feature. When the terminal determines that the real-time interactive action snapshot reaches the monitoring behavior termination action, the termination feature can be determined as a real-time control feature. At the termination feature, all interactive behavior deep learning models are destroyed, and the terminal terminates the entire interactive behavior process.

[0094] In one embodiment, the terminal may mark the interactive behavior feature as a termination feature through the interactive behavior feature machine, so as to determine the termination feature as a real-time control feature.

[0095] In one embodiment, the interactive behavior feature list also includes a termination feature; the above method further includes: when an interactive behavior termination message is obtained, determining the termination feature as a real-time control feature to start terminating the interactive behavior.

[0096] It is understandable that during the process of interactive behavior, users sometimes encounter situations where they need to terminate the interactive behavior prematurely. In this case, when the terminal obtains the interactive behavior termination message initiated by the user, it can determine the termination feature as a real-time control feature. At this time, no matter which interactive behavior deep learning model is working in real time, and regardless of whether the interactive termination behavior of the real-time behavior character is reached, the terminal will destroy the interactive behavior deep learning model and terminate the entire interactive behavior process.

[0097] In one embodiment, the feature sublist of each actor includes abnormal features of each actor; the method also includes: when the real-time interactive action snapshot deviates from the action of the real-time actor, the abnormal feature of the real-time actor is determined as a real-time control feature to start collecting action re-monitoring records starting from the real-time interactive action snapshot, ending at the interactive termination behavior and related to the real-time actor; after collecting the action re-monitoring record, the sub-interactive behavior state of the real-time actor is re-determined as a real-time control feature to start updating the target interactive behavior deep learning model according to the action re-monitoring record.

[0098] The action of the real-time behavior character refers to the action formed according to the semantic record of the sub-behavior trajectory of the real-time behavior character.

[0099] It is understandable that during the interactive behavior, the user may deviate from the interactive behavior action for various reasons. At this time, the interactive action snapshot collected by the terminal will also deviate from the action in the monitoring behavior record. The accurate interactive behavior cannot be performed according to the original action, and it is necessary to re-plan the action with the user's real-time interactive action snapshot as the starting point. Therefore, the feature sub-list of each behavioral character also includes the abnormal features of each behavioral character. When the real-time interactive action snapshot deviates from the action of the real-time behavioral character, the terminal determines the abnormal features of the real-time behavioral character as the real-time control feature. Under this abnormal feature, the terminal can be started to collect the action re-monitoring record with the real-time interactive action snapshot as the starting point, the interactive termination behavior of the real-time behavioral character as the end point and the real-time behavioral character. After collecting the action re-monitoring record, the terminal can restart the normal interactive behavior according to the action re-monitoring record, and then the terminal can re-determine the sub-interactive behavior state of the real-time behavioral character as the real-time control feature. The sub-interactive behavior state starts the terminal to re-initialize the target interactive behavior deep learning model according to the action re-monitoring record to update the target interactive behavior deep learning model. The updated target interactive behavior deep learning model can continue to perform the interactive behavior according to the user's interactive action snapshot and the action re-monitoring record. The specific process of the terminal collection action and then monitoring and recording can refer to the process of terminal collection action planning and recording in the above embodiment.

[0100] In one embodiment, the terminal can mark the interactive behavior features in the real-time behavior character as abnormal features through the sub-feature machine of the real-time behavior character, so as to determine the abnormal features of the real-time behavior character as real-time control features. Further, after collecting the action and monitoring records, the terminal can re-mark the interactive behavior features in the real-time behavior character to the sub-interactive behavior state of the real-time behavior character through the sub-feature machine of the real-time behavior character, so as to re-determine the sub-interactive behavior state of the real-time behavior character as real-time control features.

[0101] In this embodiment, by determining the abnormal features of the real-time behavior characters as real-time control features, the action plan can be re-executed when the user deviates from the action, thereby ensuring the accuracy of the interactive behavior.

[0102] In one embodiment, the target interactive behavior deep learning model is also used to report abnormal instructions carrying the real-time interactive action snapshot when the real-time interactive action snapshot deviates from the action of the real-time interactive action character; when the real-time interactive action snapshot deviates from the action of the real-time interactive action character, determining the abnormal features of the real-time interactive action character as real-time control features includes: when the real-time interactive action snapshot deviates from the action of the real-time interactive action character, in response to the abnormal instructions, determining the abnormal features of the real-time interactive action character as real-time control features.

[0103] Specifically, the terminal can determine whether the real-time interactive action snapshot deviates from the action of the real-time behavior character through the target interactive behavior deep learning model. When the real-time interactive action snapshot deviates from the action of the real-time behavior character, the target interactive behavior deep learning model reports an abnormal instruction carrying the real-time interactive action snapshot, and the terminal further responds to the abnormal instruction and determines the abnormal feature of the real-time behavior character as a real-time control feature, thereby realizing the interactive behavior function call under the abnormal feature. Since the target interactive behavior deep learning model is initialized according to the semantic record of the sub-behavior trajectory of the real-time behavior character, the target interactive behavior deep learning model can accurately determine whether the real-time interactive action snapshot deviates from the action of the real-time behavior character. Therefore, by responding to the abnormal instruction reported by the target interactive behavior deep learning model to determine the abnormal feature of the real-time behavior character as a real-time control feature, when the user deviates from the action, it can be accurately marked to the abnormal feature, and the user's abnormal behavior can be accurately predicted by calling the interactive behavior function under the abnormal feature.

[0104] In one embodiment, the target interactive behavior deep learning model is also used to respectively collect contact points of a preset number of consecutive interactive action snapshots ending at the real-time interactive action snapshot on the actions of the real-time behavior character; when the distances between a preset number of consecutive interactive action snapshots ending at the real-time interactive action snapshot and their respective existing contact points exceed a third preset distance threshold, it is determined that the real-time interactive action snapshot deviates from the action of the real-time behavior character; or when the angle between the action of a preset number of consecutive interactive action snapshots ending at the real-time interactive action snapshot and the direction of the action where each contact point is located exceeds a preset angle threshold, it is determined that the real-time interactive action snapshot deviates from the action of the real-time behavior character.

[0105] Among them, the continuous preset number of interactive action snapshots up to the real-time interactive action snapshot include the real-time interactive action snapshot and several historical interactive action snapshots that are continuous with the real-time interactive action snapshot. The contact point of the interactive action snapshot on the action of the real-time behavior character refers to the contact point on the action of the real-time behavior character that is closest to the interactive action snapshot. The determination of the contact point can refer to the description in the above embodiment. The action fragment that exists in the continuous preset number of interactive action snapshots up to the real-time interactive action snapshot refers to the action fragment obtained by connecting these interactive action snapshots.

[0106] Specifically, for the case of abnormal distance, the terminal can use the target interactive behavior deep learning model to determine whether the distance between the real-time interactive action snapshot and these historical interactive action snapshots and the respective contact points exceeds the third preset distance threshold. If so, it can be determined that the real-time interactive action snapshot deviates from the action of the real-time behavior character. For the case of abnormal angle, the terminal can use the target interactive behavior deep learning model to determine whether the direction of the action fragments in which the real-time interactive action snapshot and these historical interactive action snapshots exist and the direction of the action fragments where each contact point is located exceeds the preset angle threshold. If so, it is determined that the real-time interactive action snapshot deviates from the action of the real-time behavior character. It can be understood that the direction of the action fragment referred to here refers to the direction of the user's movement in the action fragment, and the direction of the action fragment can be uniquely determined.

[0107] In one embodiment, before performing action prediction based on the collected real-time interactive action snapshots and monitored behavior records, the interactive behavior method also includes: collecting real-time joint movement records and historical joint movement records; when the real-time joint movement records are determined to meet preset conditions based on the historical joint movement records, the interactive action snapshot in the real-time joint movement record is determined as the real-time interactive action snapshot.

[0108] Specifically, during the interactive behavior, the terminal will continuously collect the user's real-time joint activity records. Under normal circumstances, the multiple joint activity records collected by the terminal usually conform to certain rules. However, sometimes some wrong joint activity records are collected due to various reasons. These wrong joint activity records cannot reflect the user's real-time position. Then the terminal can filter these wrong joint activity records in combination with the historical joint activity records to obtain the correct joint activity records. In the embodiment of the present application, by setting preset conditions, the terminal can judge whether the real-time joint activity record meets the preset conditions based on the historical joint activity records. Only when the real-time joint activity record meets the preset conditions, the collected real-time joint activity record is considered correct, and the interactive action snapshot in the correct real-time joint activity record is determined as the real-time interactive action snapshot. Among them, the preset conditions are set as needed, as long as the set preset conditions can reflect that multiple joint activity records conform to certain rules.

[0109] In this embodiment, the terminal can filter out erroneous real-time joint movement records through historical joint movement records, so as to prevent incorrect joint movement records from affecting the accuracy of interactive behaviors.

[0110] In one embodiment, after determining the monitoring behavior record from the behavior trajectory semantic record list, the method also includes: controlling the real-time monitoring display screen to display the existing monitoring behavior according to the monitoring behavior record; performing action prediction according to the action prediction record includes: controlling the real-time monitoring display screen to display the action prediction record.

[0111] Specifically, the real-time monitoring display screen is an interface that can display the database. After determining the monitoring behavior record, the terminal can control the real-time monitoring display screen to display the existing monitoring behavior according to the monitoring behavior record, and in each behavior character, control the real-time monitoring display screen to display the action prediction record, thereby visually and intuitively performing interactive behavior predictions on the user and improving the efficiency of interactive behavior.

[0112] In a specific embodiment, a deep learning-based intelligent video security monitoring method is provided, comprising the following steps:

[0113] 1. Collect and monitor the behavior initiation action and the monitoring behavior termination action to obtain the behavior trajectory semantic record list of the monitoring behavior; the monitoring behavior has a person who is alone or interacts with multiple people.

[0114] 2. When the interactive behavior begins, the monitoring behavior record is determined from the behavior trajectory semantic record list; the monitoring behavior record includes the sub-behavior trajectory semantic records of each behavior character.

[0115] 3. Control the real-time monitoring display screen to display the existing monitoring behavior according to the monitoring behavior record.

[0116] 4. Collect the interactive behavior feature list; the interactive behavior feature list includes the feature sublist of each behavioral character; the feature sublist of each behavioral character includes the sub-interactive behavior states of each behavioral character.

[0117] Among them, the interactive behavior feature list also includes termination features; the feature sublist of each behavior character includes the abnormal features and sub-termination features of each behavior character.

[0118] 5. Determine the sub-interaction behavior state of the real-time behavior character as the real-time control feature to start creating the interactive behavior deep learning model of the real-time behavior character, and initialize the created interactive behavior deep learning model according to the semantic record of the sub-behavior trajectory to obtain the target interactive behavior deep learning model of the real-time behavior character.

[0119] 6. Use the target interactive behavior deep learning model to detect whether the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavior character.

[0120] 7. When the real-time interactive action snapshot does not reach the interaction termination behavior of the real-time behavior character, the target interactive behavior deep learning model is used to collect the action prediction record that exists in the real-time interactive action snapshot from the sub-behavior trajectory semantic record, and report the action prediction instruction carrying the action prediction record.

[0121] 8. In response to the action prediction instruction, perform action prediction according to the action prediction record.

[0122] Specifically, when executing action prediction, the real-time monitoring display screen is controlled to display the action prediction record.

[0123] 9. When judging whether the real-time interactive action snapshot deviates from the action of the real-time behavioral character through the target interactive behavior deep learning model, when the real-time interactive action snapshot deviates from the action of the real-time behavioral character, an abnormal instruction carrying the real-time interactive action snapshot is reported.

[0124] 10. In response to the abnormal instruction, the abnormal features of the real-time behavior character are determined as real-time control features to start collecting the real-time interactive action snapshot as the starting point, the interactive termination behavior as the end point, and the action re-monitoring record with the real-time behavior character.

[0125] 11. After collecting the action re-monitoring records, the sub-interaction behavior states of the real-time behavior characters are re-determined as real-time control features to start updating the target interaction behavior deep learning model based on the action re-monitoring records.

[0126] 12. When the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavior character and does not reach the monitoring behavior termination action, the sub-termination feature of the real-time behavior character is determined as the real-time control feature to start terminating the real-time interactive behavior.

[0127] 13. Determine the sub-interaction behavior state of the next actor as a real-time control feature to start creating a deep learning model of the interaction behavior of the next actor, and initialize the created deep learning model of the interaction behavior according to the semantic record of the sub-behavior trajectory of the next actor to obtain the target deep learning model of the interaction behavior of the next actor.

[0128] 14. When the real-time interactive action snapshot reaches the monitoring behavior termination action, the termination feature is determined as the real-time control feature to initiate the termination of all interactive behaviors.

[0129] In one embodiment, Figure 3 As shown, an intelligent video security monitoring system based on deep learning is provided. The system can adopt a software unit or a hardware unit, or a combination of the two to become a part of a computer device. The system specifically includes:

[0130] The behavior trajectory semantic record collection unit 802 collects the behavior trajectory semantic records of the monitoring behavior with the monitoring behavior initiation action and the monitoring behavior termination action, and obtains a behavior trajectory semantic record list; the monitoring behavior has a behavior character interacting alone or with multiple people;

[0131] A monitoring behavior record determining unit 804 is used to determine the monitoring behavior record from the behavior track semantic record list when the interactive behavior starts;

[0132] The marking unit 806 is used to perform actor marking in the order in which each actor exists, and perform action prediction in each actor according to the action prediction record; the action prediction record is determined based on the real-time interactive action snapshot and the monitoring behavior record.

[0133] In one embodiment, the marking unit is also used to perform action prediction based on the action prediction record when the real-time interaction action snapshot has not reached the interaction termination behavior of the real-time behavior character; when the real-time interaction action snapshot reaches the interaction termination behavior of the real-time behavior character and has not reached the monitoring behavior termination action, mark to the next behavior character.

[0134] In one embodiment, the above system also includes: an arrival judgment unit, which is used to determine the real-time contact point where the real-time interactive action snapshot exists on the action where the monitoring behavior record exists; when the distance between the real-time interactive action snapshot and the interactive termination behavior does not exceed a first preset distance threshold and the distance between the real-time contact point and the interactive termination behavior does not exceed a second preset distance threshold, it is determined that the real-time interactive action snapshot has reached the interactive termination behavior where the real-time behavior character exists.

[0135] In one embodiment, the monitoring behavior records include semantic records of sub-behavior trajectories of each behavior character; the above system also includes: a deep learning model creation unit, which is used to create a corresponding target interaction behavior deep learning model based on the semantic records of sub-behavior trajectories of real-time behavior characters; the target interaction behavior deep learning model is used to collect action prediction records that exist in the real-time interaction action snapshot from the sub-behavior trajectory semantic records when the real-time interaction action snapshot has not reached the interaction termination behavior of the real-time behavior character, and report action prediction instructions carrying the action prediction records; the marking unit is also used to respond to the action prediction instructions and perform action prediction according to the action prediction records when the real-time interaction action snapshot has not reached the interaction termination behavior of the real-time behavior character.

[0136] In one embodiment, the deep learning model creation unit is also used to collect an interactive behavior feature list; the interactive behavior feature list includes a feature sub-list of each behavioral character; the feature sub-list of each behavioral character includes a sub-interactive behavior state of each behavioral character; the sub-interactive behavior state of the real-time behavioral character is determined as a real-time control feature to start creating an interactive behavior deep learning model of the real-time behavioral character, and the created interactive behavior deep learning model is initialized according to the sub-behavior trajectory semantic record to obtain a target interactive behavior deep learning model of the real-time behavioral character.

[0137] In one embodiment, the interactive behavior feature list also includes a termination feature; the above system also includes a first termination unit, which is used to determine the termination feature as a real-time control feature when the real-time interactive action snapshot reaches the monitoring behavior termination action to initiate the termination of all interactive behaviors.

[0138] In one embodiment, the interactive behavior feature list also includes a termination feature; the system further includes a second termination unit, which is used to determine the termination feature as a real-time control feature to start terminating the interactive behavior when an interactive behavior termination message is obtained.

[0139] In one embodiment, the feature sublist of each actor includes a sub-termination feature of each actor; the marking unit is also used to determine the sub-termination feature of the real-time actor as a real-time control feature when the real-time interactive action snapshot reaches the interactive termination behavior of the real-time actor and does not reach the monitoring behavior termination action, so as to initiate the termination of the real-time interactive behavior.

[0140] In one embodiment, the marking unit is also used to determine the sub-interaction behavior state of the next actor as a real-time control feature to start creating an interactive behavior deep learning model for the next actor, and initialize the created interactive behavior deep learning model according to the semantic record of the sub-behavior trajectory of the next actor to obtain the target interactive behavior deep learning model of the next actor.

[0141] In one embodiment, the feature sublist of each actor includes abnormal features of each actor; the above system also includes: an abnormality handling unit, which is used to determine the abnormal features of the real-time actor as real-time control features when the real-time interactive action snapshot deviates from the action of the real-time actor, so as to start collecting action re-monitoring records starting from the real-time interactive action snapshot, ending at the interactive termination behavior and related to the real-time actor; after collecting the action re-monitoring records, re-determine the sub-interaction behavior state of the real-time actor as a real-time control feature, so as to start updating the target interactive behavior deep learning model according to the action re-monitoring records.

[0142] In one embodiment, the target interactive behavior deep learning model is also used to report an abnormal instruction carrying the real-time interactive action snapshot when the real-time interactive action snapshot deviates from the action of the real-time behavioral character; the abnormal handling unit is also used to determine the abnormal features of the real-time behavioral character as real-time control features in response to the abnormal instruction when the real-time interactive action snapshot deviates from the action of the real-time behavioral character.

[0143] In one embodiment, the above system also includes: the target interactive behavior deep learning model is also used to: respectively collect contact points of a continuous preset number of interactive action snapshots ending at the real-time interactive action snapshot on the actions of the real-time behavior character; when the distances between the continuous preset number of interactive action snapshots ending at the real-time interactive action snapshot and their respective existing contact points exceed a third preset distance threshold, determine that the real-time interactive action snapshot deviates from the action of the real-time behavior character; or when the angle between the action fragments of the continuous preset number of interactive action snapshots ending at the real-time interactive action snapshot and the direction of the action fragments where each contact point is located exceeds a preset angle threshold, determine that the real-time interactive action snapshot deviates from the action of the real-time behavior character.

[0144] In one embodiment, the above system also includes: an interactive action snapshot determination unit, which is used to collect real-time joint activity records and historical joint activity records; when the real-time joint activity record is determined to meet preset conditions based on the historical joint activity record, the interactive action snapshot in the real-time joint activity record is determined as the real-time interactive action snapshot.

[0145] In one embodiment, the above system also includes: a display control unit, which is used to control the real-time monitoring display screen to display the existing monitoring behavior according to the monitoring behavior record; the marking unit is also used to control the real-time monitoring display screen to display the action prediction record.

[0146] For the specific definition of the interactive behavior system, please refer to the definition of the interactive behavior method above, which will not be repeated here. Each unit in the above interactive behavior system can be implemented in whole or in part by software, hardware and a combination thereof. The above units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the instructions of the above units.

[0147] In the several embodiments provided in the present application, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0148] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

[0149] The above detailed description of the specific implementation of the invention is only used as an example, and the present application is not limited to the specific implementation described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the present application. Therefore, the equalization, modification, improvement, etc. made without departing from the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. An intelligent video security monitoring method based on deep learning, characterized in that: The method comprises: Collect and monitor the behavior trajectory semantic records of the monitoring behavior that has the behavior initiation action and the monitoring behavior termination action, and obtain a behavior trajectory semantic record list; the monitoring behavior has a behavior character that interacts alone or with multiple people; When the interactive behavior begins, determining the monitoring behavior record from the behavior trajectory semantic record list; Performing actor marking in the order in which each actor exists, and performing action prediction in each actor according to the action prediction record; the action prediction record is determined according to the real-time interactive action snapshot and the monitoring action record; The step of marking the actors in the order in which the actors exist, and performing action prediction for each actor according to the action prediction record includes: When the real-time interactive action snapshot does not reach the interactive termination behavior where the real-time behavior character exists, action prediction is performed according to the action prediction record; When the real-time interactive action snapshot reaches the interactive termination behavior of the real-time actor, and does not reach the monitoring behavior termination action, mark to the next actor; The monitoring behavior record includes a semantic record of sub-behavior trajectories of each behavior character; when the real-time interactive action snapshot does not reach the interactive termination behavior of the real-time behavior character, before performing action prediction according to the action prediction record, the method also includes: According to the semantic records of sub-behavior trajectories of real-time behavior characters, a corresponding deep learning model of target interactive behavior is created; The target interactive behavior deep learning model is used to collect the action prediction record existing in the real-time interactive action snapshot from the sub-behavior trajectory semantic record when the real-time interactive action snapshot has not reached the interactive termination behavior of the real-time behavior character, and report the action prediction instruction carrying the action prediction record; When the real-time interactive action snapshot does not reach the interactive termination behavior of the real-time behavior character, performing action prediction according to the action prediction record includes: When the real-time interactive action snapshot does not reach the interactive termination behavior where the real-time behavior character exists, responding to the action prediction instruction, performing action prediction according to the action prediction record; The step of creating a corresponding target interactive behavior deep learning model based on the semantic records of sub-behavior trajectories of real-time behavior characters includes: Collecting an interactive behavior feature list; the interactive behavior feature list includes a feature sublist of each actor; the feature sublist of each actor includes a sub-interactive behavior state of each actor; The sub-interactive behavior state of the real-time behavior character is determined as a real-time control feature to start creating an interactive behavior deep learning model of the real-time behavior character, and the created interactive behavior deep learning model is initialized according to the semantic record of the sub-behavior trajectory to obtain a target interactive behavior deep learning model of the real-time behavior character.

2. The method according to claim 1, characterized in that: The interactive behavior feature list also includes a termination feature; and the method further includes: When the real-time interactive action snapshot reaches the monitoring behavior termination action, determining the termination feature as a real-time control feature to start terminating all interactive behaviors; The interactive behavior feature list also includes a termination feature; and the method further includes: When an interactive behavior termination message is obtained, the termination feature is determined as a real-time control feature to start terminating the interactive behavior; The feature sublists of each actor include the sub-termination features of each actor; before marking to the next actor, the method further includes: When the real-time interactive action snapshot reaches the interactive termination behavior of the real-time behavior character and does not reach the monitoring behavior termination action, the sub-termination feature of the real-time behavior character is determined as the real-time control feature to start terminating the real-time interactive behavior.

3. The method according to claim 2, characterized in that The marking to the next actor includes: The sub-interaction behavior state of the next actor is determined as a real-time control feature to start creating a deep learning model of the interaction behavior of the next actor, and the created deep learning model of the interaction behavior is initialized according to the semantic record of the sub-behavior trajectory of the next actor to obtain the target deep learning model of the interaction behavior of the next actor.

4. The method according to claim 3, characterized in that The feature sublist of each actor includes the abnormal features of each actor; The method further comprises: When the real-time interactive action snapshot deviates from the action of the real-time behavior character, the abnormal feature of the real-time behavior character is determined as a real-time control feature to start collecting the action re-monitoring record starting from the real-time interactive action snapshot and ending at the interaction termination action and the action of the real-time behavior character; After collecting the action re-monitoring record, the sub-interaction behavior state of the real-time behavior character is re-determined as a real-time control feature to start updating the target interaction behavior deep learning model according to the action re-monitoring record.

5. The method according to claim 4, characterized in that The target interactive behavior deep learning model is also used to report abnormal instructions carrying the real-time interactive action snapshot when the real-time interactive action snapshot deviates from the action of the real-time behavior character; When the real-time interactive action snapshot deviates from the action of the real-time action character, determining the abnormal feature of the real-time action character as the real-time control feature includes: When the real-time interactive action snapshot deviates from the action of the real-time actor, in response to the abnormal instruction, the abnormal feature of the real-time actor is determined as the real-time control feature.

6. The method according to claim 5, characterized in that After determining the monitoring behavior record from the behavior trajectory semantic record list, the method further includes: Control the real-time monitoring display screen to display the existing monitoring behavior according to the monitoring behavior record; The performing action prediction according to the action prediction record comprises: Control the real-time monitoring display screen to display the action prediction record.

7. An intelligent video security monitoring system based on deep learning, characterized in that: The system comprises: A behavior trajectory semantic record collection unit collects behavior trajectory semantic records of monitoring behaviors including monitoring behavior initiation actions and monitoring behavior termination actions, and obtains a behavior trajectory semantic record list; the monitoring behaviors include behavior characters interacting alone or with multiple people; A monitoring behavior record determination unit, configured to determine a monitoring behavior record from the behavior trajectory semantic record list after the interactive behavior begins; A marking unit, used to perform actor marking in the order in which each actor exists, and perform action prediction in each actor according to an action prediction record; the action prediction record is determined based on the real-time interactive action snapshot and the monitoring action record; The deep learning-based intelligent video security monitoring system is used to execute the deep learning-based intelligent video security monitoring method as described in any one of claims 1-6.

Images