Video monitoring method and system for visual operation

Abstract

The invention relates to the technical field of digital information transmission, in particular to a video monitoring method and system for visual operation, and the method comprises the steps: obtaining the target video information of a target object, and determining the current collection efficiency-time sequence of the face information of the target object; determining a corresponding behavior state similarity between each adjacent time point, and determining each target cluster of the current collection efficiency-time sequence according to the behavior state similarity; determining a time sequence difference between a target time period corresponding to any target cluster and other time periods, and determining a continuous abnormal behavior possibility of the target object in the target time period according to the time sequence difference; and determining and marking the abnormal behavior of the target object according to the continuous abnormal behavior possibility. According to the video monitoring method for the visual operation, the problem that changes generated by static or small-amplitude actions are small and are not easy to recognize can be avoided, and the accuracy of abnormal behavior detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital information transmission, and in particular to a video monitoring method and system for visualized operation. Background Art

[0002] Visual operation refers to the process of enabling users to intuitively interact with computer systems, applications or data through GUI (Graphical User Interface) or other visual representations. Video data is collected through high-definition cameras, and based on the actual monitoring needs, the corresponding GUI interface is designed to provide tools such as thumbnails and timelines to facilitate users to quickly locate and retrieve videos. The visualization operations currently added to the processing of surveillance videos include the ability to use the timeline to reverse the video to obtain relevant monitoring information at a specific moment.

[0003] For example, in the classroom, students' movements are relatively small, and the target users of video surveillance using visual operations are usually parents of students, most of whom want to understand the historical listening status of students in class. Existing image detection technology can analyze and capture the movements of people or objects in video images, but for static or small movements, the changes produced in the video images over a continuous period of time are small, so the similarity is high, resulting in low recognition accuracy, that is, it is impossible to mark possible abnormal behaviors, such as dozing off, daydreaming, etc. Summary of the invention

[0004] In order to solve the technical problem that static and small-amplitude abnormal behaviors in surveillance videos are difficult to identify and mark, the purpose of the present invention is to provide a video surveillance method and system for visual operation, and the technical solution adopted is as follows: The present invention provides a video monitoring method for visual operation, the method comprising: Acquire target video information of the target object, and determine the current acquisition efficiency-time sequence of facial information of the target object using the target video information; Determine the behavior state similarity between each adjacent time point in the current acquisition efficiency-time series, and determine each target cluster of the current acquisition efficiency-time series according to the behavior state similarity; Determine the time difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects, and determine the possibility of continuous abnormal behavior of the target object in the target time period based on the time difference; Identify and flag anomalous behavior of target subjects based on the likelihood of continued anomalous behavior.

[0005] Furthermore, the step of determining the similarity of the behavior states corresponding to each adjacent time point in the current acquisition efficiency-time series includes: Determine the current acquisition efficiency - the efficiency difference of facial information acquisition efficiency between any two adjacent time points in the time series; Using the efficiency difference, the similarity of the behavior states between any two adjacent time points is calculated.

[0006] Furthermore, the step of calculating the similarity of the behavior states between any two adjacent time points by using the efficiency difference includes: Determine the instantaneous change rate of facial information collection efficiency corresponding to any two adjacent time points; Determine the difference in instantaneous rate of change between any two adjacent time points; The similarity of the behavior states between any two adjacent time points is calculated using the efficiency difference and the instantaneous change rate difference.

[0007] Furthermore, the step of determining each target cluster of the current acquisition efficiency-time series according to the behavior state similarity includes: The Turbopixels algorithm and behavior state similarity are used to cluster the current acquisition efficiency-time series to obtain various target clusters; among which, the target clusters represent the behavior state of the target object.

[0008] Furthermore, the step of determining the time difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects includes: Determine other time periods corresponding to other objects closest to the target time period corresponding to any target cluster; Determine the starting time interval and the ending time interval between the target time period and other time periods; The timing difference between the target time period and other time periods is determined using the start time interval and / or the end time interval.

[0009] Furthermore, the step of determining the possibility of the target object's continuous abnormal behavior in the target time period according to the time sequence difference includes: Determine the mean value of target facial information acquisition efficiency of the target object in the target time period; Determine the average efficiency of other facial information collection of other objects in other time periods closest to the target time period; The possibility of the target object's continued abnormal behavior in the target time period is calculated by using the time series difference, the mean efficiency of the target facial information collection and the mean efficiency of other facial information collection.

[0010] Further, the step of determining and marking the abnormal behavior of the target object according to the possibility of continued abnormal behavior includes: Determine the historical acquisition efficiency of the target object - time series; Determine the time period difference between the current collection efficiency-time series and the historical collection efficiency-time series; Identify and flag abnormal behavior of target objects based on time period differences and the likelihood of continued abnormal behavior.

[0011] Further, the time period difference includes: the total number of time periods difference; The steps to identify and mark the abnormal behavior of the target object based on the time period difference and the possibility of continuous abnormal behavior include: Using the total number of time period differences, the overall behavioral abnormality of the target object is calculated; Identify and flag a target's aberrant behavior based on overall behavioral aberration and likelihood of continued aberrant behavior.

[0012] Further, the step of determining and marking the abnormal behavior of the target object based on the overall abnormality of the behavior and the possibility of continued abnormal behavior includes: Calculate the abnormal behavior labeling weight obtained by normalizing the product of the overall behavior abnormality and the possibility of continuous abnormal behavior; Compare the abnormal behavior marking weight with the preset abnormal marking threshold to determine and mark the abnormal behavior of the target object.

[0013] The present invention also provides a video monitoring system for visual operation, the system is used to implement the video monitoring method for visual operation as described in any one of the above items; the system comprises: A video acquisition module, used to acquire target video information of a target object, and determine a current acquisition efficiency-time sequence of facial information of the target object using the target video information; An information clustering module is used to determine the similarity of the behavior states corresponding to each adjacent time point in the current acquisition efficiency-time series, and determine each target cluster of the current acquisition efficiency-time series according to the similarity of the behavior states; A behavior comparison module is used to determine the time difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects, and determine the possibility of continuous abnormal behavior of the target object in the target time period according to the time difference; The behavior marking module determines and marks the abnormal behavior of the target object based on the possibility of continued abnormal behavior.

[0014] The present invention has the following beneficial effects: The facial information acquisition efficiency of the characters at each time node is obtained through image processing technology, and based on the change of the facial information acquisition efficiency in the time series, the time period is segmented and clustered to obtain several time periods of the same state. This operation avoids the problem that the static or small-amplitude actions in the video images of a continuous period of time have small changes and are difficult to identify, thereby improving the accuracy of abnormal behavior detection; Furthermore, by analyzing the timing differences between the time period of the target object and the time periods of other objects, as well as the differences in the efficiency of facial information collection within the time period, the possibility of abnormal behavior of the target object in any time period can be judged; and the collection efficiency-time series of the student history can be further combined to determine the marking weight, and the rapid positioning and search of abnormal behavior of the target object can be achieved based on the marking weight. This operation solves the problem that small movements are difficult to identify in the prior art, and avoids the misjudgment of normal behavior and abnormal behavior, thereby improving the accuracy of abnormal behavior detection and the practicality of the visual operation monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of a video monitoring method for visual operation provided by an embodiment of the present invention; Figure 2 A detailed flow chart of step S3 in a video monitoring method for visual operation provided by one embodiment of the present invention; Figure 3 A detailed flow chart of step S3 in a video monitoring method for visual operation provided by another embodiment of the present invention; Figure 4 A detailed flow chart of step S4 in a video monitoring method for visual operation provided by one embodiment of the present invention; Figure 5 A schematic diagram of the structure of the hardware operating environment of a video surveillance device for visual operation according to an embodiment of the present invention; Figure 6 The schematic diagram is a framework structure of a video surveillance system for visual operation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the video monitoring method for visual operation proposed by the present invention, its specific implementation, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0018] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0019] A specific solution of a video monitoring method for visualized operation provided by the present invention is described in detail below with reference to the accompanying drawings.

[0020] Embodiment 1: For a video monitoring method for visual operation provided by the present invention, please refer to Figure 1 , which shows a step flow chart of a video monitoring method for visual operation provided by an embodiment of the present invention.

[0021] The method comprises: Step S1, obtaining target video information of a target object, and determining a current acquisition efficiency-time sequence of facial information of the target object using the target video information; The target object here can be a student, an employee, or even any object whose behavior can be identified as abnormal due to a certain range of behavior. Here, students are taken as exemplary research objects to develop the following embodiments.

[0022] Multiple cameras are installed in the space to be monitored to collect video images (video information) within a certain range of the space. This embodiment takes the classroom monitoring scene as an example; the collected video images are transmitted to the monitoring terminal based on the Internet of Things for processing and analysis; The scenario involves the behavior of students during class and between classes. Parents, as the main users of visual video surveillance, can query and locate the behavior of specified students in historical videos. In order to realize the query and positioning function, it is first necessary to locate and mark different students appearing in the video image, and then identify related actions. The specific process of positioning and marking is as follows: 1. Preprocess the video image based on existing image processing technology, including averaging, Gaussian denoising, etc.; 2. Analyze the collected current and historical video information based on semantic segmentation technology to obtain important components in the video. In this case, the components in each frame of the video image are marked separately or classified. For example, if a certain component exists in the video image at a certain time point in the historical video information, it is marked as component j, also called object j. In this embodiment, it refers to any j-th student in the space representing the target object; 3. Evaluate the facial feature acquisition efficiency (facial information acquisition efficiency) of each student. Specifically, the monitoring system pre-inputs the front ID photos of all students in the current space (class), and uses the existing image matching technology to compare the facial features of each target component with the template photo. In the process of comparison, the similarity can be obtained to identify and locate each student. The similarity and facial information acquisition efficiency will change in real time with the target's movements. Therefore, it is necessary to obtain the facial information acquisition efficiency of any target object in real time. Based on the surveillance camera, the target video information of the target object in the current classroom is obtained. Based on the above positioning and recognition process, the current acquisition efficiency-time series of the target object's facial information can be analyzed and read from the target video information. This series refers to the distribution of facial information acquisition (acquisition) efficiency at each time point, which can be reflected on the coordinate axis as each time point as the horizontal axis and the facial information acquisition efficiency as the vertical axis. Due to the change of the target object's action, the corresponding facial information acquisition efficiency reflected at each time point is often different. Here, the facial information acquisition efficiency at each time point can refer to the record of the facial information acquisition efficiency in the time interval from the adjacent previous time point to the current time point. For example, from time point 1 second to time point 2 seconds, 5 positioning and recognition of the target object's face are generated, then the corresponding facial information acquisition efficiency is 5 times / s. This is just an example for easy understanding.

[0023] It is also possible to obtain video information of D=10 historical classes, and then obtain multiple historical collection efficiency-time series, which is convenient for subsequent analysis and processing.

[0024] Step S2, determining the behavior state similarity corresponding to each adjacent time point in the current acquisition efficiency-time series, and determining each target cluster of the current acquisition efficiency-time series according to the behavior state similarity; Specifically, the step of determining the similarity of the behavior states corresponding to each adjacent time point in the current acquisition efficiency-time series includes: Determine the current acquisition efficiency - the efficiency difference of facial information acquisition efficiency between any two adjacent time points in the time series; Using the efficiency difference, the similarity of the behavior states between any two adjacent time points is calculated.

[0025] More specifically, the step of calculating the similarity of the behavior states between any two adjacent time points by using the efficiency difference includes: Determine the instantaneous change rate of facial information collection efficiency corresponding to any two adjacent time points; Determine the difference in instantaneous rate of change between any two adjacent time points; The similarity of the behavior states between any two adjacent time points is calculated using the efficiency difference and the instantaneous change rate difference.

[0026] In current classrooms, students' movements are relatively small, and the target users of visually operated video surveillance are usually parents, who mostly want to understand their students' listening status in history classes. Existing image detection technology can analyze and capture the movements of people or objects in video images, but for static or small movements, the changes produced in a continuous period of video images are small, so the similarity is high, resulting in low recognition accuracy; that is, it is impossible to mark possible abnormal behaviors, such as dozing off, daydreaming, etc. According to the analysis, the general behavior of students in the classroom includes raising their heads, lowering their heads, standing up, walking, etc., and the large movements of standing up and walking can be marked by existing technologies; the movement of raising and lowering their heads has the characteristics of small changes and long duration under normal circumstances. For example, when the teacher is lecturing, students continue to raise their heads and gaze, or continue to lower their heads to think and take notes; due to the particularity of classroom teaching, students usually look in the same direction, so the facial information acquisition efficiency of the camera installed in the direction of the student's gaze will change with time as the student's movements change, so the student's behavior can be marked according to similar efficiency time periods; For each subject, that is, each student, the facial information collection efficiency along the time series during the history class period is obtained; these data are arranged on the coordinate axis in time series, and the vertical axis of the coordinate is the facial information collection efficiency.

[0027] The specific method for measuring the similarity of behavioral states is based on the efficiency difference between the facial information collection efficiency at two time points. To distinguish, , Respectively represent two adjacent time points; It should be noted that since the process of students changing their postures is a continuous process, the efficiency of facial information acquisition increases or decreases slowly during the change of the posture. In order to avoid dividing the time points of the gradual change process into several small intervals during the time period marking process, the difference in acquisition efficiency between any moment and its adjacent previous moment is used to express the instantaneous change rate of the corresponding moment, which is recorded as , It should be noted that b-1 here is different from a, but is in , The method for obtaining the instantaneous quantity is the basic instantaneous quantity calculation method in physics. In addition, the smaller the instantaneous change rate is, the more stable the current moment is. Thus, the discriminant formula of behavior state similarity is obtained: In the formula, the instantaneous rate of change difference is obtained according to the above logic ( ), and the efficiency difference between it and the efficiency of facial information collection Using the Euclidean norm, we get , the smaller the value, the higher the similarity of the behavior state.

[0028] The step of determining each target cluster of the current acquisition efficiency-time series according to the behavior state similarity specifically includes: The Turbopixels algorithm and behavior state similarity are used to cluster the current acquisition efficiency-time series to obtain various target clusters; among which, the target clusters represent the behavior state of the target object.

[0029] According to the above-mentioned behavior state similarity discrimination method, the current collection efficiency-time series of the current student is clustered, that is, a certain behavior state similarity distinction rule can be set, such as setting a behavior state similarity threshold, and the behavior state similarity greater than or equal to the threshold is considered to be data with high behavior state similarity, which is divided into the same cluster, and the behavior state similarity less than the threshold is considered to be data with low behavior state similarity, which is divided into another cluster, thereby obtaining N target clusters. For each cluster, it represents any behavior state of the student; it should be noted that in order to make the final cluster a continuous time period, the facial information collection efficiency of each marked student can be clustered using the existing Turbopixels algorithm, which can ensure the connectivity of the final clustering result, so that the data points in each cluster are continuous in time series.

[0030] Step S3, determining the timing difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects, and determining the possibility of continuous abnormal behavior of the target object in the target time period according to the timing difference; For details, please refer to Figure 2 The step of determining the time difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects includes: Step S30, determining other time periods corresponding to other objects closest to the target time period corresponding to any target cluster; Step S31, determining the starting time interval and the ending time interval between the target time period and other time periods; Step S32: using the start time interval and / or the end time interval to determine the timing difference between the target time period and other time periods.

[0031] Because some abnormal behaviors are easily confused with each other, such as napping and daydreaming, which are easily confused with the state of looking up and the state of looking down; and according to the analysis, students' behaviors should be consistent in the same time period. For example, when the teacher is lecturing, students generally look up at the same time. There may be some students who look down during the process, but the time is short; according to the analysis, the abnormal behavior of the current student (target object) is manifested in that the duration of a certain behavior state is significantly different from the behavior of other students in the corresponding time period; Therefore, according to any time period k in the N clusters obtained in the above embodiment, the starting and ending time points (i.e., the starting time and the ending time) of the time period k are obtained; Select the other time periods of other students whose starting time or ending time is closest to the current student j's kth time period; obtain the starting time interval between the other time periods of other students r and the time period k of the current jth target student, recorded as ; and obtain the end time interval between the end time point of other time periods and the end time point of the current time period k, recorded as ; When the above two time intervals are larger, it means that the possibility that the current student j is in an abnormal behavior state is higher; This yields the timing difference for any time period k, also known as behavioral inertia: In the formula, according to the above logic, the time intervals corresponding to the starting and ending time points can be added and averaged to obtain the behavior slowness , the starting time interval and the end time interval can also be used as the time series difference and behavioral inertia respectively instead of taking the mean. The larger the value, the higher the possibility that the current student j is in an abnormal behavior state; Traversing all students, we can obtain M-1 behaviors that are related to the current student j. , M represents the total number of students in the current space.

[0032] For details, please refer to Figure 3 The step of determining the possibility of the target object's continued abnormal behavior in the target time period according to the time series difference includes: Step S300, determining the mean value of target facial information acquisition efficiency of the target object in the target time period; Step S310, determining the average values ​​of other facial information collection efficiencies of other objects in other time periods closest to the target time period; Step S320, using the time series difference, the mean value of the target facial information collection efficiency and the mean values ​​of other facial information collection efficiencies, calculate the possibility of the target object's continuous abnormal behavior in the target time period.

[0033] Based on the above embodiment, if there is an obvious difference between the facial collection efficiency of the current student j and other students in a similar time period, it means that the current student j is more likely to have continuous abnormal behavior in the current time period k; Thus, the average efficiency of other facial information collection of other students in the corresponding time period is obtained. The average efficiency of target facial information collection of student j in time period k ; Combined with the behavioral inertness (time sequence difference) in the above embodiment, it can be obtained that the current student j has the possibility of continuous abnormal behavior in time period k: In the formula, according to the above logic, the efficiency mean difference between the kth time period of the current student j and the time period r similar to that of another student is obtained and expressed as an absolute value, thus obtaining , the larger the value, the greater the possibility that the current student has abnormal behavior; at the same time, the behavioral inertia calculated above is used to weight it, and all students are traversed, thus obtaining The larger the value is, the greater the possibility that the current student j will have continuous abnormal behavior in time period k.

[0034] Step S4, determining and marking the abnormal behavior of the target object according to the possibility of continuous abnormal behavior.

[0035] For details, please refer to Figure 4 , the step S4 comprises: Step S40, determining the historical acquisition efficiency-time series of the target object; Step S41, determining the time period difference between the current acquisition efficiency-time series and the historical acquisition efficiency-time series; Step S42, determining and marking the abnormal behavior of the target object according to the time period difference and the possibility of continuous abnormal behavior.

[0036] Further, the time period difference includes: the total number of time periods difference; The steps to identify and mark the abnormal behavior of the target object based on the time period difference and the possibility of continuous abnormal behavior include: Using the total number of time period differences, the overall behavioral abnormality of the target object is calculated; Identify and flag a target's aberrant behavior based on overall behavioral aberration and likelihood of continued aberrant behavior.

[0037] More specifically, the step of determining and marking the abnormal behavior of the target object based on the overall abnormality of the behavior and the possibility of continued abnormal behavior includes: Calculate the abnormal behavior labeling weight obtained by normalizing the product of the overall behavior abnormality and the possibility of continuous abnormal behavior; Compare the abnormal behavior marking weight with the preset abnormal marking threshold to determine and mark the abnormal behavior of the target object.

[0038] Based on the operation of the above embodiment, the possibility of continuous abnormal behavior in any time period k is obtained, and this possibility reflects the difference between the behavior performance of any student in a local time period and other students; however, according to analysis, for different students in abnormal states, the frequency of switching behavior states and the frequency of historical behavior states are significantly different; in order to avoid misjudgment of the above calculated possibility of continuous abnormal behavior and abnormal behavior, the possibility of continuous abnormal behavior should be corrected in combination with the historical behavior state of the student; Therefore, based on the above-mentioned embodiment, video information of historical D=10 classes can also be obtained, and then multiple historical collection efficiency-time series can be obtained to facilitate subsequent analysis and processing.

[0039] Get the total number of time periods of the current student j in the current target class , that is, the total number of time periods obtained by clustering the current acquisition efficiency-time series; Thus, the overall abnormal behavior of the jth student in the current class is obtained: In the formula, That is, the total number of time periods corresponding to the historical collection efficiency-time series in other classes of the current student's history. represents the normalized range of the total number of current and historical time periods; D represents the number of history classes, that is, the historical collection efficiency - the number of time series.

[0040] According to the above logic, the difference between the total number of time periods in the current class and the historical classes is obtained, that is, the time period difference between the current collection efficiency-time series and each historical collection efficiency-time series, thus obtaining , and normalize its range, thus obtaining the overall behavior abnormality ,The larger the value is, the more abnormal the current student's behavior is in the current class; Combined with the possibility of persistent abnormal behavior , thus the abnormal behavior marking weight of the current k-th time period is obtained: In the formula, the above two characteristics are and Combined and normalized by the norm function, the abnormal behavior labeling weight is obtained The larger the value is, the more likely the student is to have abnormal behavior in the current time period, and the higher the weight of being marked; A preset abnormal marking threshold can be set as needed, for example, 0.7, and all time periods greater than the threshold are marked as abnormal; According to the visual operation, the starting time points of these marked time periods are assigned pointers or other prominent interactive buttons. When the user uses the visual operation video surveillance system to monitor the history classroom behavior, he can click these interactive buttons and view the abnormal behavior of the monitored object.

[0041] The present invention obtains the facial information collection efficiency of the human object at each time node through image processing technology, and based on the change of the facial information collection efficiency in the time series, divides and clusters the time period to obtain several time periods of the same state. This operation avoids the problem that the static or small-amplitude action has small changes in the video image of a continuous period of time and is difficult to identify, thereby improving the accuracy of abnormal behavior detection; Furthermore, by analyzing the timing differences between the time period of the target object and the time periods of other objects, as well as the differences in the efficiency of facial information collection within the time period, the possibility of abnormal behavior of the target object in any time period can be judged; and the collection efficiency-time series of the student history can be further combined to determine the marking weight, and the rapid positioning and search of abnormal behavior of the target object can be achieved based on the marking weight. This operation solves the problem that small movements are difficult to identify in the prior art, and avoids the misjudgment of normal behavior and abnormal behavior, thereby improving the accuracy of abnormal behavior detection and the practicality of the visual operation monitoring system.

[0042] Embodiment 2: The embodiment of the present invention also provides a video monitoring device for visualization operation. The video monitoring device for visualization operation can be a monitoring camera, a computer, a server or other data computing and processing device or a combination of multiple devices.

[0043] like Figure 5 As shown, Figure 5 It is a structural diagram of the hardware operating environment of the video monitoring device for visual operation involved in the embodiment of the present invention.

[0044] like Figure 5As shown, the video monitoring device for visualization operation may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display (Display), an input unit such as a control panel, and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WIFI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001. The memory 1005 as a computer storage medium may include a video monitoring program for visualization operation.

[0045] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0046] Continue to refer to Figure 5 , Figure 5 The memory 1005 as a computer-readable storage medium may include an operating system, a user interface module, a network communication module, and a video monitoring program for visualization operations.

[0047] exist Figure 5 In the embodiment, the network communication module is mainly used to connect to the server and can communicate data with the server; and the processor 1001 can call the video monitoring program for visualization operation stored in the memory 1005 and execute the steps in the above embodiments.

[0048] Based on the hardware structure of the video monitoring device for visualization operation, various embodiments of the video monitoring method for visualization operation of the present invention are implemented.

[0049] In addition, the present invention also provides a video monitoring system for visual operation, please refer to Figure 6 , the video monitoring system for visual operation comprises: The video acquisition module A10 is used to acquire target video information of the target object and determine the current acquisition efficiency-time sequence of the facial information of the target object using the target video information; The information clustering module A20 is used to determine the behavior state similarity corresponding to each adjacent time point in the current acquisition efficiency-time series, and determine each target cluster of the current acquisition efficiency-time series according to the behavior state similarity; The behavior comparison module A30 is used to determine the time difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects, and determine the possibility of the target object's continuous abnormal behavior in the target time period according to the time difference; The behavior marking module A40 determines and marks the abnormal behavior of the target object according to the possibility of continuous abnormal behavior.

[0050] Furthermore, the information clustering module A20 is also used for: Determine the current acquisition efficiency - the efficiency difference of facial information acquisition efficiency between any two adjacent time points in the time series; Using the efficiency difference, the similarity of the behavior states between any two adjacent time points is calculated.

[0051] Furthermore, the information clustering module A20 is also used for: Determine the instantaneous change rate of facial information collection efficiency corresponding to any two adjacent time points; Determine the difference in instantaneous rate of change between any two adjacent time points; The similarity of the behavior states between any two adjacent time points is calculated using the efficiency difference and the instantaneous change rate difference.

[0052] Furthermore, the information clustering module A20 is also used for: The Turbopixels algorithm and behavior state similarity are used to cluster the current acquisition efficiency-time series to obtain various target clusters; among which, the target clusters represent the behavior state of the target object.

[0053] Furthermore, the behavior comparison module A30 is also used for: Determine other time periods corresponding to other objects closest to the target time period corresponding to any target cluster; Determine the starting time interval and the ending time interval between the target time period and other time periods; The timing difference between the target time period and other time periods is determined using the start time interval and / or the end time interval.

[0054] Furthermore, the behavior comparison module A30 is also used for: Determine the mean value of target facial information acquisition efficiency of the target object in the target time period; Determine the average efficiency of other facial information collection of other objects in other time periods closest to the target time period; The possibility of the target object's continued abnormal behavior in the target time period is calculated by using the time series difference, the mean efficiency of the target facial information collection and the mean efficiency of other facial information collection.

[0055] Furthermore, the behavior marking module A40 is also used for: Determine the historical acquisition efficiency of the target object - time series; Determine the time period difference between the current collection efficiency-time series and the historical collection efficiency-time series; Identify and flag abnormal behavior of target objects based on time period differences and the likelihood of continued abnormal behavior.

[0056] Furthermore, the behavior marking module A40 is also used for: The steps to identify and mark the abnormal behavior of the target object based on the time period difference and the possibility of continuous abnormal behavior include: Using the total number of time period differences, the overall behavioral abnormality of the target object is calculated; Identify and flag a target's aberrant behavior based on overall behavioral aberration and likelihood of continued aberrant behavior.

[0057] Furthermore, the behavior marking module A40 is also used for: Calculate the abnormal behavior labeling weight obtained by normalizing the product of the overall behavior abnormality and the possibility of continuous abnormal behavior; Compare the abnormal behavior marking weight with the preset abnormal marking threshold to determine and mark the abnormal behavior of the target object.

[0058] The specific implementation of the video monitoring system for visualized operation of the present invention is basically the same as the above-mentioned embodiments of the video monitoring method for visualized operation, and will not be described in detail here.

[0059] In addition, the present invention also provides a computer-readable storage medium. The computer-readable storage medium of the present invention stores a video monitoring program for visualization operation, wherein when the video monitoring program for visualization operation is executed by a processor, the steps of the video monitoring method for visualization operation as described above are implemented.

[0060] The method implemented when the video monitoring program for visualization operation is executed can refer to the various embodiments of the video monitoring method for visualization operation of the present invention, which will not be described in detail here.

[0061] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0062] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0063] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] The above description is only a preferred embodiment of the present invention, and does not limit the protection scope of the present invention. All equivalent structural / method transformations made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the protection scope of the present invention.

Claims

1. A video monitoring method for visual operation, characterized in that: The method comprises: Acquire target video information of the target object, and determine the current acquisition efficiency-time sequence of facial information of the target object using the target video information; Determine the behavior state similarity between each adjacent time point in the current acquisition efficiency-time series, and determine each target cluster of the current acquisition efficiency-time series according to the behavior state similarity; Determine the time difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects, and determine the possibility of continuous abnormal behavior of the target object in the target time period based on the time difference; Identify and flag anomalous behavior of target subjects based on the likelihood of continued anomalous behavior.

2. The video monitoring method for visual operation according to claim 1, characterized in that: The step of determining the similarity of the behavior states corresponding to each adjacent time point in the current acquisition efficiency-time series includes: Determine the current acquisition efficiency - the efficiency difference of facial information acquisition efficiency between any two adjacent time points in the time series; Using the efficiency difference, the similarity of the behavior states between any two adjacent time points is calculated.

3. The video monitoring method for visual operation according to claim 2, characterized in that: The steps of calculating the similarity of the behavior states between any two adjacent time points by using the efficiency difference include: Determine the instantaneous change rate of facial information collection efficiency corresponding to any two adjacent time points; Determine the difference in instantaneous rate of change between any two adjacent time points; The similarity of the behavior states between any two adjacent time points is calculated using the efficiency difference and the instantaneous change rate difference.

4. The video monitoring method for visual operation according to claim 1, characterized in that: The steps of determining each target cluster of the current acquisition efficiency-time series according to the similarity of the behavior state include: The Turbopixels algorithm and behavior state similarity are used to cluster the current acquisition efficiency-time series to obtain various target clusters; among which, the target clusters represent the behavior state of the target object.

5. The video monitoring method for visual operation according to claim 1, characterized in that: The step of determining the time series difference between a target time period corresponding to any target cluster and other time periods corresponding to other objects includes: Determine other time periods corresponding to other objects closest to the target time period corresponding to any target cluster; Determine the starting time interval and the ending time interval between the target time period and other time periods; The timing difference between the target time period and other time periods is determined using the start time interval and / or the end time interval.

6. The video monitoring method for visual operation according to claim 1, characterized in that: The steps of determining the possibility of the target object's continued abnormal behavior in the target time period according to the time series difference include: Determine the mean value of target facial information acquisition efficiency of the target object in the target time period; Determine the average efficiency of other facial information collection of other objects in other time periods closest to the target time period; The possibility of the target object's continued abnormal behavior in the target time period is calculated by using the time series difference, the mean efficiency of the target facial information collection and the mean efficiency of other facial information collection.

7. The video monitoring method for visual operation according to claim 1, characterized in that: The steps to identify and flag the target object's abnormal behavior based on the likelihood of continued abnormal behavior include: Determine the historical acquisition efficiency of the target object - time series; Determine the time period difference between the current collection efficiency-time series and the historical collection efficiency-time series; Identify and flag abnormal behavior of target objects based on time period differences and the likelihood of continued abnormal behavior.

8. The video monitoring method for visual operation according to claim 7, characterized in that: The time period differences include: the total number of time periods; The steps to identify and mark the abnormal behavior of the target object based on the time period difference and the possibility of continuous abnormal behavior include: Using the total number of time period differences, the overall behavioral abnormality of the target object is calculated; Identify and flag a target's aberrant behavior based on overall behavioral aberration and likelihood of continued aberrant behavior.

9. The video monitoring method for visual operation according to claim 8, characterized in that: Steps to identify and flag the target subject's anomalous behavior based on overall behavioral anomalousness and likelihood of continued anomalous behavior include: Calculate the abnormal behavior labeling weight obtained by normalizing the product of the overall behavior abnormality and the possibility of continuous abnormal behavior; Compare the abnormal behavior marking weight with the preset abnormal marking threshold to determine and mark the abnormal behavior of the target object.

10. A video surveillance system for visual operation, characterized in that: The system is used to implement the video monitoring method for visual operation according to any one of claims 1 to 9; the system comprises: A video acquisition module, used to acquire target video information of a target object, and determine a current acquisition efficiency-time sequence of facial information of the target object using the target video information; An information clustering module is used to determine the similarity of the behavior states corresponding to each adjacent time point in the current acquisition efficiency-time series, and determine each target cluster of the current acquisition efficiency-time series according to the similarity of the behavior states; A behavior comparison module is used to determine the time difference between the target time period corresponding to any target cluster and other time periods corresponding to other objects, and determine the possibility of continuous abnormal behavior of the target object in the target time period based on the time difference; The behavior marking module determines and marks the abnormal behavior of the target object based on the possibility of continued abnormal behavior.

Images