System and method for detecting family violent behavior in monitoring video

Abstract

The invention relates to the field of household violent behavior detection, in particular to a household violent behavior detection system and method in a monitoring video, and the system comprises a control module which is electrically connected with a video extraction module. Signal transmission between the video extraction module and the progressive audio cutting module is achieved through electric connection, data exchange between the progressive audio cutting module and the audio analysis module is achieved through electric connection, and data communication between the audio analysis module and the multi-mode analysis module is achieved through electric connection. According to the method, the violent behaviors with consistent sound and picture are accurately identified through the pairing analysis of the video frames and the audio clips which are aligned in time, the error rate is reduced by about 60%, all violent behaviors such as limb conflicts and speech threats can be identified by means of the analysis capability of the Qwen-vl-max multi-mode large model, the integrity is improved by 45%, and the accuracy of the violent behaviors is improved. And all abnormal sounds can be captured without missing by frame-by-frame progressive analysis.

Description

Technical Field

[0001] The present invention relates to the field of domestic violence behavior detection, and in particular to a system and method for detecting domestic violence behavior in surveillance videos. Background Art

[0002] For example, in family scenarios, domestic violence usually occurs very suddenly and is difficult to monitor in ordinary families. Due to the lack of professional supervision, its real-time performance is poor and the labor cost is high. The existing monitoring systems mainly focus on common abnormal behavior detection technologies in the security field, and there is a serious lack of professional detection technologies for domestic violence.

[0003] On the one hand, it is difficult for a single video or audio to accurately reflect the characteristics of violent behavior. Visual signals and audio signals of screaming and crying need to be combined for analysis to make a more accurate judgment.

[0004] At the same time, existing algorithms do not have an effective semantic understanding of the concept of violence and cannot accurately distinguish Domestic violence can be divided into normal disputes and violent behaviors, and there is currently no quantitative classification of the degree of violence, making it difficult to make corresponding interventions. In addition, the traditional detection method is very inefficient for processing long-term videos and cannot meet real-time requirements. Therefore, we propose a system and method for detecting domestic violence in surveillance videos to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide a system and method for detecting domestic violence in surveillance videos to solve the problems raised in the above background technology.

[0006] The technical solution of the present invention is: a system and method for detecting domestic violence in surveillance videos, including a control module, wherein the control module is connected to a video extraction module via an electrical connection, the video extraction module is further connected to a progressive audio cutting module via an electrical connection to achieve signal transmission, the progressive audio cutting module is connected to an audio analysis module via an electrical connection to achieve data exchange, the audio analysis module is connected to a multimodal analysis module via an electrical connection to achieve data communication with the multimodal analysis module via an electrical connection, the multimodal analysis module is connected to a violence level determination module via an electrical connection to achieve data instructions with a response measure module via an electrical connection.

[0007] Preferably, the method comprises the following steps: Step S1: extracting video frame images from the input video by seconds, and extracting video frame images by preset time intervals; Step S2: cutting the audio in the video into progressive audio segments according to time segments; Step S3: analyze whether there are violent tendency sounds such as screams, cries, etc. in the audio segment using an audio large model; Step S4: call a multi-modal large model to judge whether there is violent behavior in the picture by combining the video frame image and the audio analysis result; Step S5: output a violence tendency level according to the analysis result through behavior and language; Step S6: expand according to corresponding reaction measures of different violence degrees.

[0008] Preferably, the video frame extraction module uses the OpenCV library to realize the extraction of one frame of image per second, and the progressive audio cutting module uses the MoviePy library to realize the generation of progressive audio segments at 1 second intervals.

[0009] Preferably, the audio analysis module uses the Qwen-audio-turbo model to analyze the audio content; and the multi-modal analysis module uses the Qwen-vl-max model to perform multi-modal analysis of images and texts.

[0010] Preferably, the progressive audio cutting step comprises: Step S1: calculate the total duration of the audio; Step S2: generate a plurality of audio segments at 1 second intervals; Step S3: each segment contains complete audio content from the start to the current time point.

[0011] Preferably, the violent behavior judgment step comprises: Step S1: extract the text description obtained by audio analysis; Step S2: input the text description and the video frame image at the corresponding time point into the multi-modal large model; Step S3: output the evaluation result according to the preset violence level standard.

[0012] Preferably, the violence level regular expression matching step comprises: Step S1: extract the violence level marked as level X from the multi-modal analysis result; Step S2: use regular expression matching to obtain the final violence level value.

[0013] Preferably, the multi-modal analysis module variable configuration step comprises: Step S1: load configuration information such as API key; Step S2: set the environment variables required for calling the large model.

[0014] The present application improves the monitoring of video in domestic violence behavior detection system and method, compared with the prior art, has the following improvements and advantages: One: the application, through the time alignment of video frame and audio segment pairing analysis, accurately identifies the sound and picture consistent violence, the error rate is reduced by about 60%, and all violent behaviors such as body conflict and verbal threat can be identified with the help of Qwen-vl-max multi-modal model analysis ability, the completeness is improved by 45%, and all abnormal sounds can be captured without missing through frame-by-frame progressive analysis.

[0015] Second: the application, through the violence level determination mechanism and automatic disposal scheme, intelligently disposes the domestic violence event, determines the disposal mode according to the five different types of violence from light to heavy according to the five-level violence determination system, the response accuracy reaches 75%, the automatic triggering disposal scheme can execute from the multi-step whole process such as early warning and automatic alarm, the original manual monitoring time is reduced from 30 minutes to about 10 seconds; the regular expression is used to automatically capture and determine the violence level, the error caused by the subjective difference of human is avoided, the consistency of the discrimination result reaches 99%, important evidence materials are automatically reserved and uploaded to the storage system, the needs of subsequent remediation and accountability are met, and the evidence efficiency is improved by 80%. BRIEF DESCRIPTION OF DRAWINGS

[0016] The application will be further explained below in combination with the drawings and examples: Figure 1 It is a domestic violence behavior detection method flow chart of the application; Figure 2 It is a progressive audio cutting principle schematic diagram of the application; DETAILED DESCRIPTION

[0017] The application will be described in detail below, and the technical solutions in the embodiments of the application will be clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0018] The application provides a domestic violence behavior detection system and method in a monitoring video by improvement, and the technical solutions of the application are: As Figure 1 - Figure 2As shown, a domestic violence behavior detection system and method in a monitoring video includes a control module, the control module is connected with a video extraction module through an electrical connection, the video extraction module is connected with a progressive audio cutting module through an electrical connection, the progressive audio cutting module is connected with an audio analysis module through an electrical connection, the audio analysis module is connected with a multi-modal analysis module through an electrical connection, the multi-modal analysis module is connected with a violence level determination module through an electrical connection, and the violence level determination module is connected with a response measure module through an electrical connection.

[0019] Further, the following steps are included: Step S1: Extract video frame images from the input video by seconds, which can be extracted according to a preset time interval; Step S2: Cut the audio in the video into progressive audio segments according to time; Step S3: Use an audio large model to analyze whether there are violent tendency sounds such as screams and cries in the audio segment; Step S4: Combine the video frame image and the audio analysis result to call a multi-modal large model to determine whether there is violent behavior in the picture; Step S5: Output the violence tendency level according to the analysis result of behavior and language; Step S6: According to the corresponding reaction measures of different violence degrees, the reaction measures are carried out.

[0020] According to the second-level frame extraction and the progressive cutting of the audio, the two are aligned in time, and The subsequent multi-modal analysis also has a time sequence basis. Based on the grading interpretation of the audio large model and the multi-modal large model, the two-layer verification mechanism of using the audio to preliminarily determine and then passing it to the multi-modal model for further determination is used to improve the acquisition speed of the violence features, and the multi-source data fusion is used to reduce the misjudgment rate. With the linkage of violence level evaluation and automatic response measures, the full-link automation from discovery to implementation of intervention is realized, which is suitable for the characteristics of timely intervention of domestic violence scenes.

[0021] Further, the video frame extraction module uses the OpenCV library to extract one frame of image per second, the progressive audio cutting module uses the MoviePy library to generate progressive audio segments at 1 second intervals, and the OpenCV and MoviePy libraries are used to realize data preprocessing. Because mature open source tools are used and these open source tools are efficient and compatible with many scenarios, there is a small error in using custom algorithms to obtain one-second-level data, and using the above open source tools can ensure that all collected data can achieve stable one-second-level accuracy. Compared with fixed segment cutting, progressive cutting can preserve the time continuity of the sound signal, which is more suitable for capturing dynamic audio features such as "screaming gradually strong" and more in line with the characteristics of the sudden outbreak of violent behavior.

[0022] Further, the audio analysis module uses the Qwen-audio-turbo model to analyze the audio content, and the multi-modal analysis module uses the Qwen-vl-max model for multi-modal analysis of images and text. The Qwen-audio-turbo model has professional recognition ability for high-frequency violent sound, and combined with its fast reasoning characteristics, it can realize real-time semantic analysis of audio signals. The multi-modal fusion capability of the Qwen-vl-max model can jointly understand the semantic understanding of the body movements in the video frames and the audio text description.

[0023] Further, the progressive audio cutting step includes: Step S1: Calculate the total duration of the audio; Step S2: Generate a plurality of audio segments at 1 second intervals; Step S3: Each segment contains complete audio content from the start to the current time point; The dynamically accumulated audio segment generation mechanism can capture the sound mutation characteristics when the violent behavior occurs in real time, avoiding the key audio segment breakage caused by fixed window cutting.

[0024] Further, the violent behavior judgment step includes: Step S1: Extract the text description obtained by audio analysis; Step S2: Input the text description and the video frame image at the corresponding time point into the multi-modal large model; Step S3: Output the evaluation result according to the preset violence level standard; By simultaneously inputting audio text and video frames, "auditory cues plus visual evidence" cross-validation is realized. If the multi-modal model detects that the audio analysis result is "screaming", it will correspondingly check whether there is a picture of body conflict in the video frame to prevent misjudgment caused by environmental noise. Different levels of violent behavior are distinguished by presetting the violence level, and the determination result is used to perform corresponding operations on different levels of violent behavior.

[0025] Further, the violence level regular expression matching step: Step S1: Extract the violence level marked as levelX from the multi-modal analysis result; Step S2: Use regular expression matching to obtain the final violence level value; By standardizing the multi-modal model output result through regular expression, the regular expression can be used to efficiently extract the quantitative violence level value from a large amount of unstructured text, speed up data processing, and realize the format difference of different model outputs for different large models. On the other hand, it also makes a good structured preparation for the subsequent violence level historical data statistics.

[0026] Further, the multi-modal analysis module variable configuration step: Step S1: Load API key and other configuration information; Step S2: Set the environment variables required for large model calling; The modular way can realize the decoupling of large model service, and facilitate the quick replacement of different API interfaces or keys according to needs, and can also improve the security of the system. Moreover, the more standardized environment variables can reduce the technical difficulty of large model calling, so that the whole system can be published according to different cloud service publishing platforms, which is conducive to better model upgrading and expansion in the later period.

[0027] The above description enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A system and method for monitoring domestic violence detection in a video, comprising a control module, characterized in that: The control module is connected with the video extraction module through electrical connection, the video extraction module is connected with the progressive audio cutting module through electrical connection, the progressive audio cutting module is connected with the audio analysis module through electrical connection, the audio analysis module is connected with the multi-modal analysis module through electrical connection, the multi-modal analysis module is connected with the violence level determination module through electrical connection, and the violence level determination module is connected with the response measure module through electrical connection.

2. The system and method for detecting domestic violence in a video according to claim 1, wherein: The method comprises the following steps: Step S1: extracting video frame images per second from the input video, which can be extracted at a preset time interval; Step S2: cutting the audio in the video into progressive audio segments by time segmentation; Step S3: using an audio large model to analyze whether there are violent tendency sounds such as screams and cries in the audio segment; Step S4: calling a multi-modal large model to determine whether there is violent behavior in the picture by combining the video frame image and the audio analysis result; Step S5: outputting the violence tendency level according to the analysis result through behavior and language; Step S6: developing corresponding reaction measures according to different violence levels.

3. The system and method for detecting domestic violence in a video according to claim 1, wherein: The video frame extraction module uses the OpenCV library to extract one frame of image per second, and the progressive audio cutting module uses the MoviePy library to generate progressive audio segments at an interval of 1 second.

4. The system and method for detecting domestic violence in a video according to claim 1, wherein: The audio analysis module uses the Qwen-audio-turbo model to analyze audio information, and the multi-modal analysis module uses the Qwen-vl-max model for multi-modal analysis of images and texts.

5. The system and method for detecting domestic violence in a video according to claim 2, wherein: The progressive audio cutting step comprises: Step S1: calculating the total duration of the audio; Step S2: generating multiple audio segments at an interval of 1 second; Step S3: each segment contains complete audio content from the start to the current time point.

6. The system and method for detecting domestic violence in a video according to claim 2, wherein: The violence behavior judgment step comprises: Step S1: extracting the text description obtained by audio analysis; Step S2: inputting the text description and the video frame image at the corresponding time point into the multi-modal large model; Step S3: outputting the evaluation result according to the preset violence level standard.

7. The system and method for detecting domestic violence in a video according to claim 2, wherein: The violence level regular expression matching step comprises: Step S1: extracting the violence level marked as level X from the multi-modal analysis result; Step S2: using regular expression matching to obtain the final violence level value.

8. The system and method for detecting domestic violence in a video according to claim 2, wherein: The multi-modal analysis module variable configuration step comprises: Step S1: loading configuration information such as API key; Step S2: setting the environment variables required for calling the large model.

Images