Campus bullying early warning method and system based on linkage of video and audio

Abstract

The invention provides a campus bullying early warning method and system based on linkage of videos and audios. The campus bullying early warning method based on video linkage audio comprises the steps that a broadcast control platform controls an audio acquisition terminal and a video acquisition terminal installed in a campus to acquire audio data and video data in the campus in real time; sending the audio data and the video data to a broadcast control platform through a wireless communication network, and performing data processing on the audio data and the video data by the broadcast control platform to obtain processed audio data and video data; information identification is carried out on video data through the video sensing platform, and information identification is carried out on audio data through the audio sensing platform to judge whether bully early warning needs to be carried out or not. The system comprises modules corresponding to the steps of the method.

Description

Technical Field

[0001] The invention proposes a campus bullying early warning method and system based on video-linked audio, belonging to the technical field of campus monitoring. Background Art

[0002] As campus safety issues become increasingly prominent, how to effectively prevent and intervene in a timely manner has become the focus of attention from all walks of life. Traditional campus safety monitoring mainly relies on manual inspections and video surveillance, but this method has problems such as blind spots, delayed response, and high labor costs, and it is difficult to meet the needs of rapid and accurate early warning of campus bullying. In recent years, with the rapid development of information technology, especially the continuous advancement of video processing and audio recognition technology, new solutions have been provided for campus bullying early warning. By integrating audio acquisition and video acquisition technology, combined with advanced video analysis and audio recognition algorithms, it is possible to achieve all-round and real-time monitoring of the internal environment of the campus, so as to timely detect and warn of potential bullying behavior.

[0003] However, most campus security monitoring systems on the market currently focus only on video monitoring, ignoring the important role of audio data in bullying warning. Audio data can capture key information such as calls for help and begging for mercy that are difficult to detect in video images, providing an important basis for judging bullying behavior. Therefore, combining video perception with audio perception to form a campus bullying warning method that combines video and audio has become an important way to improve the effectiveness of campus security monitoring.

[0004] In this context, the present invention proposes a campus bullying warning method based on video-linked audio. The method uses a broadcast control platform to control the audio acquisition terminal and video acquisition terminal installed inside the campus to collect audio data and video data in real time, and uses a wireless communication network to transmit the data to the broadcast control platform for data processing. Subsequently, the video data is identified through the video perception platform, and the audio data is identified through the linked audio perception platform to comprehensively judge whether a bullying warning is needed. This method not only improves the accuracy and timeliness of bullying warnings, but also reduces the labor intensity of manual monitoring, providing a more comprehensive and efficient guarantee for campus safety. Summary of the invention

[0005] The present invention provides a campus bullying early warning method and system based on video-linked audio, which is used to solve the above technical problems existing in the prior art. The technical solutions adopted are as follows:

[0006] A campus bullying early warning method based on video-linked audio, the campus bullying early warning method based on video-linked audio comprising:

[0007] The broadcast control platform controls the audio and video acquisition terminals installed inside the campus to collect audio and video data inside the campus in real time;

[0008] The audio data and the video data are sent to the broadcast control platform through the wireless communication network, and the broadcast control platform processes the audio data and the video data to obtain the processed audio data and the video data;

[0009] The video perception platform is used to identify information on the video data and the audio perception platform is used to identify information on the audio data to determine whether a bullying warning is needed.

[0010] Furthermore, the broadcast control platform controls the audio acquisition terminal and video acquisition terminal installed inside the campus to collect audio data and video data inside the campus in real time, including:

[0011] The broadcast control platform controls the audio acquisition terminal and the video acquisition terminal installed inside the campus to send a wireless control signal through wireless communication to control the audio acquisition terminal and the video acquisition terminal to collect audio data and video data inside the campus in real time; wherein the audio acquisition terminal includes but is not limited to a microphone and other sound receiving devices; the video acquisition terminal includes but is not limited to a high-definition camera, etc.;

[0012] The broadcast control platform monitors the current time period of the campus environment in real time;

[0013] The audio collection frequency and the video collection frequency of the audio collection terminal and the video collection terminal are adjusted according to the current time period of the campus environment.

[0014] Among them, when the campus environment is in the non-school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are reduced; when the campus environment is in the school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are increased.

[0015] Further, the audio data and the video data are sent to the broadcast control platform through the wireless communication network, and the broadcast control platform processes the audio data and the video data to obtain the processed audio data and the video data, including:

[0016] The audio acquisition terminal and the video acquisition terminal send the audio data and the video data to the broadcast control platform via the wireless communication network;

[0017] The broadcast control platform performs data processing on the audio data and the video data, wherein the data processing includes data noise reduction processing, data cleaning processing and data compression processing;

[0018] The data processing duration corresponding to the audio data and the video data is monitored in real time, and it is determined whether there is any abnormality in the data processing process of the audio data and the video data by the broadcast control platform according to the data processing duration corresponding to the audio data and the video data.

[0019] Further, judging whether there is an abnormality in the data processing process of the audio data and the video data by the broadcast control platform according to the data processing time corresponding to the audio data and the video data includes:

[0020] Real-time monitoring of the data processing duration corresponding to the audio data and the video data;

[0021] Extracting the spectral density and peak frequency of the audio data after data processing;

[0022] The first processing evaluation parameter is obtained according to the data processing time corresponding to the audio data combined with the spectrum density and peak frequency of the audio data that has completed data processing; wherein the first processing evaluation parameter is obtained by the following formula:

[0023]

[0024] Among them, P 01 represents the first processing evaluation parameter; F peak represents the peak frequency; S(f) represents the spectral density function corresponding to the audio data; f 01 and f 02 represents the frequency upper limit value and the frequency lower limit value corresponding to the frequency distribution range corresponding to the audio data; T represents the first time parameter, and the first time parameter is obtained by the following formula:

[0025]

[0026] Where, T represents the first time parameter; T peak Indicates the data processing time corresponding to the peak frequency; T p represents the average data processing time corresponding to n audio data processing times; n represents the number of audio data processing times; T i T represents the data processing time corresponding to the i-th audio data processing; z Indicates the median value of the data processing duration corresponding to n audio data processing times;

[0027] Extract the video resolution and frame rate of the video data that has completed data processing;

[0028] The second processing evaluation parameter is obtained according to the data processing time corresponding to the video data combined with the video resolution and frame rate of the video data that has completed data processing; wherein the second processing evaluation parameter is obtained by the following formula:

[0029]

[0030] Among them, P 02 represents the second processing evaluation parameter; m represents the number of video data processing times; R i represents the video resolution corresponding to the i-th video data processing; R i-1 Indicates the video resolution corresponding to the i-1th video processing; F framei represents the maximum frame rate variation of the video data corresponding to the i-th video data processing; F max Indicates the maximum allowed frame rate variation of the preset video data; T fi represents the actual processing time corresponding to the i-th video data processing; T si T represents the theoretical processing time corresponding to the i-th video data processing; x represents the second time parameter, and the second time parameter is obtained by the following formula:

[0031]

[0032] Among them, T x Represents the second time parameter; T frame represents the video data processing time corresponding to the maximum frame rate change amplitude of the video data appearing in the m-times video data processing; T m represents the average video data processing time of m video data processing times;

[0033] comparing the first processing evaluation parameter with a preset first data processing evaluation threshold;

[0034] When the first processing evaluation parameter is lower than a preset first data processing evaluation threshold, it is determined that there is an abnormality in the data processing process of the audio data;

[0035] comparing the second processing evaluation parameter with a preset second data processing evaluation threshold;

[0036] When the second processing evaluation parameter is lower than a preset second data processing evaluation threshold, it is determined that there is an abnormality in the data processing process of the video data.

[0037] Furthermore, the video perception platform is used to identify information on the video data and the audio perception platform is used to identify information on the audio data to determine whether a bullying warning is needed, including:

[0038] The broadcast control platform sends the processed audio data and video data to the campus broadcast management platform, and sends the video data to the video perception platform through the campus broadcast management platform;

[0039] The video perception platform analyzes and identifies the video data to determine whether the video data contains a preset calibration behavior; wherein the calibration behavior includes a preset bullying physical behavior;

[0040] When the video data contains a preset calibration behavior, the audio data is sent to the audio perception platform, and the audio perception platform is used to determine whether the audio data contains calibration words; wherein the calibration words include words such as calling for help and begging for mercy;

[0041] When the audio data contains the marked words, an early warning message is sent to the teacher terminal through the campus broadcast management platform.

[0042] A campus bullying early warning system based on video-linked audio, the campus bullying early warning system based on video-linked audio comprising:

[0043] The data acquisition control module is used to control the broadcast control platform to collect audio data and video data inside the campus in real time through the audio acquisition terminal and video acquisition terminal installed inside the campus;

[0044] A data processing module, used to send the audio data and video data to the broadcast control platform through a wireless communication network, and the broadcast control platform performs data processing on the audio data and video data to obtain processed audio data and video data;

[0045] The recognition and perception module is used to perform information recognition on video data through the video perception platform and to link the audio perception platform to perform information recognition on audio data to determine whether a bullying warning is needed.

[0046] Furthermore, the data acquisition control module includes:

[0047] A real-time acquisition control module is used to control the broadcast control platform to send wireless control signals to the audio acquisition terminal and the video acquisition terminal installed inside the campus through wireless communication, so as to control the audio acquisition terminal and the video acquisition terminal to collect audio data and video data inside the campus in real time; wherein the audio acquisition terminal includes but is not limited to a microphone and other sound receiving devices; the video acquisition terminal includes but is not limited to a high-definition camera, etc.;

[0048] The time period monitoring module is used for the broadcast control platform to monitor the current time period of the campus environment in real time;

[0049] The data acquisition frequency adjustment module is used to adjust the audio acquisition frequency and video acquisition frequency of the audio acquisition terminal and the video acquisition terminal according to the current time period of the campus environment.

[0050] Among them, when the campus environment is in the non-school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are reduced; when the campus environment is in the school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are increased.

[0051] Furthermore, the data processing module includes:

[0052] A data sending module, used for the audio acquisition terminal and the video acquisition terminal to send the audio data and the video data to the broadcast control platform via a wireless communication network;

[0053] A data preprocessing module, used for the broadcast control platform to perform data processing on the audio data and the video data, wherein the data processing includes data noise reduction processing, data cleaning processing and data compression processing;

[0054] The data processing quality monitoring module is used to monitor the data processing time corresponding to the audio data and video data in real time, and determine whether there is any abnormality in the data processing process of the audio data and video data by the broadcast control platform according to the data processing time corresponding to the audio data and video data.

[0055] Furthermore, the data processing quality monitoring module includes:

[0056] A real-time monitoring module, used to monitor in real time the data processing time corresponding to the audio data and the video data;

[0057] A first data extraction module, used to extract the frequency spectrum density and peak frequency of the audio data that has completed data processing;

[0058] The first processing evaluation parameter acquisition module is used to acquire the first processing evaluation parameter according to the data processing time corresponding to the audio data combined with the spectrum density and peak frequency of the audio data that has completed data processing; wherein the first processing evaluation parameter is acquired by the following formula:

[0059]

[0060] Among them, P 01 represents the first processing evaluation parameter; F peak represents the peak frequency; S(f) represents the spectral density function corresponding to the audio data; f 01 and f 02 represents the frequency upper limit value and the frequency lower limit value corresponding to the frequency distribution range corresponding to the audio data; T represents the first time parameter, and the first time parameter is obtained by the following formula:

[0061]

[0062] Where, T represents the first time parameter; T peak Indicates the data processing time corresponding to the peak frequency; T p represents the average data processing time corresponding to n audio data processing times; n represents the number of audio data processing times; T i T represents the data processing time corresponding to the i-th audio data processing; z Indicates the median value of the data processing duration corresponding to n audio data processing times;

[0063] A second data extraction module is used to extract the video resolution and frame rate of the video data that has completed data processing;

[0064] The second processing evaluation parameter acquisition module is used to acquire the second processing evaluation parameter according to the data processing time corresponding to the video data combined with the video resolution and frame rate of the video data that has completed data processing; wherein the second processing evaluation parameter is acquired by the following formula:

[0065]

[0066] Among them, P 02 represents the second processing evaluation parameter; m represents the number of video data processing times; R i represents the video resolution corresponding to the i-th video data processing; R i-1 Indicates the video resolution corresponding to the i-1th video processing; F framei represents the maximum frame rate variation of the video data corresponding to the i-th video data processing; F max Indicates the maximum allowed frame rate variation of the preset video data; T fi represents the actual processing time corresponding to the i-th video data processing; T si T represents the theoretical processing time corresponding to the i-th video data processing; x represents the second time parameter, and the second time parameter is obtained by the following formula:

[0067]

[0068] Among them, T x Represents the second time parameter; T frame represents the video data processing time corresponding to the maximum frame rate change amplitude of the video data appearing in the m-times video data processing; T m represents the average video data processing time of m video data processing times;

[0069] A first comparison module, used for comparing the first processing evaluation parameter with a preset first data processing evaluation threshold;

[0070] A first abnormality judgment module, configured to judge that an abnormality exists in the data processing process of the audio data when the first processing evaluation parameter is lower than a preset first data processing evaluation threshold;

[0071] A second comparison module, used for comparing the second processing evaluation parameter with a preset second data processing evaluation threshold;

[0072] The second abnormality judgment module is used to judge that there is an abnormality in the data processing process of the video data when the second processing evaluation parameter is lower than a preset second data processing evaluation threshold.

[0073] Furthermore, the recognition and perception module includes:

[0074] A video data transmission module, which is used for the broadcast control platform to send the processed audio data and video data to the campus broadcast management platform, and send the video data to the video perception platform through the campus broadcast management platform;

[0075] A video calibration behavior determination module is used for the video perception platform to analyze and identify video data and determine whether the video data contains a preset calibration behavior;

[0076] An audio data transmission module, used for sending the audio data to the audio perception platform when the video data contains a preset calibration behavior, and determining whether there is a calibration word in the audio data through the audio perception platform;

[0077] The audio calibration word determination module is used to send warning information to the teacher terminal through the campus broadcast management platform when the audio data contains the calibration words.

[0078] Beneficial effects of the present invention:

[0079] The present invention proposes a campus bullying early warning method and system based on video-linked audio. By real-time acquisition and processing of audio and video data, the system can quickly identify and warn of potential bullying incidents, thereby improving the timeliness and accuracy of the response. The acquisition terminals widely deployed on campus ensure the comprehensiveness of monitoring, reduce monitoring blind spots, and improve the effect of preventing bullying. Combined with the joint analysis of video and audio, the system can more accurately identify bullying behaviors and reduce the possibility of false alarms and missed reports. Once bullying behavior is identified, the system can immediately issue an early warning and provide school administrators with timely information, so that they can quickly take measures to intervene. This early warning method not only helps prevent the occurrence of bullying incidents, but also improves the overall safety level of the campus, providing students with a safer and more harmonious learning environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 is a flow chart of the method of the present invention;

[0081] Figure 2 The system block diagram of the system of the present invention. DETAILED DESCRIPTION

[0082] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0083] The embodiment of the present invention proposes a campus bullying early warning method based on video-linked audio, such as Figure 1 As shown, the campus bullying early warning method based on video-linked audio includes:

[0084] S1. The broadcast control platform controls the audio acquisition terminal and video acquisition terminal installed inside the campus to collect audio data and video data inside the campus in real time;

[0085] S2. Sending the audio data and video data to a broadcast control platform via a wireless communication network, wherein the broadcast control platform processes the audio data and video data to obtain processed audio data and video data;

[0086] S3. Use the video perception platform to identify information on the video data and use the audio perception platform to identify information on the audio data to determine whether a bullying warning is needed.

[0087] The working principle of the above technical solution is as follows: Inside the campus, audio and video data on campus are captured in real time through pre-installed audio and video acquisition terminals (such as cameras and microphones). These devices cover key areas such as corridors, playgrounds, canteens, etc. to ensure that potential bullying incidents can be fully monitored. The collected audio and video data are transmitted to the broadcast control platform in real time through wireless communication networks (such as Wi-Fi, 4G / 5G, etc.). As a data processing center, the broadcast control platform is responsible for receiving and preliminarily processing these data to remove noise, enhance signals, etc. to ensure data quality. On the broadcast control platform, the received audio and video data are further processed and analyzed. This may include compression, denoising, and contrast enhancement of video frames, as well as voice recognition and sound enhancement of audio. The processed data is easier for subsequent analysis and recognition. Through a dedicated video perception platform and audio perception platform, in-depth information recognition is performed on the processed video and audio data. The video perception platform uses image recognition technology (such as face recognition and behavior recognition) to analyze behavioral patterns in the video, while the audio perception platform interprets the speech content and emotions in the audio through voice recognition and emotion analysis technology. When the two jointly determine that there are signs of bullying (such as physical conflict in the video and insults or threats in the audio), the early warning mechanism will be triggered. Once bullying behavior is identified, the system will immediately send out an early warning signal to notify school administrators, teachers or security teams so that they can quickly intervene and deal with the incident.

[0088] The effect of the above technical solution is: by collecting and processing audio and video data in real time, the system can quickly identify and warn of potential bullying incidents, improving the timeliness and accuracy of the response. The widely deployed acquisition terminals on campus ensure the comprehensiveness of monitoring, reduce monitoring blind spots, and improve the effect of preventing bullying. Combined with the joint analysis of video and audio, the system can more accurately identify bullying behavior and reduce the possibility of false alarms and missed alarms. Once bullying behavior is identified, the system can immediately issue an early warning and provide school administrators with timely information so that they can quickly take measures to intervene. This early warning method not only helps prevent the occurrence of bullying incidents, but also improves the overall safety level of the campus and provides students with a safer and more harmonious learning environment.

[0089] In one embodiment of the present invention, the broadcast control platform controls the audio acquisition terminal and the video acquisition terminal installed inside the campus to collect audio data and video data inside the campus in real time, including:

[0090] S101, the broadcast control platform controls the audio acquisition terminal and the video acquisition terminal installed inside the campus to send a wireless control signal through wireless communication to control the audio acquisition terminal and the video acquisition terminal to collect audio data and video data inside the campus in real time; wherein the audio acquisition terminal includes but is not limited to a microphone and other sound receiving devices; the video acquisition terminal includes but is not limited to a high-definition camera, etc.;

[0091] S102, the broadcast control platform monitors the current time period of the campus environment in real time;

[0092] S103: adjusting the audio collection frequency and the video collection frequency of the audio collection terminal and the video collection terminal according to the current time period of the campus environment.

[0093] Among them, when the campus environment is in the non-school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are reduced; when the campus environment is in the school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are increased.

[0094] The working principle of the above technical solution is as follows: the broadcast control platform acts as a control center, and sends wireless control signals to the audio acquisition terminals (such as microphones) and video acquisition terminals (such as high-definition cameras) installed inside the campus through wireless communication methods (such as Wi-Fi, Zigbee, LoRa, etc.). These control signals are used to instruct the audio and video acquisition terminals to start real-time collection of audio data and video data inside the campus. The broadcast control platform has a built-in or external time synchronization device, which can monitor the current time in real time and judge the time period of the campus environment based on it. The time period can be divided into school time period (such as arriving at school in the morning to leaving school in the afternoon) and non-school time period (such as evening, weekends, holidays, etc.). According to the current time period of the campus environment, the broadcast control platform automatically adjusts the acquisition frequency of the audio acquisition terminal and the video acquisition terminal. During non-school time periods, due to the low number of people activities on campus, the risk of bullying incidents that may occur is relatively low, so the frequency of audio and video acquisition can be reduced to reduce the demand for data storage and processing, while saving energy. During school hours, the risk of bullying increases as students and faculty members are active on campus. Therefore, the frequency of audio and video collection is increased to more comprehensively capture and record campus dynamics, ensuring that potential bullying incidents can be discovered and warned of in a timely manner.

[0095] The effect of the above technical solution is: by automatically adjusting the collection frequency according to the time period, the technical solution can optimize the use of resources while ensuring the monitoring effect. Reducing the collection frequency during non-school hours can reduce the pressure of data storage and processing, reduce energy consumption, and extend the service life of the equipment. Increasing the collection frequency during school hours can ensure that key areas on campus are more intensively monitored, improve the efficiency and accuracy of monitoring, and thus more effectively prevent and respond to bullying incidents. This technical solution is highly flexible and can dynamically adjust the monitoring strategy according to different time periods and actual conditions to adapt to changes in the campus environment. Through more refined monitoring and management, this technical solution helps to improve the overall security of the campus and provide students and faculty with a safer and more harmonious learning and working environment. By combining time period monitoring and collection frequency adjustment, false alarms caused by false triggering during non-critical periods can be reduced, and the accuracy and reliability of the early warning system can be improved.

[0096] In one embodiment of the present invention, the audio data and the video data are sent to a broadcast control platform via a wireless communication network, and the broadcast control platform processes the audio data and the video data to obtain the processed audio data and the video data, including:

[0097] S201, the audio acquisition terminal and the video acquisition terminal send the audio data and the video data to the broadcast control platform via the wireless communication network;

[0098] S202, the broadcast control platform performs data processing on the audio data and the video data, wherein the data processing includes data noise reduction processing, data cleaning processing and data compression processing;

[0099] S203: monitor the data processing duration corresponding to the audio data and the video data in real time, and determine whether there is any abnormality in the data processing process of the audio data and the video data by the broadcast control platform according to the data processing duration corresponding to the audio data and the video data.

[0100] The working principle of the above technical solution is as follows: the audio acquisition terminal (such as a microphone) and the video acquisition terminal (such as a high-definition camera) send the real-time collected audio data and video data to the broadcast control platform through a wireless communication network (such as Wi-Fi, 4G / 5G, etc.). This step ensures that the data can be transmitted to the processing center in a timely and accurate manner. After receiving the audio and video data, the broadcast control platform performs a series of data processing operations, including data noise reduction, data cleaning and data compression. The algorithm is used to remove the noise components in the audio and video data, improve the signal-to-noise ratio of the data, and make subsequent analysis and recognition more accurate. Identify and correct errors or outliers in the data, remove redundant information, and ensure the integrity and accuracy of the data. The compression algorithm is used to compress the audio and video data to reduce the storage space and transmission bandwidth requirements of the data, while trying to maintain the original quality of the data.

[0101] The broadcast control platform monitors the data processing time corresponding to audio and video data in real time, that is, the time required for each data packet from reception to completion of processing. According to the preset threshold or historical data comparison, it determines whether there is an abnormality in the data processing process. If the processing time exceeds the normal range, it may mean that the system load is too high, the algorithm is inefficient, or the hardware is faulty. At this time, the system will issue an alarm or take corresponding countermeasures.

[0102] The effects of the above technical solutions are as follows: through data noise reduction and data cleaning processing, noise and outliers in audio and video data are effectively removed, and the accuracy and reliability of the data are improved. Data compression processing significantly reduces the storage space and transmission bandwidth requirements of data, reduces the operating costs of the system, and improves the real-time performance of data transmission. Real-time monitoring of data processing time and determining whether there are anomalies can help to promptly discover and solve potential problems in the system and ensure the stable operation of the system. Adding a monitoring mechanism to the processing of audio and video data enables the broadcast control platform to more comprehensively grasp the status and efficiency of data processing, providing strong support for subsequent early warning and analysis. This technical solution can be flexibly adjusted and optimized according to different application scenarios and needs to adapt to different monitoring environments and requirements.

[0103] In one embodiment of the present invention, determining whether there is an abnormality in the data processing process of the audio data and the video data by the broadcast control platform according to the data processing time corresponding to the audio data and the video data includes:

[0104] S2031, monitoring the data processing time corresponding to the audio data and the video data in real time;

[0105] S2032, extracting the frequency spectrum density and peak frequency of the audio data for which data processing has been completed;

[0106] S2033, obtaining a first processing evaluation parameter according to the data processing time corresponding to the audio data combined with the frequency spectrum density and peak frequency of the audio data for which data processing has been completed; wherein the first processing evaluation parameter is obtained by the following formula:

[0107]

[0108] Among them, P 01 represents the first processing evaluation parameter; F peak represents the peak frequency; S(f) represents the spectral density function corresponding to the audio data; f 01 and f 02 represents the frequency upper limit value and the frequency lower limit value corresponding to the frequency distribution range corresponding to the audio data; T represents the first time parameter, and the first time parameter is obtained by the following formula:

[0109]

[0110] Where, T represents the first time parameter; T peak Indicates the data processing time corresponding to the peak frequency; T p represents the average data processing time corresponding to n audio data processing times; n represents the number of audio data processing times; T i T represents the data processing time corresponding to the i-th audio data processing; z Indicates the median value of the data processing duration corresponding to n audio data processing times;

[0111] S2034, extracting the video resolution and frame rate of the video data for which data processing has been completed;

[0112] S2035. Obtain a second processing evaluation parameter according to the data processing duration corresponding to the video data combined with the video resolution and frame rate of the video data that has completed data processing; wherein the second processing evaluation parameter is obtained by the following formula:

[0113]

[0114] Among them, P 02 represents the second processing evaluation parameter; m represents the number of video data processing times; R i represents the video resolution corresponding to the i-th video data processing; R i-1 Indicates the video resolution corresponding to the i-1th video processing; F framei represents the maximum frame rate variation of the video data corresponding to the i-th video data processing; F max Indicates the maximum allowed frame rate variation of the preset video data; T fi represents the actual processing time corresponding to the i-th video data processing; T siT represents the theoretical processing time corresponding to the i-th video data processing; x represents the second time parameter, and the second time parameter is obtained by the following formula:

[0115]

[0116] Among them, T x Represents the second time parameter; T frame represents the video data processing time corresponding to the maximum frame rate change amplitude of the video data appearing in the m-times video data processing; T m represents the average video data processing time of m video data processing times;

[0117] S2036, comparing the first processing evaluation parameter with a preset first data processing evaluation threshold;

[0118] When the first processing evaluation parameter is lower than a preset first data processing evaluation threshold, it is determined that there is an abnormality in the data processing process of the audio data;

[0119] S2037, comparing the second processing evaluation parameter with a preset second data processing evaluation threshold;

[0120] S2038: When the second processing evaluation parameter is lower than a preset second data processing evaluation threshold, it is determined that an abnormality exists in the data processing process of the video data.

[0121] The working principle of the above technical solution is: real-time monitoring of the time required for audio and video data from reception to completion of processing, that is, the data processing time. Extract the spectral density (S(f)) and peak frequency (Fpeak) from the audio data that has completed data processing. Combined with the data processing time (T), spectral density function, frequency distribution range (f01, f02) and a series of time parameters (such as Tpeak, Tp, Ti, Tz, etc.) of the audio data, the first processing evaluation parameter (P01) is calculated by a specific formula. This formula comprehensively considers the frequency characteristics and processing efficiency of the audio data. Compare the calculated first processing evaluation parameter (P01) with the preset first data processing evaluation threshold. If P01 is lower than the threshold, it is judged that there is an abnormality in the data processing process of the audio data.

[0122] The video resolution (Ri) and frame rate change information (Fframei) are extracted from the video data that has completed data processing. The second processing evaluation parameter (P02) is calculated by a specific formula based on the data processing time (Tfi, Tsi), video resolution, frame rate change range (Fframei, Fmax) and second time parameter (Tx) of the video data. This formula takes into account the clarity and processing speed of the video data. The calculated second processing evaluation parameter (P02) is compared with a preset second data processing evaluation threshold. If P02 is lower than the threshold, it is determined that there is an abnormality in the data processing process of the video data.

[0123] The effect of the above technical solution is: by real-time monitoring of data processing time, potential problems in the processing process can be discovered in time to ensure timely processing and transmission of data. Combined with multi-dimensional information such as the spectral density and peak frequency of audio and the resolution and frame rate change of video, the data processing process is comprehensively and objectively evaluated, which improves the accuracy and reliability of the evaluation. Through the preset data processing evaluation threshold, it can automatically determine whether there are abnormalities in the data processing process, and issue early warnings in time, which helps to quickly locate problems and take corresponding solutions. Through continuous monitoring and evaluation of the data processing process, the system performance can be continuously optimized, the data processing efficiency and accuracy can be improved, and the system failure rate can be reduced. Ensuring the timely and accurate processing of audio and video data can improve users' satisfaction and trust in the campus security monitoring system and provide a more solid guarantee for campus safety.

[0124] In one embodiment of the present invention, a video perception platform performs information recognition on video data and an audio perception platform performs information recognition on audio data to determine whether a bullying warning is required, including:

[0125] S301, the broadcast control platform sends the processed audio data and video data to the campus broadcast management platform, and sends the video data to the video perception platform through the campus broadcast management platform;

[0126] S302, the video perception platform analyzes and identifies the video data to determine whether the video data contains a preset calibration behavior; wherein the calibration behavior includes a preset bullying physical behavior;

[0127] S303: When the video data contains a preset calibration behavior, the audio data is sent to the audio perception platform, and the audio perception platform is used to determine whether the audio data contains calibration words; wherein the calibration words include words such as calling for help and begging for mercy;

[0128] S304: When the audio data contains the marked words, a warning message is sent to the teacher terminal via the campus broadcast management platform.

[0129] The working principle of the above technical solution is as follows: the broadcast control platform first processes the collected audio and video data, including noise reduction, cleaning and compression, and then sends the processed data to the campus broadcast management platform. The campus broadcast management platform acts as a transfer station and forwards the video data to the video perception platform for subsequent analysis and recognition.

[0130] After receiving the video data, the video perception platform uses advanced video analysis algorithms to analyze the video and identify whether the video content contains preset calibration behaviors. These calibration behaviors specifically refer to physical behaviors that may constitute bullying, such as pushing, beating, etc. When the video perception platform determines that the video data does contain preset calibration behaviors, it will trigger the start of the audio perception platform. The audio perception platform then analyzes the corresponding audio data, focusing on identifying whether there are calibration words in the audio, such as voice signals such as calling for help and begging for mercy that may indicate that the victim is being bullied. If the audio perception platform detects calibration words in the audio data, it means that there is a high possibility of bullying behavior. At this time, it will send an early warning message to the teacher terminal through the campus broadcast management platform. After receiving the early warning message, the teacher terminal can immediately take corresponding measures, such as going to the scene to stop the bullying behavior, or notifying relevant personnel to deal with it.

[0131] The effect of the above technical solution is: by combining the dual recognition of video and audio, the technical solution can more accurately determine whether bullying behavior exists, reducing the possibility of false alarms and missed alarms. Once bullying behavior is detected, the system can immediately send an early warning message to the teacher's terminal to ensure that relevant personnel can respond quickly and stop the bullying behavior in time. The application of this technical solution helps to improve the level of campus security management and create a safer and more harmonious learning environment for teachers and students. Through automated identification and early warning, the burden of manual monitoring is reduced and the level of intelligence of campus security management is improved. The audio and video data collected by the system can also be used for subsequent analysis and research to provide data support for the formulation of more effective campus safety policies.

[0132] The embodiment of the present invention proposes a campus bullying early warning system based on video-linked audio. Figure 2 As shown, the campus bullying early warning system based on video-linked audio includes:

[0133] The data acquisition control module is used to control the broadcast control platform to collect audio data and video data inside the campus in real time through the audio acquisition terminal and video acquisition terminal installed inside the campus;

[0134] A data processing module, used to send the audio data and video data to the broadcast control platform through a wireless communication network, and the broadcast control platform performs data processing on the audio data and video data to obtain processed audio data and video data;

[0135] The recognition and perception module is used to perform information recognition on video data through the video perception platform and to link the audio perception platform to perform information recognition on audio data to determine whether a bullying warning is needed.

[0136] The working principle of the above technical solution is as follows: Inside the campus, audio and video data on campus are captured in real time through pre-installed audio and video acquisition terminals (such as cameras and microphones). These devices cover key areas such as corridors, playgrounds, canteens, etc. to ensure that potential bullying incidents can be fully monitored. The collected audio and video data are transmitted to the broadcast control platform in real time through wireless communication networks (such as Wi-Fi, 4G / 5G, etc.). As a data processing center, the broadcast control platform is responsible for receiving and preliminarily processing these data to remove noise, enhance signals, etc. to ensure data quality. On the broadcast control platform, the received audio and video data are further processed and analyzed. This may include compression, denoising, and contrast enhancement of video frames, as well as voice recognition and sound enhancement of audio. The processed data is easier for subsequent analysis and recognition. Through a dedicated video perception platform and audio perception platform, in-depth information recognition is performed on the processed video and audio data. The video perception platform uses image recognition technology (such as face recognition and behavior recognition) to analyze behavioral patterns in the video, while the audio perception platform interprets the speech content and emotions in the audio through voice recognition and emotion analysis technology. When the two jointly determine that there are signs of bullying (such as physical conflict in the video and insults or threats in the audio), the early warning mechanism will be triggered. Once bullying behavior is identified, the system will immediately send out an early warning signal to notify school administrators, teachers or security teams so that they can quickly intervene and deal with the incident.

[0137] The effect of the above technical solution is: by collecting and processing audio and video data in real time, the system can quickly identify and warn of potential bullying incidents, improving the timeliness and accuracy of the response. The widely deployed acquisition terminals on campus ensure the comprehensiveness of monitoring, reduce monitoring blind spots, and improve the effect of preventing bullying. Combined with the joint analysis of video and audio, the system can more accurately identify bullying behavior and reduce the possibility of false alarms and missed alarms. Once bullying behavior is identified, the system can immediately issue an early warning and provide school administrators with timely information so that they can quickly take measures to intervene. This early warning method not only helps prevent the occurrence of bullying incidents, but also improves the overall safety level of the campus and provides students with a safer and more harmonious learning environment.

[0138] In one embodiment of the present invention, the data acquisition control module includes:

[0139] A real-time acquisition control module is used to control the broadcast control platform to send wireless control signals to the audio acquisition terminal and the video acquisition terminal installed inside the campus through wireless communication, so as to control the audio acquisition terminal and the video acquisition terminal to collect audio data and video data inside the campus in real time; wherein the audio acquisition terminal includes but is not limited to a microphone and other sound receiving devices; the video acquisition terminal includes but is not limited to a high-definition camera, etc.;

[0140] The time period monitoring module is used for the broadcast control platform to monitor the current time period of the campus environment in real time;

[0141] The data acquisition frequency adjustment module is used to adjust the audio acquisition frequency and video acquisition frequency of the audio acquisition terminal and the video acquisition terminal according to the current time period of the campus environment.

[0142] Among them, when the campus environment is in the non-school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are reduced; when the campus environment is in the school period, the audio collection frequency and video collection frequency of the audio collection terminal and the video collection terminal are increased.

[0143] The working principle of the above technical solution is as follows: the broadcast control platform acts as a control center, and sends wireless control signals to the audio acquisition terminals (such as microphones) and video acquisition terminals (such as high-definition cameras) installed inside the campus through wireless communication methods (such as Wi-Fi, Zigbee, LoRa, etc.). These control signals are used to instruct the audio and video acquisition terminals to start real-time collection of audio data and video data inside the campus. The broadcast control platform has a built-in or external time synchronization device, which can monitor the current time in real time and judge the time period of the campus environment based on it. The time period can be divided into school time period (such as arriving at school in the morning to leaving school in the afternoon) and non-school time period (such as evening, weekends, holidays, etc.). According to the current time period of the campus environment, the broadcast control platform automatically adjusts the acquisition frequency of the audio acquisition terminal and the video acquisition terminal. During non-school time periods, due to the low number of people activities on campus, the risk of bullying incidents that may occur is relatively low, so the frequency of audio and video acquisition can be reduced to reduce the demand for data storage and processing, while saving energy. During school hours, the risk of bullying increases as students and faculty members are active on campus. Therefore, the frequency of audio and video collection is increased to more comprehensively capture and record campus dynamics, ensuring that potential bullying incidents can be discovered and warned of in a timely manner.

[0144] The effect of the above technical solution is: by automatically adjusting the collection frequency according to the time period, the technical solution can optimize the use of resources while ensuring the monitoring effect. Reducing the collection frequency during non-school hours can reduce the pressure of data storage and processing, reduce energy consumption, and extend the service life of the equipment. Increasing the collection frequency during school hours can ensure that key areas on campus are more intensively monitored, improve the efficiency and accuracy of monitoring, and thus more effectively prevent and respond to bullying incidents. This technical solution is highly flexible and can dynamically adjust the monitoring strategy according to different time periods and actual conditions to adapt to changes in the campus environment. Through more refined monitoring and management, this technical solution helps to improve the overall security of the campus and provide students and faculty with a safer and more harmonious learning and working environment. By combining time period monitoring and collection frequency adjustment, false alarms caused by false triggering during non-critical periods can be reduced, and the accuracy and reliability of the early warning system can be improved.

[0145] In one embodiment of the present invention, the data processing module includes:

[0146] A data sending module, used for the audio acquisition terminal and the video acquisition terminal to send the audio data and the video data to the broadcast control platform via a wireless communication network;

[0147] A data preprocessing module, used for the broadcast control platform to perform data processing on the audio data and the video data, wherein the data processing includes data noise reduction processing, data cleaning processing and data compression processing;

[0148] The data processing quality monitoring module is used to monitor the data processing time corresponding to the audio data and video data in real time, and determine whether there is any abnormality in the data processing process of the audio data and video data by the broadcast control platform according to the data processing time corresponding to the audio data and video data.

[0149] The working principle of the above technical solution is as follows: the audio acquisition terminal (such as a microphone) and the video acquisition terminal (such as a high-definition camera) send the real-time collected audio data and video data to the broadcast control platform through a wireless communication network (such as Wi-Fi, 4G / 5G, etc.). This step ensures that the data can be transmitted to the processing center in a timely and accurate manner. After receiving the audio and video data, the broadcast control platform performs a series of data processing operations, including data noise reduction, data cleaning and data compression. The algorithm is used to remove the noise components in the audio and video data, improve the signal-to-noise ratio of the data, and make subsequent analysis and recognition more accurate. Identify and correct errors or outliers in the data, remove redundant information, and ensure the integrity and accuracy of the data. The compression algorithm is used to compress the audio and video data to reduce the storage space and transmission bandwidth requirements of the data, while trying to maintain the original quality of the data.

[0150] The broadcast control platform monitors the data processing time corresponding to audio and video data in real time, that is, the time required for each data packet from reception to completion of processing. According to the preset threshold or historical data comparison, it determines whether there is an abnormality in the data processing process. If the processing time exceeds the normal range, it may mean that the system load is too high, the algorithm is inefficient, or the hardware is faulty. At this time, the system will issue an alarm or take corresponding countermeasures.

[0151] The effects of the above technical solutions are as follows: through data noise reduction and data cleaning processing, noise and outliers in audio and video data are effectively removed, and the accuracy and reliability of the data are improved. Data compression processing significantly reduces the storage space and transmission bandwidth requirements of data, reduces the operating costs of the system, and improves the real-time performance of data transmission. Real-time monitoring of data processing time and determining whether there are anomalies can help to promptly discover and solve potential problems in the system and ensure the stable operation of the system. Adding a monitoring mechanism to the processing of audio and video data enables the broadcast control platform to more comprehensively grasp the status and efficiency of data processing, providing strong support for subsequent early warning and analysis. This technical solution can be flexibly adjusted and optimized according to different application scenarios and needs to adapt to different monitoring environments and requirements.

[0152] In one embodiment of the present invention, the data processing quality monitoring module includes:

[0153] A real-time monitoring module, used to monitor in real time the data processing time corresponding to the audio data and the video data;

[0154] A first data extraction module, used to extract the frequency spectrum density and peak frequency of the audio data that has completed data processing;

[0155] The first processing evaluation parameter acquisition module is used to acquire the first processing evaluation parameter according to the data processing time corresponding to the audio data combined with the spectrum density and peak frequency of the audio data that has completed data processing; wherein the first processing evaluation parameter is acquired by the following formula:

[0156]

[0157] Among them, P 01 represents the first processing evaluation parameter; F peak represents the peak frequency; S(f) represents the spectral density function corresponding to the audio data; f 01 and f 02 represents the frequency upper limit value and the frequency lower limit value corresponding to the frequency distribution range corresponding to the audio data; T represents the first time parameter, and the first time parameter is obtained by the following formula:

[0158]

[0159] Where, T represents the first time parameter; T peak Indicates the data processing time corresponding to the peak frequency; T p represents the average data processing time corresponding to n audio data processing times; n represents the number of audio data processing times; T i T represents the data processing time corresponding to the i-th audio data processing; z Indicates the median value of the data processing duration corresponding to n audio data processing times;

[0160] A second data extraction module is used to extract the video resolution and frame rate of the video data that has completed data processing;

[0161] The second processing evaluation parameter acquisition module is used to acquire the second processing evaluation parameter according to the data processing time corresponding to the video data combined with the video resolution and frame rate of the video data that has completed data processing; wherein the second processing evaluation parameter is acquired by the following formula:

[0162]

[0163] Among them, P 02 represents the second processing evaluation parameter; m represents the number of video data processing times; R i represents the video resolution corresponding to the i-th video data processing; R i-1 Indicates the video resolution corresponding to the i-1th video processing; F framei represents the maximum frame rate variation of the video data corresponding to the i-th video data processing; F max Indicates the maximum allowed frame rate variation of the preset video data; T fi represents the actual processing time corresponding to the i-th video data processing; T si T represents the theoretical processing time corresponding to the i-th video data processing; x represents the second time parameter, and the second time parameter is obtained by the following formula:

[0164]

[0165] Among them, T x Represents the second time parameter; T frame represents the video data processing time corresponding to the maximum frame rate change amplitude of the video data appearing in the m-times video data processing; T m represents the average video data processing time of m video data processing times;

[0166] A first comparison module, used for comparing the first processing evaluation parameter with a preset first data processing evaluation threshold;

[0167] A first abnormality judgment module, configured to judge that an abnormality exists in the data processing process of the audio data when the first processing evaluation parameter is lower than a preset first data processing evaluation threshold;

[0168] A second comparison module, used for comparing the second processing evaluation parameter with a preset second data processing evaluation threshold;

[0169] The second abnormality judgment module is used to judge that there is an abnormality in the data processing process of the video data when the second processing evaluation parameter is lower than a preset second data processing evaluation threshold.

[0170] The working principle of the above technical solution is: real-time monitoring of the time required for audio and video data from reception to completion of processing, that is, the data processing time. Extract the spectral density (S(f)) and peak frequency (Fpeak) from the audio data that has completed data processing. Combined with the data processing time (T), spectral density function, frequency distribution range (f01, f02) and a series of time parameters (such as Tpeak, Tp, Ti, Tz, etc.) of the audio data, the first processing evaluation parameter (P01) is calculated by a specific formula. This formula comprehensively considers the frequency characteristics and processing efficiency of the audio data. Compare the calculated first processing evaluation parameter (P01) with the preset first data processing evaluation threshold. If P01 is lower than the threshold, it is judged that there is an abnormality in the data processing process of the audio data.

[0171] The video resolution (Ri) and frame rate change information (Fframei) are extracted from the video data that has completed data processing. The second processing evaluation parameter (P02) is calculated by a specific formula based on the data processing time (Tfi, Tsi), video resolution, frame rate change range (Fframei, Fmax) and second time parameter (Tx) of the video data. This formula takes into account the clarity and processing speed of the video data. The calculated second processing evaluation parameter (P02) is compared with a preset second data processing evaluation threshold. If P02 is lower than the threshold, it is determined that there is an abnormality in the data processing process of the video data.

[0172] The effect of the above technical solution is: by real-time monitoring of data processing time, potential problems in the processing process can be discovered in time to ensure timely processing and transmission of data. Combined with multi-dimensional information such as the spectral density and peak frequency of audio and the resolution and frame rate change of video, the data processing process is comprehensively and objectively evaluated, which improves the accuracy and reliability of the evaluation. Through the preset data processing evaluation threshold, it can automatically determine whether there are abnormalities in the data processing process, and issue early warnings in time, which helps to quickly locate problems and take corresponding solutions. Through continuous monitoring and evaluation of the data processing process, the system performance can be continuously optimized, the data processing efficiency and accuracy can be improved, and the system failure rate can be reduced. Ensuring the timely and accurate processing of audio and video data can improve users' satisfaction and trust in the campus security monitoring system and provide a more solid guarantee for campus safety.

[0173] In one embodiment of the present invention, the recognition and perception module includes:

[0174] A video data transmission module, which is used for the broadcast control platform to send the processed audio data and video data to the campus broadcast management platform, and send the video data to the video perception platform through the campus broadcast management platform;

[0175] A video calibration behavior determination module is used for the video perception platform to analyze and identify video data and determine whether the video data contains a preset calibration behavior;

[0176] An audio data transmission module, used for sending the audio data to the audio perception platform when the video data contains a preset calibration behavior, and determining whether there is a calibration word in the audio data through the audio perception platform;

[0177] The audio calibration word determination module is used to send warning information to the teacher terminal through the campus broadcast management platform when the audio data contains the calibration words.

[0178] The working principle of the above technical solution is as follows: the broadcast control platform first processes the collected audio and video data, including noise reduction, cleaning and compression, and then sends the processed data to the campus broadcast management platform. The campus broadcast management platform acts as a transfer station and forwards the video data to the video perception platform for subsequent analysis and recognition.

[0179] After receiving the video data, the video perception platform uses advanced video analysis algorithms to analyze the video and identify whether the video content contains preset calibration behaviors. These calibration behaviors specifically refer to physical behaviors that may constitute bullying, such as pushing, beating, etc. When the video perception platform determines that the video data does contain preset calibration behaviors, it will trigger the start of the audio perception platform. The audio perception platform then analyzes the corresponding audio data, focusing on identifying whether there are calibration words in the audio, such as voice signals such as calling for help and begging for mercy that may indicate that the victim is being bullied. If the audio perception platform detects calibration words in the audio data, it means that there is a high possibility of bullying behavior. At this time, it will send an early warning message to the teacher terminal through the campus broadcast management platform. After receiving the early warning message, the teacher terminal can immediately take corresponding measures, such as going to the scene to stop the bullying behavior, or notifying relevant personnel to deal with it.

[0180] The effect of the above technical solution is: by combining the dual recognition of video and audio, the technical solution can more accurately determine whether bullying behavior exists, reducing the possibility of false alarms and missed alarms. Once bullying behavior is detected, the system can immediately send an early warning message to the teacher's terminal to ensure that relevant personnel can respond quickly and stop the bullying behavior in time. The application of this technical solution helps to improve the level of campus security management and create a safer and more harmonious learning environment for teachers and students. Through automated identification and early warning, the burden of manual monitoring is reduced and the level of intelligence of campus security management is improved. The audio and video data collected by the system can also be used for subsequent analysis and research to provide data support for the formulation of more effective campus safety policies.

[0181] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A campus bullying early warning method based on video-linked audio, characterized in that: The campus bullying early warning method based on video-linked audio includes: The broadcast control platform controls the audio and video acquisition terminals installed inside the campus to collect audio and video data inside the campus in real time; The audio data and the video data are sent to the broadcast control platform through the wireless communication network, and the broadcast control platform processes the audio data and the video data to obtain the processed audio data and the video data; The video perception platform is used to identify information on the video data and the audio perception platform is used to identify information on the audio data to determine whether a bullying warning is needed.

2. According to claim 1, the campus bullying early warning method based on video-linked audio is characterized in that: The broadcast control platform controls the audio and video acquisition terminals installed inside the campus to collect audio and video data inside the campus in real time, including: The broadcast control platform controls the audio acquisition terminal and the video acquisition terminal installed inside the campus to send a wireless control signal through wireless communication to control the audio acquisition terminal and the video acquisition terminal to collect audio data and video data inside the campus in real time; The broadcast control platform monitors the current time period of the campus environment in real time; The audio collection frequency and the video collection frequency of the audio collection terminal and the video collection terminal are adjusted according to the current time period of the campus environment.

3. According to claim 1, the campus bullying early warning method based on video-linked audio is characterized in that: The audio data and the video data are sent to the broadcast control platform through the wireless communication network, and the broadcast control platform processes the audio data and the video data to obtain the processed audio data and the video data, including: The audio acquisition terminal and the video acquisition terminal send the audio data and the video data to the broadcast control platform via the wireless communication network; The broadcast control platform performs data processing on the audio data and the video data, wherein the data processing includes data noise reduction processing, data cleaning processing and data compression processing; The data processing duration corresponding to the audio data and the video data is monitored in real time, and it is determined whether there is any abnormality in the data processing process of the audio data and the video data by the broadcast control platform according to the data processing duration corresponding to the audio data and the video data.

4. According to claim 3, the campus bullying early warning method based on video-linked audio is characterized in that: Determining whether there is an abnormality in the data processing process of the audio data and the video data by the broadcast control platform according to the data processing time corresponding to the audio data and the video data includes: Real-time monitoring of the data processing duration corresponding to the audio data and the video data; Extracting the spectral density and peak frequency of the audio data after data processing; The first processing evaluation parameter is obtained according to the data processing time corresponding to the audio data combined with the spectrum density and peak frequency of the audio data that has completed data processing; wherein the first processing evaluation parameter is obtained by the following formula: Among them, P 01 represents the first processing evaluation parameter; F peak represents the peak frequency; S(f) represents the spectral density function corresponding to the audio data; f 01 and f 02 represents the frequency upper limit value and the frequency lower limit value corresponding to the frequency distribution range corresponding to the audio data; T represents the first time parameter, and the first time parameter is obtained by the following formula: Wherein, T represents the first time parameter; T peak Indicates the data processing time corresponding to the peak frequency; T p represents the average data processing time corresponding to n audio data processing times; n represents the number of audio data processing times; T i T represents the data processing time corresponding to the i-th audio data processing; z Indicates the median value of the data processing duration corresponding to n audio data processing times; Extract the video resolution and frame rate of the video data that has completed data processing; The second processing evaluation parameter is obtained according to the data processing time corresponding to the video data combined with the video resolution and frame rate of the video data that has completed data processing; wherein the second processing evaluation parameter is obtained by the following formula: Among them, P 02 represents the second processing evaluation parameter; m represents the number of video data processing times; R i represents the video resolution corresponding to the i-th video data processing; R i-1 Indicates the video resolution corresponding to the i-1th video processing; F framei represents the maximum frame rate variation of the video data corresponding to the i-th video data processing; F max Indicates the maximum allowed frame rate variation of the preset video data; T fi represents the actual processing time corresponding to the i-th video data processing; T si T represents the theoretical processing time corresponding to the i-th video data processing; x represents the second time parameter, and the second time parameter is obtained by the following formula: Among them, T x Represents the second time parameter; T frame represents the video data processing time corresponding to the maximum frame rate change amplitude of the video data appearing in the m-times video data processing; T m represents the average video data processing time of m video data processing times; comparing the first processing evaluation parameter with a preset first data processing evaluation threshold; When the first processing evaluation parameter is lower than a preset first data processing evaluation threshold, it is determined that there is an abnormality in the data processing process of the audio data; comparing the second processing evaluation parameter with a preset second data processing evaluation threshold; When the second processing evaluation parameter is lower than a preset second data processing evaluation threshold, it is determined that there is an abnormality in the data processing process of the video data.

5. According to claim 1, the campus bullying early warning method based on video-linked audio is characterized in that: The video perception platform identifies information in the video data and the audio perception platform identifies information in the audio data to determine whether a bullying warning is needed, including: The broadcast control platform sends the processed audio data and video data to the campus broadcast management platform, and sends the video data to the video perception platform through the campus broadcast management platform; The video perception platform analyzes and identifies the video data to determine whether the video data contains a preset calibration behavior; When the video data contains a preset calibration behavior, the audio data is sent to the audio perception platform, and the audio perception platform is used to determine whether there is a calibration word in the audio data; When the audio data contains the marked words, an early warning message is sent to the teacher terminal through the campus broadcast management platform.

6. A campus bullying early warning system based on video-linked audio, characterized in that: The campus bullying early warning system based on video-linked audio includes: The data acquisition control module is used to control the broadcast control platform to collect audio data and video data inside the campus in real time through the audio acquisition terminal and video acquisition terminal installed inside the campus; A data processing module, used to send the audio data and video data to the broadcast control platform through a wireless communication network, and the broadcast control platform performs data processing on the audio data and video data to obtain processed audio data and video data; The recognition and perception module is used to perform information recognition on video data through the video perception platform and to link the audio perception platform to perform information recognition on audio data to determine whether a bullying warning is needed.

7. According to claim 6, the campus bullying early warning system based on video-linked audio is characterized in that: The data acquisition control module comprises: A real-time acquisition control module is used to control the broadcast control platform to send wireless control signals to the audio acquisition terminal and the video acquisition terminal installed inside the campus through wireless communication, so as to control the audio acquisition terminal and the video acquisition terminal to collect audio data and video data inside the campus in real time; The time period monitoring module is used for the broadcast control platform to monitor the current time period of the campus environment in real time; The data acquisition frequency adjustment module is used to adjust the audio acquisition frequency and video acquisition frequency of the audio acquisition terminal and the video acquisition terminal according to the current time period of the campus environment.

8. According to claim 6, the campus bullying early warning system based on video-linked audio is characterized in that: The data processing module comprises: A data sending module, used for the audio acquisition terminal and the video acquisition terminal to send the audio data and the video data to the broadcast control platform via a wireless communication network; A data preprocessing module, used for the broadcast control platform to perform data processing on the audio data and the video data, wherein the data processing includes data noise reduction processing, data cleaning processing and data compression processing; The data processing quality monitoring module is used to monitor the data processing time corresponding to the audio data and video data in real time, and determine whether there is any abnormality in the data processing process of the audio data and video data by the broadcast control platform according to the data processing time corresponding to the audio data and video data.

9. According to claim 8, the campus bullying early warning system based on video-linked audio is characterized in that: The data processing quality monitoring module includes: A real-time monitoring module, used to monitor in real time the data processing time corresponding to the audio data and the video data; A first data extraction module, used to extract the frequency spectrum density and peak frequency of the audio data that has completed data processing; The first processing evaluation parameter acquisition module is used to acquire the first processing evaluation parameter according to the data processing time corresponding to the audio data combined with the spectrum density and peak frequency of the audio data that has completed data processing; wherein the first processing evaluation parameter is acquired by the following formula: Among them, P 01 represents the first processing evaluation parameter; F peak represents the peak frequency; S(f) represents the spectral density function corresponding to the audio data; f 01 and f 02 represents the frequency upper limit value and the frequency lower limit value corresponding to the frequency distribution range corresponding to the audio data; T represents the first time parameter, and the first time parameter is obtained by the following formula: Wherein, T represents the first time parameter; T peak Indicates the data processing time corresponding to the peak frequency; T p represents the average data processing time corresponding to n audio data processing times; n represents the number of audio data processing times; T i T represents the data processing time corresponding to the i-th audio data processing; z Indicates the median value of the data processing duration corresponding to n audio data processing times; A second data extraction module is used to extract the video resolution and frame rate of the video data that has completed data processing; The second processing evaluation parameter acquisition module is used to acquire the second processing evaluation parameter according to the data processing time corresponding to the video data combined with the video resolution and frame rate of the video data that has completed data processing; wherein the second processing evaluation parameter is acquired by the following formula: Among them, P 02 represents the second processing evaluation parameter; m represents the number of video data processing times; R i represents the video resolution corresponding to the i-th video data processing; R i-1 Indicates the video resolution corresponding to the i-1th video processing; F framei represents the maximum frame rate variation of the video data corresponding to the i-th video data processing; F max Indicates the maximum allowed frame rate variation of the preset video data; T fi represents the actual processing time corresponding to the i-th video data processing; T si T represents the theoretical processing time corresponding to the i-th video data processing; x represents the second time parameter, and the second time parameter is obtained by the following formula: Among them, T x Represents the second time parameter; T frame represents the video data processing time corresponding to the maximum frame rate change amplitude of the video data appearing in the m-times video data processing; T m represents the average video data processing time of m video data processing times; A first comparison module, used for comparing the first processing evaluation parameter with a preset first data processing evaluation threshold; A first abnormality judgment module, configured to judge that an abnormality exists in the data processing process of the audio data when the first processing evaluation parameter is lower than a preset first data processing evaluation threshold; A second comparison module, used for comparing the second processing evaluation parameter with a preset second data processing evaluation threshold; The second abnormality judgment module is used to judge that there is an abnormality in the data processing process of the video data when the second processing evaluation parameter is lower than a preset second data processing evaluation threshold.

10. The campus bullying early warning system based on video-linked audio according to claim 6 is characterized in that: The recognition and perception module comprises: A video data transmission module, which is used for the broadcast control platform to send the processed audio data and video data to the campus broadcast management platform, and send the video data to the video perception platform through the campus broadcast management platform; A video calibration behavior determination module is used for the video perception platform to analyze and identify video data and determine whether the video data contains a preset calibration behavior; An audio data transmission module, used for sending the audio data to the audio perception platform when the video data contains a preset calibration behavior, and determining whether there is a calibration word in the audio data through the audio perception platform; The audio calibration word determination module is used to send warning information to the teacher terminal through the campus broadcast management platform when the audio data contains the calibration words.

Images