Audio and video content processing method

Abstract

The invention provides an audio and video content processing method, which relates to the technical field of data processing, and comprises the following steps: dividing a safety monitoring coverage area into Y safety monitoring areas according to the performance of monitoring equipment; relay monitoring architecture configuration is carried out; after the first risk prompt is received, carrying out return node risk situation analysis to obtain a first relay linkage permission; an audio and video sequence of the first safety monitoring area is uploaded to a first risk edge node by adopting a time window rotation scheduling mechanism, risk situation monitoring is carried out, and a first access area is positioned; activating a target relay edge node according to an association relationship between the first access area and the plurality of first relay monitoring areas; and the target relay edge node performs relay monitoring of the risk situation by processing the audio and video sequence. According to the invention, the technical problem of insufficient monitoring continuity and comprehensiveness caused by the lack of seamless relay monitoring capability for the mobile monitoring object due to the independent operation of different monitoring devices in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an audio and video content processing method. Background Art

[0002] Audio and video content processing technology is widely used in many fields such as security monitoring, health monitoring, smart home, etc., especially the safety monitoring of the elderly. Through audio and video processing, the daily activities of the elderly can be monitored to promptly detect emergencies such as falls and sudden medical conditions.

[0003] However, individuals who require continuous monitoring, such as the elderly, may move out of the coverage of the monitoring equipment. Different monitoring devices in different areas of the traditional system often operate independently, lacking coordination between devices, making it difficult to achieve seamless monitoring coverage and resulting in loss of tracking. In addition, in traditional monitoring systems, all data usually needs to be sent to a central server for processing, which may cause significant delays. This is a huge safety hazard for situations that require timely response, such as falls of the elderly. Summary of the invention

[0004] The present application provides an audio and video content processing method, aiming to solve the technical problem that different monitoring devices in the prior art usually operate independently and lack the ability to seamlessly relay monitoring of mobile monitoring objects, resulting in insufficient continuity and comprehensiveness of monitoring.

[0005] The present application discloses an audio and video content processing method, the method comprising: dividing a security monitoring coverage area into Y security monitoring areas according to the performance of monitoring equipment; configuring a relay monitoring architecture for the security monitoring coverage area according to the Y security monitoring areas, wherein the Y security monitoring areas in the relay monitoring architecture are configured with Y dual-mode monitoring devices, and the Y dual-mode monitoring devices are connected to a relay analysis center through Y edge risk processing nodes; when the relay analysis center receives a first risk alert, a return node risk situation analysis is performed on the first risk alert to obtain a first relay linkage right of the first risk edge node Limit, wherein the first relay linkage authority includes monitoring activation authority for multiple first relay monitoring areas; the first dual-mode monitoring device adopts a time window rotation mechanism to upload the audio and video sequence of the first safety monitoring area to the first risk edge node, and monitors the risk situation through the first risk edge node to locate the first access area; the first risk edge node is constrained by the first relay linkage authority, and activates the target relay edge node according to the association between the first access area and multiple first relay monitoring areas; the target relay edge node performs relay monitoring of the risk situation by processing the audio and video sequence sent back by the target dual-mode monitoring device.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: Using edge computing nodes to process data from dual-mode monitoring devices can effectively reduce latency and increase data processing speed. By performing data preprocessing and preliminary analysis at local nodes, the data processing burden of the cloud center is reduced, making the entire system more efficient and responsive. A dual-engine system is adopted, including a video risk detection engine and an audio risk detection engine. The video risk detection engine analyzes visual data and identifies visual abnormalities such as falls and abnormal wandering. The audio risk detection engine processes sound data and identifies abnormal audio anomalies such as cries for help and breaking sounds. Each engine focuses on processing a specific type of data, allowing the system to allocate processing resources more efficiently. source, improving the overall data processing speed and accuracy; the event fusion engine integrates video and audio risk outputs based on timestamp alignment to provide more comprehensive risk analysis results. This fusion ensures that all relevant audio and video data can be integrated when analyzing risks, thereby improving the accuracy of risk assessment; the relay monitoring architecture can seamlessly track the monitored object. When the monitored object leaves the monitoring range of a device, the time window rotation mechanism and risk situation analysis can quickly activate the monitoring of the next related monitoring area, which ensures the continuity of monitoring. This relay monitoring mechanism ensures continuous and comprehensive monitoring coverage and enhances the effectiveness of the monitoring system.

[0007] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A flowchart of an audio and video content processing method is provided for an embodiment of the present application.

[0009] Figure 2 A schematic diagram of a process for obtaining a security monitoring area in an audio and video content processing method is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0010] The embodiment of the present application provides an audio and video content processing method to solve the technical problem that different monitoring devices in the prior art usually operate independently and lack the ability to seamlessly relay monitoring of mobile monitoring objects, resulting in insufficient continuity and comprehensiveness of monitoring.

[0011] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.

[0012] like Figure 1As shown, an embodiment of the present application provides an audio and video content processing method, the method comprising: The security monitoring coverage area is divided into Y security monitoring areas according to the performance of the monitoring equipment.

[0013] According to the model information of the dual-mode monitoring device, the device performance parameters are called, including the audio pickup range and lens focal length parameters. The effective monitoring range of each device is calculated according to the device performance parameters. The predetermined installation parameters, including device spacing, environmental restrictions, building layout, etc., are obtained through the interactive interface or preset configuration files. To ensure that there are no blind spots in monitoring, a certain overlapping coverage area is preset between adjacent devices. Using the overlapping coverage area, the effective monitoring range is adjusted to obtain the standard monitoring range of each device. Based on the standard monitoring range, a partitioning algorithm is used to divide the entire security monitoring coverage area into multiple interconnected sub-areas. Each sub-area can be covered by a monitoring device, and Y security monitoring areas are obtained. The overlapping boundary between each pair of areas is marked as the edge overlapping area, which provides a basis for the subsequent relay monitoring architecture configuration.

[0014] A relay monitoring architecture is configured for the security monitoring coverage area according to the Y security monitoring areas, wherein in the relay monitoring architecture, the Y security monitoring areas are configured with Y dual-mode monitoring devices, and the Y dual-mode monitoring devices are communicatively connected to the relay analysis center via Y edge risk processing nodes.

[0015] For each security monitoring area, a corresponding dual-mode monitoring device is configured. The dual-mode monitoring device has the function of collecting and recording two modal information of audio and video monitoring. According to the pre-configured template of the standard risk processing node, Y edge risk processing nodes are copied and generated. Each node is independently responsible for data processing and analysis of a monitoring area. A dual-mode communication link is configured for each edge risk processing node. The dual-mode communication link supports real-time transmission of audio and video data, as well as two-way communication functions. Then, all edge risk processing nodes are connected to the relay analysis center to complete the configuration of the relay monitoring architecture.

[0016] When the relay analysis center receives the first risk alert, it performs a feedback node risk situation analysis on the first risk alert and obtains the first relay linkage authority of the first risk edge node, wherein the first relay linkage authority includes monitoring activation authority of multiple first relay monitoring areas.

[0017] The first risk alert is generated by a specific edge node and sent to the relay analysis center, including potential security threats or abnormal activities detected in the monitoring area of ​​the node. According to the received first risk alert, the relay analysis center quickly locates the edge node of the risk source, that is, the first risk edge node, extracts the first audio and video analysis results contained in the first risk alert, conducts risk development trend analysis, and obtains the first risk situation.

[0018] The first relay linkage authority refers to the authority to authorize the first risk edge node to start equipment in other monitoring areas, which are adjacent to the location of the risk event, in order to strengthen monitoring and response measures. Specifically, from the predefined Y groups of edge overlapping identifiers, the first group of edge overlapping identifiers related to the first risk edge node is extracted, and according to the first risk situation and the first group of edge overlapping identifiers, multiple first relay monitoring areas, i.e., adjacent areas affected by the risk situation, are located. According to the multiple first relay monitoring areas, the first relay linkage authority is configured to the first risk edge node, including allowing the first risk edge node to trigger the dual-mode monitoring equipment in the adjacent area to enter an activated state, etc., so as to respond to and control potential security threats.

[0019] The first dual-mode monitoring device adopts a time window rotation mechanism to upload the audio and video sequences of the first security monitoring area to the first risk edge node, monitors the risk situation through the first risk edge node, and locates the first access area.

[0020] The time window rotation mechanism is a data transmission scheduling strategy that aims to effectively manage the data traffic uploaded by multiple monitoring devices to optimize network resources and data processing performance. This mechanism can ensure that each device has fair data transmission time and bandwidth while reducing data congestion. In the specific implementation, each dual-mode monitoring device is assigned a specific time window, within which the device can upload its captured audio and video sequences to the corresponding edge risk processing node.

[0021] The first dual-mode monitoring device captures the audio and video data of the monitored area in real time. Within the allocated time window, the device uploads the audio and video data to the first risk edge node. After receiving the audio and video data, the first risk edge node conducts risk situation analysis, including motion detection, sound anomaly detection, visual recognition, etc., to identify abnormal behavior or potential security threats. Based on the risk situation analysis results, the first access area is determined and marked, that is, the area that requires special attention or enhanced monitoring.

[0022] The first risk edge node uses the first relay linkage authority as a constraint and activates a target relay edge node according to an association relationship between the first access area and a plurality of first relay monitoring areas.

[0023] Based on the first relay linkage authority granted, the first risk edge node can activate other nodes and monitoring devices to perform specific security monitoring and response measures, analyze the geographical and logical relationship between the first access area and other relay monitoring areas, determine which areas may be related to or may be affected by the current risk event, and determine the target relay edge node that needs to be activated to respond to possible risk spread based on the relationship between the first access area and other monitoring areas, and send an activation signal to the selected target relay edge node to enable the node to adjust its monitoring settings or strengthen monitoring of specific areas.

[0024] The target relay edge node performs relay monitoring of risk situations by processing the audio and video sequences returned by the target dual-mode monitoring device.

[0025] The target dual-mode monitoring device transmits the captured audio and video data to the corresponding target relay edge node. The target relay edge node performs risk analysis based on the audio and video data to identify any abnormal behavior or potential threats. The analysis process is similar to that of the first risk edge node. According to the detection results, the risk situation of the target relay monitoring area is generated. The target relay edge node transmits the generated risk situation and regional monitoring data back to the relay analysis center. The analysis center further optimizes the risk monitoring layout of the entire system based on the relay results. If the risk expands further, the relay analysis center can activate more relay edge nodes to form a dynamic joint defense system.

[0026] Through such processing and monitoring mechanisms, the relay monitoring architecture ensures that the monitoring system can respond to security threats flexibly and quickly, and maintain efficient monitoring of key areas through effective data processing and risk management strategies.

[0027] Furthermore, if Figure 2 As shown, the security monitoring coverage area is divided into Y security monitoring areas according to the performance of the monitoring equipment, and the method includes: Interactively obtain predetermined installation parameters; perform network data call according to the model information of the dual-mode monitoring device to obtain the performance of the monitoring device, wherein the monitoring device performance includes audio pickup range and lens focal length parameters; solve the monitoring range according to the predetermined installation parameters and the monitoring device performance to obtain an effective monitoring range; predefine the overlapping area size, and limit the effective monitoring range according to the overlapping area size to obtain a standard monitoring range; with the standard monitoring range as a constraint, divide the security monitoring coverage area into the Y security monitoring areas, and perform overlapping identification according to the connection relationship of the Y security monitoring areas to obtain Y groups of edge overlapping identifications.

[0028] By interacting with the customer or project management team, obtain the predetermined installation parameters, including equipment spacing, environmental restrictions, building layout, etc. to ensure that the equipment can cover the expected area.

[0029] Through the model information of the dual-mode monitoring device, access the network database and query the performance parameters of the monitoring device, including the audio pickup range and lens focal length parameters. The audio pickup range indicates the radius or direction angle that the device's microphone can cover, and the lens focal length parameter determines the video monitoring range, including the field of view angle, the farthest monitoring distance, the resolution range, etc.

[0030] The monitoring range is calculated through mathematical modeling by combining the audio pickup range, lens focal length and predetermined installation parameters. Specifically, the audio coverage range of each device is generated based on the device's pickup radius and installation position. The field of view and monitoring distance of the video surveillance are calculated based on the lens' focal length and installation height. Combined with the audio and video range, the total monitoring coverage of the device, i.e. the effective monitoring range, is generated.

[0031] The overlapping area is used to avoid monitoring blind spots, ensure that key locations or personnel are always within the monitoring range of the equipment, and provide seamless connection between areas to enhance the continuity of event detection. The overlapping area size is predefined and defined according to the characteristics of the site and monitoring requirements. For example, it can be set to a certain proportion of the equipment coverage, such as 10% to 30% of the audio and video coverage. On the basis of the original effective monitoring range, the overlapping part with the coverage area of ​​other equipment is subtracted to form the standard monitoring range of the equipment.

[0032] According to the standard monitoring range, the entire security monitoring coverage area is divided into Y overlapping sub-areas. For example, based on the standard monitoring range of the equipment, the security monitoring coverage area is divided into multiple equidistant grids, each grid is a security monitoring area, and each sub-area is consistent with the standard monitoring range of a device. The monitoring equipment in each sub-area can independently complete risk detection. The value of Y is determined according to the ratio of the area of ​​the security monitoring coverage area to the standard monitoring range.

[0033] For Y security monitoring areas, record the boundary coordinates of adjacent areas and mark them as shared areas, generate Y groups of edge overlap marks, which are used to record the shared area information between sub-areas. The goal is to ensure that the elderly are always within the monitoring range of at least one device no matter where they move to, such as corridors, rooms, courtyards, etc. Furthermore, before configuring the relay monitoring architecture for the security monitoring coverage area according to the Y security monitoring areas, the method includes: Interactively obtain multiple sample behavior image sets of multiple sample risk behaviors; use the multiple sample behavior image sets to construct multiple risk behavior recognition models; pre-assign multiple risk detection tasks to the multiple sample risk behaviors, and set multiple task priorities of the multiple risk detection tasks according to the historical occurrence frequency of the multiple sample risk behaviors; pre-build a video risk detection engine, wherein the engine internal module division of the video risk detection engine includes a cascaded video stream processing module, a task management module and a model management module; load the multiple risk detection tasks into the task management module, and load the multiple risk behavior recognition models into the model management module with the multiple task priorities as constraints to complete the localization of the video risk detection engine; and so on, pre-build an audio risk detection engine and localize the audio risk detection engine; introduce an event fusion engine; connect the video risk detection engine and the audio risk detection engine in parallel, configure the event fusion engine at the output end of the video risk detection engine and the audio risk detection engine, and complete the construction of a standard risk processing node.

[0034] Risky behaviors refer to behaviors that may lead to danger or reflect emergency situations, especially abnormal behaviors that are prone to occur in the elderly, including wandering, unstable center of gravity, falling, stumbling, etc. Through historical monitoring data, public data sets or cooperative institutions, we obtain risk behavior image sets for each of the above risky behaviors to form multiple sample behavior image sets.

[0035] For each sample risk behavior, a special risk behavior identification model is constructed. Specifically, the model architecture is built based on the convolutional neural network, and the preprocessed sample behavior image set is input into the model. The loss function is optimized during the training process, and the model performance is evaluated through cross-validation and the hyperparameters are optimized. Finally, multiple risk behavior identification models that meet the accuracy requirements are obtained. Each model corresponds to a risk behavior, such as a fall detection model, a wandering detection model, etc.

[0036] For each sample risk behavior, corresponding risk detection tasks are pre-assigned, including wandering detection, unstable center of gravity detection, fall detection, etc. Each task corresponds to a specific behavior recognition model and risk behavior. Task priority is assigned according to the historical frequency of each risk behavior. For example, if the frequency of falling is high and the risk is high, the priority is the highest; if the frequency of wandering is medium and the risk is low, the priority is medium. Task priority determines the allocation of computing resources.

[0037] The pre-built video risk detection engine is used to process the video streams uploaded by the monitoring equipment in real time and identify potential risk behaviors in the video. The internal module division of the video risk detection engine includes a cascaded video stream processing module, a task management module, and a model management module. The video stream processing module is used to receive the video stream and perform pre-processing, such as denoising and downsampling; the task management module is used to allocate and schedule video risk detection tasks, for example, based on task priority, system resources are allocated to high-priority tasks; the model management module is used to store and manage a variety of risk behavior recognition models, and call the corresponding model according to task requirements.

[0038] All predefined risk detection tasks (such as wandering detection and fall detection) are loaded into the task management module, and multiple risk behavior recognition models are loaded into the model management module. The priority constraint mechanism is adopted to make high-priority tasks occupy a higher weight in the task queue. In this way, the task management module loads the models corresponding to key tasks into the memory in priority order to ensure fast response. The localization of the video risk detection engine is completed to reduce the dependence on cloud resources.

[0039] Using the same approach, for audio risk events such as screaming, falling, breaking glass, etc., an audio risk detection engine is pre-built and localized to allocate and manage audio risk detection tasks.

[0040] An event fusion engine is introduced to fuse the outputs of the video risk detection engine and the audio risk detection engine to generate more accurate and comprehensive risk analysis results.

[0041] The video risk detection engine and the audio risk detection engine are connected in parallel. That is, the two detection engines run independently and process the real-time video stream and audio stream respectively. The event fusion engine is configured at the output end of the two detection engines. In this way, the outputs of the two are uniformly processed by the event fusion engine to generate a comprehensive event assessment result and enhance the accuracy of detection. For example, when an elderly person falls, the video engine detects the falling action and the audio engine detects the collision sound. The combination of the two generates a high-confidence risk event.

[0042] Furthermore, the method configures a relay monitoring architecture for the security monitoring coverage area according to the Y security monitoring areas, and includes: The Y dual-mode monitoring devices are installed and configured in the Y security monitoring areas according to the predetermined installation parameters; the standard risk processing node is copied to obtain the Y edge risk processing nodes; after building a dual-mode communication link between the Y edge risk processing nodes and the Y dual-mode monitoring devices, the Y edge risk processing nodes are communicatively connected to the relay analysis center to complete the configuration of the relay monitoring architecture.

[0043] According to the predetermined installation parameters, corresponding dual-mode monitoring devices are installed in each security monitoring area. These parameters ensure that each device can effectively cover its designated monitoring area. Each device has video and audio monitoring functions, can capture image and sound data in real time, and provide a comprehensive monitoring perspective.

[0044] The pre-configured standard risk processing nodes are copied to create Y edge risk processing nodes so that they can process the data received from the corresponding dual-mode monitoring devices. Each edge node is responsible for preliminary data processing and analysis, reducing the load on the central server and increasing the data processing speed.

[0045] Build a dual-mode communication link between each edge risk processing node and its corresponding dual-mode monitoring device, where audio data enters the audio risk detection engine and video data enters the video risk detection engine, ensuring that data can be transmitted in real time and securely. Connect all edge risk processing nodes to the central relay analysis center to ensure that the center can receive data from all nodes for further analysis. Through the above operations, complete the configuration of the relay monitoring architecture, so that the system can monitor in real time and quickly respond to various security incidents.

[0046] Furthermore, the method comprises: In the video risk detection engine of the first risk edge node, the video stream processing module receives a first real-time video stream from the first dual-mode monitoring device; the video stream processing module pre-processes the first real-time video stream, obtains a first video frame sequence, and then passes the first video frame sequence to the task management module; the task management module assigns a detection task to the first video frame sequence according to the multiple task priorities to obtain a first task identification frame sequence; the model management module performs multi-task parallel scheduling of the multiple risk behavior recognition models according to the first task identification frame sequence to execute the multiple risk detection tasks and obtain multiple video risk outputs; and so on, the first real-time audio stream returned by the first dual-mode monitoring device is received and analyzed by the audio risk detection engine to obtain multiple audio risk outputs; the event fusion engine receives and aligns the multiple video risk outputs and the multiple audio risk outputs based on the timestamp to obtain a first audio and video analysis result; if the first audio and video analysis result is a non-empty set, the first audio and video analysis result is packaged to obtain the first risk reminder, and the first risk reminder is sent to the relay analysis center.

[0047] In the video risk detection engine of the first risk edge node, the video stream processing module receives the collected first real-time video stream from the first dual-mode monitoring device of the first risk edge node to maintain the real-time nature of monitoring.

[0048] The video stream processing module preprocesses the first real-time video stream, including decoding, format conversion, noise reduction and other processing of the video stream to facilitate subsequent video analysis. Through preprocessing, the first video frame sequence is obtained and passed to the task management module, which is responsible for task management based on the video data.

[0049] The task management module allocates analysis tasks to the first video frame sequence according to multiple preset task priorities. These priorities reflect the importance and urgency of various monitoring tasks and determine the allocation of computing resources. For example, high-priority tasks (such as fall detection tasks) will obtain more computing resources to quickly and accurately process these tasks. Each frame obtains a corresponding identifier according to the assigned detection task. This identifier determines in which analysis model each frame should be processed, and obtains the first task identification frame sequence. Through a refined processing and allocation mechanism, video data is effectively converted into information that can be used for risk analysis, optimizing the processing efficiency of data streams.

[0050] The model management module is used to schedule multiple risk behavior recognition models. These models work according to the received task identification frame sequence. Multi-task parallel scheduling uses multi-threading or asynchronous processing to allow parallel detection of different tasks. Among them, high-priority tasks will obtain more computing resources to ensure rapid response, and low-priority tasks will be scheduled at preset intervals to effectively utilize system resources. After each task is processed, the model outputs the corresponding risk analysis results, which are used for subsequent security response. Similar to video stream processing, the audio risk detection engine negatively analyzes the audio stream sent back from the first dual-mode monitoring device, such as the sound of falling, breaking windows, cries for help, etc. The audio stream is also analyzed through a parallel processing mechanism according to task priority, so as to monitor the possible sound-related risks in the environment in real time. Similar to video risk output, audio risk output includes various identified sound anomalies and alarms, and this information is used for comprehensive risk assessment.

[0051] The event fusion engine receives outputs from the video and audio risk detection engines. These outputs are timestamped. The engine aligns these outputs according to the timestamps to obtain the first audio and video analysis results. This analysis result integrates the abnormal indicators of video and audio, providing security personnel with a more comprehensive risk assessment and improving the accuracy and reliability of event identification.

[0052] If the first audio and video analysis result is not empty, that is, certain risks or abnormalities are detected, these results are packaged into a first risk alert, and the generated first risk alert is sent to the relay analysis center for further response measures, such as adjusting the monitoring strategy, initiating emergency response, etc., thereby improving the efficiency and effectiveness of security monitoring.

[0053] Further, when the relay analysis center receives the first risk alert, it performs a risk situation analysis of the first risk alert to obtain the first relay linkage authority of the first risk edge node. The method includes: When the relay analysis center receives the first risk alert, it identifies the alert return node of the first risk alert and locates the first risk edge node; performs risk development trend analysis based on the first audio and video analysis results packaged in the first risk alert to obtain a first risk situation; extracts a first group of edge overlapping identifiers from the Y groups of edge overlapping identifiers according to the first risk edge node; locates multiple first relay monitoring areas according to the first risk situation and the first group of edge overlapping identifiers, and configures the first relay linkage authority to the first risk edge node according to the multiple first relay monitoring areas.

[0054] When the relay analysis center receives the first risk alert, it first identifies which edge node the alert was sent from and locates the first risk edge node. Since each edge node corresponds to a specific monitoring area, identifying the specific edge node helps to quickly locate the geographical location of the risk event.

[0055] The first audio and video analysis results contained in the first risk alert provide preliminary risk data for the relay analysis center. This data includes multiple identified video risk outputs. Risk development trend analysis is performed based on multiple video risk outputs. Possible risk diffusion paths and potential impacts are analyzed. The direction of risk development is predicted through behavioral pattern recognition, trend analysis, etc. For example, which area the user may enter. Based on the assessment of risk development, the first risk situation is obtained.

[0056] The edge overlap identifier is used to mark the overlapping parts between monitoring areas. The first risk edge node extracts a group of overlapping identifiers related to the monitoring area corresponding to the current node from the preset Y groups of edge overlap identifiers as the first group of edge overlap identifiers, which are used to identify other monitoring areas that are geographically adjacent to or logically related to the current risk event.

[0057] According to the first risk situation and the first group of edge overlapping identifiers, determine which monitoring areas are associated with or affected by the first risk event. These areas are used as multiple first relay monitoring areas. The first relay linkage authority is configured for the first risk edge node. The first relay linkage authority allows the first risk edge node to start or strengthen monitoring of the first relay monitoring area when necessary. Such configuration ensures that the monitoring system can quickly expand its monitoring scope when preliminary risks are detected, and realize relay monitoring of potential risk areas.

[0058] Further, performing risk development situation analysis according to the first audio and video analysis result packaged in the first risk reminder to obtain a first risk situation, the method includes: The multiple video risk outputs are extracted from the first audio and video analysis results; the multiple video risk outputs are spatially identified to obtain multiple risk vectors; and the multiple risk vectors are spatially aggregated to obtain the first risk situation.

[0059] Multiple video risk outputs are extracted from the first audio and video analysis results, and these video risk outputs provide basic data for subsequent detailed analysis.

[0060] The risk vector is a mathematical representation used to describe the location, direction, and intensity of a risk event. First, potential risk events are identified from multiple video risk outputs, and the identified risk events are converted into spatial vectors, which contain the location (x, y coordinates), occurrence time (timestamp), and possible risk level of the event, to obtain multiple risk vectors.

[0061] Spatial aggregation can be used to understand the spatial distribution and association of risk events, as well as their possible interactions or development trends. Multiple risk vectors can be spatially aggregated. For example, the location of each risk event can be connected according to the timestamp to analyze how these risk events are distributed and interact with each other in the entire monitoring area. Through aggregation analysis, potential risk hotspots and risk development trends can be identified to obtain the first risk situation, thereby predicting the areas that users may enter, and activating the edge nodes in the corresponding areas accordingly, thereby enhancing the monitoring and security measures in the corresponding areas.

[0062] Further, the first dual-mode monitoring device adopts a time window rotation mechanism to upload the audio and video sequence of the first security monitoring area to the first risk edge node, monitors the risk situation through the first risk edge node, and locates the first access area. The method includes: A plurality of access vectors are constructed according to the positional relationship between the plurality of first relay monitoring areas and the first safety monitoring area; the first risk edge node adopts a time window rotation mechanism to perform risk situation analysis on the audio and video sequence uploaded by the first dual-mode monitoring device to obtain a risk direction sequence; a displacement trajectory is fitted according to the risk direction sequence, and a target displacement direction is determined according to the fitting result; the target displacement direction is used to traverse and match the plurality of access vectors, and the first access area is located in the plurality of first relay monitoring areas.

[0063] The access vector is a mathematical tool used to represent the spatial relationship between monitoring areas. These vectors are constructed based on the geographical location and logical relationship between the first relay monitoring area and the first safety monitoring area. First, the specific location and boundaries of each monitoring area are determined. Based on these location data, the direction vectors from the first safety monitoring area to each first relay monitoring area are calculated. These vectors reflect the proximity and accessibility between the areas.

[0064] The time window rotation mechanism is a data transmission scheduling strategy that aims to effectively manage the data traffic uploaded by multiple monitoring devices to optimize network resources and data processing performance. Under this mechanism, the first dual-mode monitoring device uploads audio and video data to the first risk edge node within the specified time window. After receiving the audio and video data, the first risk edge node begins to analyze the risk situation, including identifying abnormal activities and the direction of risk development, and organizes the identification results into a risk direction sequence, which reflects how the risk moves or changes in the monitoring area.

[0065] Necessary data is extracted from the risk direction sequence, including the location coordinates of the risk event, and the displacement trajectory is fitted according to the location coordinates. Machine learning algorithms, such as support vector machines and Markov models, can be used to predict future movement trajectories based on the location coordinate data. The purpose is to determine the movement path of the risk source, analyze the fitted movement trajectory, obtain the movement pattern and possible behavior trend of the risk source, and determine the target displacement direction.

[0066] Use the target displacement direction to match multiple access vectors to find the relay monitoring area that is most consistent with the movement direction of the risk source. This area will become the focus of response and may require increased monitoring intensity or specific security measures. In this way, not only can the current risk situation be tracked, but also possible future security incidents can be predicted and prepared for, thereby deploying necessary security resources and response measures in advance. This forward-looking security management can significantly improve the efficiency and effectiveness of handling security incidents.

[0067] In summary, the audio and video content processing method provided in the embodiment of the present application has the following technical effects: Using edge computing nodes to process data from dual-mode monitoring devices can effectively reduce latency and increase data processing speed. By performing data preprocessing and preliminary analysis at local nodes, the data processing burden of the cloud center is reduced, making the entire system more efficient and responsive. A dual-engine system is adopted, including a video risk detection engine and an audio risk detection engine. The video risk detection engine analyzes visual data and identifies visual abnormalities such as falls and abnormal wandering. The audio risk detection engine processes sound data and identifies abnormal audio anomalies such as cries for help and breaking sounds. Each engine focuses on processing a specific type of data, allowing the system to allocate processing resources more efficiently. source, improving the overall data processing speed and accuracy; the event fusion engine integrates video and audio risk outputs based on timestamp alignment to provide more comprehensive risk analysis results. This fusion ensures that all relevant audio and video data can be integrated when analyzing risks, thereby improving the accuracy of risk assessment; the relay monitoring architecture can seamlessly track the monitored object. When the monitored object leaves the monitoring range of a device, the time window rotation mechanism and risk situation analysis can quickly activate the monitoring of the next related monitoring area, which ensures the continuity of monitoring. This relay monitoring mechanism ensures continuous and comprehensive monitoring coverage and enhances the effectiveness of the monitoring system.

[0068] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing audio and video content, characterized in that: The method comprises: Divide the security monitoring coverage area into Y security monitoring areas according to the performance of the monitoring equipment; A relay monitoring architecture is configured for the security monitoring coverage area according to the Y security monitoring areas, wherein the Y security monitoring areas in the relay monitoring architecture are configured with Y dual-mode monitoring devices, and the Y dual-mode monitoring devices are communicatively connected to the relay analysis center via Y edge risk processing nodes; When the relay analysis center receives the first risk alert, it performs a backhaul node risk situation analysis on the first risk alert to obtain a first relay linkage authority of the first risk edge node, wherein the first relay linkage authority includes monitoring activation authorities of multiple first relay monitoring areas; The first dual-mode monitoring device uses a time window rotation mechanism to upload the audio and video sequence of the first security monitoring area to the first risk edge node, monitors the risk situation through the first risk edge node, and locates the first access area; The first risk edge node uses the first relay linkage authority as a constraint and activates a target relay edge node according to the association between the first access area and a plurality of first relay monitoring areas; The target relay edge node performs relay monitoring of risk situations by processing the audio and video sequences returned by the target dual-mode monitoring device.

2. The method for processing audio and video content according to claim 1, characterized in that: Dividing the security monitoring coverage area into Y security monitoring areas according to the performance of the monitoring equipment, the method comprising: interactively obtain predetermined installation parameters; Performing online data call according to the model information of the dual-mode monitoring device to obtain the performance of the monitoring device, wherein the performance of the monitoring device includes audio pickup range and lens focal length parameters; Solving the monitoring range according to the predetermined installation parameters and the performance of the monitoring equipment to obtain an effective monitoring range; Predefine the size of the overlapping area, and narrow the effective monitoring range according to the size of the overlapping area to obtain a standard monitoring range; After dividing the safety monitoring coverage area into the Y safety monitoring areas based on the standard monitoring range as a constraint, overlapping marks are performed according to the connection relationship of the Y safety monitoring areas to obtain Y groups of edge overlapping marks.

3. The method for processing audio and video content according to claim 2, characterized in that: Before configuring the relay monitoring architecture of the security monitoring coverage area according to the Y security monitoring areas, the method includes: Interactively obtain multiple sample behavior image sets of multiple sample risk behaviors; Using the multiple sample behavior image sets to construct multiple risk behavior recognition models; Pre-allocating a plurality of risk detection tasks for the plurality of sample risk behaviors, and setting a plurality of task priorities for the plurality of risk detection tasks according to the historical occurrence frequencies of the plurality of sample risk behaviors; Pre-constructing a video risk detection engine, wherein the internal module division of the video risk detection engine includes a cascaded video stream processing module, a task management module and a model management module; Loading the multiple risk detection tasks into the task management module, loading the multiple risk behavior recognition models into the model management module with the multiple task priorities as constraints, and completing the localization of the video risk detection engine; By analogy, an audio risk detection engine is pre-built and localized; Introducing event fusion engine; The video risk detection engine and the audio risk detection engine are connected in parallel, and the event fusion engine is configured at the output ends of the video risk detection engine and the audio risk detection engine to complete the construction of the standard risk processing node.

4. The method for processing audio and video content according to claim 3, characterized in that: The relay monitoring architecture is configured for the security monitoring coverage area according to the Y security monitoring areas, and the method includes: Performing installation and configuration of the Y dual-mode monitoring devices in the Y security monitoring areas according to the predetermined installation parameters; Copying the standard risk processing node to obtain the Y edge risk processing nodes; After building the dual-mode communication links between the Y edge risk processing nodes and the Y dual-mode monitoring devices, the Y edge risk processing nodes are communicatively connected to the relay analysis center to complete the configuration of the relay monitoring architecture.

5. The method for processing audio and video content according to claim 4, characterized in that: The method comprises: In the video risk detection engine of the first risk edge node, the video stream processing module receives a first real-time video stream from the first dual-mode monitoring device; The video stream processing module pre-processes the first real-time video stream, obtains a first video frame sequence, and then transmits the first video frame sequence to the task management module; The task management module performs detection task assignment identification on the first video frame sequence according to the multiple task priorities to obtain a first task identification frame sequence; The model management module performs multi-task parallel scheduling of the plurality of risk behavior recognition models according to the first task identification frame sequence to execute the plurality of risk detection tasks and obtain a plurality of video risk outputs; By analogy, the audio risk detection engine receives and analyzes the first real-time audio stream returned by the first dual-mode monitoring device to obtain multiple audio risk outputs; The event fusion engine receives and aligns the multiple video risk outputs and the multiple audio risk outputs based on timestamps to obtain a first audio and video analysis result; If the first audio and video analysis result is a non-empty set, the first audio and video analysis result is packaged to obtain the first risk alert, and the first risk alert is sent to the relay analysis center.

6. The method for processing audio and video content according to claim 5, characterized in that: When the relay analysis center receives the first risk alert, it performs a return node risk situation analysis on the first risk alert to obtain a first relay linkage authority of the first risk edge node. The method includes: After receiving the first risk reminder, the relay analysis center identifies the reminder return node for the first risk reminder and locates the first risk edge node; Perform risk development situation analysis according to the first audio and video analysis result packaged in the first risk reminder to obtain a first risk situation; extracting a first group of edge overlapping identifiers from the Y groups of edge overlapping identifiers according to the first risk edge node; A plurality of first relay monitoring areas are located according to the first risk situation and the first group of edge overlap identifiers, and the first relay linkage authority is configured for the first risk edge node according to the plurality of first relay monitoring areas.

7. The method for processing audio and video content according to claim 6, characterized in that: Performing risk development situation analysis according to the first audio and video analysis result packaged in the first risk reminder to obtain a first risk situation, the method comprising: Extracting and obtaining the plurality of video risk outputs from the first audio and video analysis results; Performing spatial vector identification on the multiple video risk outputs to obtain multiple risk vectors; The multiple risk vectors are spatially aggregated to obtain the first risk situation.

8. The method for processing audio and video content according to claim 6, characterized in that: The first dual-mode monitoring device uploads the audio and video sequence of the first security monitoring area to the first risk edge node using a time window rotation mechanism, performs risk situation monitoring through the first risk edge node, and locates the first access area. The method includes: constructing a plurality of admission vectors according to the positional relationship between the plurality of first relay monitoring areas and the first safety monitoring area; The first risk edge node uses a time window rotation mechanism to perform risk situation analysis on the audio and video sequence uploaded by the first dual-mode monitoring device to obtain a risk direction sequence; Fitting a displacement trajectory according to the risk direction sequence, and determining a target displacement direction according to the fitting result; The target displacement direction is used to traverse and match the multiple access vectors, and the first access area is located in the multiple first relay monitoring areas.

Images