A digital video monitoring method

Abstract

The present invention relates to the field of digital monitoring technology, and discloses a digital video monitoring method, including: digitally processing video content, identifying a real-time digital video stream corresponding to the digital video content; identifying the picture content, sound content and video flow corresponding to the real-time digital video stream, and detecting the video behavior data of the real-time digital video stream according to the picture content and sound content; identifying an abnormal traffic event corresponding to the video traffic according to an abnormal traffic triggering event, and identifying an abnormal behavior event corresponding to the video behavior data according to an abnormal behavior triggering event; generating an abnormal data stream timing sequence corresponding to the real-time digital video stream according to an abnormal traffic time point corresponding to the abnormal traffic event and an abnormal behavior time point corresponding to the abnormal behavior event; generating a two-dimensional monitoring fusion graph of the real-time digital video stream through the abnormal data stream timing sequence, and analyzing the monitoring state of the real-time digital video stream according to the two-dimensional monitoring fusion graph. The present invention can improve the accuracy of digital video monitoring.

Description

Technical Field

[0001] The present invention relates to the field of digital monitoring technology, and in particular to a digital video monitoring method. Background Art

[0002] With the development of urbanization, increasing public safety needs and technological progress, remote monitoring has become more feasible and efficient, and the demand for digital video surveillance has also continued to grow, thus promoting the widespread application of digital video surveillance. However, in order to improve the accuracy of digital video surveillance, it is necessary to monitor abnormal conditions in the video from multiple aspects, so as to achieve accurate monitoring of digital video.

[0003] Existing digital video surveillance technology is to identify abnormal data in the video by identifying sudden changes in the scene. In practical applications, only identifying abnormal video content through a single scene change may lead to an overly one-sided recognition of abnormal states of digital video surveillance, resulting in low accuracy when conducting digital video surveillance. Summary of the invention

[0004] The present invention provides a digital video monitoring method, the main purpose of which is to solve the problem of low accuracy when performing digital video monitoring.

[0005] To achieve the above object, the present invention provides a digital video monitoring method, comprising:

[0006] Acquire video content in real time, perform digital processing on the video content to obtain digital video content, and identify the real-time digital video stream corresponding to the digital video content;

[0007] Identify the picture content and sound content corresponding to the real-time digital video stream, detect the video flow of the real-time digital video stream, and detect the video behavior data of the real-time digital video stream according to the picture content and the sound content;

[0008] Identifying an abnormal traffic event corresponding to the video traffic according to a preset abnormal traffic triggering event, including: generating a traffic time sequence corresponding to the video traffic according to a preset timestamp; calculating a traffic factor corresponding to the video traffic according to each video traffic in the traffic time sequence and a traffic mean in the traffic time sequence, wherein the traffic factor calculation formula is:

[0009]

[0010] Among them, G r is the flow factor corresponding to the rth real-time digital video stream, f r is the video flow corresponding to the rth real-time digital video stream, F is the flow mean, Q is a constant, and δ is the flow standard deviation in the flow time series;

[0011] When the flow factor is a preset first factor value, extract the video flow corresponding to the flow factor, and identify the video event corresponding to the video flow; match the video event with an event in a preset abnormal flow trigger event to obtain an abnormal flow event; identify the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event;

[0012] Generate an abnormal data stream timing sequence corresponding to the real-time digital video stream according to the abnormal traffic time point corresponding to the abnormal traffic event and the abnormal behavior time point corresponding to the abnormal behavior event;

[0013] A two-dimensional monitoring fusion graph of the real-time digital video stream is generated through the abnormal data stream time sequence, and the monitoring status of the real-time digital video stream is analyzed according to the two-dimensional monitoring fusion graph.

[0014] Optionally, the digitalizing the video content to obtain digital video content includes:

[0015] Converting an original video signal corresponding to the video content into a digital signal;

[0016] Performing signal enhancement processing on the digital signal to obtain a digital enhanced signal;

[0017] The digital enhanced signal is further compressed to obtain digital video content.

[0018] Optionally, the identifying the real-time digital video stream corresponding to the digital video content includes:

[0019] Decomposing the digital video content into video frames according to a preset frame rate to obtain digital frame images;

[0020] Generate a frame sequence corresponding to the digital frame image according to a preset timestamp;

[0021] The frame sequence is determined as the real-time digital video stream.

[0022] Optionally, the identifying the picture content and the sound content corresponding to the real-time digital video stream includes:

[0023] Identify image data corresponding to each real-time digital video stream, and extract picture events corresponding to the image data;

[0024] Generate the picture content corresponding to each real-time digital video stream according to the picture event;

[0025] Extracting audio features corresponding to each real-time digital video stream, and determining sound events and sound emotions corresponding to each real-time digital video stream according to the audio features;

[0026] The sound content corresponding to each real-time digital video stream is generated according to the sound event and the sound emotion.

[0027] Optionally, the detecting the video behavior data of the real-time digital video stream according to the picture content and the sound content includes:

[0028] Extracting a stream sequence number corresponding to the real-time digital video stream;

[0029] Generate a first event index of a picture event corresponding to the picture content according to the stream sequence number;

[0030] Generate a second event index of the sound event corresponding to the sound content according to the stream sequence number;

[0031] Calculate the correlation between the image event and the sound event according to the index identifiers corresponding to the first event index and the second event index:

[0032]

[0033] Where S is the correlation degree, w k is the association weight corresponding to the k-th index pair, v ik is the screen event vector corresponding to the i-th index identifier in the k-th index pair, a jk is the sound event vector corresponding to the jth index in the kth index pair, where n is the number of index pairs;

[0034] The image event and the sound event are merged into video behavior data corresponding to the real-time digital video stream according to the correlation degree.

[0035] Optionally, matching the video event with an event in a preset abnormal traffic triggering event to obtain an abnormal traffic event includes:

[0036] Extracting abnormal event category data from preset abnormal traffic triggering events;

[0037] Classifying the video events to obtain video event category data;

[0038] Performing vector conversion on the abnormal event category data to obtain an abnormal event category vector;

[0039] Performing vector conversion on the video event category data to obtain a video event category vector;

[0040] Calculating the matching degree between the abnormal event category vector and the video event category vector;

[0041] The abnormal traffic trigger event with the highest matching degree is selected as the abnormal traffic event.

[0042] Optionally, the identifying the abnormal behavior event corresponding to the video behavior data according to a preset abnormal behavior triggering event includes:

[0043] Extracting the behavior features of each real-time digital video stream in the video behavior data, and generating the behavior time sequence corresponding to the video behavior data according to the preset event stamp and the behavior features;

[0044] The behavior factor corresponding to the video behavior data is calculated according to each behavior feature in the behavior time sequence and the standard deviation of the behavior feature in the behavior time sequence, wherein the behavior factor calculation formula is:

[0045]

[0046] Among them, H r is the behavior factor corresponding to the rth real-time digital video stream, σ rα is the feature value corresponding to the αth feature in the behavior feature in the rth real-time digital video stream, m is the number of features in the behavior feature, P is a constant, is the standard deviation of the behavioral characteristics;

[0047] When the behavior factor is a preset first factor value, extracting video behavior data corresponding to the behavior factor, and identifying a behavior event corresponding to the video behavior data;

[0048] The behavior event is matched with an event in a preset abnormal behavior trigger event to obtain an abnormal behavior event.

[0049] Optionally, generating the abnormal data stream timing corresponding to the real-time digital video stream according to the abnormal traffic time point corresponding to the abnormal traffic event and the abnormal behavior time point corresponding to the abnormal behavior event includes:

[0050] Marking the abnormal traffic time point corresponding to the abnormal traffic event on a preset traffic time axis;

[0051] Marking the abnormal behavior time point corresponding to the abnormal behavior event on a preset behavior timeline;

[0052] The event points on the traffic timeline and the time points on the behavior timeline are sorted to obtain the abnormal data stream timing corresponding to the real-time digital video stream.

[0053] Optionally, generating the two-dimensional monitoring fusion graph of the real-time digital video stream through the abnormal data stream time sequence includes:

[0054] Determine a value corresponding to a time point on a flow time axis in the abnormal data flow time series as a preset first abnormal value, and generate a first curve according to the first abnormal value;

[0055] Determine a value corresponding to a time point on a behavior time axis in the abnormal data stream time series as a preset second abnormal value, and generate a second curve according to the second abnormal value;

[0056] The first curve and the second curve are fused into a two-dimensional monitoring fusion graph of the real-time digital video stream according to a preset coordinate system.

[0057] Optionally, analyzing the monitoring status of the real-time digital video stream according to the two-dimensional monitoring fusion graph includes:

[0058] According to the curve trends of the first curve and the second curve in the two-dimensional monitoring fusion graph, the number of abnormal points corresponding to the real-time digital video stream is counted;

[0059] Determining an abnormality level corresponding to the real-time digital video stream according to the number of abnormal points and a preset abnormal point threshold;

[0060] The monitoring status of the real-time digital video stream is determined according to the abnormality level.

[0061] The embodiments of the present invention can enable digital monitoring to respond instantly by acquiring video content in real time and performing digital processing; by identifying the picture and sound content of digital video content, monitoring can more accurately analyze events and behaviors occurring in the monitoring area; by detecting video behavior data and abnormal traffic events, digital video monitoring can quickly identify and respond to potential security threats and abnormal situations, thereby improving the security and early warning capabilities of the monitoring system; generating abnormal data flow time series and two-dimensional monitoring fusion graphs can intuitively understand abnormal situations and trends in the monitoring area; by analyzing the two-dimensional monitoring fusion graph, digital video monitoring can evaluate the monitoring status in real time, improve the management efficiency and response speed of the monitoring area, and make monitoring management more efficient and reliable. Therefore, the digital video monitoring method proposed by the present invention can solve the problem of low accuracy when performing digital video monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic diagram of a flow chart of a digital video monitoring method provided by an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of a process for identifying digital video content provided by an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of a process for identifying abnormal traffic events provided by an embodiment of the present invention;

[0065] Figure 4 A functional module diagram of a digital video monitoring system provided by one embodiment of the present invention.

[0066] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0068] The embodiment of the present application provides a digital video surveillance method. The execution subject of the digital video surveillance method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the digital video surveillance method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0069] Reference Figure 1 FIG. 1 is a flow chart of a digital video monitoring method provided by an embodiment of the present invention.

[0070] In this embodiment, the digital video monitoring method includes:

[0071] S1. Acquire video content in real time, perform digital processing on the video content to obtain digital video content, and identify a real-time digital video stream corresponding to the digital video content.

[0072] In an embodiment of the present invention, the video content refers to all information and data contained in a video file, and the visual part of the video includes image data of each frame, and these image frames are combined together to form a video sequence. The video may include a sound part, such as dialogue, music, background sound effects, etc. The audio track is usually played synchronously with the video screen, wherein the video content can be obtained in real time from a pre-stored storage area through a computer statement with a data capture function (such as a Java statement, a Python statement, etc.), wherein the storage area includes but is not limited to a database and a blockchain.

[0073] Furthermore, in order to conduct instant analysis of video content, it is necessary to convert the video content into a digital format. Through the digital format, the video content of a specific time period or event can be quickly retrieved, thereby improving query efficiency.

[0074] In the embodiment of the present invention, the digital video content refers to video content obtained through digital processing, which is video data represented in digital form, including image, audio and other data.

[0075] In the embodiment of the present invention, the step of digitally processing the video content to obtain digital video content includes:

[0076] Converting an original video signal corresponding to the video content into a digital signal;

[0077] Performing signal enhancement processing on the digital signal to obtain a digital enhanced signal;

[0078] The digital enhanced signal is further compressed to obtain digital video content.

[0079] In detail, the acquisition device is used to obtain the original video signal, which is usually an analog signal. The analog video signal is converted into a digital signal through an analog-to-digital converter (ADC), that is, the continuous value of the signal is converted into a discrete value, thereby generating a digital data stream. The noise in the video is then removed by a filter to improve the image clarity, and the contrast and brightness of the digital video signal are adjusted to enhance the visual effect of the image, and a sharpening algorithm is used to improve the details and edge clarity of the image; the enhanced digital signal is also compressed through a video compression algorithm (such as H.264, H.265, etc.). Compression can reduce the volume of video data for easy storage and transmission, and the compressed bit rate is adjusted as needed to balance image quality and file size, thereby obtaining enhanced digital video content, ensuring that the monitoring system can provide clear and reliable video content and achieve the best balance in storage and transmission.

[0080] Furthermore, in order to be able to view and analyze video streams in real time and quickly respond to potential security threats or abnormal situations, it is necessary to analyze the video content frame by frame, detect abnormal behavior and issue alarms to improve monitoring efficiency. Therefore, the video content needs to be divided into video streams.

[0081] In the embodiment of the present invention, the real-time digital video stream refers to a continuously transmitted and processed digital video data stream, in which video frames are delivered quickly in chronological order and the video stream is updated in real time, ensuring that the user can see the latest picture content for instant viewing and analysis.

[0082] In the embodiment of the present invention, the step of identifying the real-time digital video stream corresponding to the digital video content includes:

[0083] Decomposing the digital video content into video frames according to a preset frame rate to obtain digital frame images;

[0084] Generate a frame sequence corresponding to the digital frame image according to a preset timestamp;

[0085] The frame sequence is determined as the real-time digital video stream.

[0086] In detail, the digital video content is decomposed into a series of digital frame images according to a preset frame rate, that is, the video content is cut into separate frame images according to a set time interval. The number of frames processed per second (i.e., the frame rate) determines the smoothness of the video. The continuous video stream can be decomposed into discrete image frames, and then a timestamp is assigned to each digital frame image to record its specific position in the video clock. The timestamp is helpful for subsequent synchronization and retrieval of the frame images. If the video frame rate is 30 frames per second, the timestamp of each frame will be assigned according to 30 time points per second. Each frame image assigned with a timestamp is arranged in chronological order to form an ordered frame sequence. The frame sequence represents the time progress of the video, ensuring that each frame image can be correctly played or analyzed in the chronological order of its appearance, and the generated frame sequence is integrated into a real-time digital video stream. The image frames are arranged in chronological order to form a continuous video stream, ensuring that the video data can be transmitted and displayed continuously and synchronously.

[0087] Furthermore, video frame images are accurately associated with time to form a dynamic video stream, which can achieve efficient video playback and real-time processing. However, in order to detect abnormal content in the video in a timely manner, the video content corresponding to the real-time digital video stream needs to be analyzed frame by frame.

[0088] S2. Identify the picture content and the sound content corresponding to the real-time digital video stream, detect the video flow rate of the real-time digital video stream, and detect the video behavior data of the real-time digital video stream according to the picture content and the sound content.

[0089] In the embodiment of the present invention, the picture content refers to the visual information captured in the video stream, including scenes, characters, objects, actions, etc., which describe the composition of the picture and the events displayed; the sound content refers to the audio information in the video stream, including sound events (such as dialogue, background noise), emotional states (such as happiness, sadness) and audio features (such as pitch, rhythm), which describe the nature and context of the sound.

[0090] In the embodiment of the present invention, refer to Figure 2 As shown, the identifying of the picture content and sound content corresponding to the real-time digital video stream includes:

[0091] S21, identifying image data corresponding to each real-time digital video stream, and extracting screen events corresponding to the image data;

[0092] S22, generating picture content corresponding to each real-time digital video stream according to the picture event;

[0093] S23, extracting audio features corresponding to each real-time digital video stream, and determining sound events and sound emotions corresponding to each real-time digital video stream according to the audio features;

[0094] S24: Generate sound content corresponding to each real-time digital video stream according to the sound event and the sound emotion.

[0095] In detail, the images in each real-time video stream are analyzed by computer vision technology (such as target detection and scene recognition), and the events in the picture (such as people, objects, and behaviors) are identified and annotated, and detailed information of the event is extracted, including the objects, actions, time, and place involved, etc., and the identified event information is integrated to generate a detailed picture content description, that is, each frame or each period of time in the video stream is analyzed to identify the event, such as detecting a car passing by, a person walking into the room, or a person making a specific action, and the identified events are classified into different types, such as traffic events (cars, pedestrians), behavioral events (running, jumping), environmental events (sunny, rainy), etc., and the picture content description is generated according to the identified events. For example, if an event of a person entering the room is identified, a description is generated, a person walked into the room, wearing a red jacket, and then multiple event information in the video stream is integrated to provide an overall picture content summary, for example, in a conference room, a person wearing a blue suit is giving a speech, and another person is taking notes, so as to combine the picture event with the contextual information of the video stream (such as time and place) to ensure that the generated picture content accurately reflects the actual situation.

[0096] Specifically, audio processing technology (such as MFCC, audio signal analysis) is used to extract audio features, analyze audio features, identify sound events, such as identifying conversations, background music, traffic noise, etc., and classify the identified sound events into different types, such as human voices, environmental noise, mechanical sounds, etc., and determine the emotional state in the sound through emotional semantic analysis, such as anger, happiness, sadness, etc., and mark the corresponding emotional information for the sound event, such as angry conversations or pleasant music, and then generate a detailed sound content description based on the sound events and emotions. For example, in the video stream, there is rapid traffic noise in the background, and several angry conversations appear. Combined with sound events and emotions, a comprehensive sound content description is created to provide contextual information. For example, there are several people arguing fiercely in a meeting, accompanied by occasional background noise and music, so as to obtain the sound content corresponding to each real-time digital video stream.

[0097] Furthermore, it is necessary not only to identify the picture content and sound content in the real-time digital video stream, but also to detect the traffic corresponding to the real-time digital video stream and monitor the traffic fluctuation in real time, which is helpful to solve the problems in video playback in time. The video traffic of the real-time digital video stream is detected, that is, real-time data including bit rate, resolution, frame rate, etc. are obtained from the video stream through a network traffic monitoring tool, so as to determine the video traffic in the real-time digital video stream according to traffic (bits / second) = resolution × frame rate × bit depth × compression, wherein resolution refers to width × height, frame rate refers to the number of frames per second, and bit depth refers to the number of bits per pixel, so as to obtain the video traffic corresponding to the real-time digital video stream.

[0098] Going further, it can identify specific events in the audio (such as music, speech, environmental noise), analyze the emotional components of the sound (such as anger, happiness), extract and analyze the speaker's voiceprint features, and synchronize the picture and sound content, thereby linking events in the picture with events in the sound for a more comprehensive understanding.

[0099] In an embodiment of the present invention, the video behavior data refers to the behavior data after synchronizing the video picture content with the sound content, combining the visual information and auditory information in the video to more comprehensively understand and analyze the behavior patterns in the video, thereby providing more accurate information for video surveillance.

[0100] In the embodiment of the present invention, the detecting the video behavior data of the real-time digital video stream according to the picture content and the sound content includes:

[0101] Extracting a stream sequence number corresponding to the real-time digital video stream;

[0102] Generate a first event index of a picture event corresponding to the picture content according to the stream sequence number;

[0103] Generate a second event index of the sound event corresponding to the sound content according to the stream sequence number;

[0104] Calculate the correlation between the image event and the sound event according to the index identifiers corresponding to the first event index and the second event index:

[0105]

[0106] Where S is the correlation degree, w k is the association weight corresponding to the k-th index pair, v ik is the screen event vector corresponding to the i-th index identifier in the k-th index pair, a jk is the sound event vector corresponding to the jth index in the kth index pair, where n is the number of index pairs;

[0107] The image event and the sound event are merged into video behavior data corresponding to the real-time digital video stream according to the correlation degree.

[0108] In detail, a stream number corresponding to each frame of an image or each audio segment is extracted from a real-time digital video stream, and the stream number can be used to uniquely identify video and audio events, and an event index of a picture event is generated according to the stream number, then the first event index of the picture event refers to a specific picture content that uniquely identifies the video event, and an event index of a sound event is generated according to the stream number, then the second event index of the sound event refers to a specific sound content that uniquely identifies the audio event. For example, the stream number corresponding to the real-time digital video stream is {A1, A2, …, A t}, the first event index corresponding to the picture event is {1,2,…,t}, and the second event index corresponding to the sound event is {1,2,…,t}.

[0109] Specifically, the index identifier corresponding to the first event index is {1,2,…,t}, and the index identifier corresponding to the second event index is {1,2,…,t}. The correlation between the real-time digital video streams corresponding to different stream numbers is calculated according to the picture events and sound events corresponding to the index identifiers. The picture events and sound events are vectorized. The picture events and sound events can be converted into picture event vectors and sound event vectors through a vector conversion model (such as a BERT model), and each index pair has an associated weight. The index identifier corresponding to the first event index is {1,2,…,t} and the index identifier corresponding to the second event index is {1,2, …,t} construct an index identification matrix. For example, in the index identification matrix, if the matrix values ​​corresponding to index 1 in the first event index identification and index 1 in the second event index identification are the picture event vector and video event vector corresponding to the stream number A1, then the weights corresponding to the index items with the same index identification are set to x / (x+y). For example, the association weight corresponding to the first index 1 and the second index 2 is 1, and the association weight corresponding to the first index 1 and the second index 2 is 1 / (1+2), thereby obtaining the association weight corresponding to each index item and identifying the importance of the index pair, thereby multiplying the inner product of the picture event vector and the sound event vector of each index pair by the weight w of the index pair. k, and then sum all index pairs. This value reflects the correlation between the weighted picture events and sound events, that is, the weighted cosine similarity is calculated to measure the similarity between the picture event vector and the sound event vector, and then the picture event and video event with the highest correlation are taken as a matching event, that is, the feature vectors of the picture event and the sound event are directly merged into a more comprehensive feature vector to represent the video behavior data corresponding to the real-time digital video stream. For example, if the correlation between the first index 1 and the second index 1 is the highest, the picture event vector and the video event vector corresponding to the corresponding stream number A1 are merged into a more comprehensive feature vector to represent the video behavior data corresponding to the stream number A1.

[0110] Furthermore, based on the detected video traffic and the detected video behavior data, abnormal events in video surveillance can be identified based on the video traffic and the video behavior data, thereby improving the intelligence level and real-time response capability of video surveillance.

[0111] S3. Identify the abnormal traffic event corresponding to the video traffic according to the preset abnormal traffic trigger event, and identify the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event.

[0112] In an embodiment of the present invention, the abnormal traffic event refers to a situation in which an abnormal video traffic occurs during digital monitoring, indicating that the video behavior corresponding to the traffic deviates from the normal pattern, such as a sudden and significant increase or decrease in traffic, which exceeds the normal fluctuation range, such as a network attack, equipment failure, or a significant change in user behavior; such as abnormal fluctuations or irregularities in traffic data, which deviates from the data pattern or expected behavior.

[0113] In the embodiment of the present invention, refer to Figure 3 As shown, the identifying the abnormal traffic event corresponding to the video traffic according to the preset abnormal traffic trigger event includes:

[0114] S31, generating a traffic time sequence corresponding to the video traffic according to a preset timestamp;

[0115] S32. Calculate the flow factor corresponding to the video flow according to each video flow in the flow time series and the flow mean in the flow time series, wherein the flow factor calculation formula is:

[0116]

[0117] Among them, G r is the flow factor corresponding to the rth real-time digital video stream, f r is the video flow corresponding to the rth real-time digital video stream, F is the flow mean, Q is a constant, and δ is the flow standard deviation in the flow time series;

[0118] S33, when the flow factor is a preset first factor value, extracting the video flow corresponding to the flow factor, and identifying the video event corresponding to the video flow;

[0119] S34: Match the video event with an event in a preset abnormal traffic triggering event to obtain an abnormal traffic event.

[0120] Specifically, the video traffic detected at each moment is generated into a traffic time sequence, such as the video traffic detected at time t1 is L1, the video traffic detected at time t2 is L2, and the video traffic detected at time t n The video traffic detected at any moment is L n , then the flow sequence is {L t1 ,L t2 ,…,L tn}, and then calculate the traffic factor according to each video traffic in the traffic time series and the traffic mean corresponding to all video traffic in the traffic time series, that is, when a video traffic f in the traffic time series r If the absolute value of the difference from the traffic mean F is greater than the product of the traffic standard deviation δ corresponding to all video traffic in the traffic time series and the constant Q, the traffic factor is defined as 1, indicating that there is video content with abnormal video traffic; and when a video traffic f in the traffic time series r If the absolute value of the difference from the traffic mean F is less than or equal to the product of the traffic standard deviation δ corresponding to all video traffic in the traffic time series and the constant Q, the traffic factor is defined as 0, indicating that there is no video content with abnormal video traffic. The traffic factor is used to indicate whether the video traffic is normal, where Q is a constant used to set the threshold range of abnormalities. It is combined with the standard deviation δ in the traffic time series to determine the traffic factor G. r The value of Q is used to identify whether the video traffic is abnormal. The constant Q is generally configured to be 0.5.

[0121] Specifically, when the flow factor is the first factor value (value 1), the corresponding video flow of the real-time digital video stream corresponding to the flow factor of 1 is collected. For example, if the flow factor corresponding to the first and fifth real-time digital video streams is 1, the video flow corresponding to the first and fifth real-time digital video streams is extracted, and the corresponding video events are identified for the extracted abnormal video flow data, that is, image processing or computer vision technology is used to check the content in the frame to identify any significant changes or activities. For example, a motion detection algorithm can be applied to find abnormal activities, and then the analysis results are compared with predefined event categories to classify the identified abnormal events. For example, abnormal activities may be classified as "traffic surge", "traffic drop" or "traffic fluctuation".

[0122] In the embodiment of the present invention, the step of matching the video event with an event in a preset abnormal traffic triggering event to obtain an abnormal traffic event includes:

[0123] Extracting abnormal event category data from preset abnormal traffic triggering events;

[0124] Classifying the video events to obtain video event category data;

[0125] Performing vector conversion on the abnormal event category data to obtain an abnormal event category vector;

[0126] Performing vector conversion on the video event category data to obtain a video event category vector;

[0127] Calculating the matching degree between the abnormal event category vector and the video event category vector;

[0128] The abnormal traffic trigger event with the highest matching degree is selected as the abnormal traffic event.

[0129] In detail, the video events are matched with events in the preset abnormal traffic trigger events, where the abnormal traffic trigger events include traffic surge, where the traffic suddenly increases to a level far higher than the normal level, which may indicate an attack (such as a DDoS attack) or a system abnormality; traffic drop, where the traffic suddenly drops, which may indicate a service failure or network problem; traffic fluctuation, where the traffic fluctuates frequently in a short period of time, which may indicate system instability or problems; abnormal pattern, where the traffic data exhibits behavior that does not conform to the normal pattern, such as an increase in traffic in an unusual time period. If the video event is a traffic surge, the abnormal traffic event is an attack.

[0130] Specifically, the category information of the abnormal event is extracted from the preset data of the abnormal traffic triggering event, and the abnormal event category data includes traffic surge, traffic drop, traffic fluctuation and abnormal pattern; the video event is classified to determine its category, and the video event category data includes traffic surge, traffic drop, traffic fluctuation and abnormal pattern, and then the abnormal event category data is converted into a vector representation, and the video event category data is also converted into a vector representation through word embedding (Word Embedding) or other feature vector representation to ensure consistency with the representation of the abnormal event category vector, and then the algorithm for calculating similarity (such as cosine similarity, Euclidean distance, etc.) is used to measure the degree of match between the abnormal event category vector and the video event category vector, and according to the calculated matching degree, the abnormal traffic triggering event that best matches the video event is selected and determined as the current abnormal traffic event.

[0131] Furthermore, in order to more comprehensively analyze abnormal events in digital video surveillance, it is necessary not only to identify abnormal traffic events in digital video surveillance, but also to analyze abnormal behavior events in digital video surveillance.

[0132] In an embodiment of the present invention, the abnormal behavior event refers to an event in which the behavioral characteristics deviate from the normal pattern or preset standard in video surveillance or data analysis, and exhibit abnormal or abnormal behavior, including unexpected activity patterns, behavior that does not conform to the norm, etc., which may indicate system problems, security threats or other abnormal situations that require attention.

[0133] In the embodiment of the present invention, the step of identifying the abnormal behavior event corresponding to the video behavior data according to a preset abnormal behavior trigger event includes:

[0134] Extracting the behavior features of each real-time digital video stream in the video behavior data, and generating the behavior time sequence corresponding to the video behavior data according to the preset event stamp and the behavior features;

[0135] The behavior factor corresponding to the video behavior data is calculated according to each behavior feature in the behavior time sequence and the standard deviation of the behavior feature in the behavior time sequence, wherein the behavior factor calculation formula is:

[0136]

[0137] Among them, H r is the behavior factor corresponding to the rth real-time digital video stream, σ rα is the feature value corresponding to the αth feature in the behavior feature in the rth real-time digital video stream, m is the number of features in the behavior feature, P is a constant, is the standard deviation of the behavioral characteristics;

[0138] When the behavior factor is a preset first factor value, extracting video behavior data corresponding to the behavior factor, and identifying a behavior event corresponding to the video behavior data;

[0139] The behavior event is matched with an event in a preset abnormal behavior trigger event to obtain an abnormal behavior event.

[0140] In detail, the behavior features include motion features, posture features, and sound features. The motion features describe how the objects or individuals in the video move and change, including speed, direction, and motion trajectory. The motion vector field of the objects in the video is analyzed by optical flow technology to describe the movement direction and speed of the objects on the image plane. The motion features are represented by speed. The posture features describe the position and posture of the individuals or objects in space, including the relative positions and angles of the individual's body parts relative to each other. The deep learning model is used to identify the key points of the human body (such as the head, shoulders, elbows, knees, etc.) and estimate their relative positions. The posture features are represented by coordinate values. The sound features describe the audio information accompanying the video, including speech, background noise, or other sound effects. The Mel spectrum features of the audio signal are extracted for speech recognition and sound classification. The sound features are represented by the frequency of each audio segment, and then the behavior features detected at each moment are generated into a behavior time series. For example, the behavior feature detected at time t1 is H 11 ,H 12 ,H 13 , the behavior feature detected at time t2 is H 21 ,H 22 ,H 23 , at t n The behavior feature detected at any moment is H n1 ,H n2 ,H n3 , then the behavior sequence is in For n Motion characteristics in momentary behavior characteristics n1 , Posture characteristics H n2 、Sound characteristics H n3 , thereby calculating the behavior factor based on each behavior feature in the behavior time series and the behavior mean corresponding to all behavior features in the behavior time series.

[0141] Specifically, when the feature value σ of the αth feature in a behavior feature in the behavior time series rα The absolute value of the difference from the behavioral mean of the αth feature is greater than the behavioral feature standard deviation corresponding to all behavioral features of the αth feature in the behavioral time series The product of the constant P, the behavior factor is defined as 1, indicating that there is abnormal behavior in the video content. For example, the corresponding behavior feature at the first moment is H 11 ,H 12 ,H 13 , then H 11 ,H 12 ,H 13The mean and standard deviation are calculated, and then the calculated mean and standard deviation are analyzed and calculated with the behavior characteristics corresponding to a certain moment, so as to determine the behavior factor corresponding to the behavior characteristics. Among them, due to the different units between the motion characteristics, posture characteristics, and sound characteristics, it is necessary to standardize the motion characteristics, posture characteristics, and sound characteristics respectively, and convert the data into a distribution with a mean of 0 and a standard deviation of 1. For example, the standardized speed is 0.63, the standardized posture characteristics are (0.26, 0.53), and the annotated sound characteristics are 0.78, then the value of the posture characteristics is Then, the motion features, posture features, and sound features are superimposed to calculate the behavior factor; and when the feature value σ of the αth feature in a behavior feature in the behavior time series rα If the absolute value of the difference from the behavioral mean of the αth feature is less than or equal to the product of the behavioral feature standard deviation φ corresponding to all behavioral features of the αth feature in the behavioral time series and the constant P, the behavioral factor is defined as 0, indicating that there is no video content with abnormal video behavior. The behavioral factor is used to indicate whether the behavior in the video is normal, where P is a constant used to set the threshold range of abnormalities. It is combined with the behavioral feature standard deviation φ in the behavioral time series to determine the behavioral factor H. r The value of P is used to identify whether the video behavior is abnormal. The constant P is generally configured to be 0.5.

[0142] Further, when the behavior factor is the first factor value (value 1), the corresponding video behavior data of the real-time digital video stream corresponding to the behavior factor of 1 is collected. For example, if the behavior factor corresponding to the first and fifth real-time digital video streams is 1, the video behavior data corresponding to the first and fifth real-time digital video streams are extracted, and the corresponding behavior events are identified for the extracted abnormal video behavior data, that is, the extracted features are analyzed using a machine learning model or a pattern recognition algorithm to identify specific behavior events, for example, walking, running, making phone calls, etc., and then the analysis results are compared with predefined event categories to classify the identified abnormal events. For example, abnormal behavior events are classified into face recognition anomalies, behavior anomalies, scene changes, audio anomalies, intrusion detection, etc., and video events are matched with events in preset abnormal behavior trigger events, where abnormal behavior trigger events include face recognition anomalies, behavior anomalies, scene changes, audio anomalies, intrusion detection, etc. Face recognition anomalies include identity recognition errors and the appearance of unknown people; behavior anomalies include abnormal action detection and behavior that deviates from the norm; scene changes include abnormal scene switching and environmental changes; audio anomalies include abnormal sound detection and audio and video asynchrony; intrusion detection includes regional intrusion and suspicious object detection.

[0143] Furthermore, in order to be able to respond promptly at the time when the anomaly occurs, the time points of abnormal traffic events and abnormal behavior events are integrated into a time series, which can more comprehensively analyze the background and causal relationship of abnormal events and effectively manage and respond to abnormal events in real-time digital video streams.

[0144] S4. Generate an abnormal data stream timing sequence corresponding to the real-time digital video stream according to the abnormal traffic time point corresponding to the abnormal traffic event and the abnormal behavior time point corresponding to the abnormal behavior event.

[0145] In the embodiment of the present invention, the abnormal data stream timing refers to sorting and displaying abnormal events appearing in the video based on events, mainly displaying abnormal activities in the data stream through time series to help identify and analyze patterns and trends of abnormal events appearing in the video.

[0146] In the embodiment of the present invention, the generating of the abnormal data stream timing corresponding to the real-time digital video stream according to the abnormal traffic time point corresponding to the abnormal traffic event and the abnormal behavior time point corresponding to the abnormal behavior event includes:

[0147] Marking the abnormal traffic time point corresponding to the abnormal traffic event on a preset traffic time axis;

[0148] Marking the abnormal behavior time point corresponding to the abnormal behavior event on a preset behavior timeline;

[0149] The event points on the traffic timeline and the time points on the behavior timeline are sorted to obtain the abnormal data stream timing corresponding to the real-time digital video stream.

[0150] In detail, the traffic timeline is used to mark the time points of abnormal traffic time, and the behavior timeline is used to mark the time points of abnormal behavior events. The event points corresponding to the abnormal traffic events are recorded on the traffic timeline, and the time points of the abnormal behavior events are recorded on the behavior timeline. The abnormal time points on the traffic timeline and the abnormal time points on the behavior timeline are merged into a unified time point set, and this time point set includes the time information of all abnormal events, whether they are traffic anomalies or behavior anomalies. The merged time points are sorted and arranged in chronological order to obtain the timing of the abnormal data stream, that is, all time points are stored in a list and sorted in ascending order, and the timing of the abnormal data stream is created according to the sorted time points. The abnormal data stream timing can be represented as a time series, in which each time point corresponds to an abnormal event, marking its source (traffic anomaly or behavior anomaly) and the specific time when it occurred, and the abnormal traffic and abnormal behavior corresponding to the same time point are regarded as the same time point.

[0151] Exemplarily, assuming that the time point of the traffic anomaly event is [T1, T2, T3], the time point of the behavior anomaly event is [T3, T4], the merged time point set is [T1, T2, T3, T4], and assuming that the sorted time points are [T1, T2, T4, T3], the generated abnormal data flow time sequence is T1 for traffic anomaly, T2 for traffic anomaly, T3 for traffic anomaly and behavior anomaly, and T4 for behavior anomaly.

[0152] Furthermore, the abnormal data stream time series is generated based on traffic events and behavioral events. In order to be able to more intuitively analyze abnormal events in digital video surveillance, the visual information in the video stream is integrated with the time series information in the abnormal data stream, providing deeper insights and real-time monitoring capabilities.

[0153] S5. Generate a two-dimensional monitoring fusion graph of the real-time digital video stream through the abnormal data stream time sequence, and analyze the monitoring status of the real-time digital video stream according to the two-dimensional monitoring fusion graph.

[0154] In an embodiment of the present invention, the two-dimensional monitoring fusion graph refers to a graph that integrates and visualizes abnormal traffic and abnormal behavior in an abnormal data flow time series in a two-dimensional coordinate system, and analyzes how different types of data are related to each other in time or space. The two-dimensional monitoring fusion graph includes an X-axis, a time axis, which shows the time distribution of data and events, a Y-axis, different data values, such as traffic anomaly values ​​and behavior anomaly values, and a curve that shows the changing trend of traffic and behavior anomalies.

[0155] In the embodiment of the present invention, the step of generating the two-dimensional monitoring fusion graph of the real-time digital video stream through the abnormal data stream time sequence includes:

[0156] Determine a value corresponding to a time point on a flow time axis in the abnormal data flow time series as a preset first abnormal value, and generate a first curve according to the first abnormal value;

[0157] Determine a value corresponding to a time point on a behavior time axis in the abnormal data stream time series as a preset second abnormal value, and generate a second curve according to the second abnormal value;

[0158] The first curve and the second curve are fused into a two-dimensional monitoring fusion graph of the real-time digital video stream according to a preset coordinate system.

[0159] In detail, a time point on the traffic timeline is extracted from the abnormal data stream sequence and determined as a first abnormal value, and the time point corresponding to the abnormal traffic event is determined as the first abnormal value. If there is an abnormal situation, the value corresponding to the time point is 1, otherwise it is 0, and the first abnormal value is a value of 1, thereby generating a first curve according to the abnormal value corresponding to the time point, and the first curve is generated by the abnormal value on the traffic timeline, showing the change of traffic data over time, reflecting the change of video traffic; and a time point on the behavior timeline is extracted from the abnormal data stream sequence and determined as a second abnormal value, and the time point corresponding to the abnormal behavior event is determined as the second abnormal value. If there is an abnormal situation, the value corresponding to the time point is 1, otherwise it is 0, and the second abnormal value is a value of 1, thereby generating a second curve according to the abnormal value corresponding to the time point, and the second curve is generated by the abnormal value on the behavior timeline, showing the change of behavior data over time, reflecting the change of video behavior, and the first curve and the second curve are fused in a preset coordinate system, combined with real-time digital video stream data, to form a two-dimensional monitoring fusion graph.

[0160] Specifically, abnormal traffic events and abnormal behavior events are displayed on a graph and can be represented by lines of different colors. If the corresponding traffic and behavior at the same time point are normal, the two curves overlap. By plotting the first curve and the second curve in the same coordinate system, the time and numerical changes of different abnormal data are observed. At the same time, combined with the visual information in the video stream, a comprehensive understanding of the events is provided. In the two-dimensional chart, screenshots or related marks of the real-time video stream can be superimposed to show the relationship between what occurs in the video and the curve changes in the abnormal data stream.

[0161] Furthermore, the temporal and numerical changes of different abnormal data can be intuitively observed through the two-dimensional monitoring fusion graph, and combined with the visual information in the video stream, a comprehensive understanding of the event can be provided. Therefore, the number of 0s and 1s in the two-dimensional monitoring fusion graph is counted to analyze the monitoring operation status of the digital video surveillance.

[0162] In an embodiment of the present invention, the monitoring status refers to the current health and stability during the monitoring process, the judgment and processing of abnormal situations, and the monitoring status can be divided into normal, slightly abnormal, moderately abnormal or severely abnormal according to the number and severity of abnormal points.

[0163] In the embodiment of the present invention, analyzing the monitoring status of the real-time digital video stream according to the two-dimensional monitoring fusion graph includes:

[0164] According to the curve trends of the first curve and the second curve in the two-dimensional monitoring fusion graph, the number of abnormal points corresponding to the real-time digital video stream is counted;

[0165] Determining an abnormality level corresponding to the real-time digital video stream according to the number of abnormal points and a preset abnormal point threshold;

[0166] The monitoring status of the real-time digital video stream is determined according to the abnormality level.

[0167] In detail, an abnormal event refers to a point in the curve with a value of 1. The first curve (traffic data) and the second curve (behavior data) are respectively identified and counted for abnormal points, the number of abnormal points in each curve is counted separately, and the total number of abnormal points is calculated. The traffic abnormal points and behavior abnormal points can be counted separately, and then combined to obtain the overall number of abnormal points. The number of abnormal points is compared with the abnormal point threshold, and the abnormal point threshold is a standard for determining the abnormality. For example, the threshold can be a certain percentage or absolute value of the total number of abnormal points.

[0168] Specifically, the abnormality level is determined as a slight abnormality when the number of abnormal points is lower than a certain standard of the threshold (such as 50% of the threshold), a moderate abnormality when the number of abnormal points is within the range of the threshold (such as 50%-100% of the threshold), and a severe abnormality when the number of abnormal points exceeds the threshold (such as 100% of the threshold). The monitoring status of the real-time digital video stream is then determined according to the abnormality level. For example, the monitoring status corresponding to a slight abnormality is normal monitoring, and the abnormality may be an isolated event; the monitoring status corresponding to a moderate abnormality is that there are certain abnormal conditions, which may require attention; the monitoring status corresponding to a severe abnormality is that the abnormal condition is serious, indicating a security risk or a monitoring system failure. For example, assuming that the number of abnormal points in traffic data is 15, the number of abnormal points in behavior data is 10, the total number of abnormal points is 25, and the preset abnormal point threshold is 20, the number of abnormal points exceeds the threshold, and it is judged to be a severe abnormality, then the monitoring status of the real-time digital video stream is a severe abnormality.

[0169] The embodiments of the present invention can enable digital monitoring to respond instantly by acquiring video content in real time and performing digital processing; by identifying the picture and sound content of digital video content, monitoring can more accurately analyze events and behaviors occurring in the monitoring area; by detecting video behavior data and abnormal traffic events, digital video monitoring can quickly identify and respond to potential security threats and abnormal situations, thereby improving the security and early warning capabilities of the monitoring system; generating abnormal data flow time series and two-dimensional monitoring fusion graphs can intuitively understand abnormal situations and trends in the monitoring area; by analyzing the two-dimensional monitoring fusion graph, digital video monitoring can evaluate the monitoring status in real time, improve the management efficiency and response speed of the monitoring area, and make monitoring management more efficient and reliable. Therefore, the digital video monitoring method proposed by the present invention can solve the problem of low accuracy when performing digital video monitoring.

[0170] like Figure 4 FIG. 1 is a functional module diagram of a digital video monitoring system provided by an embodiment of the present invention.

[0171] The digital video monitoring system 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the digital video monitoring system 100 can include a real-time digital video stream recognition module 101, a video behavior data detection module 102, an abnormal behavior event recognition module 103, an abnormal data stream timing generation module 104 and a monitoring status analysis module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0172] In this embodiment, the functions of each module / unit are as follows:

[0173] The real-time digital video stream identification module 101 is used to obtain video content in real time, perform digital processing on the video content to obtain digital video content, and identify the real-time digital video stream corresponding to the digital video content;

[0174] The video behavior data detection module 102 is used to identify the picture content and sound content corresponding to the real-time digital video stream, detect the video flow of the real-time digital video stream, and detect the video behavior data of the real-time digital video stream according to the picture content and the sound content;

[0175] The abnormal behavior event identification module 103 is used to identify the abnormal traffic event corresponding to the video traffic according to the preset abnormal traffic trigger event, and to identify the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event;

[0176] The abnormal data stream timing generation module 104 is used to generate the abnormal data stream timing corresponding to the real-time digital video stream according to the abnormal traffic time point corresponding to the abnormal traffic event and the abnormal behavior time point corresponding to the abnormal behavior event;

[0177] The monitoring status analysis module 105 is used to generate a two-dimensional monitoring fusion graph of the real-time digital video stream according to the abnormal data stream time sequence, and analyze the monitoring status of the real-time digital video stream according to the two-dimensional monitoring fusion graph.

[0178] In detail, each module described in the digital video surveillance system 100 described in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 3 The digital video surveillance method described in the present invention has the same technical means and can produce the same technical effects, so it will not be repeated here.

[0179] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0180] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0182] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0183] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is not limited only according to the above description, and it is intended that all changes within the meaning and scope of equivalent elements within the scope of protection are included in the present invention.

[0184] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0185] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A digital video monitoring method, characterized in that: The method comprises: Acquire video content in real time, perform digital processing on the video content to obtain digital video content, and identify the real-time digital video stream corresponding to the digital video content; Identify the picture content and sound content corresponding to the real-time digital video stream, detect the video flow of the real-time digital video stream, and detect the video behavior data of the real-time digital video stream according to the picture content and the sound content; Identifying an abnormal traffic event corresponding to the video traffic according to a preset abnormal traffic triggering event, including: generating a traffic time sequence corresponding to the video traffic according to a preset timestamp; calculating a traffic factor corresponding to the video traffic according to each video traffic in the traffic time sequence and a traffic mean in the traffic time sequence, wherein the traffic factor calculation formula is: Among them, G r is the flow factor corresponding to the rth real-time digital video stream, f r is the video flow corresponding to the rth real-time digital video stream, F is the flow mean, Q is a constant, and δ is the flow standard deviation in the flow time series; When the flow factor is a preset first factor value, extract the video flow corresponding to the flow factor, and identify the video event corresponding to the video flow; match the video event with an event in a preset abnormal flow trigger event to obtain an abnormal flow event; identify the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event; Mark the abnormal traffic time point corresponding to the abnormal traffic event on the preset traffic time axis; mark the abnormal behavior time point corresponding to the abnormal behavior event on the preset behavior time axis; sort the event points on the traffic time axis and the time points on the behavior time axis to obtain the abnormal data flow time sequence corresponding to the real-time digital video stream; The numerical value corresponding to the time point in the abnormal data stream time series is determined as the abnormal value, the time point is used as the horizontal coordinate, and the abnormal value is used as the vertical coordinate to generate a two-dimensional monitoring fusion map, and the monitoring status of the real-time digital video stream is analyzed according to the two-dimensional monitoring fusion map.

2. The digital video monitoring method according to claim 1, characterized in that: The step of digitally processing the video content to obtain digital video content includes: Converting an original video signal corresponding to the video content into a digital signal; Performing signal enhancement processing on the digital signal to obtain a digital enhanced signal; The digital enhanced signal is further compressed to obtain digital video content.

3. The digital video monitoring method according to claim 1, characterized in that: The step of identifying the real-time digital video stream corresponding to the digital video content comprises: Decomposing the digital video content into video frames according to a preset frame rate to obtain digital frame images; Generate a frame sequence corresponding to the digital frame image according to a preset timestamp; The frame sequence is determined as the real-time digital video stream.

4. The digital video monitoring method according to claim 1, characterized in that: The identifying of the picture content and the sound content corresponding to the real-time digital video stream includes: Identify image data corresponding to each real-time digital video stream, and extract picture events corresponding to the image data; Generate the picture content corresponding to each real-time digital video stream according to the picture event; Extracting audio features corresponding to each real-time digital video stream, and determining sound events and sound emotions corresponding to each real-time digital video stream according to the audio features; The sound content corresponding to each real-time digital video stream is generated according to the sound event and the sound emotion.

5. The digital video monitoring method according to claim 1, characterized in that: The detecting of the video behavior data of the real-time digital video stream according to the picture content and the sound content includes: Extracting a stream sequence number corresponding to the real-time digital video stream; Generate a first event index of a picture event corresponding to the picture content according to the stream sequence number; Generate a second event index of the sound event corresponding to the sound content according to the stream sequence number; Calculate the correlation between the image event and the sound event according to the index identifiers corresponding to the first event index and the second event index: Where S is the correlation degree, w k is the association weight corresponding to the k-th index pair, v ik is the screen event vector corresponding to the i-th index identifier in the k-th index pair, a jk is the sound event vector corresponding to the jth index in the kth index pair, where n is the number of index pairs; The image event and the sound event are merged into video behavior data corresponding to the real-time digital video stream according to the correlation degree.

6. The digital video monitoring method according to claim 1, characterized in that: The identifying the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event includes: Extracting the behavior features of each real-time digital video stream in the video behavior data, and generating the behavior time sequence corresponding to the video behavior data according to the preset event stamp and the behavior features; The behavior factor corresponding to the video behavior data is calculated according to each behavior feature in the behavior time sequence and the standard deviation of the behavior feature in the behavior time sequence, wherein the behavior factor calculation formula is: Among them, H r is the behavior factor corresponding to the rth real-time digital video stream, σ rα is the feature value corresponding to the αth feature in the behavior feature in the rth real-time digital video stream, m is the number of features in the behavior feature, P is a constant, is the standard deviation of the behavioral characteristics; When the behavior factor is a preset first factor value, extracting video behavior data corresponding to the behavior factor, and identifying a behavior event corresponding to the video behavior data; The behavior event is matched with an event in a preset abnormal behavior trigger event to obtain an abnormal behavior event.

7. The digital video monitoring method according to claim 1, characterized in that: Analyzing the monitoring status of the real-time digital video stream according to the two-dimensional monitoring fusion graph includes: According to the curve trends of the first curve and the second curve in the two-dimensional monitoring fusion graph, the number of abnormal points corresponding to the real-time digital video stream is counted; Determining an abnormality level corresponding to the real-time digital video stream according to the number of abnormal points and a preset abnormal point threshold; The monitoring status of the real-time digital video stream is determined according to the abnormality level.

8. A digital video monitoring system, characterized in that: For executing the digital video monitoring method according to any one of claims 1 to 7, the system comprises: A real-time digital video stream identification module is used to obtain video content in real time, perform digital processing on the video content to obtain digital video content, and identify the real-time digital video stream corresponding to the digital video content; A video behavior data detection module, used to identify the picture content and sound content corresponding to the real-time digital video stream, detect the video flow of the real-time digital video stream, and detect the video behavior data of the real-time digital video stream according to the picture content and the sound content; The abnormal behavior event identification module is used to identify the abnormal traffic event corresponding to the video traffic according to the preset abnormal traffic trigger event, including: generating the traffic time sequence corresponding to the video traffic according to the preset timestamp; calculating the traffic factor corresponding to the video traffic according to each video traffic in the traffic time sequence and the traffic mean in the traffic time sequence, wherein the traffic factor calculation formula is: Among them, G r is the flow factor corresponding to the rth real-time digital video stream, f r is the video flow corresponding to the rth real-time digital video stream, F is the flow mean, Q is a constant, and δ is the flow standard deviation in the flow time series; When the flow factor is a preset first factor value, extract the video flow corresponding to the flow factor, and identify the video event corresponding to the video flow; match the video event with an event in a preset abnormal flow trigger event to obtain an abnormal flow event; identify the abnormal behavior event corresponding to the video behavior data according to the preset abnormal behavior trigger event; The abnormal data stream timing generation module is used to mark the abnormal traffic time point corresponding to the abnormal traffic event on the preset traffic time axis; mark the abnormal behavior time point corresponding to the abnormal behavior event on the preset behavior time axis; sort the event points on the traffic time axis and the time points on the behavior time axis to obtain the abnormal data stream timing corresponding to the real-time digital video stream; The monitoring status analysis module is used to determine the numerical value corresponding to the time point in the abnormal data stream time series as the abnormal value, use the time point as the horizontal coordinate and the abnormal value as the vertical coordinate to generate a two-dimensional monitoring fusion graph, and analyze the monitoring status of the real-time digital video stream according to the two-dimensional monitoring fusion graph.

Images