Video stream event analysis and wearable device instant alarm system

Abstract

The invention relates to the technical field of video stream event analysis, in particular to a video stream event analysis and wearable device instant alarm system. According to the method, the video stream is segmented through key frame identification and frequency domain feature extraction, and the event segments with significant temporal and spatial changes are identified, so that the event analysis efficiency is improved; based on a model of a recurrent neural network, abnormal events in a video are identified and evaluated through accurate time series data modeling and event label matching, and joint optimization is performed on an event identification model and a matching result analysis model through a generative adversarial network by an adversarial joint training module. The model has higher robustness and generalization ability in the face of a complex environment, so that the event identification precision is improved; alarm information is generated according to the reliability probability of the event tag, and timely feedback is provided for a user; according to the invention, through close combination of video stream event analysis and the wearable device, the real-time performance and accuracy of event detection are improved.

Description

Technical Field

[0001] This invention relates to the field of video stream event analysis technology, specifically to a video stream event analysis and wearable device real-time alarm system. Background Technology

[0002] With the development of intelligent monitoring and security technologies, video stream-based anomaly detection and alarm systems are gradually being widely used. Traditional monitoring systems detect events through static monitoring and manual analysis, but as the amount of video data continues to increase, the efficiency and accuracy of manual analysis are becoming bottlenecks. Furthermore, traditional monitoring systems can typically only detect events with known patterns and cannot accurately identify new and unknown anomalies in ever-changing real-world scenarios.

[0003] To address this challenge, modern video stream event analysis systems have begun to incorporate deep learning and computer vision technologies to perform real-time event recognition in an automated and intelligent manner. These systems can quickly extract features, analyze events, and generate alerts from large-scale video stream data. However, existing video stream event analysis systems are often limited by the amount of data, especially since processing real-time video streams requires efficient computing and storage resources, resulting in high latency and an inability to respond promptly at critical moments.

[0004] Secondly, existing alarm systems often rely on fixed alarm rules, such as threshold judgment based on event matching tags, which cannot flexibly adjust alarm responses. Traditional alarm systems fail to effectively consider alarm levels and users' actual needs, resulting in serious false alarms and missed alarms.

[0005] To address the aforementioned issues, it is necessary to propose a video stream event analysis and wearable device real-time alarm system. Summary of the Invention

[0006] The purpose of this invention is to solve the problems existing in the background art, and to propose a video stream event analysis and wearable device real-time alarm system.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A video stream event analysis and wearable device real-time alarm system includes a video stream acquisition module, an event analysis module, an analysis result scoring module, an adversarial joint training module, a signal matching and alarm module, and a wearable device execution response module.

[0008] The video stream acquisition module acquires raw video signals and entropy coding compression algorithm to encode the acquired raw video signals in real time to obtain standard video stream data. Keyframe recognition algorithm extracts feature frames from the standard video stream data. The raw video signal is then segmented using these feature frames to obtain several video slices.

[0009] It can access webcams and CCTV cameras in real time and capture raw video signals at preset frame rates and video encoding formats.

[0010] In a preferred embodiment of the present invention, the acquired raw video signal is encoded in real time using an entropy coding compression algorithm to obtain standard video stream data. The specific process is as follows: The original video signal is transformed to the frequency domain using Discrete Cosine Transform (DCT), yielding the frequency domain coefficients of each image frame. All frequency domain coefficients of each image frame are then arranged according to their frequency domain indices to obtain a frequency domain coefficient matrix. Finally, the frequency domain coefficient matrices corresponding to each image frame are arranged according to their image frame indices to obtain the standard video stream data.

[0011] As a preferred embodiment of the present invention, feature frames are extracted from standard video stream data using a keyframe recognition algorithm. The specific process is as follows: In standard video stream data, the sum of the upper diagonal portion of the frequency domain coefficient matrix corresponding to each image frame is calculated to obtain the high-frequency feature values ​​corresponding to each image frame. For the high-frequency feature values ​​of each adjacent image frame, the coefficient of variation of the high-frequency features for each image frame is calculated.

[0012] If the coefficient of variation of a high-frequency feature of an image frame is found to be greater than a preset threshold, the image frame is determined to be a feature frame.

[0013] The original video signal is segmented based on feature frames to obtain several video slices. The segmentation rules are as follows: If there are several consecutive feature frames, that is, if the image frame indices of multiple feature frames are an arithmetic sequence with a common difference of 1, then the consecutive feature frames are grouped into the same video slice. All image frames between two non-contiguous feature frames are grouped into the same video slice; All generated video slices are sent to the event analysis module and the adversarial joint training module.

[0014] The event analysis module inputs the received video slices into the event recognition model based on a recurrent neural network to obtain event matching labels for each feature frame.

[0015] Event matching labels for each video slice are calculated using an event recognition model based on a recurrent neural network. The preset event matching labels point to: no abnormal situation, traffic collision, running a red light, speeding, people being hit, fire, or damage to infrastructure structures. Send the event recognition model to the adversarial joint training module.

[0016] The analysis results scoring module inputs the event matching labels of each video slice into the matching results analysis model based on a recurrent neural network to obtain the reliability probability of the event matching labels.

[0017] The credibility of the matching results is obtained by using a recurrent neural network-based matching result analysis model to score the credibility of the image frames in the video slices and the event matching labels output by the corresponding event analysis module.

[0018] The label confidence level ranges from 0 to 1 and is used to evaluate whether the event matching labels output by the event analysis module are reliable. A label confidence level of 1 means that the event matching label describes the image frame completely accurately; a label confidence level of 0 means that the event matching label describes the image frame completely incorrectly.

[0019] The matching result analysis model is sent to the adversarial joint training module.

[0020] The adversarial joint training module uses a GA generative adversarial network to jointly train the event recognition model and the matching result analysis model, and sends the model parameters of the trained event recognition model to the signal matching and alarm module.

[0021] The Gan generative adversarial network consists of a generator G and a discriminator D.

[0022] Generator G contains several event recognition models. Each model generates pseudo-event matching labels based on video slices, simulating the event recognition model's operation on event matching labels for image frames within the video slices. The pseudo-event matching labels and their corresponding video frames from the video slices are then sent to discriminator D. The goal of generator G is to make the discriminator unable to distinguish between the generated pseudo-event matching labels and real event matching labels. Each event recognition model generates pseudo-event matching labels based on video slices, simulating the event recognition model's operation on event matching labels for image frames within the video slices.

[0023] The loss function of generator G optimizes the set of training parameters E1 of each event recognition model in the generator by minimizing the probability that the discriminator determines whether the label is real; The discriminator D comprises a matching result analysis model and a set of preset standard video slices. It performs label confidence calculations on all pseudo-event matching labels and their corresponding image frames, and compares the confidence calculation results with the confidence calculation results of the preset standard video slices and their corresponding real event matching labels. The discriminator's goal is to correctly distinguish between real event matching labels and pseudo-event matching labels.

[0024] The loss function of discriminator D is optimized by maximizing the difference between the generated label and the real label; In a preferred embodiment of the present invention, adversarial training is performed between a generator G and a discriminator D, allowing the generator and discriminator to continuously optimize each other. This adversarial training is achieved by minimizing the loss functions of both the generator G and the discriminator D. The generator continuously generates data with a level of credibility closer to the true labels, while the discriminator continuously improves its ability to identify fake data.

[0025] In a preferred embodiment of the present invention, in each training iteration, the generator and discriminator optimize the set of parameters to be trained using the backpropagation algorithm until the loss function of the generator G and the loss function of the discriminator D both reach their minimum values. At this point, the set of parameters to be trained for each event recognition model is saved, and this is marked as training complete. The set of model parameters of the trained event recognition model is then sent to the signal matching and alarm module.

[0026] The signal matching and alarm module saves the model parameters of the trained event recognition model. Video slices output in real-time from the video stream acquisition module are input into the trained event recognition model to obtain event matching labels for each feature frame in each video slice. The percentage of repeated occurrences of the same event matching label within the same video slice is calculated to obtain event matching information, including alarm content and alarm level.

[0027] Record the event matching labels for each feature frame in each video slice, and count the event matching labels that appear most frequently.

[0028] The specific process of matching alarm information based on events is as follows: If the most frequently occurring event is identified as having no abnormality, no alarm information will be generated, and no subsequent alarm information level matching will be performed. If the most frequently occurring event matching tag is not 0, then the alarm information content is matched according to the accident type pointed to by the event matching tag.

[0029] As a preferred embodiment of the present invention, the alarm information level is obtained based on event matching information, and the specific process is as follows: If the most frequently occurring event matching tag accounts for 80% or more of all event matching tags, then the alarm level is set to Level 3. If the most frequently occurring event matching tag accounts for less than 80% but greater than or equal to 50% of all event matching tags, then the alarm information level is set to level two. If the most frequently occurring event matching tag accounts for less than 50% but greater than or equal to 30% of all event matching tags, then the alarm level is set to Level 1. The wearable device's response module triggers an automatic response mechanism based on the content and level of the received alarm information.

[0030] If the alarm information level is identified as Level 1, the time and location of the event are recorded and output through the display screen of the wearable device; If the alarm information level is identified as level two, detailed event information and the steps the user should take will be displayed on the screen and output through the wearable device's display screen; and the corresponding video clip will be played through the wearable device. If the alarm information level is detected as level three, the wearable device will vibrate violently and emit a high-decibel sound warning to ensure that the user's attention is drawn; and the corresponding video clip will be played through the wearable device.

[0031] Compared with the prior art, the beneficial effects of the present invention are: This invention provides a video stream-based event analysis and wearable device real-time alarm system. Through the collaborative work of multiple modules, it significantly improves the accuracy and real-time performance of event analysis. First, the video stream acquisition module acquires video signals using high frame rates and high-quality encoding formats, ensuring the clarity and accuracy of the video data, facilitating subsequent feature extraction and abnormal event identification. Through keyframe recognition and frequency domain feature extraction, the system can efficiently segment the video stream and identify event segments with significant spatiotemporal variations, thereby improving the efficiency of event analysis.

[0032] The event analysis module and analysis result scoring module proposed in this invention adopt a model based on recurrent neural networks. Through accurate time-series data modeling and event label matching, they can promptly identify and evaluate abnormal events in videos and provide reliable event information for subsequent alarms. The adversarial joint training module uses a generative adversarial network (GAN) to jointly optimize the event recognition model and the matching result analysis model, making the model more robust and generalizing in complex environments, thereby improving the accuracy of event recognition. Attached Figure Description

[0033] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0035] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A or solution B, or a solution that simultaneously satisfies A and B. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0036] Please see Figure 1 As shown, a video stream event analysis and wearable device real-time alarm system includes a video stream acquisition module, an event analysis module, an analysis result scoring module, an adversarial joint training module, a signal matching and alarm module, and a wearable device execution response module.

[0037] The video stream acquisition module accesses network cameras and closed-circuit television cameras in real time, acquiring raw video signals at a preset frame rate and video encoding format. The acquired raw video signals are then encoded in real time using an entropy coding compression algorithm to obtain standard video stream data. A keyframe recognition algorithm extracts feature frames from the standard video stream data to capture abnormal actions. The raw video signal is then segmented using these feature frames to obtain several video slices. Historical data from all extracted video slices is sent to the event analysis module. Finally, all real-time extracted video slices are sent to the adversarial joint training module and the signal matching and alarm module.

[0038] Access webcams and CCTV cameras in real time to capture raw video signals vi={v1, v2, v3, ..., vn} at a preset frame rate of 30fps and H.264 video encoding format.

[0039] Where i is the image frame index, i = 1, 2, ..., n; Where v1, v2, v3, ..., vn are the image frames contained in the original video signal, and n is the total number of image frames contained in the original video signal vi.

[0040] It's important to note that in image processing, the choice of frame rate is closely related to the object's movement speed and the spatiotemporal resolution of the image. Generally, the maximum displacement of an object between two adjacent frames should not exceed the minimum size of the target object. In video stream event analysis applications, when objects such as people and vehicles are captured at 30fps, the time interval between two adjacent frames is 33.33 milliseconds. At this time, the object's displacement typically does not exceed 2 meters. Therefore, 30fps ensures sufficiently high temporal resolution, avoiding the loss or blurring of key features due to insufficient frame rate. Furthermore, the H.264 video encoding format can provide high-quality video images, especially performing well at low bitrates. It can maintain high video clarity even with lower bandwidth.

[0041] Furthermore, the acquired raw video signal is encoded in real time using an entropy coding compression algorithm to obtain standard video stream data. The specific process is as follows: The original video signal V is transformed into the frequency domain by using Discrete Cosine Transform (DCT) to obtain the frequency domain coefficients vi(u, v) of each image frame vi. The transformation formula is as follows:

[0042] Where x and y are the height and width components of the pixel coordinates (x, y) in each image frame vi, respectively; the value range is x=0, 1, 2, ..., M-1; y=0, 1, 2, ..., N-1; M is the image width of image frame vi, and N is the image height of image frame vi; Among them The frequency domain coefficients obtained after transformation represent the low-frequency and high-frequency components of the image in the frequency domain, where u and v are the transformed frequency domain indices, with values ​​ranging from u = 0, 1, 2, ..., M-1; y = 0, 1, 2, ..., N-1; when the frequency domain coefficients The closer the frequency domain indices u and v are to 0, the closer they are to the low-frequency part of image frame vi, i.e., the main contour and structural information of the image, which will serve as key parameters for subsequent abnormal motion capture calculations; when the frequency domain coefficients... The further the frequency domain indices u and v are from 0, the closer they are to the high-frequency part of the image frame vi, that is, the detailed information contained in the edges and textures of the image, which can be reduced or discarded during image compression.

[0043] Among them and Let be the predefined normalized coefficients of the DCT discrete cosine transform, and let its expression be:

[0044] Arrange all frequency domain coefficients of each image frame vi according to their frequency domain indices u and v to obtain the frequency domain coefficient matrix:

[0045] Arrange the frequency domain coefficient matrix Vi corresponding to each image frame vi according to the image frame index i=1,2,...,n to obtain the standard video stream data Vi=V1,V2,...,Vn.

[0046] Furthermore, feature frames are extracted from standard video stream data using a keyframe recognition algorithm. The specific process is as follows: Calculate the sum of the upper diagonal portions of the frequency domain coefficient matrix Vi corresponding to each image frame vi in ​​the standard video stream data:

[0047] Obtain the high-frequency feature values ​​corresponding to each image frame vi. .

[0048] It should be noted that high-frequency eigenvalues This is the summation of the frequency domain coefficients in the frequency domain coefficient matrix where the frequency domain indices u and v are close to 0, containing the main contour and structural information of all images. When a significant change in the high-frequency feature values ​​of adjacent image frames is detected, it indicates that there has been a significant contour displacement, increase or decrease, or change in the object structure between these two adjacent image frames.

[0049] Furthermore, for the high-frequency feature values ​​of each adjacent image frame vi and vi+1 and By using a preset formula:

[0050] Calculate the coefficient of variation of high-frequency features for each image frame vi. .

[0051] If the coefficient of variation of the high-frequency features of image frame vi is found to be greater than the preset threshold of 0.5, it means that the change in the high-frequency feature value from the image frame to the next image frame is greater than 50%, and image frame vi is determined to be a feature frame.

[0052] The original video signal vi = {v1, v2, v3, ..., vn} is segmented based on feature frames to obtain several video slices. The segmentation rule is as follows: If there are several consecutive feature frames, that is, if the image frame index i of multiple feature frames is an arithmetic sequence with a common difference of 1, then the consecutive feature frames are grouped into the same video slice. All image frames between two non-contiguous feature frames are grouped into the same video slice; All generated video slices are sent to the event analysis module and the adversarial joint training module.

[0053] It should be noted that video slicing divides the original video stream into several smaller temporal segments by segmenting continuous feature frames. Each video slice represents a segment of an event or behavior with significant temporal and spatial variations, facilitating subsequent event analysis and anomaly detection. By extracting and segmenting feature frames, the originally long video data can be simplified into small segments with key spatiotemporal features, greatly improving analysis efficiency.

[0054] It should be further explained that video slicing identifies significant changes between image frames based on the high-frequency feature variation coefficient between frames, timely captures sudden movements or structural changes of objects in the scene, and identifies video slices containing continuous events. This not only preserves the temporal continuity of events but also effectively reduces redundant data, thereby improving the computational efficiency and response speed of subsequent analysis modules when processing video streams.

[0055] The event analysis module inputs the received video slices into the event recognition model based on a recurrent neural network to obtain event matching labels for each feature frame.

[0056] An event recognition model based on recurrent neural networks:

[0057] Calculate event matching tags for each video slice ; in, The hidden state of the event recognition model; where Output in hidden state The tanh activation function is used. and Let be the first and second weight matrices of the hidden state to be trained, which are used to map the features of image frame vi in ​​the input video slice and the hidden state at the previous time step to the current state, respectively. Output the hidden state from the previous loop. The hidden state bias term to be trained; in, This is the output layer of the event recognition model, responsible for outputting the hidden states. Perform a fully connected computation and map it to an event label space, matching it with preset event matching labels. ;in For the sigmoid activation function, where and These are the weight matrix and bias term of the output layer to be trained, respectively; Among them, the preset event matching tags include: =0 indicates no abnormal situation; =1 represents a traffic collision accident; =2 indicates that a red light violation occurred; =3 indicates that speeding has occurred; =5 indicates that someone was hit; =6 indicates that a fire has occurred; =7 indicates that there is damage to the infrastructure structure; Send the event recognition model to the adversarial joint training module.

[0058] The analysis results scoring module inputs the event matching labels of each video slice into the matching results analysis model based on a recurrent neural network to obtain the reliability probability of the event matching labels.

[0059] The model is analyzed based on the matching results from a recurrent neural network.

[0060] The event matching label is output by the event analysis module for the image frame vi in ​​the video slice and the corresponding event. Perform a credibility score to obtain the label credibility. .

[0061] in The hidden states of the matching result analysis model; where For output in the hidden state, The tanh activation function is used. The first weight matrix for the hidden state to be trained is used to match the image frame vi in ​​the input video slice with the event labels. The combination of these is mapped to a higher-dimensional space; This is the second weight matrix for the hidden state to be trained, responsible for taking the output of the hidden state from the previous iteration. Mapped to the current state, The hidden state bias term to be trained; in The output gate of the matching result analysis model, For the credibility of the label, For the sigmoid activation function, where and These are the weight matrix and bias term of the output layer to be trained, respectively; Tag credibility The numerical range is 0 to 1, used to evaluate the event matching tags output by the event analysis module. Is it reliable? How trustworthy is the label? A value of 1 indicates that the event matches the tag. The description of image frame vi is completely accurate; the label reliability is high. A value of 0 indicates that the event matches the tag. The description of image frame vi is completely wrong; label credibility. A value greater than 0.8 indicates that the event matches the tag. The description of image frame vi reaches the minimum acceptable probability of credibility.

[0062] The matching result analysis model is sent to the adversarial joint training module.

[0063] The adversarial joint training module uses a GA generative adversarial network to jointly train the event recognition model and the matching result analysis model, and sends the model parameters of the trained event recognition model to the signal matching and alarm module.

[0064] The Gan generative adversarial network consists of a generator G and a discriminator D.

[0065] Generator G contains several event recognition models. Each model generates pseudo-event matching labels based on video slices, simulating the event recognition model's operation on event matching labels for image frames within the video slices. The pseudo-event matching labels and their corresponding video frames from the video slices are then sent to discriminator D. The goal of generator G is to make the discriminator unable to distinguish between the generated pseudo-event matching labels and real event matching labels. Each event recognition model generates pseudo-event matching labels based on video slices, simulating the event recognition model's operation on event matching labels for image frames within the video slices.

[0066] The loss function of generator G optimizes the set of training parameters E1 of each event recognition model in the generator by minimizing the probability that the discriminator determines whether a label is real. Its loss function is:

[0067] Where E1 is the set of parameters to be trained for each event recognition model, E1={ , , , , }; that is, the hidden state of each event recognition model, the first weight matrix, the second weight matrix, the weight matrix of the output layer to be trained, and the bias term of the output layer to be trained; Among them This indicates that the pseudo-events output by the generator match the tags. in This indicates that the discriminator matches the generated pseudo-events with tags. The discrimination result is in the range of 0 to 1, where P(vi) is the set of inputs to the generator, i.e., all video slices.

[0068] The discriminator D comprises a matching result analysis model and a set of preset standard video slices. It performs label confidence calculations on all pseudo-event matching labels and their corresponding image frames vi, and compares the confidence calculation results with the confidence calculation results of the preset standard video slices and their corresponding real event matching labels. The discriminator's goal is to correctly distinguish between real event matching labels and pseudo-event matching labels.

[0069] The loss function of discriminator D is optimized by maximizing the difference between the generated label and the real label. Its loss function is:

[0070] E2 is the set of parameters to be trained for the matching result analysis model, E2={ , , , , }, including the hidden state of the matching result analysis model, the first weight matrix, the second weight matrix, the weight matrix of the output layer to be trained, and the bias terms of the output layer to be trained; Where x is a preset standard video slice, and G(x) is the discrimination result of the discriminator on the standard video slice x; Adversarial training is performed between the generator G and the discriminator D, allowing the generator and discriminator to continuously optimize each other by minimizing the loss function of the generator G. The loss function of discriminator D Implement adversarial training between the generator and the discriminator. The generator continuously generates data with a level of credibility closer to the real labels, while the discriminator continuously improves its ability to identify fake data.

[0071] Furthermore, in each training iteration, the generator and discriminator are optimized using the backpropagation algorithm on the sets E1 and E2 of parameters to be trained, until the loss function of the generator G is optimized. The loss function of discriminator D Once all parameters reach their minimum values, the set E1 containing the training parameters of each event recognition model is saved, marking the training as complete. The set E1 containing the model parameters of the trained event recognition models is then sent to the signal matching and alarm module.

[0072] It's important to note that Generative Adversarial Networks (GANs) can generate high-quality data samples in an unsupervised environment through adversarial training. The game-like mechanism between the generator and discriminator allows the model to self-optimize during training without relying on large amounts of labeled data. Traditional machine learning models typically depend on manually labeled, large-scale datasets, while GANs, through adversarial training, can process unlabeled data more efficiently, reducing the cost and time of data labeling.

[0073] The signal matching and alarm module saves the model parameters of the trained event recognition model. Video slices output in real-time from the video stream acquisition module are input into the trained event recognition model to obtain event matching labels for each feature frame in each video slice. The percentage of repeated occurrences of the same event matching label within the same video slice is calculated to obtain event matching information, including alarm content and alarm level.

[0074] Record the event matching labels for each feature frame in each video slice, and count the event matching labels that appear most frequently.

[0075] The specific process of matching alarm information based on events is as follows: If the most frequently occurring event is identified as matching the tag If the value is 0, no alarm information will be generated, and no subsequent alarm information level matching will be performed. If the most frequently occurring event is identified as matching the tag =1, then a traffic collision alarm message is generated; if the most frequently occurring event matching tag is identified, then... =2, then a red light violation warning message is generated; if the most frequently occurring event is identified, the matching tag is =3, then an overspeed alarm message is generated; if the most frequently occurring event is identified, the matching tag is =5, then generate a personnel collision alarm message; if the most frequently occurring event matching tag is identified as =6, then a fire alarm message is generated; if the most frequently occurring event matching tag is identified as If the value is 7, an infrastructure structure damage alarm message will be generated. Furthermore, the alarm information level is obtained based on the event matching information. The specific process is as follows: If the most frequently occurring event matching tag accounts for 80% or more of all event matching tags, then the alarm level is set to Level 3. If the most frequently occurring event matching tag accounts for less than 80% but greater than or equal to 50% of all event matching tags, then the alarm information level is set to level two. If the most frequently occurring event matching tag accounts for less than 50% but greater than or equal to 30% of all event matching tags, then the alarm level is set to Level 1. The wearable device's response module triggers an automatic response mechanism based on the content and level of the received alarm information.

[0076] If the alarm information level is identified as Level 1, the time and location of the event are recorded and output through the display screen of the wearable device; If the alarm information level is identified as level two, detailed event information and the steps the user should take will be displayed on the screen and output through the wearable device's display screen; and the corresponding video clip will be played through the wearable device. If the alarm information level is detected as level three, the wearable device will vibrate violently and emit a high-decibel sound warning to ensure that the user's attention is drawn; and the corresponding video clip will be played through the wearable device.

Claims

1. A video stream event analysis and wearable device real-time alarm system, comprising a video stream acquisition module, an event analysis module, an analysis result scoring module, and an adversarial joint training module, characterized in that: The video stream acquisition module acquires raw video signals and entropy coding compression algorithm to encode the acquired raw video signals in real time to obtain standard video stream data; feature frames are extracted from the standard video stream data through keyframe recognition algorithm. The original video signal is segmented by feature frames to obtain several video slices; The event analysis module inputs the received video slices into an event recognition model based on a recurrent neural network to obtain event matching labels for each feature frame; The analysis result scoring module inputs the event matching labels of each video slice into the matching result analysis model based on a recurrent neural network to obtain the reliability probability of the event matching labels; The adversarial joint training module uses a GA generative adversarial network to jointly train the event recognition model and the matching result analysis model, and sends the model parameters of the trained event recognition model to the signal matching and alarm module.

2. The video stream event analysis and wearable device real-time alarm system according to claim 1, characterized in that, It also includes a signal matching and alarm module and a wearable device execution response module: The signal matching and alarm module saves the model parameters of the trained event recognition model; it inputs the video slices output in real time by the video stream acquisition module into the trained event recognition model to obtain the event matching labels of each feature frame in each video slice; it counts the proportion of repeated occurrences of the same event matching labels in the same video slice to obtain event matching information, including alarm information content and alarm information level. The wearable device's response module triggers an automatic response mechanism based on the content and level of the received alarm information.

3. The video stream event analysis and wearable device real-time alarm system according to claim 2, characterized in that, The specific process of real-time encoding of the acquired raw video signal using the entropy coding compression algorithm is as follows: The original video signal is converted to the frequency domain by using Discrete Cosine Transform (DCT) to obtain the frequency domain coefficients of each image frame. All the frequency domain coefficients of each image frame are arranged according to the frequency domain index to obtain the frequency domain coefficient matrix. The frequency domain coefficient matrix corresponding to each image frame is arranged according to the image frame index to obtain the standard video stream data.

4. The video stream event analysis and wearable device real-time alarm system according to claim 3, characterized in that, The specific process of extracting feature frames from standard video stream data using keyframe recognition algorithms is as follows: In standard video stream data, the sum of the upper diagonal part of the frequency domain coefficient matrix corresponding to each image frame is calculated to obtain the high-frequency feature value corresponding to each image frame; for the high-frequency feature values ​​of each adjacent image frame, the high-frequency feature variation coefficient of each image frame is calculated. If the coefficient of variation of a high-frequency feature of an image frame is found to be greater than a preset threshold, the image frame is determined to be a feature frame.

5. A video stream event analysis and wearable device real-time alarm system according to claim 1, characterized in that, The specific process of segmenting the original video signal based on feature frames is as follows: If there are several consecutive feature frames, that is, if the image frame indices of multiple feature frames are an arithmetic sequence with a common difference of 1, then the consecutive feature frames are grouped into the same video slice. All image frames between two non-contiguous feature frames are grouped into the same video slice; All generated video slices are sent to the event analysis module and the adversarial joint training module.

6. The video stream event analysis and wearable device real-time alarm system according to claim 1, characterized in that, The reliability probability of event matching tags is as follows: The label confidence level ranges from 0 to 1 and is used to evaluate whether the event matching labels output by the event analysis module are reliable. A label confidence level of 1 means that the event matching label describes the image frame completely accurately; a label confidence level of 0 means that the event matching label describes the image frame completely incorrectly.

7. A video stream event analysis and wearable device real-time alarm system according to claim 1, characterized in that, The specific process of jointly training the event recognition model and the matching result analysis model using a Gaining Adversarial Network (GA) is as follows: A generator G and a discriminator D are generated. The generator G contains several event recognition models. Each event recognition model generates pseudo-event matching labels based on video slices, simulating the event recognition model's operation on the event matching labels of image frames in the video slices. The pseudo-event matching labels and their corresponding video frames in the video slices are then sent to the discriminator D. The goal of the generator G is to make the discriminator unable to distinguish between the generated pseudo-event matching labels and the real event matching labels. Each event recognition model generates pseudo-event matching labels based on the video slice, simulating the event recognition model's operation on event matching labels for image frames in the video slice.

8. A video stream event analysis and wearable device real-time alarm system according to claim 7, characterized in that, The generator and discriminator are specifically as follows: Generator G contains several event recognition models. Each event recognition model generates pseudo-event matching labels based on video slices, simulating the event recognition model's operation on event matching labels of image frames in the video slices. The pseudo-event matching labels and their corresponding video frames in the video slices are then sent to discriminator D. The goal of generator G is to make the discriminator unable to distinguish between the generated pseudo-event matching labels and the real event matching labels. Each event recognition model generates pseudo-event matching labels based on the video slice, simulating the event recognition model's operation on the event matching labels of image frames in the video slice; The loss function of generator G optimizes the set of training parameters E1 of each event recognition model in the generator by minimizing the probability that the discriminator determines whether the label is real; The discriminator D includes a matching result analysis model and a set of preset standard video slices; it performs label confidence calculation on all pseudo-event matching labels and their corresponding image frames, and compares the confidence calculation results with the confidence calculation results of the preset standard video slices and their corresponding real event matching labels; the goal of the discriminator is to correctly distinguish between real event matching labels and pseudo-event matching labels. The loss function of discriminator D is optimized by maximizing the difference between the generated label and the real label; Adversarial training is performed using a generator G and a discriminator D.

9. A video stream event analysis and wearable device real-time alarm system according to claim 8, characterized in that, The specific process of adversarial training using generator G and discriminator D is as follows: The generator and discriminator continuously optimize each other by minimizing the loss functions of the generator G and the discriminator D to achieve adversarial training between them. The generator will continuously generate data that is closer to the credibility of the real labels, while the discriminator will continuously improve its ability to identify fake data. In each training iteration, the generator and discriminator optimize the set of parameters to be trained using the backpropagation algorithm until the loss function of the generator G and the loss function of the discriminator D both reach their minimum values. At this point, the set of parameters to be trained for each event recognition model is saved, marking the training as complete. The set of model parameters of the trained event recognition model is then sent to the signal matching and alarm module.

Images