Method and terminal for monitoring early warning model in standard video

Abstract

The invention discloses a standard video-in-channel monitoring early warning model method and terminal, and the method specifically comprises the steps: creating a universal video access platform, and configuring an early warning model, specifically, the configuration of the early warning model comprises the steps: analyzing a video stream frame by frame, judging whether an image is abnormal or not, and immediately giving out an early warning by a system once the abnormality is detected, real-time playback or storage of related frames is triggered; the video corresponds to a static wide angle, video analysis is carried out through static background comparison and sound analysis, whether a change occurs is judged, and a change result is recorded; a video middle station realizes deep understanding and alarm management of video contents through an intelligent analysis technology and in combination with multi-dimensional information, an algorithm model is trained and optimized by constructing a positive sample set and an error sample set, configuration is adjusted through a visual interface, and the alarm accuracy and efficiency are improved; dynamic video electronic fence early warning is established, and through real-time monitoring and intelligent analysis, abnormity identification is realized, so that an alarm mechanism is triggered.

Description

Technical Field

[0001] The present invention relates to the field of video intelligent monitoring and analysis, and in particular, mainly to a method and a terminal for a standard video mid-stage monitoring early warning model. Background Art

[0002] The traditional video middle platform has the problem of cumbersome and non-uniform access to video terminals, time-consuming access, and requires dedicated personnel to access each terminal device. The traditional video middle platform has a single function, and most of them can only perform live video broadcasts and cannot perform complex business operations. The traditional video middle platform uses a manual mode to observe the real-time picture and perform manual intervention, which results in high costs. The traditional video middle platform cannot meet the current requirements of real-time warning, and achieve early warning, early detection, and early intervention. The traditional video middle platform cannot perform real-time playback of warnings, frame-by-frame analysis, and cannot perform post-event backtracking. The traditional video middle platform has no concept of a sample library for video acquisition and analysis.

[0003] 1. The cumbersome and non-uniform access to video terminals. When accessing video terminals, traditional video middleware often faces a variety of different device models, brands, and communication protocols, which makes the access process cumbersome and difficult to unify. Each terminal may require separate configuration and debugging, which is not only time-consuming and labor-intensive, but also requires dedicated personnel with professional skills to operate. This non-standardized access method not only increases maintenance costs, but also reduces the scalability and flexibility of the system.

[0004] 2. Single function: The functions of traditional video platforms are relatively single, mainly focusing on live video. With the diversification of business needs, users often need more diversified functional support, such as video on demand, video conferencing, video editing, video analysis, etc. However, traditional video platforms often cannot meet these complex needs, limiting their application in a wider range of scenarios.

[0005] 3. The cost of manual monitoring and intervention is high. Traditional video platforms rely on manual real-time image observation and intervention, which not only requires staff to maintain a high degree of concentration and vigilance, but also increases labor costs. In long-term, large-scale monitoring tasks, the efficiency and accuracy of manual monitoring are difficult to guarantee. In addition, manual intervention may also have the problem of response delay, and it is impossible to respond and process immediately.

[0006] 4. Insufficient real-time warning and intervention capabilities. With the improvement of security awareness and changes in business needs, users have higher and higher requirements for real-time warning and intervention capabilities. However, traditional video platforms often cannot meet these requirements. They lack intelligent analysis and warning mechanisms, and cannot analyze and judge video content in real time, thus failing to achieve early warning, early detection, and early intervention. This may result in potential safety hazards and business risks not being handled and resolved in a timely manner.

[0007] 5. Lack of early warning playback and frame-by-frame analysis. After an abnormal situation occurs, users often need to replay and analyze the video frame by frame to understand the overall picture and details of the event. However, traditional video platforms often cannot provide such functional support. They cannot achieve real-time playback and frame-by-frame analysis of early warnings, resulting in users being unable to conduct in-depth review and analysis of events. This not only affects users' understanding and judgment of the event, but also limits the possibility of subsequent processing and improvement.

[0008] 6. The gap between sample library and video acquisition and analysis. Traditional video platforms also have deficiencies in video acquisition and analysis. They often do not have the concept of establishing a sample library and cannot effectively manage and utilize the collected video data. At the same time, they also lack advanced video analysis technology and cannot conduct in-depth analysis and mining of video content. This results in users being unable to obtain more useful information and value from video data, and also limits the development of video platforms in terms of intelligence and automation.

[0009] 7. Lack of visualization and configuration flexibility. Traditional video platforms also have deficiencies in configuration and display. The lack of a visual configuration interface makes rule configuration complicated and error-prone. At the same time, the inability to drag and drop configuration to generate a rule engine in real time increases the difficulty and cost of configuration. In addition, the rule display is not intuitive enough, which increases the difficulty of understanding and maintenance.

[0010] 8. Lack of real-time early warning for dynamic video security verification. Traditional video middle-end monitoring systems have significant limitations when dealing with dynamic image monitoring tasks such as water level monitoring and water leakage detection. These systems often have difficulty in real-time capturing and accurately analyzing water level fluctuations or subtle signs of water leakage, making it impossible to implement an immediate early warning mechanism. This deficiency directly weakens the effectiveness of video monitoring in preventing safety accidents, making it difficult for potential risks to be detected and effectively controlled in a timely manner, thereby increasing the possibility of safety accidents.

[0011] In summary, traditional video platforms have obvious deficiencies and limitations in terms of access to video terminals, functional diversity, real-time monitoring and intervention, early warning playback and frame-by-frame analysis, and sample libraries and video acquisition and analysis. These problems limit the application and development of traditional video platforms in modern society. Therefore, with the continuous advancement of technology and changes in user needs, we need to continue to explore and innovate to promote the development of video platforms in a more intelligent, automated and efficient direction. Summary of the invention

[0012] Based on the problems existing in the prior art, according to one aspect of the present invention, a method for monitoring and early warning model of a standard video center is proposed, including:

[0013] A universal video access platform is created, and an early warning model is configured, wherein the configuration of the early warning model includes:

[0014] S1. Analyze the video stream frame by frame, take screenshots every second for processing, match the previous frame with the next frame, and determine whether there is any abnormality in the picture. Once an abnormality is detected, the system immediately issues an early warning and triggers real-time playback or saves the relevant frames;

[0015] S2, the video corresponds to a static wide angle, and the video is analyzed through static background comparison and sound analysis to determine whether a change has occurred and record the change result;

[0016] S3, the video middle platform uses intelligent analysis technology and combines multi-dimensional information to achieve in-depth understanding of video content and alarm management. It trains and optimizes the algorithm model by building positive sample sets and error sample sets, and then adjusts the configuration through a visual interface to improve the accuracy and efficiency of alarms.

[0017] S4. Establish dynamic video electronic fence warning, through real-time monitoring and intelligent analysis, to identify abnormalities and trigger the alarm mechanism.

[0018] Furthermore, the specific steps of creating a universal video access platform also include:

[0019] Preparation stage: Make sure the IPC camera or other video source device supports the GB / T28181-2016 standard, or forward the video stream to WVP-PRO.

[0020] Configuration stage: According to the network environment and device conditions, modify the WVP-PRO configuration file, including the media server address, port number and device registration information; if you plan to use multiple streaming media nodes for load balancing, you need to add the corresponding node information in the configuration file;

[0021] Registration and authentication: Register the IPC camera or other video source device to the WVP-PRO platform. If the device is successfully registered, it will be authenticated by the corresponding access video manufacturer.

[0022] Video stream access: If the device supports the GB / T28181 standard, the video stream will be pushed to the WVP-PRO platform directly through the 28181 protocol; if the device does not support the GB / T28181 standard, the video stream will be forwarded to the WVP-PRO platform through the RTSP or RTMP protocol;

[0023] Testing and debugging: On the front-end page of WVP-PRO, check whether the connected video stream is displayed normally, and test whether the functions of the platform are running normally;

[0024] Deployment and use: Deploy the WVP-PRO platform to the production environment, perform necessary configuration and optimization, and conduct video management and control through a visual interface.

[0025] WVP-PRO supports multiple video streaming protocols, so different types of video sources can be easily integrated.

[0026] Furthermore, the frame-by-frame analysis of the video stream specifically includes:

[0027] S1.1, Video stream capture: Real-time video stream capture via camera or video file;

[0028] S1.2, frame-by-frame capture: decompose the video stream into individual frames, capture multiple video frames per second, and the number of video frames depends on the frame rate of the video;

[0029] S1.3. Inter-frame matching: Compare adjacent frames to detect changes in the picture, including pixel-level changes or feature-level changes.

[0030] Furthermore, the determining whether there is an abnormality in the picture includes: determining whether a change in the picture constitutes an abnormality according to a preset threshold or model, and the abnormality includes object movement, color mutation or shape change.

[0031] Furthermore, the picture matching adopts a picture matching formula algorithm, specifically a pixel difference method and a block matching method, wherein the pixel difference method directly compares the difference between corresponding pixels of adjacent frames, and the specific formula is as follows:

[0032]

[0033] Among them, P t (x,y) and P t-1 (x, y) are the pixel values ​​at position (x, y) of the current frame and the previous frame respectively, t represents the current frame, t-1 represents the previous frame, W represents the width, H represents the height, and D represents the total difference between the two frames.

[0034] The block matching method divides the image into several small blocks and compares the similarity of these small blocks in adjacent frames. The matching criterion adopts the mean square error MAE. The specific formula is as follows:

[0035]

[0036] Among them, B t (x,y) and B t-1 (x, y) represents the pixel value of a block at position (x, y) in the current frame and the previous frame respectively, N represents the width of the block, and M represents the height of the block.

[0037] In addition to the pixel difference method and block matching method, the picture matching formula algorithm can also use the feature matching method and the optical flow method. The feature matching method extracts features (such as corners, edges, textures, etc.) in the image and compares the changes of these features in adjacent frames. This method is usually more robust than the pixel difference method and the block matching method, but the computational complexity is also higher. The specific algorithm of feature matching depends on the feature extractor and matching strategy used.

[0038] The optical flow method is used to estimate the motion of pixels or feature points in an image. By calculating the optical flow field between adjacent frames, the trajectory and speed of moving objects can be detected. The implementation of the optical flow method is relatively complex, but it can provide richer motion information.

[0039] After detecting the difference between frames, it is necessary to determine whether it constitutes an abnormality based on the preset threshold. The threshold setting can be based on historical data, experience values, or machine learning models. If the difference exceeds the threshold, the early warning mechanism is triggered.

[0040] Furthermore, the step S2 specifically includes:

[0041] S2.1. Obtain video byte stream data through Java code and store the video stream data in the specified OSS file system; specify the change type and set the upper threshold of the sound;

[0042] S2.2, performing video analysis through static background comparison and sound analysis, wherein the static background comparison uses a difference comparison or frame difference function to compare each frame in the video with the initial frame or background template to detect whether there is any obvious change;

[0043] The sound analysis includes measuring the audio level in the video using an audio analysis tool of the software; setting a reference level based on the ambient noise or a preset threshold, and comparing whether the audio level in the video exceeds the reference level, thereby determining whether the sound is too loud;

[0044] S2.3. Record the time of the change, the type of change and the details of the change; record the time period when the sound is too loud, as well as the specific characteristics and source of the sound; record the number of times the picture changes, as well as the time and characteristics of each change;

[0045] S2.4. Automatically summarize the report at regular intervals, and organize all the information recorded during the analysis process into a detailed summary report, which includes the basic information of the video, the analysis method, the changes found, the sound conditions and the number of screen changes, as well as the corresponding conclusions and suggestions; and intuitively display the analysis results through charts, screenshots or video clips.

[0046] Among them, clarify the type of change, such as whether an object moves, the color changes, a new object appears, etc.

[0047] Furthermore, the step S3 specifically includes:

[0048] S3.1. Construct multi-dimensional information, including people, places, events, things and organizations;

[0049] S3.2. Construct a sample set, including a regular sample set and a violation sample set; the regular sample set contains video clips that comply with the specifications or normal situations, such as employees wearing work clothes correctly, normal activities in a safe area, etc.; the violation sample set contains video clips of violations or abnormal situations, such as not wearing work clothes, entering a restricted area, etc.

[0050] S3.3, Algorithm selection and training, including: using deep learning algorithms (such as CNN, ResNet) to extract features of faces, objects and scenes in video frames; using classifiers (such as SVM, Softmax) or target detection algorithms (YOLO, SSD) to classify and detect the extracted features; using loss functions for optimization training;

[0051] S3.4, Visual interface and algorithm adjustment configuration, including designing a user-friendly interface to display real-time video streams, alarm information and historical records; adjusting algorithm configuration by dragging sliders or entering numerical values, and viewing the adjustment effects in real time;

[0052] S3.5, Deployment and testing, including: deploying the trained model to the video platform and integrating it with the video stream processing system; testing the performance and accuracy of the system in actual scenarios, collecting feedback and continuously optimizing.

[0053] Among them, building multi-dimensional information requires demand analysis and concept definition:

[0054] People: Identify faces, body postures, clothing, etc. in videos.

[0055] Location: Locate the place, area, etc. in the video.

[0056] Events: Identify events in the video, such as abnormal behavior, accidents, etc.

[0057] Objects: Identify objects in the video, such as signs of people not wearing work clothes, dangerous goods, etc.

[0058] Organization: Associate people and events to specific organizations or departments for easier management.

[0059] Furthermore, the loss function includes positioning loss and classification loss, and the specific formula is as follows:

[0060]

[0061] Among them, S 2 represents the number of grids, B represents the number of prediction boxes for each grid, Indicates whether the j-th prediction box of the i-th grid is responsible for the target, x i and i Represents the center coordinates of the prediction box, and represents the center coordinates of the real box, p i (c) represents the predicted category probability, represents the true category probability, λ coord represents the positioning loss weight, and c represents the category in the classification loss.

[0062] The localization loss detects the coordinates and size of the bounding box, and the classification loss detects the category probability.

[0063] Furthermore, the step S4 specifically includes:

[0064] S4.1. According to the actual height of the monitoring pool and the viewing angle of the camera, a virtual water level is set in the video image, wherein the virtual water level is realized by a fixed reference line or a grid;

[0065] S4.2, using an image recognition algorithm in the video analysis system to identify the actual water level in the video, wherein the image recognition algorithm includes edge detection, color analysis or pattern recognition;

[0066] S4.3, by calculating the pixel coordinates of the actual water level in the video image, extracting the position information of the realized water level

[0067] S4.4. Calculate the ratio of the actual water level to the virtual water level. If the calculated water level ratio is greater than or equal to 1 / 3, it means that the water level has exceeded 1 / 3 of the virtual water level, and an alarm mechanism is triggered.

[0068] According to a second aspect of the present invention, a terminal of a standard video middle station monitoring and early warning model is proposed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0069] A universal video access platform is created, and an early warning model is configured, wherein the configuration of the early warning model includes:

[0070] The video stream is analyzed frame by frame, and screenshots are taken every second for processing. The previous frame is matched with the next frame to determine whether there is any abnormality in the picture. Once an abnormality is detected, the system immediately issues an early warning and triggers real-time playback or saves the relevant frames.

[0071] The video corresponds to a static wide angle, and the video is analyzed through static background comparison and sound analysis to determine whether changes have occurred and record the change results;

[0072] The video platform uses intelligent analysis technology and multi-dimensional information to achieve in-depth understanding of video content and alarm management. It trains and optimizes algorithm models by building positive sample sets and error sample sets, and then adjusts configurations through a visual interface to improve the accuracy and efficiency of alarms.

[0073] Establish dynamic video electronic fence early warning, through real-time monitoring and intelligent analysis, identify anomalies and trigger the alarm mechanism.

[0074] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0075] 1. Unified access to video management platform: The system supports seamless access to mainstream camera equipment such as Uniview and Hikvision, and realizes unified management and centralized scheduling of video resources. This design greatly simplifies the deployment and maintenance process of the video surveillance system and improves the compatibility and scalability of the system.

[0076] 2. Frame-by-frame analysis of video streams: Using advanced video processing technology, the video streams are analyzed frame by frame to ensure that no key information is missed. This function is of great significance for timely detection of abnormal situations and prevention of security incidents. At the same time, the system supports real-time playback function, which can quickly locate and view relevant video clips after an incident occurs.

[0077] 3. Video and static wide-angle comparison: The system can automatically compare the video screen with the preset static wide-angle image to detect changes in the scene. This function helps to identify unauthorized activities, environmental changes and other abnormal situations, providing strong support for security management.

[0078] 4. Modular design: The system adopts the concepts of people, places, things, objects, and organizations for modular design, breaking down complex video surveillance tasks into multiple manageable modules. This design approach not only improves the flexibility and configurability of the system, but also enables users to easily adjust and optimize monitoring strategies according to actual needs.

[0079] 5. Intelligent identification based on sample sets: The system uses matching regular sample sets and illegal sample sets for intelligent identification and judgment. By continuously learning and optimizing the algorithm model, the system can more accurately identify the target objects and behavior patterns in the video, thereby improving the accuracy and reliability of the early warning.

[0080] 6. Alarm task operation management: The system supports custom alarm rules and task operation management functions. For example, you can set alarm rules for specific scenarios such as "not wearing work clothes", and trigger alarm notifications once relevant behaviors are detected. This function helps enterprises strengthen internal management and improve work efficiency and safety.

[0081] 7. Algorithm library management and adjustment: The system has a rich built-in algorithm library and supports dynamic management and adjustment of the algorithm library. Users can select appropriate algorithm models for video analysis and processing according to actual needs, and can also optimize the analysis effect by adjusting algorithm parameters.

[0082] 8. In order to overcome the shortcomings of traditional video surveillance in dynamic early warning, especially in monitoring factory equipment problems such as rising water levels and water leakage, we can use innovative technical means, such as video virtual rulers and virtual electronic fences, to preset dynamic early warning mechanisms on the screen, so as to achieve timely response and efficient solutions.

[0083] 9. Visual interface configuration: The system provides an intuitive and easy-to-use visual interface configuration tool, which allows users to easily configure and adjust monitoring strategies without writing code. This design greatly reduces the system's usage threshold and learning cost, allowing non-professional technicians to quickly get started and effectively use system functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.

[0085] Figure 1 A schematic diagram of the standard video middle station monitoring and early warning model process according to an embodiment of the present invention is shown.

[0086] Figure 2 A schematic diagram of a general video access platform process according to an embodiment of the present invention is shown.

[0087] Figure 3 A schematic diagram of a video stream frame-by-frame analysis process according to an embodiment of the present invention is shown.

[0088] Figure 4 A schematic diagram of a process for determining a change in a video corresponding to a static wide-angle according to an embodiment of the present invention is shown.

[0089] Figure 5 A schematic diagram of the video multi-dimensional information alarm task management process according to an embodiment of the present invention is shown.

[0090] Figure 6 A schematic diagram of a dynamic video electronic fence warning process according to an embodiment of the present invention is shown.

[0091] Figure 7It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0092] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0093] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0094] Figure 1 FIG. 4 shows a schematic diagram of a standard video middle station monitoring and early warning model process according to an embodiment of the present invention. Figure 1 As shown:

[0095] First, Figure 2 As shown in the figure, a universal video access platform is created. The specific steps include:

[0096] Preparation stage: Make sure the IPC camera or other video source device supports the GB / T28181-2016 standard, or forward the video stream to WVP-PRO.

[0097] Configuration stage: According to the network environment and device conditions, modify the WVP-PRO configuration file, including the media server address, port number and device registration information; if you plan to use multiple streaming media nodes for load balancing, you need to add the corresponding node information in the configuration file;

[0098] Registration and authentication: Register the IPC camera or other video source device to the WVP-PRO platform. If the device is successfully registered, it will be authenticated by the corresponding access video manufacturer.

[0099] Video stream access: If the device supports the GB / T28181 standard, the video stream will be pushed to the WVP-PRO platform directly through the 28181 protocol; if the device does not support the GB / T28181 standard, the video stream will be forwarded to the WVP-PRO platform through the RTSP or RTMP protocol;

[0100] Testing and debugging: On the front-end page of WVP-PRO, check whether the connected video stream is displayed normally, and test whether the functions of the platform are running normally;

[0101] Deployment and use: Deploy the WVP-PRO platform to the production environment, perform necessary configuration and optimization, and conduct video management and control through a visual interface.

[0102] WVP-PRO supports multiple video streaming protocols, so different types of video sources can be easily integrated.

[0103] Next, configure the early warning model, including:

[0104] S1. Analyze the video stream frame by frame, take screenshots every second for processing, match the previous frame with the next frame, and determine whether there is any abnormality in the picture. Once an abnormality is detected, the system immediately issues an early warning and triggers real-time playback or saves the relevant frames;

[0105] Further, such as Figure 3 As shown in the figure, the frame-by-frame analysis of the video stream specifically includes:

[0106] S1.1, Video stream capture: Real-time video stream capture via camera or video file;

[0107] S1.2, frame-by-frame capture: decompose the video stream into individual frames, capture multiple video frames per second, and the number of video frames depends on the frame rate of the video;

[0108] S1.3. Inter-frame matching: Compare adjacent frames to detect changes in the picture, including pixel-level changes or feature-level changes.

[0109] The determining whether there is an abnormality in the picture includes: determining whether a picture change constitutes an abnormality according to a preset threshold or model, and the abnormality includes object movement, color mutation or shape change.

[0110] Furthermore, the picture matching adopts a picture matching formula algorithm, which specifically includes a pixel difference method and a block matching method. The pixel difference method directly compares the difference between corresponding pixels of adjacent frames. The specific formula is as follows:

[0111]

[0112] Among them, P t (x,y) and P t-1 (x, y) are the pixel values ​​at position (x, y) of the current frame and the previous frame respectively, t represents the current frame, t-1 represents the previous frame, W represents the width, H represents the height, and D represents the total difference between the two frames.

[0113] The block matching method divides the image into several small blocks and compares the similarity of these small blocks in adjacent frames. The matching criterion adopts the mean square error MAE. The specific formula is as follows:

[0114]

[0115] Among them, B t (x,y) and B t-1(x, y) represents the pixel value of a block at position (x, y) in the current frame and the previous frame respectively, N represents the width of the block, and M represents the height of the block.

[0116] In addition to the pixel difference method and block matching method, the picture matching formula algorithm can also use the feature matching method and the optical flow method. The feature matching method extracts features (such as corners, edges, textures, etc.) in the image and compares the changes of these features in adjacent frames. This method is usually more robust than the pixel difference method and the block matching method, but the computational complexity is also higher. The specific algorithm of feature matching depends on the feature extractor and matching strategy used.

[0117] The optical flow method is used to estimate the motion of pixels or feature points in an image. By calculating the optical flow field between adjacent frames, the trajectory and speed of moving objects can be detected. The implementation of the optical flow method is relatively complex, but it can provide richer motion information.

[0118] After detecting the difference between frames, it is necessary to determine whether it constitutes an abnormality based on the preset threshold. The threshold setting can be based on historical data, experience values, or machine learning models. If the difference exceeds the threshold, the early warning mechanism is triggered.

[0119] S2, video corresponds to static wide angle, and video analysis is performed through static background comparison and sound analysis to determine whether changes have occurred and record the change results, such as Figure 4 As shown, the specific steps are as follows:

[0120] S2.1. Obtain video byte stream data through Java code and store the video stream data in the specified OSS file system; specify the change type and set the upper threshold of the sound;

[0121] S2.2, performing video analysis through static background comparison and sound analysis, wherein the static background comparison uses a difference comparison or frame difference function to compare each frame in the video with the initial frame or background template to detect whether there is any obvious change;

[0122] The sound analysis includes measuring the audio level in the video using an audio analysis tool of the software; setting a reference level based on the ambient noise or a preset threshold, and comparing whether the audio level in the video exceeds the reference level, thereby determining whether the sound is too loud;

[0123] S2.3. Record the time of the change, the type of change and the details of the change; record the time period when the sound is too loud, as well as the specific characteristics and source of the sound; record the number of times the picture changes, as well as the time and characteristics of each change;

[0124] S2.4. Automatically summarize the report at regular intervals, and organize all the information recorded during the analysis process into a detailed summary report, which includes the basic information of the video, the analysis method, the changes found, the sound conditions and the number of screen changes, as well as the corresponding conclusions and suggestions; and intuitively display the analysis results through charts, screenshots or video clips.

[0125] Among them, clarify the type of change, such as whether an object moves, the color changes, a new object appears, etc.

[0126] S3. The video middle platform uses intelligent analysis technology and multi-dimensional information to achieve in-depth understanding of video content and alarm management. It trains and optimizes the algorithm model by building positive sample sets and error sample sets, and then adjusts the configuration through a visual interface to improve the accuracy and efficiency of alarms. Figure 5 As shown, the specific steps are as follows:

[0127] S3.1. Construct multi-dimensional information, including people, places, events, things and organizations;

[0128] S3.2. Construct a sample set, including a regular sample set and a violation sample set; the regular sample set contains video clips that comply with the specifications or normal situations, such as employees wearing work clothes correctly, normal activities in a safe area, etc.; the violation sample set contains video clips of violations or abnormal situations, such as not wearing work clothes, entering a restricted area, etc.

[0129] S3.3, Algorithm selection and training, including: using deep learning algorithms (such as CNN, ResNet) to extract features of faces, objects and scenes in video frames; using classifiers (such as SVM, Softmax) or target detection algorithms (YOLO, SSD) to classify and detect the extracted features; using loss functions for optimization training;

[0130] S3.4, Visual interface and algorithm adjustment configuration, including designing a user-friendly interface to display real-time video streams, alarm information and historical records; adjusting algorithm configuration by dragging sliders or entering numerical values, and viewing the adjustment effects in real time;

[0131] S3.5, Deployment and testing, including: deploying the trained model to the video platform and integrating it with the video stream processing system; testing the performance and accuracy of the system in actual scenarios, collecting feedback and continuously optimizing.

[0132] Among them, building multi-dimensional information requires demand analysis and concept definition:

[0133] People: Identify faces, body postures, clothing, etc. in videos.

[0134] Location: Locate the place, area, etc. in the video.

[0135] Events: Identify events in the video, such as abnormal behavior, accidents, etc.

[0136] Objects: Identify objects in the video, such as signs of people not wearing work clothes, dangerous goods, etc.

[0137] Organization: Associate people and events to specific organizations or departments for easier management.

[0138] Furthermore, the loss function includes positioning loss and classification loss, and the specific formula is as follows:

[0139]

[0140] Among them, S 2 represents the number of grids, B represents the number of prediction boxes for each grid, Indicates whether the j-th prediction box of the i-th grid is responsible for the target, x i and i Represents the center coordinates of the prediction box, and represents the center coordinates of the real box, p i (c) represents the predicted category probability, represents the true category probability, λ coord represents the positioning loss weight, and c represents the category in the classification loss.

[0141] The localization loss detects the coordinates and size of the bounding box, and the classification loss detects the category probability.

[0142] S4. Establish dynamic video electronic fence warning, through real-time monitoring and intelligent analysis, to identify abnormalities and trigger alarm mechanisms, such as Figure 6 As shown, the specific steps include:

[0143] S4.1. According to the actual height of the monitoring pool and the viewing angle of the camera, a virtual water level is set in the video image, wherein the virtual water level is realized by a fixed reference line or a grid;

[0144] S4.2, using an image recognition algorithm in the video analysis system to identify the actual water level in the video, wherein the image recognition algorithm includes edge detection, color analysis or pattern recognition;

[0145] S4.3, by calculating the pixel coordinates of the actual water level in the video image, extracting the position information of the realized water level

[0146] S4.4. Calculate the ratio of the actual water level to the virtual water level. If the calculated water level ratio is greater than or equal to 1 / 3, it means that the water level has exceeded 1 / 3 of the virtual water level, and an alarm mechanism is triggered.

[0147] In this embodiment, a dynamic water leakage electronic fence early warning is adopted, including:

[0148] 1) Define virtual electronic fence areas. Define one or more virtual electronic fence areas in the video screen. These areas are usually set according to the actual layout of the monitoring scene and potential leakage points. You can use video analysis software or custom algorithms to draw these areas in the picture and save their coordinate information.

[0149] 2) Image preprocessing: Preprocessing the video frames, including grayscale, denoising, contrast enhancement and other steps, to improve the accuracy and efficiency of subsequent processing.

[0150] 3) Water leakage feature detection

[0151] 3.1) Static feature detection

[0152] Color analysis: Analyzes color changes in pixels within the fence area, especially areas that differ greatly from the normal background color. Water leaks usually cause darker colors (such as dark water stains) or reflective (such as wet surfaces).

[0153] Texture analysis: Detects abnormal changes in texture within the fence area, such as the sudden appearance of irregular textures or blurred areas.

[0154] 3.2) Dynamic feature detection

[0155] Motion detection: Use techniques such as frame difference or optical flow to detect motion changes within the fence area. Water leaks may cause dynamic changes in water flow.

[0156] Change detection: Continuously monitor changes in pixel values ​​within the fence area, especially sudden and significant changes, which may indicate the occurrence of a water leak.

[0157] 4) Leakage judgment logic, combining static and dynamic feature detection results, designing leakage judgment logic. This usually involves setting one or more thresholds to determine whether the detected features are sufficient to indicate a leak.

[0158] Threshold setting: Set appropriate thresholds for color differences, texture changes, motion intensity, etc. based on the experiment and scene characteristics.

[0159] Comprehensive judgment: If multiple water leakage characteristics are met simultaneously in the fence area (such as darkening of color and abnormal texture, or detection of significant movement changes), it is judged as a water leakage event.

[0160] 5) Alarm and response

[0161] Once a water leak is detected, the alarm mechanism is triggered immediately, such as sending alarm information to relevant personnel, recording event logs, etc. At the same time, corresponding emergency response measures can be initiated as needed.

[0162] 6) Continuous Optimization

[0163] Based on feedback and effect evaluation in actual applications, the settings of the virtual electronic fence, image processing algorithms and water leakage judgment logic are continuously optimized to improve the accuracy and reliability of the system.

[0164] Specifically, by integrating advanced image recognition and intelligent analysis algorithms into the video surveillance system, a virtual ruler and virtual electronic fence can be constructed. The virtual ruler can track and measure the height changes of the water level in real time. Once the water level exceeds the preset safety threshold, the system will immediately trigger an early warning signal to notify relevant personnel to take countermeasures. At the same time, the virtual electronic fence is set around the key equipment area. When abnormal water flow, water leakage or other abnormal conditions are detected, an alarm will also be quickly issued to ensure that the problem can be discovered and handled immediately.

[0165] This dynamic early warning system based on video virtual scale and virtual electronic fence not only improves the intelligence level of video surveillance, but also significantly enhances its response speed and efficiency in solving factory equipment problems. It enables monitoring personnel to remotely and in real time grasp the on-site situation, quickly judge and respond to potential safety hazards, thereby effectively reducing the occurrence of safety accidents and ensuring the safe production and operation of the factory.

[0166] In summary, the video intelligent monitoring and analysis system of the present invention provides enterprises with efficient, intelligent and flexible video monitoring solutions through a number of innovative technologies such as unified access, frame-by-frame analysis, static comparison, modular design, intelligent recognition, alarm management, algorithm library adjustment and visual configuration.

[0167] According to a second aspect of the present invention, a terminal of a standard video middle station monitoring and early warning model is proposed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0168] A universal video access platform is created, and an early warning model is configured, wherein the configuration of the early warning model includes:

[0169] The video stream is analyzed frame by frame, and screenshots are taken every second for processing. The previous frame is matched with the next frame to determine whether there is any abnormality in the picture. Once an abnormality is detected, the system immediately issues an early warning and triggers real-time playback or saves the relevant frames.

[0170] The video corresponds to a static wide angle, and the video is analyzed through static background comparison and sound analysis to determine whether changes have occurred and record the change results;

[0171] The video platform uses intelligent analysis technology and multi-dimensional information to achieve in-depth understanding of video content and alarm management. It trains and optimizes algorithm models by building positive sample sets and error sample sets, and then adjusts configurations through a visual interface to improve the accuracy and efficiency of alarms.

[0172] Establish dynamic video electronic fence early warning, through real-time monitoring and intelligent analysis, identify anomalies and trigger the alarm mechanism.

[0173] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 suitable for implementing an electronic device of an embodiment of the present application. Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0174] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0175] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0176] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0177] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0178] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0179] The modules involved in the embodiments of the present application may be implemented by software or by hardware.

[0180] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist alone and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: creates a general video access platform and configures an early warning model, wherein the configuration early warning model includes: analyzing the video stream frame by frame to determine whether there is an abnormality in the picture, once an abnormality is detected, the system immediately issues an early warning, and triggers real-time playback or saves the relevant frame; the video corresponds to a static wide angle, and performs video analysis through static background comparison and sound analysis to determine whether a change has occurred and record the change result; the video middle station uses intelligent analysis technology and combines multi-dimensional information to achieve in-depth understanding and alarm management of video content, trains and optimizes the algorithm model by constructing a positive sample set and an error sample set, and then adjusts the configuration through a visual interface to improve the accuracy and efficiency of the alarm; establishes a dynamic video electronic fence early warning, and recognizes abnormalities through real-time monitoring and intelligent analysis to trigger an alarm mechanism.

[0181] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for monitoring and early warning model of a standard video platform, characterized in that: The following steps are involved: A universal video access platform is created, and an early warning model is configured, wherein the configuration of the early warning model includes: S1. Analyze the video stream frame by frame, take screenshots every second for processing, match the previous frame with the next frame, and determine whether there is any abnormality in the picture. Once an abnormality is detected, the system immediately issues an early warning and triggers real-time playback or saves the relevant frames; S2, the video corresponds to a static wide angle, and the video is analyzed through static background comparison and sound analysis to determine whether a change has occurred and record the change result; S3, the video middle platform uses intelligent analysis technology and combines multi-dimensional information to achieve in-depth understanding of video content and alarm management. It trains and optimizes the algorithm model by building positive sample sets and error sample sets, and then adjusts the configuration through a visual interface to improve the accuracy and efficiency of alarms. S4. Establish dynamic video electronic fence warning, through real-time monitoring and intelligent analysis, to identify abnormalities and trigger the alarm mechanism.

2. The method according to claim 1, characterized in that The specific steps of creating a universal video access platform also include: Preparation stage: Make sure the IPC camera or other video source device supports the GB / T28181-2016 standard, or forward the video stream to WVP-PRO. Configuration stage: According to the network environment and device conditions, modify the WVP-PRO configuration file, including the media server address, port number and device registration information; if you plan to use multiple streaming media nodes for load balancing, you need to add the corresponding node information in the configuration file; Registration and authentication: Register the IPC camera or other video source device to the WVP-PRO platform. If the device is successfully registered, it will be authenticated by the corresponding access video manufacturer. Video stream access: If the device supports the GB / T28181 standard, the video stream will be pushed to the WVP-PRO platform directly through the 28181 protocol; if the device does not support the GB / T28181 standard, the video stream will be forwarded to the WVP-PRO platform through the RTSP or RTMP protocol; Testing and debugging: On the front-end page of WVP-PRO, check whether the connected video stream is displayed normally, and test whether the functions of the platform are running normally; Deployment and use: Deploy the WVP-PRO platform to the production environment, perform necessary configuration and optimization, and conduct video management and control through a visual interface.

3. The method according to claim 1, characterized in that The frame-by-frame analysis of the video stream specifically includes: S1.1, Video stream capture: Real-time video stream capture via camera or video file; S1.2, frame-by-frame capture: decompose the video stream into individual frames, capture multiple video frames per second, and the number of video frames depends on the frame rate of the video; S1.

3. Inter-frame matching: Compare adjacent frames to detect changes in the picture, including pixel-level changes or feature-level changes.

4. The method according to claim 1, characterized in that: The determining whether there is an abnormality in the picture includes: determining whether a picture change constitutes an abnormality according to a preset threshold or model, and the abnormality includes object movement, color mutation or shape change.

5. The method according to claim 1, characterized in that The picture matching adopts a picture matching formula algorithm, specifically a pixel difference method and a block matching method. The pixel difference method directly compares the difference between corresponding pixels of adjacent frames. The specific formula is as follows: Among them, P t (x,y) and P t-1 (x, y) are the pixel values ​​at position (x, y) of the current frame and the previous frame respectively, t represents the current frame, t-1 represents the previous frame, W represents the width, H represents the height, and D represents the total difference between the two frames.

6. The method according to claim 5, characterized in that The block matching method divides the image into several small blocks and compares the similarity of these small blocks in adjacent frames. The matching criterion adopts the mean square error MAE. The specific formula is as follows: Among them, B t (x,y) and B t-1 (x, y) represents the pixel value of a block at position (x, y) in the current frame and the previous frame respectively, N represents the width of the block, and M represents the height of the block.

7. The method according to claim 1, characterized in that The step S2 specifically includes: S2.

1. Obtain video byte stream data through Java code and store the video stream data in the specified OSS file system; specify the change type and set the upper threshold of the sound; S2.2, performing video analysis through static background comparison and sound analysis, wherein the static background comparison uses a difference comparison or frame difference function to compare each frame in the video with the initial frame or background template to detect whether there is any obvious change; The sound analysis includes measuring the audio level in the video using an audio analysis tool of the software; setting a reference level based on the ambient noise or a preset threshold, and comparing whether the audio level in the video exceeds the reference level, thereby determining whether the sound is too loud; S2.

3. Record the time of the change, the type of change and the details of the change; record the time period when the sound is too loud, as well as the specific characteristics and source of the sound; record the number of times the picture changes, as well as the time and characteristics of each change; S2.

4. Automatically summarize the report at regular intervals, and organize all the information recorded during the analysis process into a detailed summary report, which includes the basic information of the video, the analysis method, the changes found, the sound conditions and the number of screen changes, as well as the corresponding conclusions and suggestions; and intuitively display the analysis results through charts, screenshots or video clips.

8. The method according to claim 1, characterized in that The step S3 specifically includes: S3.

1. Construct multi-dimensional information, including people, places, events, things and organizations; S3.2, construct sample sets, including regular sample sets and illegal sample sets; S3.3, Algorithm selection and training, including: using deep learning algorithms to extract features of faces, objects and scenes in video frames; using classifiers or target detection algorithms to classify and detect the extracted features; using loss functions for optimization training; S3.4, Visual interface and algorithm adjustment configuration, including designing a user-friendly interface to display real-time video streams, alarm information and historical records; adjusting algorithm configuration by dragging sliders or entering numerical values, and viewing the adjustment effects in real time; S3.5, Deployment and testing, including: deploying the trained model to the video platform and integrating it with the video stream processing system; testing the performance and accuracy of the system in actual scenarios, collecting feedback and continuously optimizing.

9. The method according to claim 8, characterized in that The loss function includes positioning loss and classification loss, and the specific formula is as follows: Among them, S 2 represents the number of grids, B represents the number of prediction boxes for each grid, Indicates whether the j-th prediction box of the i-th grid is responsible for the target, x i and i Represents the center coordinates of the prediction box, and represents the center coordinates of the real box, p i (c) represents the predicted category probability, represents the true category probability, λ coord represents the positioning loss weight, and c represents the category in the classification loss.

10. The method according to claim 1, characterized in that The step S4 specifically includes: S4.

1. According to the actual height of the monitoring pool and the viewing angle of the camera, a virtual water level is set in the video image, wherein the virtual water level is realized by a fixed reference line or a grid; S4.2, using an image recognition algorithm in the video analysis system to identify the actual water level in the video, wherein the image recognition algorithm includes edge detection, color analysis or pattern recognition; S4.3, by calculating the pixel coordinates of the actual water level in the video image, extracting the position information of the realized water level S4.

4. Calculate the ratio of the actual water level to the virtual water level. If the calculated water level ratio is greater than or equal to 1 / 3, it means that the water level has exceeded 1 / 3 of the virtual water level, and an alarm mechanism is triggered.

11. A terminal for monitoring and warning model of a standard video middle station, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: A universal video access platform is created, and an early warning model is configured, wherein the configuration of the early warning model includes: The video stream is analyzed frame by frame, and screenshots are taken every second for processing. The previous frame is matched with the next frame to determine whether there is any abnormality in the picture. Once an abnormality is detected, the system immediately issues an early warning and triggers real-time playback or saves the relevant frames. The video corresponds to a static wide angle, and the video is analyzed through static background comparison and sound analysis to determine whether changes have occurred and record the change results; The video platform uses intelligent analysis technology and multi-dimensional information to achieve in-depth understanding of video content and alarm management. It trains and optimizes algorithm models by building positive sample sets and error sample sets, and then adjusts configurations through a visual interface to improve the accuracy and efficiency of alarms. Establish dynamic video electronic fence early warning, through real-time monitoring and intelligent analysis, identify anomalies and trigger the alarm mechanism.

12. A computer program product, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images