Nuclear power station security situation early warning scheduling system based on multispectral video perception

Abstract

The invention relates to a nuclear power station security situation early warning scheduling system based on multispectral video perception, and the system comprises an image collection unit which is used for collecting a multimodal video image of a preset region in a nuclear power station; the image preprocessing unit is used for carrying out de-noising processing according to the multi-mode video image and reconstructing a real image in the radiation environment of the nuclear power station; the deep learning unit is used for extracting a feature map in the real image based on the real image, performing target detection according to the feature map, obtaining semantic classification of a detected target and identifying an abnormal behavior of the detected target; and the scheduling decision engine and the plurality of linkage systems are used for controlling the plurality of linkage systems to start emergency response according to the abnormal behavior of the detection target.

Description

Technical Field

[0001] This invention relates to the field of nuclear power plant safety monitoring and dispatching technology, and in particular to a nuclear power plant safety situation early warning and dispatching system based on multispectral video perception. Background Technology

[0002] To ensure the safe operation of nuclear power plants, the plant area needs to implement 24-hour safety monitoring and emergency dispatch.

[0003] The shortcomings of existing technology are: (1) High rates of missed detections, false detections, and high labor costs associated with manual identification and processing of video surveillance: Large-scale video surveillance systems are typically deployed in key areas of the plant, such as reactor buildings, turbine buildings, entrances and exits, corridors, and around high-risk equipment. These systems consist of hundreds or even thousands of cameras, forming a visual perception network covering the entire plant, and are usually equipped with video walls and dedicated monitoring rooms, where security and operations personnel conduct 24-hour uninterrupted manual monitoring. Monitoring personnel need to stare at multiple video screens for extended periods, which easily leads to visual fatigue and decreased attention, causing monitoring efficiency to decrease sharply as working hours increase. Faced with massive video streams, the ability of manual information processing is nearing its limit. Human attention cannot continuously cover all monitoring screens, and the rate of missed detection is extremely high for instantaneous, localized, or simultaneous abnormal events (such as light smoke from minor leaks, momentary violations by personnel, or abnormal flickering of distant images). In addition, judgment criteria vary from person to person, lacking an objective and unified standard.

[0004] (2) Delay in emergency response: From the discovery of anomalies, confirmation of the nature of the event, reporting, to the activation of emergency procedures, the entire process relies on manual decision-making and operation, resulting in a significant delay in emergency response. In nuclear safety incidents where every second counts, these delays of a few minutes or even longer can lead to a dramatic expansion of the consequences of the accident.

[0005] (3) The problem of “information isolation and inability to link” in the system: video surveillance, security alarm, access control, fire protection system, process control system, etc. are usually independent of each other, forming information islands. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to solve a series of technical challenges in existing nuclear power plant safety monitoring systems, such as insufficient initiative, poor real-time performance, low reliability, and information silos, which are caused by reliance on manual interpretation.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, this application provides a nuclear power plant safety situation early warning and dispatch system based on multispectral video perception, comprising: The image acquisition unit is used to acquire multimodal video images of a preset area in a nuclear power plant; The image preprocessing unit is used to perform noise reduction processing on the multimodal video images to reconstruct a real image under the radiation environment of the nuclear power plant. The deep learning unit is used to extract feature maps from the real image, perform target detection based on the feature maps, obtain the semantic classification of the detected target, and identify the abnormal behavior of the detected target. The scheduling decision engine and several linkage systems are used to control the linkage systems to initiate emergency response based on the abnormal behavior of the detected target.

[0008] In one implementation, the multimodal video images include visible light images, infrared images, and gamma-ray images.

[0009] In one implementation, the image preprocessing unit processes the image based on a GAN network, with the input being a multimodal image damaged under the radiation environment of a nuclear power plant, and the output being a real image with radiation filtered out.

[0010] In one implementation, the deep learning unit aligns and fuses multimodal real images through a feature pyramid network, extracts features from different modal real images and fuses them to generate an enhanced feature map containing visible light details, heatmaps and radioactivity intensity information.

[0011] In one implementation, a detection branch of the deep learning unit is used to identify detection targets, the categories of which include people and equipment.

[0012] In one implementation, another branch of the deep learning unit performs continuous frame analysis via a 3D convolutional network to identify predefined abnormal behaviors of the detected target.

[0013] In one implementation, the scheduling decision engine is interconnected with several linked systems based on an API gateway.

[0014] In one implementation, the scheduling decision engine generates control commands corresponding to each linked system based on the abnormal behavior of the detected target, and sends the control commands to each linked system via an API gateway.

[0015] In one implementation, the linkage system includes a DCS, an access control system, and a fire protection system.

[0016] The present invention has the following advantages due to the adoption of the above technical solutions: 1. By employing a GAN-based irradiation noise suppression algorithm to preprocess multispectral video streams, effective filtering of image snowflake noise in high-radiation environments is achieved. This solves the technical problem that traditional image processing algorithms suffer from a sharp drop in signal-to-noise ratio and become unusable in strong irradiation scenarios, providing high-quality image input for subsequent accurate recognition.

[0017] 2. By using multispectral feature fusion technology based on Feature Pyramid Network (FPN), a deep weighted fusion of visible light details, infrared thermal map and gamma-ray intensity information is achieved, generating an enhanced feature map that cannot be directly perceived by the human eye. This overcomes the shortcomings of single-spectral sensors, which have limited perception dimensions and incomplete information, and greatly improves the depth and breadth of situational awareness.

[0018] 3. By adopting a multi-task branch parallel analysis structure, the synchronous detection and identification of multiple targets such as personnel status, equipment defects, and dynamic anomalies are realized. This avoids the waste of computing resources and coordination delays caused by deploying independent models for each task, significantly improving analysis efficiency and reducing system complexity and deployment costs.

[0019] 4. By building a unified API gateway and scheduling decision engine, visual perception information is automatically converted into executable control commands across systems (DCS, access control, fire protection, etc.), breaking down information silos and forming a millisecond-level automatic closed loop from "perception-decision-execution", which completely changes the traditional low-efficiency emergency mode that relies on manual judgment and manual operation. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the multimodal video processing flow in one embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0022] This application provides a nuclear power plant safety situation early warning and dispatch system based on multispectral video perception, including: The image acquisition unit is used to acquire multimodal video images of a preset area in a nuclear power plant; The image preprocessing unit is used to perform noise reduction processing on the multimodal video images to reconstruct a real image under the radiation environment of the nuclear power plant. The deep learning unit is used to extract feature maps from the real image, perform target detection based on the feature maps, obtain the semantic classification of the detected target, and identify the abnormal behavior of the detected target. The scheduling decision engine and several linkage systems are used to control the linkage systems to initiate emergency response based on the abnormal behavior of the detected target.

[0023] In a more detailed embodiment, the system architecture of this application is described.

[0024] The system of this application includes an image acquisition unit, an image preprocessing unit, a deep learning unit, a scheduling decision engine, and several linkage systems.

[0025] The image acquisition unit is used to acquire multimodal video images of a preset area in the nuclear power plant, including visible light images, infrared images, and gamma-ray images.

[0026] The image preprocessing unit is used to perform noise reduction processing on the multimodal video images to reconstruct a real image under the radiation environment of the nuclear power plant.

[0027] like Figure 1 The algorithm inputs visible light, infrared, and gamma-ray video streams from a camera to edge nodes, and processes the images using a GAN-based network. The generator learns and reconstructs image details corrupted by radiation noise, while the discriminator distinguishes the processed image from a clean, real image. Through repeated adversarial training, the model effectively filters random snowflake noise in high-radiation environments.

[0028] Specifically, a GAN network consists of two neural networks trained adversarially: a generator (G) and a discriminator (D). The generator uses a U-Net architecture, taking an image contaminated by irradiation noise as input and outputting a processed, "denoised" image. It strives to generate images as realistic as possible to "fool" the discriminator. The discriminator uses a PatchGAN classifier, taking either the "denoised" image generated by the generator or a real, clean image as input. Its output is a probability value indicating the probability that the input image is a real, clean image. It attempts to distinguish between the "fake" sharp images generated by the generator and real, sharp images. Through repeated iterations of this adversarial process, the generator's denoising ability becomes increasingly stronger, enabling the model to effectively filter random snowflake noise in high-radiation environments.

[0029] The deep learning unit is used to extract feature maps from the real image, perform target detection based on the feature maps, obtain the semantic classification of the detected target, and identify the abnormal behavior of the detected target.

[0030] like Figure 1This method aligns and fuses different denoised spectral images of the same scene using a Feature Pyramid Network (FPN). The network extracts features from different spectral images and performs weighted fusion to generate an enhanced feature map that includes visible light details, heatmaps, and radioactivity intensity information.

[0031] The fused feature maps are fed into a multi-task deep learning model for parallel analysis. One branch, based on the YOLOv8 framework, is used for object detection, identifying people, equipment, tools, etc., and determining whether people are wearing safety helmets, protective clothing, etc. The Neck part employs a Path Aggregation Network (PANet) structure to further enhance the top-down and bottom-up information flow in the feature pyramid, better fusing features at different scales. This is crucial for detecting targets of different sizes (such as small tools and large human bodies). The second branch is dedicated to anomaly detection. It uses a 3D convolutional network to analyze the temporal change patterns of each pixel in consecutive frames, marking which regions belong to dynamic anomalies and assigning anomaly categories, identifying dynamic features of smoke, flames, and liquid leaks.

[0032] The scheduling decision engine and several linkage systems are used to control the linkage systems to initiate emergency response based on the abnormal behavior of the detected target.

[0033] The platform layer integrates with the various existing subsystems of the nuclear power plant (such as the DCS distributed control system, access control system, fire protection system, and public address system) through a unified API gateway. The instruction sequence generated by the scheduling decision engine is directly converted into standard control instructions for each subsystem through the API gateway. This design makes the system of this invention a "smart hub" connecting information silos, realizing deep automatic collaboration across systems.

[0034] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0035] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0036] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A nuclear power plant safety situation early warning and dispatch system based on multispectral video perception, characterized in that, include: The image acquisition unit is used to acquire multimodal video images of a preset area in a nuclear power plant; The image preprocessing unit is used to perform noise reduction processing on the multimodal video images to reconstruct a real image under the radiation environment of the nuclear power plant. The deep learning unit is used to extract feature maps from the real image, perform target detection based on the feature maps, obtain the semantic classification of the detected target, and identify the abnormal behavior of the detected target. The scheduling decision engine and several linkage systems are used to control the linkage systems to initiate emergency response based on the abnormal behavior of the detected target.

2. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 1, characterized in that, The multimodal video images include visible light images, infrared images, and gamma-ray images.

3. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 2, characterized in that, The image preprocessing unit processes images based on a GAN network. The input is a multimodal image damaged by radiation in a nuclear power plant environment, and the output is a real image with radiation filtered out.

4. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 3, characterized in that, The deep learning unit aligns and fuses multimodal real images through a feature pyramid network, extracts features from real images of different modalities and fuses them to generate an enhanced feature map containing visible light details, heatmaps and radioactivity intensity information.

5. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 4, characterized in that, One detection branch of the deep learning unit is used to identify detection targets, and the classification of detection targets includes people and equipment.

6. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 5, characterized in that, Another branch of the deep learning unit performs continuous frame analysis through a 3D convolutional network to identify predefined abnormal behaviors of the detected target.

7. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 1, characterized in that, The scheduling decision engine is interconnected with several linked systems based on the API gateway.

8. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 7, characterized in that, The scheduling decision engine generates control commands corresponding to each linked system based on the abnormal behavior of the detected target, and sends the control commands to each linked system via the API gateway.

9. The nuclear power plant safety situation early warning and dispatch system based on multispectral video perception according to claim 4 or 8, characterized in that, The linkage system includes DCS, access control system and fire protection system.

Images