Foreign matter detection edge device, visual management device, foreign matter monitoring device and method

Abstract

The invention provides foreign matter detection edge equipment, visual management equipment and a foreign matter monitoring device and method. The foreign matter detection edge equipment comprises a sensor module and a foreign matter falling data edge processing module connected with the sensor module. The sensor module comprises an event camera and an industrial camera. The foreign matter falling data edge processing module comprises an edge storage module and a GPU data processing unit. And the GPU data processing unit is used for inputting the event stream and the image video stream data into a small foreign matter falling detection algorithm, receiving a prediction result output by the small foreign matter falling detection algorithm, generating alarm push information when the prediction result is that foreign matter falling is detected, and transmitting the alarm push information to the visual management equipment. According to the invention, cross-modal information fusion of the event camera and the industrial camera is adopted to solve the problem of difficult monitoring of fine foreign object targets in a key foreign object prevention area of a nuclear power plant.

Description

Technical Field

[0001] The present application belongs to the technical field of nuclear power foreign matter monitoring, and specifically relates to a foreign matter detection edge device, a visualization management device, a foreign matter monitoring apparatus and a method. Background Art

[0002] During the overhaul, the primary reactor is in an open state, the reactor pool is filled with water, and the various cross-operations in the core area are complicated. There are many maintenance personnel and equipment tools, and foreign objects often fall into the refueling pool. If foreign objects fall into the refueling pool and enter the primary system, it will pose a serious threat to the safe production and operation of the nuclear power plant. Preventing foreign objects is an important task in the maintenance process of a nuclear power plant and an important guarantee for the safe operation of the unit. The reactor refueling pool is a first-level key foreign object prevention area. In order to reduce the risk of foreign objects affecting the safe production and operation of nuclear power plants, it is necessary to monitor and locate foreign objects in key areas such as the reactor refueling pool of a nuclear power plant, detect foreign object falling incidents as soon as possible, and handle foreign objects to avoid safety hazards.

[0003] Falling foreign objects in the nuclear power plant reactor refueling pool business scenario are characterized by small size, fast falling speed and complex image background. Manual review of monitoring videos or foreign object falling detection solutions based on traditional cameras are often difficult to automatically capture tiny high-speed falling foreign objects in a timely manner due to the limitations of camera frame rate and time resolution. This is the business pain point of foreign object monitoring in nuclear power plant reactor refueling pools. Summary of the invention

[0004] In view of this, the present application is committed to providing a foreign object detection edge device and visualization management equipment, a foreign object monitoring device and method, which adopts event cameras and industrial cameras as perception sensors, uses edge processing technology and high-performance machine vision algorithm design, and develops visualization software to solve the problem of difficult monitoring of small foreign object targets in key foreign object prevention areas of nuclear power plants.

[0005] In a first aspect, the present application provides a foreign object detection edge device, which includes a sensor module and a foreign object falling data edge processing module connected to the sensor module. The sensor module includes an event camera and an industrial camera. The event camera is used to collect event stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant. The industrial camera is used to collect image video stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant. The foreign object falling data edge processing module includes an edge storage module and a GPU data processing unit. The edge storage module is used to obtain event stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant from the event camera, obtain image video stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant from the industrial camera, transmit the event stream data and image video stream data to the GPU data processing unit running a small foreign object falling detection algorithm, and receive and store the prediction results transmitted by the GPU data processing unit. The GPU data processing unit is used to input event stream and image video stream data into the small foreign object falling detection algorithm, receive the prediction results output by the small foreign object falling detection algorithm, and generate an alarm push information and transmit it to the visualization management device when the prediction result is that a foreign object has been detected.

[0006] In a specific embodiment of the present application, the edge storage module is also used to pre-process the image video data stream obtained from the industrial camera, and then compress it through hardware hard coding and store it as a file in mp4 format.

[0007] In a specific embodiment of the present application, the GPU data processing unit runs a small foreign object fall detection algorithm, and the GPU data processing unit interacts with the edge storage module. The GPU data processing unit is used to process the event stream data and image video stream data transmitted by the edge storage module using the small foreign object fall detection algorithm, and transmit the processed prediction results to the edge storage module for storage.

[0008] In a specific embodiment of the present application, reference Figure 2 ,The small foreign object falling detection algorithm includes an ,event processing module, a spatial and temporal feature mixing module, a ,multi-layer and multi-stage feature extraction module, a feature transfer and ,anchor generation module, a classification and regression module, and a post-processing module.

[0009] The event processing module is used to encode the polarity, time and position of the event point. The event is represented as a four-tuple (x, y, t, p), where (x, y) is the pixel position where the event occurs, t is the timestamp, and p is the polarity of the event (positive or negative). It is converted into an event spike tensor (EST), where the first dimension represents polarity, the second dimension represents time, the third dimension represents the height of the event camera, and the fourth dimension represents the width of the event camera, so as to be compatible with the subsequent network structure; then, by flattening the time dimension and polarity dimension, a 3D tensor compatible with 2D convolution is obtained.

[0010] The spatial and temporal feature hybrid module is used to extract the spatial and temporal features of event stream data. It uses convolutional layers and long short-term memory (LSTM) units to extract spatial features and aggregate temporal features, respectively, and interacts local and global features through a multi-axis self-attention mechanism.

[0011] The hierarchical multi-stage feature extraction module combines multiple recurrent visual transformer modules to form a multi-stage hierarchical backbone network to learn the feature graph of event stream data stage by stage.

[0012] The feature transfer and anchor generation module passes the feature map obtained by multi-stage network processing to the object detection framework and generates a series of anchors of potential object positions and sizes on the feature map.

[0013] The classification and regression module is the target detection framework of the small foreign object falling detection algorithm, which consists of a feature pyramid network (FPN) and a detection head. The detection head contains two branches: a classification head and a bounding box regression head. Among them, the feature pyramid network (FPN) is used for multi-scale feature map fusion to improve the algorithm's small target detection performance. The classification head is used to predict whether each anchor point contains a target and the target category, and the bounding box regression head is used to fine-tune the position of the anchor point to determine the exact position of the target.

[0014] The post-processing module applies post-processing techniques such as non-maximum suppression (NMS) to select the best bounding box from a series of overlapping bounding boxes output by the target detection framework to obtain the final prediction result.

[0015] In a specific embodiment of the present application, the spatial and temporal feature hybrid module includes an improved spatial attention module (Spatial Attention Module, SAM) and a gated recurrent unit (Gated Recurrent Unit, GRU) module. The improved spatial attention module integrates the local attention mechanism and the global attention mechanism to extract local and global features. The gated recurrent unit module is used to extract the temporal feature information of the event stream data to improve the long-term dependency of the event sequence data.

[0016] In a specific embodiment of the present application, the hierarchical multi-stage feature extraction module is also used to, when combining multiple recurrent visual transformer modules, receive the features of the previous stage as input at each stage, and can selectively use the long short-term memory network (LSTM) state of the previous time step to calculate the features of the next stage.

[0017] The second aspect of the present application provides a visual management device, which is integrated with visual management software. The visual management device interacts with the foreign object detection edge device of the first aspect of the present application. The visual management software is used to receive the alarm push information transmitted by the foreign object detection edge device of the first aspect of the present application when the prediction result is that a foreign object has fallen, and to perform visual display.

[0018] In a specific embodiment of the present application, the visual management device includes a fall alarm display and management system. The fall alarm display and management system is used to generate alarm information according to the alarm push information transmitted by the foreign object detection edge device of the embodiment of the present application, and communicate with the monitoring server, and display it in real time on the interface of the device integrated visual management software for users to view, and at the same time has a query function for retrieving foreign object fall alarm information according to time.

[0019] In a specific embodiment of the present application, the visual management device further includes a video review system. The video review system is used to call videos stored on the local hard disk of the device. The video review system has one or more functions of full screen, screenshot, start and stop, revisit positioning, and fast and slow playback of the viewed picture.

[0020] In a specific implementation of the present application, the visual management device further includes a multi-angle display monitoring system. The multi-angle display monitoring system is used to synchronously play multi-angle monitoring videos.

[0021] In a specific embodiment of the present application, the visualization management device also includes a system status and alarm hardware management system. The system status and alarm hardware management system is used to display the number of successfully connected foreign object detection edge devices, and has the function of configuring whether the sound and light alarm device is turned on and whether the remote computer room is allowed to log in and access the system through the internal LAN.

[0022] In a specific implementation of the present application, the visual management device further includes a user management system. The user management system is used to manage system user information, including user number, user name, mobile phone number, status and creation time, and supports administrator user's add, delete, modify and query operations.

[0023] In a specific implementation of the present application, the visual management device further includes a streaming media component, a codec component and a database. The streaming media component is used to read, transmit and push streaming media data; the codec component is used to encode and decode streaming media data; the database is used to store and read user information, configuration status and alarm information of the system.

[0024] The third aspect of the present application provides a nuclear power plant reactor refueling pool foreign body monitoring device, which includes the foreign body detection edge device of the first aspect of the present application and the visualization management device of the second aspect of the present application. A small foreign body fall detection algorithm is run in the foreign body detection edge device. The visualization management device is integrated with visualization management software. The foreign body detection edge device is used to collect event and image video stream data in the monitoring area of ​​the nuclear power plant reactor refueling pool, use the small foreign body fall detection algorithm to process the event and image video stream, and generate an alarm push information and transmit it to the visualization management device when the predicted result after processing is that a foreign body falls. The visualization management device interacts with the foreign body detection edge device of the first aspect of the present application for data, and is used to receive the alarm push information transmitted by the foreign body detection edge device of the first aspect of the present application when the predicted result is that a foreign body falls, and perform visual display.

[0025] A fourth aspect of the present application provides a method for monitoring foreign objects in a reactor refueling pool of a nuclear power plant, the method comprising: using an event camera in a foreign object detection edge device in a foreign object monitoring device for a reactor refueling pool of a nuclear power plant to obtain event stream data in a monitoring area of ​​the reactor refueling pool of the nuclear power plant; using an industrial camera in the foreign object detection edge device to obtain image video stream data in the monitoring area of ​​the reactor refueling pool of the nuclear power plant; using an edge storage module in the foreign object detection edge device to obtain event stream data in the monitoring area of ​​the reactor refueling pool of the nuclear power plant from the event camera, and obtaining image video stream data in the monitoring area of ​​the reactor refueling pool of the nuclear power plant from the industrial camera. , transmit the event stream data and image video stream data to the GPU data processing unit in the foreign object detection edge device; use the GPU data processing unit in the foreign object detection edge device to process the event stream data and image video stream data transmitted by the edge storage module using a small foreign object falling detection algorithm, and transmit the processed prediction result to the edge storage module, and when the prediction result is that a foreign object is detected falling, an alarm push information is generated and transmitted to the visualization management device in the foreign object monitoring device of the reactor refueling pool of the nuclear power plant; use the edge storage module in the foreign object detection edge device to receive and store the prediction result transmitted by the GPU data processing unit; use the visualization management device for visualization display.

[0026] The beneficial effect of the technical solution of the present application is that event cameras and industrial cameras are used as perception sensors, and through edge processing technology and high-performance machine vision algorithm design, as well as the development of visualization software, the problem of accurate monitoring of high-speed weak falling foreign objects in key foreign object prevention areas of nuclear power plants is effectively solved, and alarm information and alarm monitoring videos are output in real time, providing an effective engineering technical solution for foreign object monitoring in the reactor core refueling pool of nuclear power plants, and realizing intelligent improvement of foreign object prevention management. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Shown is a schematic structural diagram of a foreign matter monitoring device for a nuclear power plant reactor refueling pool provided by an embodiment of the present application.

[0028] Figure 2 Shown is a schematic diagram of a small foreign object falling detection algorithm provided in an embodiment of the present application.

[0029] Figure 3 Shown is a schematic flow chart of a method for monitoring foreign matter in a nuclear power plant reactor refueling pool provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] At least one embodiment of the present application provides a foreign object detection edge device, referring to Figure 1 , the foreign object detection edge device includes a sensor module and a foreign object falling data edge processing module connected to the sensor module. The sensor module includes an event camera and an industrial camera. The event camera is used to collect event stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant. The industrial camera is used to collect image video stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant. The foreign object falling data edge processing module includes an edge storage module and a GPU data processing unit. The edge storage module is used to obtain event stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant from the event camera, obtain image video stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant from the industrial camera, transmit event stream data and image video stream data to a GPU data processing unit running a small foreign object falling detection algorithm, and receive and store the prediction results transmitted by the GPU data processing unit. The GPU data processing unit is used to input event stream and image video stream data into the small foreign object falling detection algorithm, receive the prediction results output by the small foreign object falling detection algorithm, and generate an alarm push information and transmit it to the visualization management device when the prediction result is that a foreign object has been detected.

[0032] It should be noted that event cameras, due to their extremely high refresh rate and precise time resolution, can achieve a refresh rate equivalent to more than 10,000 frames per second and capture very fast moving objects. Therefore, the solution of using event cameras as important perception sensors can effectively solve the problem of monitoring small foreign objects in key foreign object protection areas of nuclear power plants.

[0033] In at least one embodiment of the present application, the edge storage module is also used to pre-process the image video data stream obtained from the industrial camera, and then compress it through hardware hard coding and store it as an mp4 format file. This is conducive to looping detection and pulling video directories on the monitoring server (also known as the server side) to achieve decoding and playback on the monitoring server.

[0034] In at least one embodiment of the present application, the GPU data processing unit runs a small foreign object fall detection algorithm, and the GPU data processing unit interacts with the edge storage module. The GPU data processing unit is used to process the event stream data and image video stream data transmitted by the edge storage module using the small foreign object fall detection algorithm, and transmit the processed prediction results to the edge storage module for storage.

[0035] It should be noted that the small foreign object falling detection algorithm uses the Recurrent Vision Transformer (RVT) module to process the event stream data to determine whether there is a fallen foreign object in the monitoring area.

[0036] In at least one embodiment of the present application, a small foreign object falling detection algorithm includes an event processing module, a spatial and temporal feature mixing module, a multi-layer and multi-stage feature extraction module, a feature transfer and anchor point generation module, a classification and regression module, and a post-processing module.

[0037] The event processing module is used to encode the polarity, time and position of the event point. The event is represented as a four-tuple (x, y, t, p), where (x, y) is the pixel position where the event occurs, t is the timestamp, and p is the polarity of the event (positive or negative). It is converted into an event spike tensor (EST), where the first dimension represents polarity, the second dimension represents time, the third dimension represents the height of the event camera, and the fourth dimension represents the width of the event camera, so as to be compatible with the subsequent network structure; then, by flattening the time dimension and polarity dimension, a 3D tensor compatible with 2D convolution is obtained.

[0038] The spatial and temporal feature hybrid module is used to extract the spatial and temporal features of event stream data. It uses convolutional layers and long short-term memory (LSTM) units to extract spatial features and aggregate temporal features, respectively, and interacts local and global features through a multi-axis self-attention mechanism.

[0039] The hierarchical multi-stage feature extraction module combines multiple recurrent visual transformer modules to form a multi-stage hierarchical backbone network to learn the feature graph of event stream data stage by stage.

[0040] The feature transfer and anchor generation module passes the feature map obtained by multi-stage network processing to the object detection framework and generates a series of anchors of potential object positions and sizes on the feature map.

[0041] The classification and regression module is the target detection framework of the small foreign object falling detection algorithm, which consists of a feature pyramid network (FPN) and a detection head. The detection head contains two branches: a classification head and a bounding box regression head. Among them, the feature pyramid network (FPN) is used for multi-scale feature map fusion to improve the algorithm's small target detection performance. The classification head is used to predict whether each anchor point contains a target and the target category, and the bounding box regression head is used to fine-tune the position of the anchor point to determine the exact position of the target.

[0042] The post-processing module applies post-processing techniques such as non-maximum suppression (NMS) to select the best bounding box from a series of overlapping bounding boxes output by the target detection framework to obtain the final prediction result.

[0043] For example, the spatial and temporal feature hybrid module can use convolutional layers to extract spatial features from the input features while downsampling the spatial resolution.

[0044] In at least one embodiment of the present application, the spatial and temporal feature hybrid module includes an improved spatial attention module (Spatial Attention Module, SAM) and a gated recurrent unit (Gated Recurrent Unit, GRU) module. The improved spatial attention module integrates the local attention mechanism and the global attention mechanism to extract local and global features. The gated recurrent unit module is used to extract the temporal feature information of the event stream data to improve the long-term dependency of the event sequence data.

[0045] In at least one embodiment of the present application, the hierarchical multi-stage feature extraction module is further used to receive the features of the previous stage as input when combining multiple recurrent visual transformer modules, and optionally use the long short-term memory network (LSTM) state of the previous time step to calculate the features of the next stage. In this way, by saving the LSTM network state, each stage retains the time information for the entire feature map.

[0046] At least one embodiment of the present application further provides a visual management device, referring to Figure 1 The visualization management device is integrated with the visualization management software. The visualization management device interacts with the data of the foreign object detection edge device of the embodiment of the present application. The visualization management software is used to receive the alarm push information transmitted by the foreign object detection edge device of the embodiment of the present application when the prediction result is that a foreign object has been detected to have fallen, and to perform a visual display. In this way, the alarm information can be displayed to the on-site foreign object detection safety officer in a timely manner when a foreign object has fallen.

[0047] In at least one embodiment of the present application, the visual management device includes a fall alarm display and management system. The fall alarm display and management system is used to generate alarm information according to the alarm push information transmitted by the foreign object detection edge device of the embodiment of the present application, and communicate with the monitoring server, and display it in real time on the interface of the device integrated visual management software for users to view, and at the same time has a query function for retrieving foreign object fall alarm information according to time.

[0048] In at least one embodiment of the present application, the visual management device further includes a video review system. The video review system is used to call videos stored on the local hard disk of the device. The video review system has one or more functions of full screen, screenshot, start and stop, revisit positioning, and fast and slow playback of the viewed screen. In this way, the video revisit viewing function of the video is realized by using the video review system.

[0049] In at least one embodiment of the present application, the visual management device further includes a multi-angle display monitoring system. The multi-angle display monitoring system is used to synchronously play multi-angle monitoring videos. In this way, the multi-angle display monitoring system is used to facilitate users to view and monitor abnormal conditions in real time.

[0050] In at least one embodiment of the present application, the visualization management device also includes a system status and alarm hardware management system. The system status and alarm hardware management system is used to display the number of successfully connected foreign object detection edge devices, and has the function of configuring whether the sound and light alarm device is turned on and whether the remote computer room is allowed to log in and access the system through the internal LAN.

[0051] In at least one embodiment of the present application, the visual management device further includes a user management system. The user management system is used to manage system user information, including but not limited to user number, user name, mobile phone number, status and creation time, and supports the addition, deletion, modification and query operations of administrator users.

[0052] In at least one embodiment of the present application, the visual management device further includes a streaming media component, a codec component and a database. The streaming media component is used to read, transmit and push streaming media data; the codec component is used to encode and decode streaming media data; the database is used to store and read user information, configuration status and alarm information of the system.

[0053] At least one embodiment of the present application provides a foreign matter monitoring device for a nuclear power plant reactor refueling pool, which includes a foreign matter detection edge device and a visualization management device of the above-mentioned embodiment of the present application. A small foreign matter falling detection algorithm is run in the foreign matter detection edge device. The visualization management device is integrated with visualization management software.

[0054] The foreign object detection edge device is used to collect event and image video stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant. It uses a small foreign object falling detection algorithm to process the event and image video stream. The predicted result after processing is that when a foreign object falling is detected, an alarm push information is generated and transmitted to the visualization management device.

[0055] The visualization management device interacts with the foreign object detection edge device of the above-mentioned embodiment of the present application for data, and is used to receive the alarm push information transmitted by the foreign object detection edge device of the embodiment of the present application when the prediction result is that a foreign object has been detected falling, and to perform a visualization display.

[0056] According to the technical solution provided in the embodiment of the present application, event cameras and industrial cameras are used as perception sensors. Through edge processing technology and high-performance machine vision algorithm design, as well as the development of visualization software, the problem of accurate monitoring of high-speed weak falling foreign objects in key foreign object prevention areas of nuclear power plants is effectively solved, and alarm information and alarm monitoring videos are output in real time, providing an effective engineering technical solution for foreign object monitoring in the reactor core refueling pool of a nuclear power plant, and realizing the intelligent improvement of foreign object prevention management.

[0057] At least one embodiment of the present application provides a method for monitoring foreign matter in a nuclear power plant reactor refueling pool, which is performed using a nuclear power plant reactor refueling pool foreign matter monitoring device provided in an embodiment of the present application as an execution subject. The method for monitoring foreign matter in a nuclear power plant reactor refueling pool includes the following steps.

[0058] S1: Use the event camera in the foreign object detection edge device of the nuclear power plant reactor refueling pool foreign object monitoring device to obtain the event stream data in the monitoring area of ​​the nuclear power plant reactor refueling pool.

[0059] S2: Use the industrial camera in the foreign object detection edge device to obtain image video stream data in the monitoring area of ​​the reactor refueling pool of the nuclear power plant.

[0060] S3: Use the edge storage module in the foreign object detection edge device to obtain the event stream data in the monitoring area of ​​the reactor refueling pool of the nuclear power plant from the event camera, obtain the image video stream data in the monitoring area of ​​the reactor refueling pool of the nuclear power plant from the industrial camera, and transmit the event stream data and the image video stream data to the GPU data processing unit in the foreign object detection edge device.

[0061] For example, the edge storage module receives event stream data obtained by the event camera through the OpenEB event camera driver compiled under the Advanced RISC Machine (ARM) architecture.

[0062] S4: The GPU data processing unit in the foreign object detection edge device uses a small foreign object falling detection algorithm to process the event stream data and image video stream data transmitted by the edge storage module, and transmits the processed prediction results to the edge storage module.

[0063] For example, the GPU data processing unit runs the small foreign object falling detection algorithm, performs feature extraction and detection, and outputs the prediction results to the edge storage module.

[0064] For another example, after the GPU data processing unit receives the event stream data, it uses the small foreign object falling detection algorithm to perform event processing, spatial and temporal feature mixing, hierarchical multi-stage feature extraction, feature transfer and anchor point generation, classification and regression, and post-processing operations on the event data in sequence, and outputs the timestamp of the detected foreign object falling to the edge storage module.

[0065] S5: The GPU data processing unit in the foreign object detection edge device is used to receive the prediction results output by the small foreign object falling detection algorithm, and when the prediction result is that a foreign object is detected falling, an alarm push message is generated and transmitted to the visualization management device in the foreign object monitoring device of the nuclear power plant reactor refueling pool.

[0066] For example, refer to Figure 3 ,The GPU data processing unit in the foreign object detection edge ,device assigns the output timestamp as the variable value to the global ,variable, the storage process accesses to monitor the changes of the ,variable, and generates a json message or a video of 5s before and after the alarm and ,outputs it to the visualization management device.

[0067] S6: Use visual management equipment for visual display.

[0068] For example, refer to Figure 3 The generated json message can be forwarded to the monitoring server to be pushed to the database to trigger an alarm; the generated alarm video is pulled by the Web software backend and displayed visually.

[0069] According to the technical solution provided in the embodiment of the present application, through the cross-modal information fusion of event cameras and industrial cameras, combined with edge processing technology and machine vision neural network methods, the monitoring and identification of fallen foreign objects in the entire range of the reactor refueling pool of a nuclear power plant is realized, and an alarm is issued when a foreign object is found to have fallen, thereby realizing remote monitoring and automatic alarm functions, ensuring the effective implementation of on-site foreign object prevention work in the reactor refueling pool of a nuclear power plant, and minimizing or reducing the occurrence of foreign object incidents caused by personnel and equipment.

[0070] It should be noted that the combination of the various technical features in the embodiments of the present application is not limited to the combination described in the embodiments of the present application or the combination described in the specific embodiments, and all technical features described in the present application can be freely combined or combined in any way unless there is a contradiction between them.

[0071] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the term "comprising" only indicates that the steps and elements that have been clearly identified are included, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0072] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A foreign object detection edge device, characterized in that: It includes a sensor module and a foreign object falling data edge processing module connected to the sensor module. The sensor module includes an event camera and an industrial camera. The event camera is used to collect event stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant, and the industrial camera is used to collect image video stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant. The foreign object falling data edge processing module includes an edge storage module and a GPU data processing unit. The edge storage module is used to obtain event stream data within the monitoring area of ​​the nuclear power plant reactor refueling pool from the event camera, and obtain image video stream data within the monitoring area of ​​the nuclear power plant reactor refueling pool from the industrial camera, and transmit the event stream data and image video stream data to the GPU data processing unit running the small foreign object falling detection algorithm, and receive and store the prediction results transmitted by the GPU data processing unit; the GPU data processing unit is used to input the event stream and image video stream data into the small foreign object falling detection algorithm, receive the prediction results output by the small foreign object falling detection algorithm, and generate an alarm push information when the prediction result is that a foreign object is detected and transmit it to the visualization management device.

2. The foreign object detection edge device according to claim 1, characterized in that: The GPU data processing unit runs a small foreign object fall detection algorithm, and the GPU data processing unit interacts with the edge storage module for data. The GPU data processing unit is used to process the event stream data and image video stream data transmitted by the edge storage module using a small foreign object falling detection algorithm, and transmit the processed prediction results to the edge storage module for storage.

3. The foreign object detection edge device according to claim 1, characterized in that: The small foreign body falling detection algorithm includes event processing module, spatial and temporal feature mixing module, multi-layer and multi-stage feature extraction module, feature transfer and anchor point generation module, classification and regression module, and post-processing module. The event processing module is used to encode the polarity, time and position of the event point. The event is represented as a four-tuple and converted into an event pulse tensor representation. Then, by flattening the time dimension and polarity dimension, a 3D tensor compatible with 2D convolution is obtained. The spatial and temporal feature hybrid module is used to extract the spatial and temporal features of event stream data. It uses convolutional layers and long short-term memory network units to extract spatial features and aggregate temporal features, and interacts local and global features through a multi-axis self-attention mechanism. The hierarchical multi-stage feature extraction module combines multiple recurrent visual transformer modules to form a multi-stage hierarchical backbone network to learn the feature graph of event stream data stage by stage; The feature transfer and anchor generation module transfers the feature map obtained by multi-stage network processing to the object detection framework and generates a series of anchors of potential object positions and sizes on the feature map; The classification and regression module consists of a feature pyramid network and a detection head. The detection head includes two branches: a classification head and a bounding box regression head. The feature pyramid network is used to fuse multi-scale feature maps to improve the algorithm's small target detection performance. The classification head is used to predict whether each anchor point contains a target and the target category. The bounding box regression head is used to fine-tune the position of the anchor point to determine the exact position of the target. The post-processing module applies post-processing techniques to select the best bounding box from a series of overlapping bounding boxes output by the object detection framework to obtain the final prediction result.

4. The foreign object detection edge device according to claim 1, characterized in that: The spatial and temporal feature hybrid module in the small foreign object falling detection algorithm includes an improved spatial attention module and a gated recurrent unit module. The improved spatial attention module integrates the local attention mechanism and the global attention mechanism to extract local and global features. The gated recurrent unit module is used to extract the temporal feature information of event stream data to improve the long-term dependency of event sequence data.

5. A visual management device, characterized in that: The visualization management device is integrated with visualization management software, and the visualization management device interacts with the data of the foreign object detection edge device described in claim 1; the visualization management software is used to receive the alarm push information transmitted by the foreign object detection edge device described in claim 1 when the prediction result is that a foreign object has been detected falling, and to perform visual display.

6. The visual management device according to claim 5, characterized in that: Including fall alarm display and management system, The fall alarm display and management system is used to generate alarm information based on the alarm push information transmitted by the foreign object detection edge device according to claim 1, and communicate with the monitoring server, and display it in real time on the interface of the visual management software for users to view, and at the same time has a query function for retrieving foreign object fall alarm information according to time.

7. The visual management device according to claim 5, characterized in that: It also includes one or more of a video review system, a multi-angle display monitoring system, a system status and alarm hardware management system, and a user management system. The video review system is used to call the video stored on the local hard disk of the device. The video review system has one or more functions of full screen, screenshot, start and stop, return positioning, fast and slow playback of the viewed picture; The multi-angle display monitoring system is used to synchronously play multi-angle monitoring videos; The system status and alarm hardware management system is used to display the number of successfully connected foreign object detection edge devices, and has the function of configuring whether the sound and light alarm device is turned on and whether the remote computer room is allowed to log in and access the system through the internal LAN; The user management system is used to manage system user information, which includes user ID, user name, mobile phone number, status and creation time, and supports the administrator user's add, delete, modify and query operations.

8. The visual management device according to claim 5, characterized in that: It also includes streaming media components, encoding and decoding components and databases. The streaming media component is used to read, transmit and push streaming media data; the codec component is used to encode and decode streaming media data; the database is used to store and read the system's user information, configuration status, and alarm information.

9. A foreign body monitoring device for a nuclear power plant reactor refueling pool, characterized in that: A foreign object detection edge device comprising any one of claims 1 to 4 and a visualization management device according to any one of claims 5 to 8, wherein a small foreign object falling detection algorithm is run in the foreign object detection edge device, and the visualization management device is integrated with visualization management software; Foreign object detection edge device, used to collect event and image video stream data in the monitoring area of ​​the reactor refueling pool of a nuclear power plant, process the event and image video stream using a small foreign object falling detection algorithm, and generate an alarm push message when a foreign object falling is detected after processing and transmit it to the visualization management device; A visualization management device that interacts with the foreign object detection edge device described in claim 1 for data, and is used to receive the alarm push information transmitted by the foreign object detection edge device described in claim 1 and perform visual display when the prediction result is that a foreign object has been detected falling.

10. A method for monitoring foreign matter in a nuclear power plant reactor refueling pool, characterized in that: include: Using the event camera in the foreign matter detection edge device in the nuclear power plant reactor refueling pool foreign matter monitoring device as described in claim 9 to obtain event stream data in the monitoring area of ​​the nuclear power plant reactor refueling pool; Use the industrial camera in the foreign object detection edge device to obtain image video stream data in the monitoring area of ​​the nuclear power plant reactor refueling pool; The edge storage module in the foreign object detection edge device is used to obtain event stream data in the monitoring area of ​​the nuclear power plant reactor refueling pool from the event camera, and the image video stream data in the monitoring area of ​​the nuclear power plant reactor refueling pool is obtained from the industrial camera, and the event stream data and the image video stream data are transmitted to the GPU data processing unit in the foreign object detection edge device; The GPU data processing unit in the foreign object detection edge device uses a small foreign object falling detection algorithm to process the event stream data and image video stream data transmitted by the edge storage module, and transmits the processed prediction result to the edge storage module, and generates an alarm push message when the prediction result is that a foreign object is detected falling, and transmits it to the visualization management device in the foreign object monitoring device for the nuclear power plant reactor refueling pool as described in claim 9; Using the edge storage module in the foreign object detection edge device to receive and store the prediction results transmitted by the GPU data processing unit; Use visual management equipment for visual display.

Images