Intelligent inspection method based on nuclear power plant

Abstract

The invention relates to the field of intelligent inspection, in particular to an intelligent inspection method based on a nuclear power plant, which comprises the following steps: S1, identifying road boundaries and obstacle types through asynchronous information matching, 3D imaging of a 2D laser radar, color point cloud map construction and ENet semantic segmentation; s2, acquiring a substation inspection image, performing bilateral filtering and adaptive edge compensation processing on the image, inputting the processed image into a Faster R-CNN algorithm for target detection, and performing remote recognition on suspended foreign matters; s3, intelligent inspection of the nuclear power plant is completed based on a laser radar and camera fusion method, and the intelligent inspection scheme not only can effectively identify road boundaries and obstacle types, but also can efficiently and accurately identify suspended foreign matters.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection, and in particular to an intelligent inspection method based on a nuclear power plant. Background Art

[0002] At present, the operation and maintenance status of nuclear power plants is directly related to the safety performance of the entire power grid. In order to improve the inspection efficiency, robot inspection of nuclear power plants has gradually become a research hotspot.

[0003] However, although the existing high-precision LiDAR sensors have high navigation accuracy, they cannot identify obstacle types and road boundaries, which affects inspection performance and efficiency. At the same time, high-precision three-dimensional LiDAR is too expensive to be popularized. The complexity of the internal environment of nuclear power plants and the diversity of equipment have led to the increasing prominence of the problem of suspended foreign objects, which poses a major threat to the safe operation of the power grid. The nuclear power plant environment is complex and the inspection work is time-consuming and laborious.

[0004] Therefore, researching and applying intelligent inspection technology for nuclear power plants is an inevitable trend in technological progress in the power industry, and is also the key to ensuring grid safety and improving operation and maintenance efficiency. Summary of the invention

[0005] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an intelligent inspection method based on a nuclear power plant.

[0006] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: an intelligent inspection method based on a nuclear power plant, comprising: S1, identifying road boundaries and obstacle types through asynchronous information matching, 3D imaging of 2D lidar, color point cloud map construction and ENet semantic segmentation; S2, collect substation inspection images, perform bilateral filtering and adaptive edge compensation on the images, input the processed images into the FasterR-CNN algorithm for target detection, and remotely identify hanging foreign objects; S3. Complete intelligent inspection of nuclear power plants based on the fusion method of lidar and camera.

[0007] The asynchronous information matching method is used to align the data of different sensors through timestamps. The implementation details of the algorithm are as follows: (1) Collect messages from different sensors at each time point When each message Timestamped ; For different sensor messages, create different arrays To save the corresponding message; (2) Only the latest messages are saved , that is, when receiving Discard messages ; (3) Set the set S, when different Contains a sensor message When, different is stored in the set S; (4) For the set S, different arrays Medium sensor message Timestamp With timestamp For comparison, use the latest sensor message as the center point for matching time. ; (5) Calculation The current timestamp of the point With the initial moment Medium sensor message Timestamp The time difference between ;if The matching tolerance is Different sensor data in ;if If the matching error is within the tolerance, the data set is discarded and the above steps (1) to (4) are repeated until all sensor messages are processed.

[0008] Three-dimensional imaging is achieved by fusing 2D lidar data with inertial measurement unit (IMU) data, performing coordinate transformation and data stitching.

[0009] Visible light sensor data is integrated with environmental point cloud information to construct a color 3D lidar point cloud map of the working environment.

[0010] Find the positional relationship between the lidar sensor and the visible light sensor, find the projection position of the laser point in space on the visible light sensor screen, and assign the pixel color of the position to the laser point.

[0011] The ENet network is used to perform semantic segmentation on the color point cloud map to identify different types of obstacles.

[0012] The collected substation inspection images are enhanced using bilateral filters and adaptive edge compensation techniques to remove interference and noise. The bilateral filter combines spatial and grayscale filtering to retain the edge information of the image while removing noise. The adaptive edge compensation enhances the image edge by calculating the grayscale value difference of each pixel in the image.

[0013] , let I(x,y) represent the inspection image, then the filtering and denoising expression formula of the inspection image is as follows: ; In the above formula, represents the inspection image after filtering and denoising, where Represents the coordinates of the pixel; Represents the pixel points of the inspection image; Represents a set of pixels of the inspection image; , Represent the spatial proximity factor and spatial grayscale similarity factor respectively.

[0014] The FasterR-CNN algorithm is used for target detection. The algorithm integrates the region proposal network (RPN) and the target detection network into a unified framework to achieve end-to-end target detection. VGG16 is used as the feature extraction network, and feature extraction, classification and regression operations are performed through convolutional layers, RPN, RoI pooling layers and fully connected layers.

[0015] Through edge computing, the FasterR-CNN algorithm is optimized to run on devices close to the data source, quickly processing and analyzing image data and generating detection results.

[0016] The beneficial effects of the present invention are: The robot's autonomous environment understanding capability is improved through asynchronous information matching, 3D imaging of 2D lidar, color point cloud map construction and ENet semantic segmentation. This method can effectively identify road boundaries and obstacle types while reducing data volume and hardware costs, and can also efficiently and accurately identify suspended foreign objects.

[0017] Through edge optimization, the FasterR-CNN algorithm significantly improves the detection speed and system responsiveness while maintaining high detection accuracy. This method not only efficiently and accurately detects foreign objects, but also reduces the workload of operation and maintenance personnel and improves the automation and intelligence level of substation inspection.

[0018] The intelligent inspection plan of nuclear power plants based on the fusion method of lidar and camera can not only effectively identify road boundaries and obstacle types, but also efficiently and accurately hang foreign objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention is a schematic flow chart of an intelligent inspection method based on a nuclear power plant. DETAILED DESCRIPTION

[0020] The following will refer to the attached Figure 1 The embodiments of the present invention are described in detail. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0021] An intelligent inspection method based on a nuclear power plant, comprising: S1, identifying road boundaries and obstacle types through asynchronous information matching, 3D imaging of 2D lidar, color point cloud map construction and ENet semantic segmentation; S2, collect substation inspection images, perform bilateral filtering and adaptive edge compensation on the images, input the processed images into the FasterR-CNN algorithm for target detection, and remotely identify hanging foreign objects; S3. Complete intelligent inspection of nuclear power plants based on the fusion method of lidar and camera.

[0022] First, asynchronous information matching was developed to solve the problem of incomplete information matching when fusing sensor information. Second, in order to avoid the impact of information redundancy caused by 3D LiDAR on system stability, 2D LiDAR combined with an inertial measurement unit was used to replace 3D LiDAR, which can also reduce costs. This method can achieve 3D imaging by fusing 2D LiDAR data and inertial measurement unit data, and performing coordinate transformation and data splicing. The visible light information collected by the visible light sensor is fused with the environmental point cloud information to construct a color 3D LiDAR point cloud map for detecting the working environment of the nuclear power plant.

[0023] Based on the 3D laser radar color point cloud image, the proposed method uses the ENet network to perform semantic segmentation on the color point cloud image. Therefore, the detection of nuclear power plants can not only obtain the distance and direction information of obstacles, but also identify the type of obstacles. The details are as follows: (1) Visible light sensor data, IMU data, and 2D lidar sensor data are fused through an asynchronous information matching method; (2) Three-dimensional imaging can be obtained by transforming the coordinate axes and fusing the matched IMU data and two-dimensional lidar data; (3) By fusing the matching visible light sensor data, the 3D lidar point cloud image of the working environment becomes colored; (4) Use the ENet network to perform semantic segmentation on the color point cloud image.

[0024] The common multi-sensor fusion method mainly focuses on synchronous data fusion. However, in actual work, due to the frequency difference of different sensors, the limitation of measurement frequency and the delay in data transmission, the information of different sensors cannot be completely matched in time scale. In order to truly and accurately restore the working scene, it is necessary to match the multi-sensor information in approximate time.

[0025] information Represents information from different sensors, and the message Represents the information of sensor i at different times. Due to the different frequencies of sensor information acquisition and the different amounts of information data from different sensors, the different sensor information collected by the system will be presented as an asynchronous sensor system.

[0026] The time axis goes from left to right, and the first sensor message received is on the left side of the time axis. In an asynchronous sensor system, the information of different sensors cannot be completely aligned on the same time scale. If only the message of each sensor in each frame is matched, the time difference between different sensor information will become larger and larger. The implementation details of the algorithm are as follows: (1) Collect messages from different sensors at each time point When each message Timestamped ; For different sensor messages, create different arrays To save the corresponding message; (2) Only the latest messages are saved , that is, when receiving Discard messages ; (3) Set the set S, when different Contains a sensor message When, different is stored in the set S; (4) For the set S, different arrays Medium sensor message Timestamp With timestamp For comparison, use the latest sensor message as the center point for matching time. ; (5) Calculation The current timestamp of the point With the initial moment Medium sensor message Timestamp The time difference between ;if The matching tolerance is Different sensor data in ;if If the matching error is within the tolerance, the data set is discarded and the above steps (1) to (4) are repeated until all sensor messages are processed.

[0027] Through multi-sensor asynchronous information matching, asynchronous sensor information can be effectively processed. In this way, different sensor information that meets the time error requirements of the asynchronous sensor system can be retained, thereby achieving approximate time matching of asynchronous sensor information.

[0028] Compared with 2D LiDAR sensors, 3D LiDAR sensors can obtain richer environmental information in one scan. In addition, 3D LiDAR can directly construct front, back, left, right, and height information for inspecting the surrounding environment of a nuclear power plant. In practical applications, 3D LiDAR sensors lead to information redundancy. However, with a 2D LiDAR sensor, only the boundary information of one obstacle in the environment can be obtained after one scan, so it is difficult to restore the real environment.

[0029] In order to construct a 3D point cloud of the environment, the 2D LiDAR sensor must be mounted in a special way. The 2D LiDAR sensor is set at a height h from the ground, and then rotated around the Y1 axis to generate an angle θ between the X1-Y1 plane and the horizontal plane. Assume that a point on the obstacle is P, and the coordinates of the point in the LiDAR coordinate system are P(xp, yp, zp). By calculating the coordinates of the point in the world coordinate system, a 3D point cloud of the environment is constructed.

[0030] Through coordinate transformation calculation, the actual position relationship of the point cloud data frame of the 2D LiDAR sensor in the 3D space can be restored. A frame of data from a 2D LiDAR sensor contains less information and is not enough to restore the real environment. Therefore, on this basis, it is necessary to splice multiple frames of 2D LiDAR sensor data.

[0031] During the robot's motion, the inertial measurement unit (IMU) accurately records the robot's motion posture. The 2D LiDAR sensor and the IMU are relatively fixed on the robot body. Therefore, through the multi-sensor asynchronous information matching method proposed above, the 2D LiDAR data at similar times are fused with the IMU data, and the multi-frame data is converted to the world coordinate system. The 2D LiDAR data is spliced ​​to achieve 3D imaging of the 2D LiDAR data.

[0032] Since the coordinate system of the 2D LiDAR sensor coincides with the world coordinate system, the IMU coordinate system needs to be rotated first to coincide with the LiDAR coordinate system.

[0033] Due to the limitation of the working principle of LiDAR sensors, the robot can only judge the direction and distance of obstacles in the driving direction. It is difficult to understand the environment simply through point cloud information. In order to restore the environmental information more realistically, a monocular visible light sensor is used. The environmental point cloud information is fused with the visible light information to construct a color 3D LiDAR point cloud map of the working environment.

[0034] The key to constructing a color LiDAR point cloud is to find the positional relationship between the LiDAR sensor and the visible light sensor. Then find the projection position of the laser point in space on the visible light sensor screen. Finally, assign the pixel color at that position to the laser point. Therefore, first of all, it is necessary to clarify the relationship between the coordinates of the visible light sensor and the world coordinate system and the positional relationship with the LiDAR sensor.

[0035] Convert from the world coordinate system to the camera coordinate system. Since both are rigid body coordinate systems, they can be achieved through rotation and translation.

[0036] The lidar point cloud is assigned RGB colors to the corresponding pixels. The 2D lidar point cloud containing RGB colors is then imaged three-dimensionally to construct a color 3D lidar point cloud map of the working environment.

[0037] Color point cloud map segmentation based on ENet. After completing the ENet network training, for the acquired visible light sensor data, the learned semantic segmentation model can be used to understand the image, segment different targets, and cover different types of targets with different colors, and output images with semantic information.

[0038] At this point, the lidar points are projected into the semantic image using the principle of color point cloud construction. The corresponding pixels are colored to construct a lidar point cloud with semantic information. The semantic segmentation of the color point cloud is completed. In the point cloud with semantic information, each lidar point contains six-dimensional information, namely the coordinate information of the point in space and the semantic color information of the target in the environment represented by the point, namely (x, y, z, R, G, B). At this point, not only the distance and direction information of obstacles in the nuclear power plant can be obtained, but also the type of obstacles can be identified with the help of semantic information. Based on the recognition results, further navigation decisions can be made.

[0039] In order to remotely and intelligently identify the detection images of foreign objects hanging in nuclear power plants, a bilateral filter detection image enhancement method based on adaptive edge compensation is proposed. During the detection process, the robot carries a high-definition camera to monitor and collect data of key equipment and areas in the nuclear power plant in real time, while the ground control station receives the detection image. Based on the detection image, the feature vector of the foreign objects hanging in the detection image is extracted using the machine vision image processing algorithm. The specific process is as follows: Due to the acquisition of the inspection image environment and the performance limitations of the robot-mounted camera, the collected inspection image contains interference noise, resulting in low resolution and insufficient contrast. The texture information defects in the inspection image are more blurred, affecting the accuracy of remote identification of foreign objects hanging in the inspection image, which is information that needs to be collected. The inspection image needs to be denoised and edge compensated to improve its resolution and contrast.

[0040] First, the inspection image is denoised using a bilateral filter, which combines spatial and grayscale filtering to retain the edge information of the image while denoising. The principle of the bilateral filter is to determine the filter weight based on the spatial distance and grayscale value difference of the pixel points to achieve denoising while retaining the edge details of the image. The spatial proximity factor is usually defined based on the Euclidean distance between pixels. Let I(x,y) represent the inspection image, then the filter denoising expression formula of the inspection image is as follows: ; In the above formula, represents the inspection image after filtering and denoising, where Represents the coordinates of the pixel; Represents the pixel points of the inspection image; Represents a set of pixels of the inspection image; , Represent the spatial proximity factor and spatial grayscale similarity factor respectively.

[0041] When calculating the filtered pixel value, the pixels in the local area centered on the point are weighted and summed. The weights are determined by the spatial proximity factor and the spatial grayscale similarity factor to consider both the spatial position relationship and grayscale similarity of the pixels. When conducting inspections, details such as the edges of equipment are very important for judging the status of the equipment. Bilateral filtering denoising helps to accurately identify key equipment in nuclear power plants and possible abnormalities, such as hanging foreign objects.

[0042] After filtering the test image and removing the interference noise contained in the test image, the test image is edge enhanced using the adaptive edge compensation method. The deviation is based on the difference in grayscale value at each pixel between the original image and the bilaterally filtered image. By summing this difference in a local area centered on the pixel, a quantity that reflects the degree of change between the filtered image and the original image in the local area can be obtained. The deviation value helps to identify the edge position. In the subsequent edge compensation operation, the deviation value is an important reference for determining the area that needs edge enhancement.

[0043] When the bilateral filter window is located at the edge of the detection image, its spatial grayscale similarity factor value is small, vice versa, the spatial grayscale similarity factor value is large. Therefore, the image edge is described by adjusting the sum of the spatial grayscale similarity factors of the detection image. The sum of the spatial grayscale similarity factors is calculated by adding the spatial grayscale similarity factors in the local area centered on the pixel point and multiplying them by the coefficient.

[0044] When the detection image is over-smoothed or over-sharpened, it will cause excessive image edge enhancement or grayscale jump. To avoid this phenomenon, adaptive edge compensation can be performed based on the detection image.

[0045] By introducing adaptive parameters, adaptive edge compensation can be dynamically adjusted according to the actual edge of the image to avoid these undesirable phenomena. This ensures that the final detection image maintains a natural and accurate visual effect while enhancing the edge, thereby providing high-quality image data for detection for accurate analysis and judgment.

[0046] After the above steps, the denoising and edge enhancement of the detection image are achieved, making the contour features of the hanging foreign objects in the detection image clearer.

[0047] The acquired enhanced detection images are taken as input to construct a set of enhanced detection images, and then a FasterR-CNN algorithm network model is established to achieve remote recognition of hanging foreign objects in the detection images.

[0048] The FasterR-CNN algorithm combines the parallel RPN and FasterR-CNN algorithms, both of which can be trained end-to-end. The emergence of the FasterR-CNN algorithm has changed the development process of object detection algorithms. It adds convolutional feature acquisition, region candidate frame generation, and target classification and regression to the network, thereby achieving comprehensive network correction. FasterR-CNN mainly consists of four parts: a convolutional layer (ConvLayers), a region proposal network (RPN), a region of interest pooling (RoIPooling) layer, and a fully connected network for classification and calculation of regressors.

[0049] The convolutional layer structure of the FasterR-CNN algorithm is VGG16, which is used as a feature extraction network. The network contains 13 convolutional layers, 13 ReLU activation function layers, and four maximum pooling layers. The enhanced detection image is input into the convolutional layer structure of the FasterR-CNN algorithm. After convolution and pooling, the feature map of the image is finally obtained. This feature map is shared with the subsequent RPN and fully connected layers. FasterR-CNN supports image input of any size, such as PxQ. The image is normalized and scaled before entering the network. For example, the short side of the image can be set to no more than 600 and the long side to no more than 1000. Therefore, it can be assumed that MxN=1000x600 (if the image is smaller than this size, the edge can be filled with 0, that is, the image shows a black edge). When inspecting, the features of the device and foreign objects can be effectively extracted. The convolution operation can capture local features in the image, such as the edges and textures of the device. The ReLU activation function enables the network to learn more complex nonlinear relationships and enhances the network's expressive power. The maximum pooling layer can reduce the amount of data while retaining important feature information, which helps improve network computational efficiency and prevent overfitting.

[0050] After using the convolutional layer structure to obtain the inspection image features, the RPN layer of the FasterR-CNN algorithm extracts multiple target candidate regions of the inspection image. RPN can efficiently generate candidate regions that may contain targets (such as hanging foreign objects in the inspection image), thereby narrowing the search range for subsequent classification and regression. The classification branch helps to screen truly meaningful candidate regions by accurately distinguishing between targets and backgrounds. The regression branch can more accurately locate the target by precisely adjusting the anchor box, and improve the detection accuracy by precisely adjusting the anchor box.

[0051] The introduction of channel attention mechanism and spatial attention mechanism can better utilize the information in the feature map, reduce the influence of irrelevant feature channels, highlight the semantic and spatial location information of the target, and further improve the performance of RPN. RPN uses a sliding window technique to generate k rectangles at each position of the feature map. By regressing an anchor box that may contain the target, the anchor box obtains an accurate target bounding box. RPN contains two branches: classification and regression. We perform deep convolution on the features of the target template and search in the image to provide these two branches for training. The task of the classification branch is to determine whether each anchor box belongs to the target or the background, that is, a binary classification task. Therefore, after deep cross-correlation, we get 2k features. The classification branch performs pre-background distinction on these features. The task of the regression branch is to regress the generated anchor box frame to a more accurate bounding box. Because [x, y, w, h] is used to represent the range of the anchor box, where x and y are the coordinates of the origin, and w and h represent the width and height respectively, the k anchor boxes eventually get 4k features, and the regression branch converts these features into more boxes. Because [x, y, w, h] is used to represent the semantic part of the feature map of interest. Different channels in the feature map respond differently to specific targets, that is, for different targets, some feature channels may be more important, while some channels may be completely irrelevant. Such feature channels are irrelevant to the target semantics and usually affect the similarity calculation process of the model, which in turn affects the tracking results. Therefore, channel attention is used to enhance the dependencies between feature channels by modeling and extracting the weights of each channel. Specific channels respond to semantics and reduce the influence of irrelevant channels. In contrast, the regression branch focuses on the spatial position of the target in the feature map. Naturally, spatial attention is used to enhance the feature map and highlight the spatial position of the target to obtain better regression results.

[0052] Under normal circumstances, the FasterR-CNN algorithm can accomplish the purpose of target recognition, but when it is applied to the remote recognition of hanging foreign objects in inspection images, if the target area is larger than the image boundary, the target area needs to be discarded, resulting in the training process not converging, which reduces the final recognition accuracy. Therefore, the FasterR-CNN algorithm must be optimized. Using edge computing to optimize the FasterR-CNN algorithm can effectively adjust the network structure, reduce model parameters and calculation amount, and be more efficient in edge processing.

[0053] The purpose of calculating the gradient amplitude of an image is to obtain the change in pixel intensity information in the image. In the detection image, areas such as the edge of the device and the outline of hanging foreign objects usually have large grayscale changes, that is, the gradient amplitude is large. By calculating the gradient amplitude, we can preliminarily locate areas in the image that may contain important information and provide basic data for subsequent edge detection and algorithm optimization.

[0054] Because the grayscale fluctuation at the edge of the detected image is discontinuous and the gradient amplitude is significant, the gradient difference between the edge points of the detected image and the surrounding non-edge points is significant.

[0055] In the inspection image, after gradient calculation and edge point judgment, some wide edge areas may appear. By suppressing non-maxima, the edges can be refined to a width of only one pixel (ideally), making the edges clearer and more accurate. This is very important for the FasterR-CNN algorithm to accurately identify hanging foreign objects, because clear edges help the algorithm better locate and identify targets, reduce misjudgments, and improve the accuracy of remote identification of hanging foreign objects in inspection images.

[0056] Finally, the intelligent inspection of the nuclear power plant is completed based on the lidar and camera fusion method.

[0057] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0058] It should be noted that, in the description of the present invention, the terms "first", "second" and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0059] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0060] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0061] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0062] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0063] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0064] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An intelligent inspection method based on a nuclear power plant, characterized in that: include: S1, identifying road boundaries and obstacle types through asynchronous information matching, 3D imaging of 2D lidar, color point cloud map construction and ENet semantic segmentation; S2, collect substation inspection images, perform bilateral filtering and adaptive edge compensation on the images, input the processed images into the FasterR-CNN algorithm for target detection, and remotely identify hanging foreign objects; S3. Complete intelligent inspection of nuclear power plants based on the fusion method of lidar and camera.

2. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: The asynchronous information matching method is used to align the data of different sensors through timestamps. The implementation details of the algorithm are as follows: (1) Collect messages from different sensors at each time point When each message Timestamped ; For different sensor messages, create different arrays To save the corresponding message; (2) Only the latest messages are saved , that is, when receiving Discard messages ; (3) Set the set S, when different Contains a sensor message When, different is stored in the set S; (4) For the set S, different arrays Medium sensor message Timestamp With timestamp For comparison, use the latest sensor message as the center point for matching time. ; (5) Calculation The current timestamp of the point With the initial moment Medium sensor message Timestamp The time difference between ;if The matching tolerance is Different sensor data in ;if If the matching error is within the tolerance, the data set is discarded and the above steps (1) to (4) are repeated until all sensor messages are processed.

3. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: Three-dimensional imaging is achieved by fusing 2D lidar data with inertial measurement unit (IMU) data, performing coordinate transformation and data stitching.

4. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: Visible light sensor data is integrated with environmental point cloud information to construct a color 3D lidar point cloud map of the working environment.

5. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: Find the positional relationship between the lidar sensor and the visible light sensor, find the projection position of the laser point in space on the visible light sensor screen, and assign the pixel color of the position to the laser point.

6. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: The ENet network is used to perform semantic segmentation on the color point cloud map to identify different types of obstacles.

7. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: The collected substation inspection images are enhanced using bilateral filters and adaptive edge compensation technology to remove interference and noise. The bilateral filter combines spatial and grayscale filtering to retain the edge information of the image while removing noise. Adaptive edge compensation enhances image edges by calculating the gray value difference of each pixel in the image.

8. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: Assume I(x,y) represents the inspection image, then the filtering and denoising expression of the inspection image is as follows: ； In the above formula, represents the inspection image after filtering and denoising, where Represents the coordinates of the pixel; Represents the pixel points of the inspection image; Represents a set of pixels of the inspection image; , Represent the spatial proximity factor and spatial grayscale similarity factor respectively.

9. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: The FasterR-CNN algorithm is used for target detection. The algorithm integrates the region proposal network (RPN) and the target detection network into a unified framework to achieve end-to-end target detection. VGG16 is used as the feature extraction network, and feature extraction, classification and regression operations are performed through convolutional layers, RPN, RoI pooling layers and fully connected layers.

10. The intelligent inspection method based on a nuclear power plant according to claim 1, characterized in that: Through edge computing, the FasterR-CNN algorithm is optimized to run on devices close to the data source, quickly processing and analyzing image data and generating detection results.

Images