A method and system for identifying foreign object risks based on deep learning object detection

Abstract

This application relates to the field of computer vision technology, and in particular to a method and system for foreign object risk identification based on deep learning-based object detection. The method includes: preprocessing image data collected from a nuclear power plant work site; constructing an improved YOLOv5 object detection model, and inputting the preprocessed image into the improved YOLOv5 object detection model for feature extraction and foreign object target detection; outputting foreign object target detection results through the improved YOLOv5 object detection model, the detection results including the foreign object target category, target bounding box position, and detection confidence; performing confidence filtering and non-maximum suppression processing on the foreign object target detection results to obtain the final foreign object target detection results; calculating the spatial features of the foreign object based on the final foreign object target detection results and constructing a foreign object risk contribution value; calculating the comprehensive foreign object risk value of the work site based on the risk contribution value of each foreign object target, and determining the foreign object risk level based on the comprehensive foreign object risk value.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and system for identifying foreign object risks based on deep learning target detection. Background Technology

[0002] During the overhaul, maintenance, and modification of nuclear power plants, foreign objects such as tools, parts, and debris are typically left inside equipment or systems. Therefore, a foreign object clearance (FOD) process is usually required to manage and verify the work site. Currently, the identification and assessment of FOD risks primarily rely on on-site inspections by operators and manual review by management personnel.

[0003] The above method has the following shortcomings in practical applications: Foreign object identification relies on human experience, is highly subjective, and makes it difficult to guarantee the consistency and stability of identification results; The work site environment is complex, with large changes in lighting, severe obstruction, and small and diverse foreign objects, making it easy for manual inspection to miss or misjudge. Existing general-purpose image-based target detection methods are mostly designed for general scenarios, with a large number of model parameters and high computational complexity, making it difficult to run stably on mobile terminals or edge devices at nuclear power plant operation sites. The lack of automated and standardized support for identification results in the foreign object prevention management process makes it difficult to provide objective and quantitative basis for foreign object prevention permit decisions.

[0004] Therefore, existing technologies cannot simultaneously meet the comprehensive requirements of nuclear power plant operation sites for accuracy, real-time performance, and lightweight deployment of foreign object identification. There is an urgent need for a more effective automatic foreign object risk identification technology. Summary of the Invention

[0005] The purpose of this application is to overcome the above-mentioned defects of the prior art and provide a method and system for foreign object risk identification based on deep learning target detection, especially for the complex environment of nuclear power plant operation sites, and to achieve automatic foreign object identification and risk assessment by adopting a lightweight improved YOLOv5 network structure.

[0006] To address the aforementioned technical problems, the technical solution provided in this application offers an automatic foreign object risk identification method based on deep learning object detection, comprising: Step 1: Collect image data of the nuclear power plant operation site; Step 2: Perform image preprocessing on the collected image data of the nuclear power plant operation site; Step 3: Construct an improved YOLOv5 target detection model, and input the preprocessed image into the improved YOLOv5 target detection model for feature extraction and foreign object target detection; Step 4: By improving the YOLOv5 target detection model, output the foreign object detection results, which include: foreign object target category, target bounding box position, and detection confidence. Step 5: Perform confidence screening and non-maximum suppression processing on the foreign object detection results to obtain the final foreign object detection results; Step 6: Calculate the spatial characteristics of foreign objects and construct the foreign object risk contribution value based on the final foreign object target detection results; Step 7: Calculate the comprehensive foreign object risk value at the work site based on the risk contribution value of each foreign object target, and determine the foreign object prevention risk level based on the comprehensive foreign object risk value.

[0007] As an improvement to the above technical solution, in step 2, the image preprocessing includes: size normalization, pixel normalization, and Gaussian filtering smoothing.

[0008] As an improvement to the above technical solution, in step 3, the improved YOLOv5 object detection model includes: a backbone network, a feature fusion network, and a detection head. The backbone network uses the ShuffleNetV2 structure to lightweight improve the feature extraction structure in the original YOLOv5s model. The feature fusion network uses the GSConv lightweight convolutional module to replace at least some of the standard convolutional layers.

[0009] As an improvement to the above technical solution, the ShuffleNetV2 structure includes: a channel splitting processing unit, a lightweight feature transformation unit, and a channel splicing and shuffling unit, wherein, The channel splitting processing unit is used to split the input feature map The feature map is divided into two parts along the channel dimension; the first part is the feature map obtained by channel splitting. The second part of the feature map is obtained by directly using it as a bypass output and channel splitting. Entering the feature transformation branch: ; The lightweight feature transformation unit is used to transform the second part of the feature map. Perform channel-wise convolution and point-wise convolution sequentially to obtain the second part of the feature map after convolution. To reduce the number of parameters and multiply-accumulate operations: ; in, This indicates a channel-wise convolution operation. This represents pointwise convolution operation; The channel splicing and shuffling unit is used to splice the first part of the feature map. The second part of the feature map after convolution Perform splicing to obtain the spliced ​​feature map. Furthermore, the information exchange between different channels is enhanced through channel shuffling operations: ; in, This indicates a channel shuffling operation, used to enhance information exchange between channels; This indicates a feature concatenation operator, used to concatenate the first part of the feature map. The second part of the feature map after convolution The components are stitched together along the channel dimension. The GSConv lightweight convolutional module includes: a main feature extraction unit, a Ghost feature generation unit, and a feature fusion and channel shuffling unit, wherein... The main feature extraction unit is used to perform standard convolution on the input feature map to obtain the main feature map. : ; in, This represents the standard convolution operation; The Ghost feature generation unit is based on the main feature mapping. Redundant features are generated through inexpensive linear transformation. : ; in, Represents inexpensive linear transformation symbols; The feature fusion and channel shuffling unit is used to map the main features. With redundancy features The data is then concatenated and a channel shuffling operation is performed to form the final output features. : .

[0010] As an improvement to the above technical solution, in step 4, the detection result of a single foreign object target is expressed as follows: ; in, No. The detection results of the foreign object target, Indicates the first The bounding box of a foreign object target. Indicates the first The category to which the foreign object belongs. This indicates that the improved YOLOv5 object detection model is for the first... The confidence level of the detection results for each foreign object target; among which, ; in, For the first The x-coordinate of the center point of the bounding box of the foreign object target. The ordinate of the center point of the bounding box of the i-th foreign object target; For the first The width of the bounding box of each foreign object target. Let be the height of the bounding box of the i-th foreign object target; A collection of foreign object detection results Represented as: ,in, The detection result for the first foreign object target. The detection result for the second foreign object target. For the first The detection results of the foreign object target, This represents the total number of detection results for foreign object targets.

[0011] As an improvement to the above technical solution, the confidence screening condition in step 5 is as follows: ; in, This indicates that the improved YOLOv5 object detection model is for the first... The confidence level of the detection results for a foreign object target. The preset confidence threshold; when Then it means the first The detection results of each foreign object target are filtered by confidence level; otherwise, they are considered to have failed the confidence level filter and the detection results of foreign objects targets that have failed the confidence level filter are discarded. In step 5, non-maximum suppression is applied to overlapping bounding boxes of the same foreign object category: ; in, Indicates the first The bounding box of a foreign object target. No. The bounding box of a foreign object target. Represents bounding box and bounding box The intersection, union, and ratio operations, To preset the overlap threshold, Indicates comparison bounding boxes and bounding box The corresponding confidence levels and Among them, detection results with higher confidence are retained, while detection results with lower confidence are deleted.

[0012] As an improvement to the above technical solution, in step 6, the foreign object space characteristics include area ratio. Distance from key areas ,in, The area ratio for; ; in, Indicates the first The width of the bounding box of each foreign object target. Indicates the first The height of the bounding box of the foreign object target. Indicates the first The area of ​​the bounding box of each foreign object in the image. Indicates the width of the input image. Indicates the height of the input image; The distance of the key area for: If the key areas of the equipment are predefined Then calculate the distance between the foreign object target and the critical area. : ; in, For the first The x-coordinate of the center point of the bounding box of the foreign object target. Let be the ordinate of the center point of the bounding box of the i-th foreign object target. For predefined critical areas of the device The x-coordinate of the center point, For predefined critical areas of the device The ordinate of the center point; Step 6, in which the foreign object risk contribution value is constructed, specifically includes: For each foreign object target, define a corresponding risk contribution value. : ; in, For the first Foreign object target belongs to foreign object category The corresponding risk weights, For the first The confidence level of the detection results for a foreign object target. This represents the distance decay function.

[0013] As an improvement to the above technical solution, step 7 specifically includes: Step 7.1: Sum the risk contribution values ​​of all foreign objects within the same work site to obtain the comprehensive foreign object risk value. : ; in, This indicates the total number of foreign objects detected. Step 7.2: Assess the overall foreign object risk value Normalization is performed: ; in: This represents the maximum overall risk value obtained from historical statistics. This represents the minimum comprehensive risk value obtained from historical statistics. This is the normalized overall foreign object risk value. The value range is [0,1]; Step 7.3: Introduce foreign object quantity factor, foreign object type distribution factor, and spatial location enhancement factor to the normalized comprehensive foreign object risk value. Make corrections to obtain the final risk assessment value. : Step 7.4: Based on the final risk assessment value The risk of foreign objects is classified into the following levels: ; in, For configurable high-risk thresholds, This is a configurable medium-risk threshold.

[0014] As an improvement to the above technical solution, step 7.3 specifically includes: Step 7.3.1: Utilize the foreign object quantity correction factor The normalized overall foreign object risk value is corrected as follows: ; in, The overall foreign object risk value after adjusting for the number of foreign objects, and the foreign object number adjustment factor. for: ; Step 7.3.2: Utilize the foreign object type distribution factor Overall foreign object risk value after adjusting for the number of foreign objects Make corrections: ; in, The overall foreign object risk value after correction for foreign object type, and the foreign object type distribution factor. for: ; in, The ordinal number of the foreign object's category. The total number of categories of foreign object targets, Indicates the first The number of foreign objects in each category, For the first Risk sensitivity coefficients for each category of foreign object; the categories of foreign object protection targets include at least: tool-type foreign objects, fastener-type foreign objects, soft foreign objects, and debris-type foreign objects; Step 7.3.3: Utilizing the spatial location enhancement factor Overall foreign object risk value after correction for foreign object type Make corrections: ; in, The spatial location enhancement factor serves as the final risk assessment value. for: .

[0015] To achieve another objective of this invention, this invention provides an automatic foreign object risk identification system based on deep learning target detection, comprising: an image acquisition module, an image preprocessing module, a foreign object target detection module, a risk quantification and analysis module, and a risk level determination module; wherein, The image acquisition module is used to acquire image data of the nuclear power plant operation site; The image preprocessing module performs image preprocessing on the collected image data from the nuclear power plant operation site; The foreign object detection module has a built-in improved YOLOv5 target detection model, which is used to extract features and detect foreign objects from the received preprocessed image data, and output the foreign object detection results. The foreign object detection results include: foreign object category, target bounding box position and detection confidence. The risk quantification and analysis module is used to perform confidence screening and non-maximum suppression processing on the foreign object target detection results to obtain the final foreign object target detection results, calculate the foreign object spatial characteristics based on the final foreign object target detection results and construct the foreign object risk contribution value. The risk level determination module is used to calculate the comprehensive foreign object risk value at the work site based on the risk contribution value of each foreign object target, and to determine the foreign object prevention risk level based on the comprehensive foreign object risk value.

[0016] The advantages of this application lie in its use of YOLOv5 as the basic target detection model, combined with the characteristics of small size, diverse types, and complex backgrounds of foreign objects at nuclear power plant operation sites, to specifically improve the network structure. First, a lightweight network, ShuffleNetV2, is introduced into the backbone network to reconstruct the original feature extraction structure. Through channel splitting and shuffling mechanisms, the model's parameter count and computational complexity are reduced while maintaining effective representation of key features, thereby improving the model's operating efficiency on mobile terminals and edge devices. Second, a lightweight convolutional network, GSConv, is introduced in the feature fusion stage to optimize the multi-scale feature fusion structure of the Neck part. While ensuring sufficient fusion of multi-scale feature information, redundant computation is further reduced, improving the overall real-time detection performance of the model. Based on the above improved model, this invention constructs a target detection method for foreign object prevention scenarios. By automatically analyzing images of the operation site, it accurately identifies multiple types of targets, including tool-type foreign objects, fastener-type foreign objects, and soft foreign objects, and outputs the location and category information of the foreign object targets, providing an objective and quantitative data foundation for foreign object prevention risk assessment. Compared with existing technologies, this invention significantly reduces model complexity while ensuring detection accuracy, and balances the requirements of identification accuracy and lightweight deployment, effectively improving the intelligence and standardization of foreign object prevention management at nuclear power plant operation sites. Attached Figure Description

[0017] Figure 1 The overall architecture diagram of the foreign object risk identification system based on deep learning target detection provided by this invention; Figure 2 A flowchart for improving the detection of foreign objects using the YOLOv5 model. Detailed Implementation

[0018] The technical solutions provided in this application are further illustrated below with reference to the embodiments.

[0019] I. System Overall Structure The automatic foreign object risk identification system of the present invention includes: The image acquisition module is used to acquire image or video data at the nuclear power plant operation site; The image preprocessing module is used to standardize the acquired images; The foreign object detection module identifies foreign objects based on an improved YOLOv5 model. The risk analysis module is used to perform statistical analysis and risk quantification on the test results; The results output module is used to output the foreign object identification results and provide data support for the foreign object protection authorization process.

[0020] The modules mentioned above can be deployed on mobile terminals, edge computing devices, or servers, and interact with each other via the network.

[0021] II. Image Acquisition and Preprocessing (a) Image Acquisition Images of the work site are acquired using mobile terminal cameras or fixed industrial video equipment; the acquired images are denoted as:

[0022] Where x represents the x-axis coordinate of a pixel, y represents the y-axis coordinate of a pixel, and c represents the color channel.

[0023] (II) Image Preprocessing To improve the model's adaptability to complex field environments, the following preprocessing steps are performed on the original images: 1. Size normalization The input image is scaled to the fixed size required by the YOLOv5s model to obtain the scaled image. :

[0024] in, Represents the image size transformation function. The width of the input image. The height of the input image.

[0025] 2. Pixel normalization Normalize the image pixel values:

[0026] in, Let be the image ordinal number, indicating the first digit. Zhang preprocessed image, This represents the normalized pixel value of the nth preprocessed image.

[0027] 3. Brightness and noise suppression processing Gaussian filtering is used to smooth the image, resulting in a smoothed image. :

[0028] in, The radius of the filtering window. The offset of the x-axis within the convolution window, This represents the offset of the y-axis within the convolution window. This represents the normalized pixel value at position (x+m, y+g). The pixel value is the result of the Gaussian kernel function.

[0029] III. Improved YOLOv5 model structure This embodiment addresses the characteristics of foreign object detection targets at nuclear power plant sites, which are characterized by "small size, diverse types, complex backgrounds, and high real-time requirements." It makes targeted improvements to the network structure of the YOLOv5s target detection model, achieving both model lightweighting and improved inference efficiency while ensuring detection accuracy.

[0030] (I) Description of the basic model structure The YOLOv5s model consists of a backbone network, a feature fusion network (Neck), and a detection head. Specifically: The backbone network is used to extract semantic features at different scales; The Neck network employs FPN and PAN structures to achieve multi-scale feature fusion; The detection head outputs the target's category and location prediction results based on fused features.

[0031] Although YOLOv5s has certain advantages in terms of lightweight design compared to other object detection models, its backbone network and feature fusion part still have computational redundancy in the deployment scenarios of mobile terminals and edge devices at nuclear power plant sites, which affects the model's real-time performance and deployment adaptability.

[0032] (II) Improvements to the backbone network based on ShuffleNetV2 To reduce the computational complexity of the backbone network, this embodiment introduces the ShuffleNetV2 lightweight network structure to reconstruct the YOLOv5s backbone network.

[0033] In the original YOLOv5s backbone network, multiple C3 modules are composed of several Bottleneck units connected in series. Although this has strong feature representation capabilities, it suffers from redundant computation in scenarios with dense small targets. Therefore, this embodiment replaces the C3 modules, which are located in the low-to-mid-level feature extraction stage and whose output features are used for subsequent multi-scale feature fusion, with a ShuffleNetV2 structure while maintaining the network depth and feature output scale, to reduce the computational complexity of the backbone network.

[0034] The ShuffleNetV2 unit includes the following steps: 1. Channel splitting processing Input feature map It is divided into two parts along the channel dimension, one part of which is directly used as a bypass output, i.e. The other part enters the feature transformation branch, that is... :

[0035] 2. Lightweight Feature Transformation For feature branches Perform channel-wise convolution and point-wise convolution sequentially to obtain the feature branches after convolution. To reduce the number of parameters and multiply-accumulate operations:

[0036] in, This indicates a channel-wise convolution operation. This indicates pointwise convolution operation.

[0037] 3. Channel splicing and shuffling The bypass feature X1 is concatenated with the transformed feature Y2 to obtain the concatenated feature. Furthermore, the information exchange between different channels is enhanced through channel shuffling operations:

[0038] in, This indicates a "channel shuffling operation," used to enhance information exchange between channels. This represents the "feature concatenation operator", which is used to concatenate two feature maps along the channel dimension.

[0039] Through the above structural improvements, the parameter size and computational load of the backbone network are effectively reduced without significantly reducing the feature representation capability, thereby improving the inference efficiency of the model on nuclear power plant field equipment.

[0040] IV. Optimization of Neck Feature Fusion Structure Based on GSConv In the feature fusion stage of YOLOv5s, the Neck network performs multiple upsampling, downsampling and convolution operations on features at different scales through FPN and PAN structures. This process has a significant impact on the overall computational cost of the model.

[0041] To further reduce computational complexity, this embodiment introduces the GSConv lightweight convolutional module into the Neck network to replace at least some of the standard convolutional layers.

[0042] The GSConv module includes: Main feature extraction branch: Performs standard convolution on the input features to obtain the main feature map. :

[0043] in, This represents the standard convolution operation.

[0044] Ghost feature generation branch: Based on the main feature map, redundant features are generated through inexpensive linear transformation. :

[0045] in, This represents the inexpensive linear transformation symbol.

[0046] Feature fusion and channel shuffling: The main feature and the Ghost feature are concatenated, and a channel shuffling operation is performed to form the final output feature.

[0047] By introducing GSConv into the Neck section, this embodiment significantly reduces the number of parameters and computational complexity of the convolutional layer while ensuring the multi-scale feature fusion effect, thereby enhancing the model's real-time response capability in small-scale foreign object target detection.

[0048] V. Foreign Object Target Detection and Result Calculation This embodiment is based on an improved YOLOv5 object detection model, such as... Figure 2 As shown, foreign object targets in images of nuclear power plant operation sites are automatically identified, and the detection results are then structured and risk quantified to support intelligent decision-making for foreign object prevention permit management.

[0049] 1) Definition of Foreign Object Target Categories To identify common types of foreign objects at nuclear power plant work sites, a set of target categories for foreign object prevention is pre-constructed. :

[0050] in, For the first type of foreign object protection target, For the second type of foreign object protection target, For the first Type of foreign object target, Total number of foreign object target categories.

[0051] The target categories include at least: tool-type foreign objects, fastener-type foreign objects, soft foreign objects, and debris-type foreign objects.

[0052] The above classification is consistent with the foreign object prevention management standards, and is used to ensure the consistency between test results and on-site management requirements.

[0053] (II) Output format for foreign object detection After performing inference on the input image, the improved YOLOv5 model outputs a set of detection results for multiple foreign objects. :

[0054] The detection result for the first foreign object target. The detection results for the second foreign object target. For the first The detection results of foreign object targets, This represents the total number of detection results for foreign object targets.

[0055] Each test result Defined as:

[0056] In the formula:

[0057] Indicates the bounding box location of the foreign object target. , and , respectively, are the x and y coordinates of the center point of the bounding box of the i-th foreign object target; , These are the width and height of the bounding box, respectively; Indicates the category to which the foreign object belongs; This indicates the model's confidence level in the detection result; This is an index for foreign object detection results.

[0058] (iii) Screening and deduplication of test results To reduce false positives and duplicate detections, this embodiment performs screening and deduplication on the original detection results.

[0059] Confidence screening: Set confidence threshold ,when If the result is not found, discard the corresponding test result.

[0060] Non-maximum suppression processing: For overlapping detection boxes of the same category, the non-maximum suppression (NMS) algorithm is used for processing.

[0061] in, Indicates the intersection-union ratio of two bounding boxes; For the detection frame, To preset the overlap threshold, This indicates a comparison of the confidence levels corresponding to the two detection boxes. and Retain the detection results with higher confidence and delete the detection results with lower confidence; Represents the detection box The corresponding confidence level.

[0062] (iv) Calculation of spatial characteristics of foreign objects Based on the detected results after screening, the spatial characteristics of the foreign object target are further calculated.

[0063] Area percentage calculate:

[0064] in, Indicates the first The proportion of the pixel area of ​​a foreign object to the total pixel area of ​​the input image. This represents the width of the bounding box of the i-th foreign object target. This represents the height of the bounding box of the i-th foreign object. The bounding box of the i-th foreign object target is the area in the image, i.e. Indicates the first The pixel area of ​​a foreign object in the image. , These represent the width and height of the input image, respectively.

[0065] Critical area distance calculation: If the location of the critical area of ​​the equipment is predefined. Then calculate the distance between the foreign object target and the critical area. :

[0066] The function representing the spatial distance calculation between the i-th foreign object target and the critical area is:

[0067] in, For the first The x-coordinate of the center point of the bounding box of the foreign object target. Let be the ordinate of the center point of the bounding box of the i-th foreign object target. For predefined critical areas of the device The x-coordinate of the center point, For predefined critical areas of the device The ordinate of the center point; Distance between foreign object target and critical area It is used to assess the potential impact of foreign objects on critical equipment.

[0068] (v) Weighted Calculation Method for Foreign Object Risk Based on the detection results and spatial characteristics, a foreign object risk weighted calculation model is constructed.

[0069] For each foreign object target, define its risk contribution value. :

[0070] in: For the first Foreign object target belongs to foreign object category The corresponding risk weights, Indicates the detection confidence level. This indicates the percentage of the area occupied by foreign objects.

[0071] In embodiments with critical region constraints, a distance factor is further introduced:

[0072] in, This is the distance decay function.

[0073] (vi) Calculation of comprehensive risk value The total foreign object risk value is obtained by summing up all foreign object risk contribution values ​​within the same work site. :

[0074] The comprehensive risk value is used to reflect the overall risk level of foreign objects at the current work site.

[0075] VI. Methods for Determining the Risk Level of Foreign Objects This embodiment, based on the completion of foreign object target detection and risk quantification calculation, further determines the level of foreign object risk at the work site in order to achieve intelligent control of the foreign object clearance process.

[0076] (I) General Approach to Risk Level Determination Foreign object risk level is determined based on comprehensive risk value. With foreign objects as the core, and combined with information on the quantity, type, and spatial location of foreign objects, a multi-dimensional assessment of foreign object prevention risks at the work site is conducted to avoid misjudgments or omissions caused by a single indicator.

[0077] (ii) Normalization of basic risk values Since the dimensions and numerical range of the comprehensive risk value may differ under different work scenarios and equipment types, this embodiment first normalizes the comprehensive risk value:

[0078] in: and These represent the maximum and minimum comprehensive risk values ​​obtained from historical statistics, respectively. This is the normalized risk value, with a range of [0,1].

[0079] (iii) Foreign object quantity correction factor In some embodiments, relying solely on risk-weighted values ​​may not accurately reflect the potential risks posed by a "dense distribution of multiple foreign objects," therefore, a foreign object quantity correction factor is introduced. :

[0080] in, This indicates the number of foreign objects detected.

[0081] Corrected risk value Represented as:

[0082] IV) Correction of Foreign Object Type Distribution To further differentiate the impact of different foreign object combinations on safety, this embodiment introduces a foreign object type distribution correction factor. :

[0083] in: Indicates the first The number of foreign objects, To prevent the total number of foreign object target categories, This represents the risk sensitivity coefficient corresponding to the type of foreign object.

[0084] Corrected risk value for:

[0085] (v) Spatial Position Enhancement Factor When foreign objects are near critical equipment or critical work areas, the risk is significantly higher than in ordinary areas. This embodiment can introduce a spatial location enhancement factor p:

[0086] in: Indicates the distance between the foreign object target and the critical area. This is the distance decay function.

[0087] The final risk assessment value is:

[0088] (vi) Risk Level Mapping Rules Based on the final risk assessment value The risk of foreign objects is divided into multiple levels:

[0089] in, For configurable high-risk thresholds, This is a configurable medium-risk threshold.

[0090] VII. Automatic Foreign Object Identification Method and Flowchart Based on the above technical solutions, such as Figure 1 As shown, the method flow of the present invention includes the following steps: S1: Acquire image data of the nuclear power plant operation site; S2: Preprocess the acquired image data; S3: Input the preprocessed image into the improved YOLOv5 model for feature extraction and object detection; S4: Output the detection results of foreign objects, including category, location, and confidence level; S5: Calculate the overall risk value based on the type of foreign object and the detection results; S6: Determine the risk level of foreign object prevention based on the comprehensive risk value and output it to the foreign object prevention permit management system.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An automatic foreign object risk identification method based on deep learning object detection, comprising: Step 1: Collect image data of the nuclear power plant operation site; Step 2: Perform image preprocessing on the collected image data of the nuclear power plant operation site; Step 3: Construct an improved YOLOv5 target detection model, and input the preprocessed image into the improved YOLOv5 target detection model for feature extraction and foreign object target detection; Step 4: By improving the YOLOv5 target detection model, output the foreign object detection results, which include: foreign object target category, target bounding box position, and detection confidence. Step 5: Perform confidence screening and non-maximum suppression processing on the foreign object detection results to obtain the final foreign object detection results; Step 6: Calculate the spatial characteristics of foreign objects and construct the foreign object risk contribution value based on the final foreign object target detection results; Step 7: Calculate the comprehensive foreign object risk value at the work site based on the risk contribution value of each foreign object target, and determine the foreign object prevention risk level based on the comprehensive foreign object risk value.

2. The automatic foreign object risk identification method based on deep learning target detection according to claim 1, characterized in that, In step 2, the image preprocessing includes: size normalization, pixel normalization, and Gaussian filtering smoothing. 3.The foreign matter risk automatic identification method based on deep learning target detection of claim 1, wherein, In step 3, the improved YOLOv5 object detection model includes: a backbone network, a feature fusion network, and a detection head. The backbone network uses the ShuffleNetV2 structure to lightweight improve the feature extraction structure in the original YOLOv5s model. The feature fusion network uses the GSConv lightweight convolutional module to replace at least some of the standard convolutional layers.

4. The foreign matter risk automatic identification method based on deep learning target detection according to claim 3, characterized in that, The ShuffleNetV2 architecture includes: a channel splitting processing unit, a lightweight feature transformation unit, and a channel splicing and shuffling unit, wherein... The channel splitting processing unit is configured to split the input feature map into two parts along a channel dimension The first part of the feature map obtained by channel splitting is directly output as a bypass output The second part of the feature map obtained by channel splitting is directly output as a bypass output Enter a feature transformation branch: ； The lightweight feature variable unit is configured to process the second partial feature map The channel-wise convolution and the point-wise convolution are sequentially performed to obtain the second partial feature map after convolution To reduce the parameter quantity and multiply-add operation: ； wherein, denotes a channel-wise convolution operation, denotes a point-wise convolution operation; The channel splicing and shuffling unit is used to splice the first part of the feature map. The second part of the feature map after convolution Perform splicing to obtain the spliced ​​feature map. Furthermore, the information exchange between different channels is enhanced through channel shuffling operations: ； in, This indicates a channel shuffling operation, used to enhance information exchange between channels; This indicates a feature concatenation operator, used to concatenate the first part of the feature map. The second part of the feature map after convolution The components are stitched together along the channel dimension. The GSConv lightweight convolutional module includes: a main feature extraction unit, a Ghost feature generation unit, and a feature fusion and channel shuffling unit, wherein... The main feature extraction unit is used to perform standard convolution on the input feature map to obtain the main feature map. : ； in, This represents the standard convolution operation; The Ghost feature generation unit is based on the main feature mapping. Redundant features are generated through inexpensive linear transformations. : ； in, Represents inexpensive linear transformation symbols; The feature fusion and channel shuffling unit is used to map the main features. With redundancy features The data is then concatenated and a channel shuffling operation is performed to form the final output features. : 。 5. The automatic foreign object risk identification method based on deep learning target detection according to claim 1, characterized in that, In step 4, the detection result of a single foreign object target is expressed as follows: ； in, No. The detection results of the foreign object target, Indicates the first The bounding box of a foreign object target. Indicates the first The category to which the foreign object belongs. This indicates that the improved YOLOv5 object detection model is for the first... The confidence level of the detection results for each foreign object target; among which, ； in, For the first The x-coordinate of the center point of the bounding box of the foreign object target. The ordinate of the center point of the bounding box of the i-th foreign object target; For the first The width of the bounding box of each foreign object target. Let be the height of the bounding box of the i-th foreign object target; A collection of foreign object detection results Represented as: ,in, The detection result for the first foreign object target. The detection result for the second foreign object target. For the first The detection results of the foreign object target, This represents the total number of detection results for foreign object targets.

6. The automatic foreign object risk identification method based on deep learning target detection according to claim 5, characterized in that, In step 5, the confidence level screening criteria are as follows: ； in, This indicates that the improved YOLOv5 object detection model is for the first... The confidence level of the detection results for a foreign object target. The preset confidence threshold; when Then it means the first The detection results of each foreign object target are filtered by confidence level; otherwise, they are considered to have failed the confidence level filter and the detection results of foreign objects targets that have failed the confidence level filter are discarded. In step 5, non-maximum suppression is applied to overlapping bounding boxes of the same foreign object category: ； in, Indicates the first The bounding box of a foreign object target. No. The bounding box of a foreign object target. Represents bounding box and bounding box The intersection, union, and ratio operations, To preset the overlap threshold, Indicates comparison bounding boxes and bounding box The corresponding confidence levels and Among them, detection results with higher confidence are retained, while detection results with lower confidence are deleted.

7. The automatic foreign object risk identification method based on deep learning target detection according to claim 1, characterized in that, In step 6, the characteristics of the foreign object space include the area ratio. Distance from key areas ,in, The area ratio for; ； in, Indicates the first The width of the bounding box of each foreign object target. Indicates the first The height of the bounding box of the foreign object target. Indicates the first The area of ​​the bounding box of each foreign object in the image. Indicates the width of the input image. Indicates the height of the input image; The distance of the key area for: If the key areas of the equipment are predefined Then calculate the distance between the foreign object target and the critical area. : ； in, For the first The x-coordinate of the center point of the bounding box of the foreign object target. Let be the ordinate of the center point of the bounding box of the i-th foreign object target. For predefined critical areas of the device The x-coordinate of the center point, For predefined critical areas of the device The ordinate of the center point; Step 6, in which the foreign object risk contribution value is constructed, specifically includes: For each foreign object target, define a corresponding risk contribution value. : ； in, For the first Foreign object target belongs to foreign object category The corresponding risk weights, For the first The confidence level of the detection results for a foreign object target. This represents the distance decay function.

8. The automatic foreign object risk identification method based on deep learning target detection according to claim 7, characterized in that, Step 7 specifically includes: Step 7.1: Sum the risk contribution values ​​of all foreign objects within the same work site to obtain the comprehensive foreign object risk value. : ； in, This indicates the total number of foreign objects detected. Step 7.2: Assess the overall foreign object risk value Normalization is performed: ； in: This represents the maximum overall risk value obtained from historical statistics. This represents the minimum comprehensive risk value obtained from historical statistics. This is the normalized overall foreign object risk value. The value range is [0,1]; Step 7.3: Introduce foreign object quantity factor, foreign object type distribution factor, and spatial location enhancement factor to the normalized comprehensive foreign object risk value. Make corrections to obtain the final risk assessment value. : Step 7.4: Based on the final risk assessment value The risk of foreign objects is classified into the following levels: ； in, For configurable high-risk thresholds, This is a configurable medium-risk threshold.

9. The automatic foreign object risk identification method based on deep learning target detection according to claim 8, characterized in that, Step 7.3 specifically includes: Step 7.3.1: Utilize the foreign object quantity correction factor The normalized overall foreign object risk value is corrected as follows: ； in, The overall foreign object risk value after adjusting for the number of foreign objects, and the foreign object number adjustment factor. for: ； Step 7.3.2: Utilize the foreign object type distribution factor Overall foreign object risk value after adjusting for the number of foreign objects Make corrections: ； in, The overall foreign object risk value after correction for foreign object type, and the foreign object type distribution factor. for: ； in, The ordinal number of the foreign object's category. The total number of categories of foreign object targets, Indicates the first The number of foreign objects in each category, For the first Risk sensitivity coefficients for each category of foreign object; the categories of foreign object protection targets include at least: tool-type foreign objects, fastener-type foreign objects, soft foreign objects, and debris-type foreign objects; Step 7.3.3: Utilizing the spatial location enhancement factor Overall foreign object risk value after correction for foreign object type Make corrections: ； in, The spatial location enhancement factor serves as the final risk assessment value. for: 。 10. An automatic foreign object risk identification system based on deep learning target detection, characterized in that, include: The system comprises an image acquisition module, an image preprocessing module, a foreign object detection module, a risk quantification and analysis module, and a risk level determination module; among which, The image acquisition module is used to acquire image data of the nuclear power plant operation site; The image preprocessing module performs image preprocessing on the collected image data from the nuclear power plant operation site; The foreign object detection module has a built-in improved YOLOv5 target detection model, which is used to extract features and detect foreign objects from the received preprocessed image data, and output the foreign object detection results. The foreign object detection results include: foreign object category, target bounding box position and detection confidence. The risk quantification and analysis module is used to perform confidence screening and non-maximum suppression processing on the foreign object target detection results to obtain the final foreign object target detection results. Based on the final foreign object target detection results, the foreign object spatial characteristics are calculated and a foreign object risk contribution value is constructed. The risk level determination module is used to calculate the comprehensive foreign object risk value at the work site based on the risk contribution value of each foreign object target, and to determine the foreign object prevention risk level based on the comprehensive foreign object risk value.

Images