Potential safety hazard detection method, device, equipment, medium and product
By using a detection model that includes an input layer, a feature enhancement layer, and an upsampling layer during data center inspections, the problem of inaccurate detection of small-scale security vulnerabilities has been solved, achieving higher detection accuracy.
Patent Information
- Application Number
- CN202510992817.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-18
- Publication Date
- 2025-10-31
AI Technical Summary
Existing intelligent inspection solutions for data centers are not accurate enough in detecting small-scale security risks.
A detection model consisting of an input layer, a feature enhancement layer, and an upsampling layer is adopted. Feature fusion is performed through convolution processing, feature enhancement, and upsampling to improve detection accuracy.
By preserving image details and performing multiple feature fusions, the accuracy of detecting small-scale security risks is improved.
Smart Images
Figure CN120876831A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, equipment, medium and product for detecting safety hazards. Background Technology
[0002] The purpose of computer room inspection is to promptly identify and address any abnormalities in the equipment according to a pre-set inspection plan, thereby eliminating potential safety hazards in the operating equipment and ensuring stable production. The inspection content includes equipment inspection, environmental monitoring, and power safety.
[0003] Currently, intelligent inspection of data centers uses deep learning-based detection models to detect potential security risks. However, the current solution is inaccurate when small-scale security risks exist. Summary of the Invention
[0004] This application provides a method, apparatus, equipment, medium, and product for detecting safety hazards, in order to improve the accuracy of safety hazard detection.
[0005] In a first aspect, this application provides a method for detecting security risks, comprising: acquiring an image to be detected; inputting the image to be detected into a trained detection model for processing; wherein the detection model includes an input layer, a feature enhancement layer, and an upsampling layer; the input layer is used to perform convolution processing on the image to be detected to obtain multiple first feature images of different sizes; each of the multiple first feature images of different sizes is obtained by convolution based on the first feature image of the previous size; the feature enhancement layer is used to perform feature enhancement on the multiple first feature images of different sizes to obtain multiple feature enhancement images of different sizes; the upsampling layer is used to upsample the multiple feature enhancement images of different sizes to obtain multiple upsampled images of different sizes; feature fusion is performed based on the multiple upsampled images of different sizes and the multiple feature enhancement images of different sizes to obtain multiple first feature fusion images; and security risk target detection is performed based on the multiple first feature fusion images to obtain a detection result.
[0006] Secondly, this application provides a safety hazard detection device, comprising: an acquisition module for acquiring an image to be detected; and a processing module for inputting the image to be detected into a trained detection model for processing; wherein the detection model includes an input layer, a feature enhancement layer, and an upsampling layer; the input layer is used to perform convolution processing on the image to be detected to obtain multiple first feature images of different sizes; each of the multiple first feature images of different sizes is obtained by convolution based on the first feature image of the previous size; the feature enhancement layer is used to perform feature enhancement on the multiple first feature images of different sizes to obtain multiple feature enhancement images of different sizes; the upsampling layer is used to upsample the multiple feature enhancement images of different sizes to obtain multiple upsampled images of different sizes; and the detection module is used to perform feature fusion based on the multiple upsampled images of different sizes and the multiple feature enhancement images of different sizes to obtain multiple first feature fusion images; and to perform safety hazard target detection based on the multiple first feature fusion images to obtain a detection result.
[0007] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;
[0008] The memory stores computer-executed instructions;
[0009] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.
[0010] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.
[0011] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.
[0012] The safety hazard detection method, apparatus, equipment, medium, and product provided in this application acquire an image to be detected and input it into a trained detection model for processing to obtain a detection result. The detection model includes an input layer, a feature enhancement layer, and an upsampling layer. The input layer performs convolution processing on the image to be detected, obtaining multiple first feature images of different sizes. Each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size. The feature enhancement layer enhances the features of the multiple first feature images of different sizes, obtaining multiple feature-enhanced images of different sizes. The upsampling layer upsamples the multiple feature-enhanced images of different sizes, obtaining multiple upsampled images of different sizes. Based on the multiple upsampled images and the multiple feature-enhanced images of different sizes, feature fusion is performed to obtain multiple first feature fused images. Safety hazard target detection is then performed based on the multiple first feature fused images to obtain a detection result. Upsampling the obtained feature-enhanced images of multiple sizes preserves more detailed information in the image, improving the accuracy of the subsequent detection model's detection results. Subsequently, multiple upsampled images of different sizes and corresponding feature-enhanced images of different sizes are fused to obtain multiple first feature fused images of different sizes. Then, security hazard target detection is performed based on these multiple first feature fused images, yielding detection results and enabling the detection of security hazards of various sizes. The multiple first feature fused images further incorporate more image detail information, further improving the accuracy of the detection model in detecting security hazard targets. Attached Figure Description
[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0014] Figure 1 An exemplary flowchart of a safety hazard detection method is shown.
[0015] Figure 2 This is a schematic diagram of the architecture of an example detection model in this application;
[0016] Figure 3 This is a schematic diagram of the structure of a feature enhancement module as an example;
[0017] Figure 4 An exemplary schematic diagram of a safety hazard detection device is shown;
[0018] Figure 5 The diagram above illustrates the structure of an electronic device.
[0019] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation
[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0021] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.
[0022] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.
[0023] It should be noted that the safety hazard detection methods, devices, equipment, storage media and products provided in this application can be used in the field of artificial intelligence, or in any field other than artificial intelligence. The application fields of the safety hazard detection methods, devices, equipment, storage media and products in this application are not limited.
[0024] The purpose of computer room inspection is to promptly identify and address any abnormalities in the equipment according to a pre-set inspection plan, thereby eliminating potential safety hazards in the operating equipment and ensuring stable production. The inspection content includes equipment inspection, environmental monitoring, and power safety.
[0025] Currently, intelligent inspection of data centers uses deep learning-based detection models to detect potential security risks. However, the current solution is inaccurate when small-scale security risks exist.
[0026] The safety hazard detection method, apparatus, equipment, medium, and product provided in this application aim to solve the aforementioned technical problems of the prior art. The safety hazard detection method, apparatus, equipment, medium, and product provided in this application acquire an image to be detected and input it into a trained detection model for processing to obtain a detection result. The detection model includes an input layer, a feature enhancement layer, and an upsampling layer. The input layer performs convolution processing on the image to be detected to obtain multiple first feature images of different sizes. Each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size. The feature enhancement layer enhances the features of the multiple first feature images of different sizes to obtain multiple feature-enhanced images of different sizes. The upsampling layer upsamples the multiple feature-enhanced images of different sizes to obtain multiple upsampled images of different sizes. Feature fusion is performed based on the multiple upsampled images and the multiple feature-enhanced images of different sizes to obtain multiple first feature fused images. Safety hazard target detection is then performed based on the multiple first feature fused images to obtain a detection result. Upsampling the obtained feature-enhanced images of multiple sizes preserves more detailed information in the image, improving the accuracy of the subsequent detection model's detection results. Subsequently, multiple upsampled images of different sizes and corresponding feature-enhanced images of different sizes are fused to obtain multiple first feature fused images of different sizes. Then, security hazard target detection is performed based on these multiple first feature fused images, yielding detection results and enabling the detection of security hazards of various sizes. The multiple first feature fused images further incorporate more image detail information, further improving the accuracy of the detection model in detecting security hazard targets.
[0027] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.
[0028] Example 1
[0029] Figure 1 An exemplary flowchart of a safety hazard detection method is shown; including:
[0030] Step 101: Obtain the image to be detected;
[0031] Step 102: Input the image to be detected into the trained detection model for processing; wherein, the detection model includes an input layer, a feature enhancement layer, and an upsampling layer; the input layer is used to perform convolution processing on the image to be detected to obtain multiple first feature images of different sizes; each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size; the feature enhancement layer is used to enhance the features of the multiple first feature images of different sizes to obtain multiple feature-enhanced images of different sizes; the upsampling layer is used to upsample the multiple feature-enhanced images of different sizes to obtain multiple upsampled images of different sizes;
[0032] Step 103: Based on multiple upsampled images of different sizes and multiple feature-enhanced images of different sizes, perform feature fusion to obtain multiple first feature fusion images; and perform safety hazard target detection based on the multiple first feature fusion images to obtain detection results.
[0033] Specifically, the process begins by acquiring images to be detected. These images are pictures of various devices or components in a computer room that require security hazard detection. In practical applications, these images can be obtained through methods such as taking photos or downloading. Next, the images are input into a trained detection model for processing. The detection model includes an input layer, a feature enhancement layer, and an upsampling layer. The input layer performs convolution processing on the images input to the model, resulting in multiple first feature images of different sizes. The feature enhancement layer enhances the received first feature images of different sizes, resulting in multiple enhanced feature images of different sizes. The upsampling layer upsamples these enhanced feature images, resulting in multiple upsampled feature images of different sizes. Based on the obtained upsampled feature images and the multiple enhanced feature images of different sizes, feature fusion is performed to obtain multiple fused first feature images. Further, security hazard target detection is performed based on these fused first feature images to obtain the final detection result. For example, the obtained multiple fused first feature images can be of different sizes, allowing for the detection of security hazard targets of different sizes, thus improving detection accuracy. In this example, a feature enhancement layer is used to enhance the features of the feature image output from the input layer. Then, the enhanced image is upsampled to obtain upsampled images of different sizes. Furthermore, feature fusion is performed based on these multiple enhanced and upsampled images to obtain multiple first feature fused images. Finally, security detection is performed based on these multiple fused images to obtain the detection result. This process preserves more detailed feature information in the processed image, improving the accuracy of detecting potential security threats.
[0034] Figure 2 This is a schematic diagram of the architecture of an example detection model in this application; Figure 2 Taking a detection model that includes four levels of depthwise separable convolutional modules, four levels of feature enhancement modules, three levels of upsampling modules, three levels of first fusion modules, three levels of third convolutional modules, and three levels of second fusion modules as an example. Please refer to the following. Figure 2 ;
[0035] The input layer includes a first convolutional module and multiple levels of depthwise separable convolutional modules;
[0036] The first convolutional module is used to perform convolution processing on the image to be detected and output the initial feature image.
[0037] Each level of depthwise separable convolutional module is used to perform convolution processing on the first feature image output by the previous level of depthwise separable convolutional module, and output a first feature image of the corresponding size.
[0038] The primary depth-separable convolution module is used to perform convolution processing on the initial feature image.
[0039] Specifically, the first convolutional module receives the image to be detected and performs convolution processing on it to obtain an initial feature image. Each level of the multi-level depthwise separable convolutional module performs convolution processing on the first feature image output by the previous level's depthwise separable convolutional module, outputting a first feature image of the corresponding size. The first-level depthwise separable convolutional module performs convolution processing on the initial feature image. The depthwise separable convolutional module first performs channel-by-channel convolution on the received image, then performs spatial convolution on each channel individually (without mixing channel information), followed by pointwise convolution (1×1), and finally uses 1×1 convolution to mix channel information. This increases the speed of feature extraction and improves the detection efficiency of the detection model.
[0040] Optionally, the feature enhancement layer includes a multi-level feature enhancement module and a second convolutional module;
[0041] The multi-level feature enhancement module is used to enhance the features of multiple first feature images of different sizes to obtain feature-enhanced images of the corresponding sizes; the multi-level feature enhancement module corresponds one-to-one with the depthwise separable convolution module;
[0042] The second convolution module is used to perform convolution processing on the first feature image output by the last-stage depthwise separable convolution module to output the second feature image; the feature enhancement module corresponding to the last-stage depthwise separable convolution module is used to enhance the features of the second feature image to obtain the corresponding feature-enhanced image.
[0043] Specifically, the feature enhancement layer includes a multi-level feature enhancement module and a second convolution module. The multi-level feature enhancement module enhances multiple first feature images of different sizes to obtain a feature-enhanced image of the corresponding size. Each multi-level feature enhancement module corresponds one-to-one with a depthwise separable convolution module. The second convolution module performs convolution processing on the first feature image output by the last depthwise separable convolution module to output a second feature image. The feature enhancement module corresponding to the last depthwise separable convolution module then enhances the second feature image to obtain the corresponding feature-enhanced image. In this example, image enhancement of the first feature image improves the expressiveness of the features, thereby increasing the accuracy of the detection results during subsequent security vulnerability detection.
[0044] Optional, Figure 3 Here is a schematic diagram of the structure of a feature enhancement module as an example; such as Figure 3 As shown, the feature enhancement module includes multiple convolutional units and fusion units; the multi-level feature enhancement module performs feature enhancement on multiple first feature images of different sizes to obtain a feature-enhanced image of the corresponding size, specifically for:
[0045] The first feature image of the corresponding size is input into multiple convolutional units for convolution processing to obtain the feature images output by multiple convolutional units;
[0046] The feature images output by multiple convolutional units are input into the fusion unit to perform feature fusion and obtain a feature-enhanced image of the corresponding size.
[0047] Specifically, each feature enhancement module includes multiple convolutional units and one fusion unit; for example... Figure 3 As shown, the feature enhancement module includes three convolutional units and one fusion unit. When the feature enhancement module receives a first feature image of a corresponding size, it inputs the first feature image of that size into the three convolutional units respectively for convolution processing, obtaining the feature images output by the three convolutional units. Then, the feature images output by the three convolutional units are input into the fusion unit for feature fusion, obtaining the feature-enhanced image of the corresponding size. For example, the three convolutional units can be: the first branch is a 1×1 convolution, the second branch is a 1×1 and 3×3 convolution, and the third branch is a 1×1, 3×3, and 3×3 convolution. In this example, by using the convolutional units on each branch to convolve the first feature image, the receptive field of the feature is broadened, and the obtained multi-path features are fused, improving the richness and expressive power of the features, while also improving the efficiency of the convolution.
[0048] Continue to refer to Figure 2 The upsampling layer includes multiple upsampling modules; the detection model also includes multiple first fusion modules, multiple third convolution modules, and multiple second fusion modules.
[0049] The multi-level first fusion module is used to perform feature fusion on multiple feature-enhanced images of different sizes and corresponding upsampled images of the same size, and output multiple second feature fusion images of different sizes; wherein, the feature-enhanced image of the smallest size is used as the second feature fusion image of that size;
[0050] The multi-level upsampling module is used to upsample multiple second feature fusion images of different sizes and output multiple upsampled images of different sizes.
[0051] The multi-level third convolution module is used to perform convolution processing on other second feature fusion images besides the second feature fusion image in multiple second feature fusion images of different sizes, and output multiple third feature images of different sizes.
[0052] The multi-level second fusion module is used to perform feature fusion on multiple third feature images of different sizes and corresponding second feature fusion images of the same size to obtain multiple first feature fusion images.
[0053] Specifically, each level of depthwise separable convolutional module outputs a first feature image; the first feature image output by the last level of depthwise separable convolutional module is processed by the second convolutional module to output a second feature image; then, the second feature image is input to the corresponding feature enhancement module for feature enhancement; the first feature images output by the first three levels of depthwise separable convolutional modules are input to the corresponding feature enhancement modules for feature enhancement. Each feature enhancement module outputs a corresponding feature enhancement image. The feature enhancement image output by the last-level feature enhancement module is input to the third-level upsampling module for upsampling processing to obtain the corresponding upsampled image. The third-level upsampled image and the feature enhancement image output by the third-level feature enhancement module are then passed through the third-level first fusion module for feature fusion, outputting the second feature fusion image corresponding to the third-level first feature fusion module. The second feature fusion image corresponding to the third level is then input to the corresponding second-level upsampling module for upsampling processing to obtain the second-level upsampled image. The second-level upsampled image is then passed through the second-level first feature fusion module again with the feature enhancement image output by the second-level feature enhancement module for feature fusion, obtaining the second feature fusion image corresponding to the second level. The second-level feature fusion image is then input to the corresponding first-level upsampling module for upsampling processing to obtain the second-level upsampled image. The second-level upsampled image and the feature enhancement image output by the first-level feature enhancement module are then fused to obtain the first-level second feature fusion image. The fourth-level second feature fusion image is the feature enhancement image corresponding to the fourth-level feature enhancement module. Furthermore, the first-level second feature fusion image is passed through the first-level third convolution module for convolution processing to obtain the first-level third feature image. The obtained first-level third feature image and the second-level second feature fusion image are then passed through the first-level second feature fusion module for feature fusion again to obtain the second-level first feature fusion image. The second-level first feature fusion image is then passed through the second-level third convolution module for convolution processing to obtain the second-level third feature image. The second-level third feature image and the third-level second feature fusion image are then passed through the third-level second fusion feature module for fusion processing to obtain the third-level first feature fusion image. The third-level first feature fusion image is then passed through the fourth-level third convolution module for convolution processing to obtain the fourth-level third feature image. The fourth-level third feature image and the fourth-level second feature fusion image are then passed through the fourth-level second feature fusion module for feature fusion to obtain the fourth-level first feature fusion image. The first-level first feature fusion image is essentially the same as the first-level second feature fusion image. In addition to being transmitted to the corresponding third convolution module, the first feature fusion image for each level is also output. Subsequently, based on the obtained first fusion feature images of different sizes, security hazard target detection is performed to obtain the final detection result.For example, different first-feature fused images correspond to different sizes, but have the same number of channels. For instance, the size of the first-level first-feature fused image is 152×152, the size of the second-level first-feature fused image is 76×76, the size of the third-level first-feature fused image is 38×38, and the size of the fourth-level first-feature image is 19×19, all with 150 channels. Subsequently, based on the first-feature fused images of different sizes, security risks of different sizes can be detected. This example can accurately detect small-sized target risks.
[0054] Optionally, the method also includes:
[0055] Obtain the training set for the detection model; the training set includes multiple training images;
[0056] Based on the training set, the prior bounding box parameters of the detection model are optimized using a clustering algorithm to obtain the optimized prior bounding box parameters.
[0057] A detection model is constructed based on the optimized prior box parameters.
[0058] Specifically, images with security risks are acquired and used as the training set for the detection model. The training set includes multiple training images, each containing small-sized security risks. Based on the acquired training set, a clustering algorithm is used to optimize the prior bounding box (anchor) parameters of the detection model. For example, a K-means (K-means++) algorithm can be used, where K represents the number of cluster centers. The width and height of the target boxes in the training set are clustered to obtain more representative prior bounding boxes. The clustering algorithm first randomly selects k cluster centers, then calculates the distances between other targets and the centers, and assigns them to the nearest centers to obtain k clusters. Through iterative adjustments, the distances between targets within each cluster decrease, while the distances between clusters increase. The cluster centers are then recalculated, and this process is repeated multiple times until the cluster centers no longer change, resulting in the optimized anchor parameters. Based on the optimized prior bounding box parameters, the detection model can be constructed. This example optimizes the prior bounding box parameters, improving the accuracy of the detection model.
[0059] Optionally, the method also includes:
[0060] The training set is input into the detection model to be trained, and the detection model is trained to obtain the trained detection model.
[0061] Specifically, after constructing the detection model, the training set is input into the model to be trained, resulting in a trained detection model. In this example, the training set is used to train the model, improving its accuracy.
[0062] Optionally, the method also includes:
[0063] Obtain the validation set of the detection model; the validation set includes multiple validation images; input the validation set into the trained detection model to optimize the model parameters;
[0064] Obtain the test set of the detection model; input the test set into the trained detection model. If the error between the output of the trained detection model and the actual observation result is not greater than the preset value, then the current detection model is used as the trained detection model.
[0065] Specifically, after inputting the training dataset into the detection model to be trained, a validation set is obtained. This validation set includes multiple validation images, which are images with small security vulnerabilities. The validation set is then input into the trained model to optimize its parameters, resulting in a trained detection model. Further, to verify the performance and generalization ability of the optimized model, a test set is obtained, consisting of multiple test images. This test set is input into the trained model. If the error between the output of the trained model and the actual observation results is no greater than a preset value, the current detection model is considered to have been successfully trained and is considered a successfully trained model. In this example, obtaining a validation set for parameter optimization and a test set for model performance testing improves the accuracy of the detection model training.
[0066] The security hazard detection method provided in this embodiment acquires an image to be detected and inputs it into a trained detection model for processing to obtain a detection result. The detection model includes an input layer, a feature enhancement layer, and an upsampling layer. The input layer performs convolution processing on the image to be detected, obtaining multiple first feature images of different sizes. Each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size. The feature enhancement layer enhances the features of the multiple first feature images of different sizes, obtaining multiple feature-enhanced images of different sizes. The upsampling layer upsamples the multiple feature-enhanced images of different sizes, obtaining multiple upsampled images of different sizes. Based on the multiple upsampled images and the multiple feature-enhanced images of different sizes, feature fusion is performed to obtain multiple first feature fusion images. Security hazard target detection is then performed based on the multiple first feature fusion images to obtain a detection result. Upsampling the obtained feature-enhanced images of multiple sizes preserves more detailed information in the image, improving the accuracy of the subsequent detection model's detection results. Subsequently, multiple upsampled images of different sizes and corresponding feature-enhanced images of different sizes are fused to obtain multiple first feature fused images of different sizes. Then, security hazard target detection is performed based on these multiple first feature fused images, yielding detection results and enabling the detection of security hazards of various sizes. The multiple first feature fused images further incorporate more image detail information, further improving the accuracy of the detection model in detecting security hazard targets.
[0067] Example 2
[0068] Figure 4 An exemplary schematic diagram of a safety hazard detection device is shown; including:
[0069] Acquisition module 21 is used to acquire the image to be detected;
[0070] Processing module 22 is used to input the image to be detected into a trained detection model for processing; wherein, the detection model includes an input layer, a feature enhancement layer, and an upsampling layer; the input layer is used to perform convolution processing on the image to be detected to obtain multiple first feature images of different sizes; each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size; the feature enhancement layer is used to perform feature enhancement on the multiple first feature images of different sizes to obtain multiple feature enhancement images of different sizes; the upsampling layer is used to upsample the multiple feature enhancement images of different sizes to obtain multiple upsampled images of different sizes;
[0071] The detection module 23 is used to perform feature fusion based on multiple upsampled images of different sizes and multiple feature-enhanced images of different sizes to obtain multiple first feature fusion images; and to perform safety hazard target detection based on the multiple first feature fusion images to obtain detection results.
[0072] Specifically, the process begins by acquiring images to be detected. These images are pictures of various devices or components in a computer room that require security hazard detection. In practical applications, these images can be obtained through methods such as taking photos or downloading. Next, the images are input into a trained detection model for processing. The detection model includes an input layer, a feature enhancement layer, and an upsampling layer. The input layer performs convolution processing on the images input to the model, resulting in multiple first feature images of different sizes. The feature enhancement layer enhances the received first feature images of different sizes, resulting in multiple enhanced feature images of different sizes. The upsampling layer upsamples these enhanced feature images, resulting in multiple upsampled feature images of different sizes. Based on the obtained upsampled feature images and the multiple enhanced feature images of different sizes, feature fusion is performed to obtain multiple fused first feature images. Further, security hazard target detection is performed based on these fused first feature images to obtain the final detection result. For example, the obtained multiple fused first feature images can be of different sizes, allowing for the detection of security hazard targets of different sizes, thus improving detection accuracy. In this example, a feature enhancement layer is used to enhance the features of the feature image output from the input layer. Then, the enhanced image is upsampled to obtain upsampled images of different sizes. Furthermore, feature fusion is performed based on these multiple enhanced and upsampled images to obtain multiple first feature fused images. Finally, security detection is performed based on these multiple fused images to obtain the detection result. This process preserves more detailed feature information in the processed image, improving the accuracy of detecting potential security threats.
[0073] The input layer includes a first convolutional module and multiple levels of depthwise separable convolutional modules;
[0074] The first convolutional module is used to perform convolution processing on the image to be detected and output the initial feature image.
[0075] Each level of depthwise separable convolutional module is used to perform convolution processing on the first feature image output by the previous level of depthwise separable convolutional module, and output a first feature image of the corresponding size.
[0076] The primary depth-separable convolution module is used to perform convolution processing on the initial feature image.
[0077] Specifically, the first convolutional module receives the image to be detected and performs convolution processing on it to obtain an initial feature image. Each level of the multi-level depthwise separable convolutional module performs convolution processing on the first feature image output by the previous level's depthwise separable convolutional module, outputting a first feature image of the corresponding size. The first-level depthwise separable convolutional module performs convolution processing on the initial feature image. The depthwise separable convolutional module first performs channel-by-channel convolution on the received image, then performs spatial convolution on each channel individually (without mixing channel information), followed by pointwise convolution (1×1), and finally uses 1×1 convolution to mix channel information. This increases the speed of feature extraction and improves the detection efficiency of the detection model.
[0078] Optionally, the feature enhancement layer includes a multi-level feature enhancement module and a second convolutional module;
[0079] The multi-level feature enhancement module is used to enhance the features of multiple first feature images of different sizes to obtain feature-enhanced images of the corresponding sizes; the multi-level feature enhancement module corresponds one-to-one with the depthwise separable convolution module;
[0080] The second convolution module is used to perform convolution processing on the first feature image output by the last-stage depthwise separable convolution module to output the second feature image; the feature enhancement module corresponding to the last-stage depthwise separable convolution module is used to enhance the features of the second feature image to obtain the corresponding feature-enhanced image.
[0081] Specifically, the feature enhancement layer includes a multi-level feature enhancement module and a second convolution module. The multi-level feature enhancement module enhances multiple first feature images of different sizes to obtain a feature-enhanced image of the corresponding size. Each multi-level feature enhancement module corresponds one-to-one with a depthwise separable convolution module. The second convolution module performs convolution processing on the first feature image output by the last depthwise separable convolution module to output a second feature image. The feature enhancement module corresponding to the last depthwise separable convolution module then enhances the second feature image to obtain the corresponding feature-enhanced image. In this example, image enhancement of the first feature image improves the expressiveness of the features, thereby increasing the accuracy of the detection results during subsequent security vulnerability detection.
[0082] The feature enhancement module includes multiple convolutional units and fusion units; the multi-level feature enhancement module enhances the features of multiple first feature images of different sizes to obtain a feature-enhanced image of the corresponding size, specifically for:
[0083] The first feature image of the corresponding size is input into multiple convolutional units for convolution processing to obtain the feature images output by multiple convolutional units;
[0084] The feature images output by multiple convolutional units are input into the fusion unit to perform feature fusion and obtain a feature-enhanced image of the corresponding size.
[0085] Specifically, each feature enhancement module includes multiple convolutional units and one fusion unit. The feature enhancement module comprises three convolutional units and one fusion unit. When the feature enhancement module receives a first feature image of a corresponding size, it inputs the first feature image of that size into the three convolutional units for convolution processing, obtaining the feature images output by the three convolutional units. Then, the feature images output by the three convolutional units are input into the fusion unit for feature fusion, resulting in a feature-enhanced image of the corresponding size. For example, the three convolutional units can be: the first branch is a 1×1 convolution, the second branch is a 1×1 and 3×3 convolution, and the third branch is a 1×1, 3×3, and 3×3 convolution. In this example, by using the convolutional units on each branch to convolve the first feature image, the receptive field of the feature is broadened, and the resulting multi-path features are fused, improving the richness and expressive power of the features while also increasing the efficiency of the convolution.
[0086] The upsampling layer includes multiple upsampling modules; the detection model also includes multiple first fusion modules, multiple third convolution modules, and multiple second fusion modules.
[0087] The multi-level first fusion module is used to perform feature fusion on multiple feature-enhanced images of different sizes and corresponding upsampled images of the same size, and output multiple second feature fusion images of different sizes; wherein, the feature-enhanced image of the smallest size is used as the second feature fusion image of that size;
[0088] The multi-level upsampling module is used to upsample multiple second feature fusion images of different sizes and output multiple upsampled images of different sizes.
[0089] The multi-level third convolution module is used to perform convolution processing on other second feature fusion images besides the second feature fusion image in multiple second feature fusion images of different sizes, and output multiple third feature images of different sizes.
[0090] The multi-level second fusion module is used to perform feature fusion on multiple third feature images of different sizes and corresponding second feature fusion images of the same size to obtain multiple first feature fusion images.
[0091] Specifically, each level of depthwise separable convolutional module outputs a first feature image; the first feature image output by the last level of depthwise separable convolutional module is processed by the second convolutional module to output a second feature image; then, the second feature image is input to the corresponding feature enhancement module for feature enhancement; the first feature images output by the first three levels of depthwise separable convolutional modules are input to the corresponding feature enhancement modules for feature enhancement. Each feature enhancement module outputs a corresponding feature enhancement image. The feature enhancement image output by the last-level feature enhancement module is input to the third-level upsampling module for upsampling processing to obtain the corresponding upsampled image. The third-level upsampled image and the feature enhancement image output by the third-level feature enhancement module are then passed through the third-level first fusion module for feature fusion, outputting the second feature fusion image corresponding to the third-level first feature fusion module. The second feature fusion image corresponding to the third level is then input to the corresponding second-level upsampling module for upsampling processing to obtain the second-level upsampled image. The second-level upsampled image is then passed through the second-level first feature fusion module again with the feature enhancement image output by the second-level feature enhancement module for feature fusion, obtaining the second feature fusion image corresponding to the second level. The second-level feature fusion image is then input to the corresponding first-level upsampling module for upsampling processing to obtain the second-level upsampled image. The second-level upsampled image and the feature enhancement image output by the first-level feature enhancement module are then fused to obtain the first-level second feature fusion image. The fourth-level second feature fusion image is the feature enhancement image corresponding to the fourth-level feature enhancement module. Furthermore, the first-level second feature fusion image is passed through the first-level third convolution module for convolution processing to obtain the first-level third feature image. The obtained first-level third feature image and the second-level second feature fusion image are then passed through the first-level second feature fusion module for feature fusion again to obtain the second-level first feature fusion image. The second-level first feature fusion image is then passed through the second-level third convolution module for convolution processing to obtain the second-level third feature image. The second-level third feature image and the third-level second feature fusion image are then passed through the third-level second fusion feature module for fusion processing to obtain the third-level first feature fusion image. The third-level first feature fusion image is then passed through the fourth-level third convolution module for convolution processing to obtain the fourth-level third feature image. The fourth-level third feature image and the fourth-level second feature fusion image are then passed through the fourth-level second feature fusion module for feature fusion to obtain the fourth-level first feature fusion image. The first-level first feature fusion image is essentially the same as the first-level second feature fusion image. In addition to being transmitted to the corresponding third convolution module, the first feature fusion image for each level is also output. Subsequently, based on the obtained first fusion feature images of different sizes, security hazard target detection is performed to obtain the final detection result.For example, different first-feature fused images correspond to different sizes, but have the same number of channels. For instance, the size of the first-level first-feature fused image is 152×152, the size of the second-level first-feature fused image is 76×76, the size of the third-level first-feature fused image is 38×38, and the size of the fourth-level first-feature image is 19×19, all with 150 channels. Subsequently, based on the first-feature fused images of different sizes, security risks of different sizes can be detected. This example can accurately detect small-sized target risks.
[0092] Optionally, the device further includes: a building module 24, which is used for:
[0093] Obtain the training set for the detection model; the training set includes multiple training images;
[0094] Based on the training set, the prior bounding box parameters of the detection model are optimized using a clustering algorithm to obtain the optimized prior bounding box parameters.
[0095] A detection model is constructed based on the optimized prior box parameters.
[0096] Specifically, images with security risks are acquired and used as the training set for the detection model. The training set includes multiple training images, each containing small-sized security risks. Based on the acquired training set, a clustering algorithm is used to optimize the prior bounding box (anchor) parameters of the detection model. For example, a K-means (K-means++) algorithm can be used, where K represents the number of cluster centers. The width and height of the target boxes in the training set are clustered to obtain more representative prior bounding boxes. The clustering algorithm first randomly selects k cluster centers, then calculates the distances between other targets and the centers, and assigns them to the nearest centers to obtain k clusters. Through iterative adjustments, the distances between targets within each cluster decrease, while the distances between clusters increase. The cluster centers are then recalculated, and this process is repeated multiple times until the cluster centers no longer change, resulting in the optimized anchor parameters. Based on the optimized prior bounding box parameters, the detection model can be constructed. This example optimizes the prior bounding box parameters, improving the accuracy of the detection model.
[0097] Optionally, the device also includes a training module 25, which is used for:
[0098] The training set is input into the detection model to be trained, and the detection model is trained to obtain the trained detection model.
[0099] Specifically, after constructing the detection model, the training set is input into the detection model to be trained, and the model is trained to obtain the trained detection model.
[0100] Optionally, the device further includes a verification module 26, which is used for:
[0101] Obtain the validation set of the detection model; the validation set includes multiple validation images; input the validation set into the trained detection model to optimize the model parameters;
[0102] Obtain the test set of the detection model; input the test set into the trained detection model. If the error between the output of the trained detection model and the actual observation result is not greater than the preset value, then the current detection model is used as the trained detection model.
[0103] Specifically, after inputting the training dataset into the detection model to be trained, a validation set is obtained. This validation set includes multiple validation images, which are images with small security vulnerabilities. The validation set is then input into the trained model to optimize its parameters, resulting in a trained detection model. Further, to verify the performance and generalization ability of the optimized model, a test set is obtained, consisting of multiple test images. This test set is input into the trained model. If the error between the output of the trained model and the actual observation results is no greater than a preset value, the current detection model is considered to have been successfully trained and is considered a successfully trained model. In this example, obtaining a validation set for parameter optimization and a test set for model performance testing improves the accuracy of the detection model training.
[0104] The safety hazard detection device provided in this embodiment acquires an image to be detected and inputs it into a trained detection model for processing to obtain a detection result. The detection model includes an input layer, a feature enhancement layer, and an upsampling layer. The input layer performs convolution processing on the image to be detected, obtaining multiple first feature images of different sizes. Each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size. The feature enhancement layer enhances the features of the multiple first feature images of different sizes, obtaining multiple feature-enhanced images of different sizes. The upsampling layer upsamples the multiple feature-enhanced images of different sizes, obtaining multiple upsampled images of different sizes. Based on the multiple upsampled images and the multiple feature-enhanced images of different sizes, feature fusion is performed to obtain multiple first feature fused images. Safety hazard target detection is then performed based on the multiple first feature fused images to obtain a detection result. Upsampling the obtained feature-enhanced images of multiple sizes preserves more detailed information in the image, improving the accuracy of the subsequent detection model's detection results. Subsequently, multiple upsampled images of different sizes and corresponding feature-enhanced images of different sizes are fused to obtain multiple first feature fused images of different sizes. Then, security hazard target detection is performed based on these multiple first feature fused images, yielding detection results and enabling the detection of security hazards of various sizes. The multiple first feature fused images further incorporate more image detail information, further improving the accuracy of the detection model in detecting security hazard targets.
[0105] Example 3
[0106] Figure 5 The diagram above illustrates the structure of an electronic device, which includes:
[0107] The device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, memory 292, and communication interface 293 can communicate with each other via the bus 294. The communication interface 293 can be used for information transmission. The processor 291 can invoke logical instructions stored in the memory 292 to execute the methods described in the example above.
[0108] Furthermore, the logic instructions in the aforementioned memory 292 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.
[0109] The memory 292, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 292, that is, it implements the methods in the above method examples.
[0110] The memory 292 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 292 may include high-speed random access memory and may also include non-volatile memory.
[0111] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method in any of the embodiments.
[0112] This application also provides a computer program product, including a computer program that, when executed by a processor, is used to implement the method in any of the embodiments.
[0113] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0114] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0115] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.
[0116] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.
[0117] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.
[0118] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0119] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.
[0120] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0121] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A method for detecting safety hazards, characterized in that, include: Acquire the image to be detected; The image to be detected is input into a trained detection model for processing. The detection model includes an input layer, a feature enhancement layer, and an upsampling layer. The input layer performs convolution processing on the image to be detected to obtain multiple first feature images of different sizes. Each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size. The feature enhancement layer enhances the features of the multiple first feature images of different sizes to obtain multiple feature-enhanced images of different sizes. The upsampling layer upsamples the multiple feature-enhanced images of different sizes to obtain multiple upsampled images of different sizes. Based on the multiple upsampled images of different sizes and the multiple feature-enhanced images of different sizes, feature fusion is performed to obtain multiple first feature fusion images; and based on the multiple first feature fusion images, security hazard target detection is performed to obtain detection results.
2. The method according to claim 1, characterized in that, The input layer includes a first convolutional module and a multi-level depthwise separable convolutional module; The first convolution module is used to perform convolution processing on the image to be detected and output an initial feature image; Each level of depthwise separable convolutional module is used to perform convolution processing on the first feature image output by the previous level of depthwise separable convolutional module, and output a first feature image of the corresponding size. The primary depth-separable convolution module is used to perform convolution processing on the initial feature image.
3. The method according to claim 2, characterized in that, The feature enhancement layer includes a multi-level feature enhancement module and a second convolution module; The multi-level feature enhancement module is used to enhance the features of multiple first feature images of different sizes to obtain feature-enhanced images of corresponding sizes; the multi-level feature enhancement module corresponds one-to-one with the depthwise separable convolution module; The second convolution module is used to perform convolution processing on the first feature image output by the last-stage depthwise separable convolution module to output the second feature image; the feature enhancement module corresponding to the last-stage depthwise separable convolution module is used to perform feature enhancement on the second feature image to obtain the corresponding feature-enhanced image.
4. The method according to claim 3, characterized in that, The feature enhancement module includes multiple convolutional units and fusion units; when the multi-level feature enhancement module performs feature enhancement on multiple first feature images of different sizes to obtain a feature-enhanced image of the corresponding size, it is specifically used for: The first feature image of the corresponding size is input into the multiple convolutional units for convolution processing to obtain the feature images output by the multiple convolutional units; The feature images output by multiple convolutional units are input into the fusion unit to perform feature fusion and obtain a feature-enhanced image of the corresponding size.
5. The method according to claim 4, characterized in that, The upsampling layer includes a multi-level upsampling module; the detection model also includes a multi-level first fusion module, a multi-level third convolution module, and a multi-level second fusion module. The multi-level first fusion module is used to perform feature fusion on multiple feature-enhanced images of different sizes and corresponding upsampled images of the same size, and output multiple second feature-fused images of different sizes; wherein, the feature-enhanced image of the smallest size is used as the second feature-fused image of that size; The multi-level upsampling module is used to perform upsampling processing on multiple second feature fusion images of different sizes and output multiple upsampled images of different sizes. The multi-level third convolution module is used to perform convolution processing on other second feature fusion images besides the second feature fusion image in multiple second feature fusion images of different sizes, and output multiple third feature images of different sizes. The multi-level second fusion module is used to perform feature fusion on multiple third feature images of different sizes and corresponding second feature fusion images of the same size to obtain multiple first feature fusion images.
6. The method according to any one of claims 1-5, characterized in that, The method further includes: Obtain the training set of the detection model; the training set includes multiple training images; Based on the training set, the prior bounding box parameters of the detection model are optimized using a clustering algorithm to obtain the optimized prior bounding box parameters. The detection model is constructed based on the optimized prior box parameters.
7. The method according to claim 6, characterized in that, The method further includes: The training set is input into the detection model to be trained, and the detection model is trained to obtain the trained detection model.
8. The method according to claim 7, characterized in that, The method further includes: Obtain a validation set for the detection model; the validation set includes multiple validation images; input the validation set into the trained detection model to optimize the model parameters; Obtain a test set of the detected images; input the test set into the trained detection model; if the error between the output of the trained detection model and the actual observation result is not greater than a preset value, then the current detection model is used as the trained detection model.
9. A safety hazard detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected; A processing module is used to input the image to be detected into a trained detection model for processing; wherein, the detection model includes an input layer, a feature enhancement layer, and an upsampling layer; the input layer is used to perform convolution processing on the image to be detected to obtain multiple first feature images of different sizes; each of the multiple first feature images of different sizes is obtained by convolving the first feature image of the previous size; the feature enhancement layer is used to perform feature enhancement on the multiple first feature images of different sizes to obtain multiple feature-enhanced images of different sizes; the upsampling layer is used to upsample the multiple feature-enhanced images of different sizes to obtain multiple upsampled images of different sizes; The detection module is used to perform feature fusion based on the multiple upsampled images of different sizes and the multiple feature-enhanced images of different sizes to obtain multiple first feature fusion images; and to perform safety hazard target detection based on the multiple first feature fusion images to obtain detection results.
10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.