A small sample target detection network device guided by acoustic cloud information

Abstract

A few-sample target detection network device guided by acoustic cloud information is disclosed. The target detection network uses Faster R-CNN as the base detection model. The feature learning components include an acoustic cloud information guidance module, a feature fusion module, a region candidate network, a region of interest pooling module, and a region of interest feature extractor. The acoustic cloud information guidance module includes an acoustic feature guidance branch and a global feature guidance branch. The global feature guidance branch generates weights by extracting texture information from the feature map, guiding global feature selection. The weights obtained from the two branches are multiplied by the feature image output by the backbone network, and the channels are compressed after stacking. This effectively guides the network to focus on the texture information of the acoustic cloud coverage area and important boundaries, enhancing the network's feature learning ability. This invention solves the problem of insufficient acoustic image data and difficulty in network recognition and utilization of acoustic information, thereby improving the efficiency of substation operation and maintenance.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a small-sample target detection network device guided by acoustic cloud information. Background Technology

[0002] With the continuous development of deep learning, devices are becoming increasingly intelligent, and unstructured data such as images, videos, and audio are crucial information for recognition and detection. Currently, most mainstream device anomaly detection methods are traditional image-based target detection methods, which can only determine whether a device has external defects. They struggle with acoustic signal feature extraction and audio-guided visual localization, exhibiting blind spots and failing to accurately analyze the actual operating status of the device. Therefore, it is necessary to integrate multi-source data, including visual and auditory data, to collaboratively recognize device attributes. In the context of rapid intelligent development, utilizing acoustic information to improve the accuracy of target recognition and to address the issue of identifying the operational status of devices is becoming increasingly important. Figure 1 As shown, in 2014, Girshick et al. proposed the R-CNN algorithm, which achieved a significant performance improvement in the field of object detection, marking the beginning of the deep learning era for object detection. In the existing technical document "REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C] / / Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal: NIPS, 2015: 91-99.", Ren et al. further proposed a two-stage algorithm, Faster R-CNN, based on a large number of candidate regions in 2015.

[0003] In existing technical documents such as "REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection [C] / / Proceedings of 2016 IEEE Conference on ComputerVision and Pattern Recognition. Las Vegas: IEEE, 2016: 779-788.", classic one-stage networks include the YOLO series, SSD, RetinaNet, etc., which predefine anchor boxes of different sizes and proportions on the image to replace two-stage algorithms.

[0004] In recent years, the transformer, which has shone brightly in the field of natural language processing, has also been successfully applied to object detection. Researchers at Facebook AI proposed DETR, a fusion version of CNN and transformer, for object detection and panoptic segmentation, achieving end-to-end prediction. This is described in detail in the existing technical document "CHEN H, WANG Y, WANG G, et al. LSTD: A Low-ShotTransfer Detector for Object Detection [EB / OL]. 2021-08-10. https: / / doi.org / 10.48550 / arXiv.1803.01529".

[0005] The aforementioned mainstream object detection methods primarily rely on training with a large number of samples. When there are only a few labeled samples, these methods struggle to achieve satisfactory results. However, in real-world scenarios, it is difficult to obtain a sufficient number of high-quality acoustic image registration samples. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a small sample target detection network device guided by acoustic cloud information. By introducing an acoustic cloud information guidance mechanism into the Faster R-CNN target detection network, the improved model's feature extraction capability is enhanced, and the target detection accuracy is improved.

[0007] The present invention adopts the following technical solution.

[0008] A small sample target detection network device guided by acoustic cloud information includes:

[0009] The two-stage object detection framework Faster R-CNN serves as the base detection model, and the feature learning component of the base detection model is used to execute the training process.

[0010] The training process consists of two phases:

[0011] Phase 1: Obtain a pre-trained model using labeled data from the substation image database;

[0012] The second stage involves fine-tuning the pre-trained model using small sample data from the sound field cloud map database.

[0013] Preferably, the feature learning component of the basic detection model includes:

[0014] The system includes an acoustic cloud information guidance module, an FPN module as a feature fusion module, a region candidate network, a region of interest pooling module, and a ROI feature extractor.

[0015] Preferably, the acoustic cloud information guidance module is used for feature selection;

[0016] The feature fusion module is used to fuse information from multiple scales;

[0017] Region candidate networks are used to generate candidate boxes and refine them to obtain more accurate coordinates.

[0018] The Region of Interest (ROI) pooling module is used to collect candidate boxes generated by the region candidate network.

[0019] The ROI feature extractor is used to extract candidate boxes from the feature map.

[0020] Preferably, the acoustic cloud information guidance module is used to generate acoustic feature guidance weights and global feature guidance weights.

[0021] A method for training a few-sample target detection network guided by acoustic cloud information includes:

[0022] Execute the training process;

[0023] The training process consists of two phases:

[0024] Phase 1: Obtain a pre-trained model using labeled data from the substation image database;

[0025] The second stage involves fine-tuning the pre-trained model using small sample data from the sound field cloud map database.

[0026] Preferably, the first stage specifically includes:

[0027] Obtain the first-stage input image from the substation image database;

[0028] The first-stage input image is fed into the backbone network and the feature image is output.

[0029] The feature image is input into the FPN module for feature fusion.

[0030] The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database as a one-stage model.

[0031] Randomly initialized weights are assigned to the region candidate network, while the ROI feature extractor is fixed.

[0032] Preferably, the backbone network is a ResNet101 network, and the first-stage input image is the labeled data of substation images in the substation image database.

[0033] Preferably, the second stage specifically includes:

[0034] Obtain the second-stage input image from the sound field cloud map database and the substation image database;

[0035] Acquire acoustic cloud information from the acoustic cloud image database to guide the image;

[0036] The second-stage input image is fed into the backbone network and the feature image is output.

[0037] The acoustic cloud information guidance image is input into the acoustic cloud information guidance module to generate acoustic feature guidance weights;

[0038] The acoustic feature-guided weights are multiplied by the feature image to perform acoustic feature selection on the feature image;

[0039] The feature image is input into the acoustic cloud information guidance module to generate global feature guidance weights;

[0040] Global feature-guided weights are multiplied by the feature image to perform global feature selection on the feature image;

[0041] The feature images generated after acoustic feature selection and global feature selection are stacked together;

[0042] The stacked feature images are then subjected to channel compression via a convolution operation.

[0043] The compressed feature image is input into the FPN module for feature fusion.

[0044] The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database, forming a two-stage model.

[0045] Preferably, the second-stage input image is the first-stage model in the substation image database and the sound field data on the sound field cloud map of the substation in the sound field cloud map database; the sound cloud information guiding image is the RBG three-channel sound cloud image on the sound field cloud map of the substation in the sound field cloud map database.

[0046] Preferably, the method for inputting the acoustic cloud information guidance image into the acoustic cloud information guidance module to generate acoustic feature guidance weights includes:

[0047] The RGB three-channel acoustic cloud image, which serves as the guiding image for acoustic cloud information, is normalized using the following formula:

[0048]

[0049]

[0050]

[0051] Where R, G, and B are the RGB three-channel pixel values ​​of the acoustic cloud information guided image, and r, g, and b are the normalized pixel values ​​of the RGB three channels.

[0052] Preferably, the method for inputting the acoustic cloud information guidance image into the acoustic cloud information guidance module to generate acoustic feature guidance weights further includes:

[0053] After acoustic image normalization, the normalized pixel values ​​are converted into normalized acoustic weights, and the calculation formula is as follows:

[0054]

[0055]

[0056] N = 0.5 + 0.5k

[0057] Where r, g, and b are the normalized pixel values ​​of the RGB three channels, k is the normalized sound field intensity, and N is the normalized acoustic weight.

[0058] Preferably, the method for inputting the acoustic cloud information guidance image into the acoustic cloud information guidance module to generate acoustic feature guidance weights further includes:

[0059] Normalized acoustic weights are fed into a max pooling layer to generate acoustic feature-guided weights.

[0060] Preferably, the method for generating global feature-guided weights includes:

[0061] The feature images are sequentially compressed using average pooling and max pooling layers. After feature compression by these two pooling layers, the compressed feature images are stacked. Then, the stacked feature images are fed into a 7x7 convolutional layer to generate a single-channel feature map. This single-channel feature map is then fed into the Sigmoid function, calculated using the following formula:

[0062]

[0063] Where AvgP and MaxP represent average pooling and max pooling, respectively, F represents the input feature map; M(F) represents the output global feature guidance weights; f 7×7 This represents a convolution operation with a kernel size of 7×7; σ represents the Sigmoid function.

[0064] Preferably, the method for channel compression of a feature image includes:

[0065] The feature images generated after acoustic feature selection and global feature selection are stacked together, and channel compression is performed on the feature images using a convolution operation. The calculation formula is as follows:

[0066] F m =f 1×1 {Concatenate[F1,F2]}

[0067] Where F1 and F2 represent the feature images generated after acoustic feature selection and global feature selection, respectively; F m Represents the output feature image; Concatenate represents a stacking operation; f 1×1 This indicates a convolution operation with a kernel size of 1×1.

[0068] A terminal, comprising a processor and a storage medium;

[0069] The storage medium is used to store instructions;

[0070] The processor is configured to operate according to the instructions to execute the steps of the acoustic cloud information-guided few-sample target detection network training method.

[0071] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the acoustic cloud information-guided few-sample target detection network training method.

[0072] The beneficial effects of this invention are as follows: Compared with the prior art, the acoustic cloud information-guided small-sample target detection network device provided by this invention addresses the main difficulty of the scarcity of existing acoustic image samples. To alleviate the problems of scarce acoustic images and low recognition accuracy under implementation conditions, acoustic cloud information is introduced into the small-sample target detection network. The acoustic cloud information guidance module effectively extracts the position and weight information contained in the acoustic image. Acoustic feature guidance weights are generated through acoustic cloud image normalization processing to guide the network to focus on the acoustic cloud coverage area. Global feature guidance weights are generated by extracting feature map texture information to guide the network to focus on areas in the image itself that are conducive to recognition. By multiplying the acoustic feature guidance weights and global feature guidance weights with the feature image output by the backbone network, acoustic and global feature selection is achieved, which can effectively guide the network to focus on the acoustic cloud coverage area and important boundary texture information, thereby enhancing the network's feature learning ability. The target detection network training method provided by this invention introduces acoustic cloud information into the small-sample target detection network, enhances the network's representation ability, and effectively improves the detection accuracy of acoustic images. Attached Figure Description

[0073] Figure 1 This is a schematic diagram of a small-sample target detection network in existing technologies.

[0074] Figure 2This is a network schematic diagram of a small sample target detection network training method guided by acoustic cloud information according to the present invention.

[0075] Figure 3 This is a structural diagram of the acoustic cloud information guidance module in the acoustic cloud information guidance few-sample target detection network training method of the present invention;

[0076] Figure 4 This invention provides a dataset and its preprocessing results for a few-sample target detection network training method guided by acoustic cloud information.

[0077] Figure 5 This is the target detection result of a small sample target detection network training method guided by acoustic cloud information according to the present invention. Detailed Implementation

[0078] The present application will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be construed as limiting the scope of protection of the present application.

[0079] The present invention discloses a small sample target detection network device guided by acoustic cloud information, comprising:

[0080] The object detection network uses the two-stage object detection framework Faster R-CNN as the base detection model, and the feature learning component of the base detection model is used to execute the training process.

[0081] The training process consists of two phases:

[0082] Phase 1: A pre-trained model is obtained using a large amount of labeled data from the substation image database;

[0083] The second stage involves fine-tuning the pre-trained model using small sample data from the sound field cloud map database.

[0084] In a preferred but non-limiting embodiment of the present invention, the feature learning component of the basic detection model includes:

[0085] The acoustic cloud information guidance module serves as the FPN module of the Feature Pyramid Networks (FPN), the Region Proposal Network (RPN), the Region of Interest Pooling module, and the Region of Interest Feature Extractor.

[0086] In a preferred but non-limiting embodiment of the present invention, the acoustic cloud information guidance module is used for feature selection;

[0087] The feature fusion module is used to fuse information from multiple scales;

[0088] The region candidate network is used to generate candidate boxes, which has a binary classification function to determine whether there is a target in all preset boxes; and corrects the candidate boxes to obtain more accurate coordinates;

[0089] The Region of Interest (ROI) pooling module is used to collect candidate boxes generated by the region candidate network.

[0090] The ROI feature extractor is used to extract candidate boxes from the feature map.

[0091] In a preferred but non-limiting embodiment of the present invention, the acoustic cloud information guidance module is used to generate acoustic feature guidance weights and global feature guidance weights.

[0092] The present invention discloses a method for training a few-sample target detection network guided by acoustic cloud information, comprising:

[0093] Execute the training process;

[0094] The training process consists of two phases:

[0095] Phase 1: A pre-trained model is obtained using a large amount of labeled data from the substation image database;

[0096] The second stage involves fine-tuning the pre-trained model using small sample data from the sound field cloud map database.

[0097] In a preferred but non-limiting embodiment of the present invention, the first stage specifically includes:

[0098] The first-stage input image is obtained from the substation image database; the annotation data for the substation images is generated as a VOC format XML file using Labelimg annotation, and the YOLO annotation format is as follows:

[0099] <object-class> <x> <y> <width> <height>Where object-class represents the object category; x and y represent the ratio of the center point coordinates (x, y) of the bounding box of the substation image to the width and height of the substation image, ranging from 0 to 1; width and height represent the ratio of the width and height of the bounding box of the substation image to the width and height of the substation image, ranging from 0 to 1.

[0100] The first-stage input image is fed into the backbone network and the feature image is output.

[0101] The feature image is input into the FPN module for feature fusion.

[0102] The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database as a one-stage model.

[0103] Randomly initialized weights are assigned to the region candidate network, while the ROI feature extractor is fixed.

[0104] In a preferred but non-limiting embodiment of the present invention, the backbone network is a ResNet101 network, and the first-stage input image is the labeled data of substation images in the substation image database.

[0105] In a preferred but non-limiting embodiment of the present invention, the second stage specifically includes:

[0106] Obtain the second-stage input image from the sound field cloud map database and the substation image database;

[0107] Acquire acoustic cloud information from the acoustic cloud image database to guide the image;

[0108] The second-stage input image is fed into the backbone network and the feature image is output.

[0109] The acoustic cloud information guidance image is input into the acoustic cloud information guidance module to generate acoustic feature guidance weights;

[0110] The acoustic feature-guided weights are multiplied by the feature image to perform acoustic feature selection on the feature image;

[0111] The feature image is input into the acoustic cloud information guidance module to generate global feature guidance weights;

[0112] Global feature-guided weights are multiplied by the feature image to perform global feature selection on the feature image;

[0113] The feature images generated after acoustic feature selection and global feature selection are stacked together;

[0114] The stacked feature images are then subjected to channel compression via a convolution operation.

[0115] The compressed feature image is input into the FPN module for feature fusion.

[0116] The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database, forming a two-stage model.

[0117] In a preferred but non-limiting embodiment of the present invention, the second-stage input image consists of a first-stage model from the substation image database and sound field data from the sound field cloud map of the substation in the sound field cloud map database; the sound cloud information guiding image is an RBG three-channel sound cloud image from the sound field cloud map of the substation in the sound field cloud map database. The sound field data includes data such as the sound pressure level, sound pressure, and frequency characteristics of the substation sound field.

[0118] like Figure 2 As shown, specifically, the acoustic cloud information-guided few-sample target detection network training method of this invention selects the widely used two-stage target detection framework Faster R-CNN as the basic detection model. The backbone network is ResNet101 to increase network depth. An acoustic cloud information-guided module is introduced for feature selection. The feature learning component also includes an FPN module to fuse multi-scale information. A Region Candidate Network (RPN) is used to generate candidate boxes, which has a binary classification function to determine whether any preset boxes contain targets; and it corrects the candidate boxes to obtain more accurate coordinates. Region of Interest Pooling (ROIPooling) is used to collect the candidate boxes generated by the RPN and extract them from the feature map to generate regions of interest, which are then fed into subsequent fully connected layers for further classification and regression. Therefore, the features learned by the model from the substation image database can be transferred to the target category in the acoustic cloud image database without further parameter updates. The method of this invention divides the network training process into two stages: the first stage uses a large amount of labeled data from the substation image database to obtain a pre-trained model; the second stage adds few-sample data from the acoustic cloud image database to fine-tune the model. In the first stage, only the substation image database is fed into the backbone network for feature extraction to obtain the corresponding feature map. After feature fusion, the feature map is input into the RPN to generate multiple regions of interest. After ROI pooling and ROI feature extraction network, classification and regression are performed, and randomly initialized weights are assigned to the region candidate network to fix the ROI feature extractor. In the second stage, small sample data from the sound field cloud map database are added to fine-tune the model.

[0119] like Figure 3 As shown, in a preferred but non-limiting embodiment of the present invention, the method for inputting an acoustic cloud information guidance image into an acoustic cloud information guidance module to generate acoustic feature guidance weights includes:

[0120] The RGB three-channel acoustic cloud image, which serves as the guiding image for acoustic cloud information, is normalized using the following formula:

[0121]

[0122]

[0123]

[0124] Where R, G, and B are the RGB three-channel pixel values ​​of the acoustic cloud information guided image, and r, g, and b are the normalized pixel values ​​of the RGB three channels.

[0125] In a preferred but non-limiting embodiment of the present invention, the method for inputting an acoustic cloud information guidance image into an acoustic cloud information guidance module to generate acoustic feature guidance weights further includes:

[0126] After acoustic image normalization, the normalized pixel values ​​are converted into normalized acoustic weights, and the calculation formula is as follows:

[0127]

[0128]

[0129] N = 0.5 + 0.5k

[0130] Where r, g, and b are the normalized pixel values ​​of the RGB three channels, k is the normalized sound field intensity, and N is the normalized acoustic weight.

[0131] In a preferred but non-limiting embodiment of the present invention, the method for inputting an acoustic cloud information guidance image into an acoustic cloud information guidance module to generate acoustic feature guidance weights further includes:

[0132] Normalized acoustic weights are fed into a max pooling layer to generate acoustic feature-guided weights.

[0133] In a preferred but non-limiting embodiment of the present invention, the method for generating global feature-guided weights includes:

[0134] The feature images are sequentially compressed using average pooling and max pooling layers. After feature compression by these two pooling layers, the compressed feature images are stacked. Then, the stacked feature images are fed into a 7x7 convolutional layer to generate a single-channel feature map. This single-channel feature map is then fed into the Sigmoid function, calculated using the following formula:

[0135] M(F)=σ{f 7×7 [AvgP(F); MaxP(F)]}

[0136] Where AvgP and MaxP represent average pooling and max pooling, respectively, F represents the input feature map; M(F) represents the output global feature guidance weights; f 7×7 This represents a convolution operation with a kernel size of 7×7; σ represents the Sigmoid function.

[0137] In a preferred but non-limiting embodiment of the present invention, a method for channel compression of a feature image includes:

[0138] The feature images generated after acoustic feature selection and global feature selection are stacked together, and channel compression is performed on the feature images using a convolution operation. The calculation formula is as follows:

[0139] F m =f 1×1 {Concatenate[F1,F2]}

[0140] Where F1 and F2 represent the feature images generated after acoustic feature selection and global feature selection, respectively; F m Represents the output feature image; Concatenate represents a stacking operation; f 1×1 This indicates a convolution operation with a kernel size of 1×1.

[0141] The terminal described in this invention includes a processor and a storage medium;

[0142] The storage medium is used to store instructions;

[0143] The processor is configured to operate according to the instructions to execute the steps of the acoustic cloud information-guided few-sample target detection network training method.

[0144] The present invention discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the acoustic cloud information-guided few-sample target detection network training method.

[0145] Experimental Analysis:

[0146] 1.1 Experimental Environment Configuration: The experimental environment for this invention is as follows: Windows 10 system, Intel(R) Core(TM) i5-8250U@1.80GHz CPU, 16GB RAM. The server is a Dell Precision T3430, running Linux (specifically Ubuntu 20), with two Nvidia 1080Ti graphics cards, Python 3.8, and the PyTorch framework. Stochastic Gradient Descent (SGD) algorithm is used to update the gradient in the experiment, with an initial learning rate set to 2.5 × 10⁻⁶. -4 The learning rate decays to 1e-4 after each training cycle. The batch size is set to 4 samples.

[0147] 1.2 Dataset

[0148] The experimental data used in this invention includes 8347 images under different lighting and weather conditions. The base class substation multi-target detection dataset mainly includes 17 types of targets such as meters, insulators, and breathers. Using the roLabelImg tool, 8307 images were annotated for network model training and testing. The small sample dataset includes three types of targets: radiators, main transformer bushings, and main transformers, containing 20 annotated images for network model training and testing, and 20 registration images for feature guidance.

[0149] 1.3 Structural Composition: This invention uses Faster R-CNN as the basic detection model, such as... Figure 4 As shown, the backbone network uses ResNet101 to increase network depth. The target detection network feature learning components include an acoustic cloud information guidance module, an FPN module, a region candidate network, a region of interest pooling module, and a ROI feature extractor. The target network training process of this invention is divided into two stages. In the first stage, a large amount of labeled data from the substation image database is input into the backbone network and feature images are output. The feature images are then input into the FPN module for feature fusion. The fused feature images are then sequentially input into the region candidate network and the region of interest module for classification and regression, and the first-stage model is stored. Images from the acoustic cloud image database and the substation image database are input into the backbone network and feature images are output. Acoustic cloud information guidance images are input into the acoustic cloud information guidance module to generate acoustic feature guidance weights, which are multiplied with the feature images output by the backbone network for acoustic feature selection. Feature images are input into the acoustic cloud information guidance module to generate global feature guidance weights, which are multiplied with the feature images output by the backbone network for global feature selection. The feature images generated by acoustic feature selection and global feature selection are stacked together. Convolution operation is used to compress the channels of the stacked feature images. The channel-compressed feature images are input into the FPN module for feature fusion. The feature images after feature fusion are then input into the region candidate network and the region of interest module for classification and regression. The two-stage model is then stored.

[0150] In this invention, the acoustic cloud information guidance module effectively extracts the location and weight information contained in the acoustic image. Acoustic feature guidance weights are generated through acoustic cloud image normalization processing, guiding the network to focus on the acoustic cloud coverage area. Global feature guidance weights are generated by extracting feature map texture information, guiding the network to focus on areas within the image that are conducive to recognition. By multiplying the acoustic feature guidance weights and global feature guidance weights respectively with the feature image output by the backbone network, acoustic and global feature selection is achieved. This effectively guides the network to focus on the acoustic cloud coverage area and important boundary texture information, enhancing the network's feature learning ability. The above methods effectively improve the network's target detection accuracy; the recognition results are shown in the attached figure. Figure 5 The performance index comparison results are shown in Table 1.

[0151] Table 1 Target Detection Performance Indicators

[0152]

[0153] In this paper, FSOD is a traditional few-shot target detection algorithm, ACG is the acoustic cloud information-guided method proposed in this paper, AP is the average accuracy, bAP is the average detection accuracy of the classes contained in the substation image database, and nAP is the average detection accuracy of the classes contained in the acoustic cloud image database. As shown in Table 1, the few-shot target detection network training method of this invention significantly improves the detection accuracy compared to the traditional few-shot target detection algorithm. The target detection method of this invention improves the AP index by 0.245 and the AP50 index by 0.772 compared to the original FSOD algorithm. Due to the addition of the ACG module, the network pays more attention to few-shot feature information, resulting in a 0.109 reduction in bAP and a 0.065 improvement in bAP50 compared to the original FSOD algorithm; an improvement of 2.248 in nAP and a 4.773 improvement in nAP50 compared to the original FSOD algorithm. Therefore, the target detection method trained in this paper achieves a significant improvement in detection accuracy on acoustic image datasets.

[0154] This invention presents a small-sample target detection network guided by acoustic cloud information, using Faster R-CNN as the basic detection model. The target detection network's feature learning components include an acoustic cloud information guidance module, an FPN module, a region candidate network, a region of interest pooling module, and a ROI feature extractor. Through a two-stage training process, it is fine-tuned using acoustic cloud maps. This alleviates the problem of low detection accuracy caused by insufficient sample size. The improved network achieves higher detection accuracy on acoustic cloud map databases, and significantly improves the recognition accuracy for targets covered by acoustic clouds.

[0155] Working Principle: The acoustic cloud information-guided few-sample target detection network training method provided in this invention generates acoustic feature-guided weights by normalizing the acoustic cloud image. These weights are then multiplied with the feature image output by the backbone network to achieve feature selection guided by acoustic cloud information. Average pooling and max pooling are used to compress the feature image, and the resulting layers are stacked and connected to a set of convolutional and normalization layers to generate global feature-guided weights for filtering global features, effectively guiding the network to focus on the acoustic cloud coverage area. Simultaneously, multiplying the global feature-guided weights with the feature image output by the backbone network prevents the loss of global texture information. This allows the model to combine the important location and weight information contained in the acoustic image, leveraging its crucial role in feature selection, resulting in better model feature learning and a significant improvement in the accuracy of acoustic image target detection.

[0156] The beneficial effects of this invention are as follows: Compared with the prior art, the acoustic cloud information-guided small-sample target detection network device provided by this invention addresses the main difficulty of the scarcity of existing acoustic image samples. To alleviate the problems of scarce acoustic images and low recognition accuracy under implementation conditions, acoustic cloud information is introduced into the small-sample target detection network. The acoustic cloud information guidance module effectively extracts the position and weight information contained in the acoustic image. Acoustic feature guidance weights are generated through acoustic cloud image normalization processing to guide the network to focus on the acoustic cloud coverage area. Global feature guidance weights are generated by extracting feature map texture information to guide the network to focus on areas in the image itself that are conducive to recognition. By multiplying the acoustic feature guidance weights and global feature guidance weights with the feature image output by the backbone network, acoustic and global feature selection is achieved, which can effectively guide the network to focus on the acoustic cloud coverage area and important boundary texture information, thereby enhancing the network's feature learning ability. The target detection network training method provided by this invention introduces acoustic cloud information into the small-sample target detection network, enhances the network's representation ability, and effectively improves the detection accuracy of acoustic images.

[0157] The few-shot learning method of this invention enables feature extraction with a small number of labeled samples, training a model with good generalization performance and reducing dependence on large-scale labeled data. By improving training strategies, adjusting model structure, and reconstructing the loss function, the model can learn generalized features of the target category using only a small number of labeled samples.

[0158] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0163] In the description of this specification, references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0164] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.< / height> < / width> < / y> < / x> < / object-class>

Claims

1. A small-sample target detection network device guided by acoustic cloud information, characterized in that, include: The two-stage object detection framework Faster R-CNN serves as the base detection model, and the feature learning component of the base detection model is used to execute the training process. The training process consists of two phases: Phase 1: Obtain a pre-trained model using labeled data from the substation image database; Second stage: Fine-tuning the pre-trained model by adding small sample data from the sound field cloud map database; The first phase specifically includes: Obtain the first-stage input image from the substation image database; The first-stage input image is fed into the backbone network and the feature image is output. The feature image is input into the FPN module for feature fusion. The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database as a one-stage model. Assign randomly initialized weights to the region candidate network and fix the ROI feature extractor; The backbone network is a Resnet101 network, and the first-stage input image is the labeled data of substation images in the substation image database. The second phase specifically includes: Obtain the second-stage input image from the sound field cloud map database and the substation image database; Acquire acoustic cloud information from the acoustic cloud image database to guide the image; The second-stage input image is fed into the backbone network and the feature image is output. The acoustic cloud information guidance image is input into the acoustic cloud information guidance module to generate acoustic feature guidance weights; The acoustic feature-guided weights are multiplied by the feature image to perform acoustic feature selection on the feature image; The feature image is input into the acoustic cloud information guidance module to generate global feature guidance weights; Global feature-guided weights are multiplied by the feature image to perform global feature selection on the feature image; The feature images generated after acoustic feature selection and global feature selection are stacked together; The stacked feature images are then subjected to channel compression via a convolution operation. The compressed feature image is input into the FPN module for feature fusion. The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database as a two-stage model. The second-stage input image consists of the first-stage model in the substation image database and the sound field data on the sound field cloud map of the substation in the sound field cloud map database; the sound cloud information guiding image is the RBG three-channel sound cloud image on the sound field cloud map of the substation in the sound field cloud map database. The method for inputting an acoustic cloud information-guided image into an acoustic cloud information-guided module to generate acoustic feature-guided weights includes: The RGB three-channel acoustic cloud image, which serves as the guiding image for acoustic cloud information, is normalized using the following formula: Where R, G, and B are the RGB three-channel pixel values ​​of the acoustic cloud information guided image, and r, g, and b are the normalized pixel values ​​of the RGB three channels. The method for generating an acoustic map with acoustic feature guidance weights by inputting an image guided by acoustic cloud information into an acoustic cloud information guidance module also includes: After normalization, the normalized pixel values ​​are converted into normalized acoustic weights, and the calculation formula is as follows: Where r, g, and b are the normalized pixel values ​​of the RGB three channels, k is the normalized sound field intensity, and N is the normalized acoustic weight. The method of inputting an acoustic cloud information-guided image into an acoustic cloud information-guided module to generate acoustic feature-guided weights also includes: Normalized acoustic weights are fed into a max pooling layer to generate acoustic feature-guided weights. Methods for generating global feature-guided weights include: The feature images are sequentially compressed using average pooling and max pooling layers. After feature compression by these two pooling layers, the compressed feature images are stacked. Then, the stacked feature images are fed into a 7x7 convolutional layer to generate a single-channel feature map. This single-channel feature map is then fed into the Sigmoid function, calculated using the following formula: in and These represent average pooling and max pooling, respectively. The feature map representing the input; This represents the global feature guidance weights of the output; Indicates the kernel size as Convolution operations; This represents the Sigmoid function; Methods for channel compression of feature images include: The feature images generated after acoustic feature selection and global feature selection are stacked together, and channel compression is performed on the feature images using a convolution operation. The calculation formula is as follows: in These represent the feature images generated after acoustic feature selection and global feature selection, respectively. Represents the output feature image; Concatenate represents a stacking operation; Indicates the kernel size as The convolution operation.

2. The acoustic cloud information-guided small sample target detection network device according to claim 1, characterized in that, The feature learning components of the basic detection model include: The system includes an acoustic cloud information guidance module, an FPN module as a feature fusion module, a region candidate network, a region of interest pooling module, and a ROI feature extractor.

3. The acoustic cloud information-guided small sample target detection network device according to claim 2, characterized in that, The acoustic cloud information guidance module is used for feature selection; The feature fusion module is used to fuse information from multiple scales; Region candidate networks are used to generate candidate boxes and refine them to obtain more accurate coordinates. The Region of Interest (ROI) pooling module is used to collect candidate boxes generated by the region candidate network. The ROI feature extractor is used to extract candidate boxes from the feature map.

4. The acoustic cloud information-guided small sample target detection network device according to claim 2, characterized in that, The acoustic cloud information guidance module is used to generate acoustic feature guidance weights and global feature guidance weights.

5. A method for training a few-sample target detection network guided by acoustic cloud information, characterized in that, include: Execute the training process; The training process consists of two phases: Phase 1: Obtain a pre-trained model using labeled data from the substation image database; Second stage: Fine-tuning the pre-trained model by adding small sample data from the sound field cloud map database; The first phase specifically includes: Obtain the first-stage input image from the substation image database; The first-stage input image is fed into the backbone network and the feature image is output. The feature image is input into the FPN module for feature fusion. The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database as a one-stage model. Assign randomly initialized weights to the region candidate network and fix the ROI feature extractor; The backbone network is a Resnet101 network, and the first-stage input image is the labeled data of substation images in the substation image database. The second phase specifically includes: Obtain the second-stage input image from the sound field cloud map database and the substation image database; Acquire acoustic cloud information from the acoustic cloud image database to guide the image; The second-stage input image is fed into the backbone network and the feature image is output. The acoustic cloud information guidance image is input into the acoustic cloud information guidance module to generate acoustic feature guidance weights; The acoustic feature-guided weights are multiplied by the feature image to perform acoustic feature selection on the feature image; The feature image is input into the acoustic cloud information guidance module to generate global feature guidance weights; Global feature-guided weights are multiplied by the feature image to perform global feature selection on the feature image; The feature images generated after acoustic feature selection and global feature selection are stacked together; The stacked feature images are then subjected to channel compression via a convolution operation. The compressed feature image is input into the FPN module for feature fusion. The feature images after feature fusion are sequentially input into the region candidate network, the region of interest pooling module, and the ROI feature extractor for classification and regression. The classification and regression results are stored in the substation image database as a two-stage model. The second-stage input image consists of the first-stage model in the substation image database and the sound field data on the sound field cloud map of the substation in the sound field cloud map database; the sound cloud information guiding image is the RBG three-channel sound cloud image on the sound field cloud map of the substation in the sound field cloud map database. The method for inputting an acoustic cloud information-guided image into an acoustic cloud information-guided module to generate acoustic feature-guided weights includes: The RGB three-channel acoustic cloud image, which serves as the guiding image for acoustic cloud information, is normalized using the following formula: Where R, G, and B are the RGB three-channel pixel values ​​of the acoustic cloud information guided image, and r, g, and b are the normalized pixel values ​​of the RGB three channels. The method for generating an acoustic map with acoustic feature guidance weights by inputting an image guided by acoustic cloud information into an acoustic cloud information guidance module also includes: After normalization, the normalized pixel values ​​are converted into normalized acoustic weights, and the calculation formula is as follows: Where r, g, and b are the normalized pixel values ​​of the RGB three channels, k is the normalized sound field intensity, and N is the normalized acoustic weight. The method of inputting an acoustic cloud information-guided image into an acoustic cloud information-guided module to generate acoustic feature-guided weights also includes: Normalized acoustic weights are fed into a max pooling layer to generate acoustic feature-guided weights. Methods for generating global feature-guided weights include: The feature images are sequentially compressed using average pooling and max pooling layers. After feature compression by these two pooling layers, the compressed feature images are stacked. Then, the stacked feature images are fed into a 7x7 convolutional layer to generate a single-channel feature map. This single-channel feature map is then fed into the Sigmoid function, calculated using the following formula: in and These represent average pooling and max pooling, respectively. The feature map representing the input; This represents the global feature guidance weights of the output; Indicates the kernel size as Convolution operations; This represents the Sigmoid function; Methods for channel compression of feature images include: The feature images generated after acoustic feature selection and global feature selection are stacked together, and channel compression is performed on the feature images using a convolution operation. The calculation formula is as follows: in These represent the feature images generated after acoustic feature selection and global feature selection, respectively. Represents the output feature image; Concatenate represents a stacking operation; Indicates the kernel size as The convolution operation.

6. A terminal, comprising a processor and a storage medium; The storage medium is used to store instructions; Its features are, The processor is configured to operate according to the instructions to execute the steps of the acoustic cloud information-guided few-sample target detection network training method according to claim 5.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the acoustic cloud information-guided few-sample target detection network training method according to claim 5.

Images