A method for building a marine biological target detection algorithm model and a recognition method
Through the EfficientDet network model and the bidirectional feature pyramid network BiFPN, the problem of low recognition in marine biological detection is solved, and the recognition accuracy and adaptability are improved, especially the ability to extract multi-scale features in complex underwater environments.
Patent Information
- Application Number
- CN202210779589.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-04
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-07-04
AI Technical Summary
The existing underwater object detection algorithm has problems such as low recognition and incomplete recognition in marine biological detection. Especially in complex underwater environments, the target size, shape and posture are diverse, and the image is blurred and occluded. This results in inconspicuous features and is difficult to effectively identify.
The EfficientDet network model is used to build a deep learning object detection algorithm model, including the backbone feature extraction network, the enhanced feature extraction network and the classification prediction network. Feature fusion is carried out through the bidirectional feature pyramid network BiFPN, using average pooling, standard deviation pooling and maximum pooling. and other technologies extract multi-scale features to improve the ability to identify marine biological targets.
The identification accuracy rate of marine biological target detection is significantly improved, especially when dealing with stacking, occlusion and incomplete information, and the ability to extract details and background features is enhanced, making the network more adaptable to the underwater environment.
Smart Images

Figure CN114937201B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of underwater target detection, and in particular relates to a method for building a marine biological target detection algorithm model and a method for identifying the same. Background Art
[0002] Underwater target detection is one of the important areas of marine technology research. In the past, most underwater target detection relied on divers, and long-term underwater operations and complex underwater environments had serious effects on their bodies. Therefore, underwater target detection has important research value and application prospects, and can provide technical support for the monitoring, protection and sustainable development of marine fish and other biological resources.
[0003] The task of object detection is to find all objects of interest in an image and determine their locations and categories. In addition to image classification, the core problems that object detection needs to solve are:
[0004] 1. The target may appear anywhere in the image.
[0005] 2. Targets come in various sizes.
[0006] 3. Targets come in a variety of shapes.
[0007] Existing target detection algorithms are divided into traditional target detection methods and target detection methods based on deep learning. Traditional target detection algorithms have the disadvantages of difficulty in extracting features and weak generalization ability, and their detection accuracy is difficult to achieve ideal results. With the continuous development of artificial intelligence, target detection algorithms based on deep learning have gradually become a new research focus. Target detection algorithms based on deep learning can be roughly divided into two categories: one-stage algorithms based on regression, such as YOLO and SSD; two-stage algorithms based on classification, such as Faster R-CNN. Convolutional neural networks are the basis of deep learning. They use backpropagation algorithms for feedback learning and improve learning ability through automated feature extraction. Compared with traditional detection methods based on manual feature extraction, target detection methods based on deep learning have better learning effects. As a branch of computer vision, target detection algorithms based on deep learning have been widely used in transportation, agriculture, remote sensing and other fields.
[0008] Due to the complex underwater environment, various objects have different sizes, shapes, and postures, and there are many occluded targets and targets with incomplete information. In addition, the underwater images are blurred and the contrast is low, resulting in unclear features. Commonly used target detection algorithms used in marine biological target detection generally have problems such as low recognition and incomplete recognition. Summary of the invention
[0009] In view of the above problems, the first aspect of the present invention provides a method for building a marine biological target detection algorithm model, comprising the following steps:
[0010] Step 1: obtain a dataset of several marine biological images, label the target species using labeling software, and divide the dataset into a training set, a validation set, and a test set;
[0011] Step 2: Build a deep learning target detection algorithm model based on the EfficientDet network model. The algorithm model includes three parts: a backbone feature extraction network, an enhanced feature extraction network, and a classification prediction network. The backbone feature extraction network is an EfficientNet network, which is used to perform preliminary extraction of image features. The enhanced feature extraction network is used to further extract feature information of the details, texture, and background of the target object. The classification prediction network is used for positioning and classification.
[0012] Step 3: Set the training parameters of the algorithm model, use the training set and validation set obtained in step 1 to train the model, and continuously optimize the model parameters during the training process;
[0013] Step 4: Input the images of the test set into the trained model and output the detection results for verification.
[0014] In a possible design, the specific structure and processing process of the enhanced feature extraction network are as follows:
[0015] The intermediate feature layers P4, P5 and P6 of the backbone feature extraction network are used as input, and the P4, P5 and P6 layers correspond to the low-level, intermediate-level and high-level feature maps respectively; the P4 and P5 layers obtain the global information of the features through average pooling Avgpool and standard deviation pooling Stdpool, and the P6 layer obtains the global information of the features through maximum pooling Maxpool; the P4, P5 and P6 layers are respectively subjected to 1×1 convolutional layers to obtain the mutual dependence between channels; the P4 and P5 layers are standardized by batch normalization, and the Sigmoid function is taken to obtain the channel weights, and the weights are multiplied with the P4, P5 and P6 layers as new feature layers and input into the bidirectional feature pyramid network BiFPN;
[0016] The specific formula is as follows:
[0017] The P4 and P5 layers use average pooling Avgpool and standard deviation pooling Stdpool:
[0018]
[0019]
[0020] g(x)=[μ c ,ξ c ]
[0021] Among them, X i,j is the value of the input channel, μ c is the feature vector after average pooling, ξ c is the vector representation after standard deviation pooling of the feature layer, g(x) is the feature information after the concatenation of average pooling and standard deviation pooling, H and W are the width and height of the input image;
[0022] After obtaining the pooling information of the feature map, the weights of the feature maps of each channel are obtained:
[0023] S=σ(BN(C1D(g(x))))
[0024] In the formula, σ is the Sigmoid function, BN is used for standardization, C1D represents one-dimensional convolution, which is used to obtain the correlation between channels, and S represents the weight of the channel direction;
[0025] The original input X is recalibrated by the weight S, so the output formula of the P4 and P5 layers is:
[0026] Y 4,5 =SX
[0027] The P6 layer uses the maximum pooling Maxpool, the formula is as follows:
[0028]
[0029] S=σ(C1D(M c ))
[0030] M c is the feature vector after maximum pooling, X i,j is the value of the input channel, σ is the Sigmoid function, C1D represents one-dimensional convolution, which is used to obtain the correlation between channels, and S represents the weight in the channel direction. The output of the P6 layer is:
[0031] Y6=SX
[0032] Finally, the feature layers Y4, Y5, Y6 and P3 and P7 in the backbone feature extraction network are input into the bidirectional feature pyramid network BiFPN, and top-down and bottom-up bidirectional feature fusion is performed repeatedly.
[0033] In a possible design, the specific process of the bidirectional feature pyramid network BiFPN for feature fusion is:
[0034] S1, feature layers P3, Y4, Y5, Y6 and P7 are used as new inputs P3_in, P4_in, P5_in, P6_in and P7_in, and two P4_in, P5_in and P6_in are constructed at the same time;
[0035] S2, after obtaining P3_in, P4_in_1, P4_in_2, P5_in_1, P5_in_2, P6_in_1, P6_in_2 and P7_in, upsample P7_in, and stack it with P6_in_1 after upsampling to obtain P6_td; then upsample P6_td, and stack it with P5_in_1 after upsampling to obtain P5_td; then upsample P5_td, and stack it with P4_in_1 after upsampling to obtain P4_td; then upsample P4_td, and stack it with P3_in after upsampling to obtain P3_out;
[0036] S3, after obtaining P3_out, P4_td, P4_in_2, P5_td, P5_in_2, P6_td, P6_in_2 and P7_in, downsample P3_out, and stack it with P4_td and P4_in_2 to obtain P4_out; then downsample P4_out, and stack it with P5_td and P5_in_2 to obtain P5_out; then downsample P5_out, and stack it with P6_td and P6_in_2 to obtain P6_out; then downsample P6_out, and stack it with P7_in to obtain P7_out;
[0037] S4, use the obtained P3_out, P4_out, P5_out, P6_out and P7_out as P3_in, P4_in, P5_in, P6_in, P7_in, and repeat steps S2 and S3 to stack them.
[0038] In one possible design, the classification prediction network consists of a class prediction network ClassNet and a bounding box prediction network BoxNet to generate object classes and bounding box locations, respectively;
[0039] The class prediction network ClassNet consists of three 64-channel convolutions and one num_anchors x num_classes convolution, which is used to predict the type of each prediction box at each grid point on the feature layer. The num_anchors refers to the number of prior boxes in the feature layer, and the num_classes refers to the total number of target classes detected by the network.
[0040] The bounding box prediction network BoxNet consists of 3 64-channel convolutions and 1 num_anchors x 4 convolution, which is used to predict the changes of each prior box at each grid point on the feature layer. The num_anchors refers to the number of prior boxes owned by the feature layer, and the 4 refers to the adjustment of the prior box.
[0041] Among them, the ClassNet used in each feature layer is the same ClassNet; the BoxNet used in each feature layer is the same BoxNet.
[0042] In a possible design, the training parameters are set in step 3, the training cycle is set to 200, the batch size is set to 16, the initial learning rate is set to 0.01, and the learning rate is reduced tenfold when the cycle is 100 and 150.
[0043] The second aspect of the present invention also provides a method for identifying marine biological targets, comprising the following steps: acquiring marine biological images; inputting the acquired images into the detection algorithm model constructed by the construction method described in the first aspect; and obtaining the type of marine biological targets after model processing.
[0044] The third aspect of the present invention also provides a device for identifying marine biological targets, which includes at least one processor and at least one memory; the memory stores a program of the detection algorithm model built by the building method described in the first aspect; when the processor executes the program stored in the memory, marine biological target detection and identification can be achieved.
[0045] The fourth aspect of the present invention also provides a computer-readable storage medium, which stores a computer execution program of the detection algorithm model constructed by the construction method described in the first aspect, and when the computer execution program is executed by a processor, it can realize marine biological target detection and identification.
[0046] Compared with the prior art, the present invention provides a method for building a marine biological target detection algorithm model and a recognition method. In the present invention, the construction of the enhanced feature extraction network improves the convolutional neural network's ability to extract target feature information, especially for stacked targets, occluded targets and other targets with incomplete information and camouflaged targets, greatly improving the recognition accuracy. And the processing results of the enhanced feature extraction network are used in the bidirectional feature pyramid network BiFPN, which enhances the network's multi-scale feature fusion capability and makes the network more adaptable to the actual underwater environment. Specifically, the top feature map has rich semantic information, and the maximum pooling Maxpool can well retain the texture detail features, while the bottom feature map has high resolution, and the average pooling Avgpool and the standard deviation Stdpool can well retain the background information. By constructing an enhanced feature extraction network, the network can further extract feature information such as details, textures, and backgrounds of targets of different scales, and strengthen semantic information. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 This is a flowchart of the method for building the detection algorithm model of the present invention.
[0048] Figure 2 This is a schematic diagram of the overall structure of the algorithm model constructed by the present invention.
[0049] Figure 3 This is a schematic diagram of the structure of the backbone network EfficientNet.
[0050] Figure 4 Schematic diagram of the structure of the enhanced feature extraction network.
[0051] Figure 5 Schematic diagram of the bidirectional feature pyramid network BiFPN structure.
[0052] Figure 6 Schematic diagram of the structure for the classification prediction network to classify and locate the target object.
[0053] Figure 7 It is a comparison chart of the mAP values of the commonly used target detection algorithms and the algorithm of the present invention.
[0054] Figure 8 The figure is a simplified block diagram of the structure of the marine biological target identification device in the present invention. DETAILED DESCRIPTION
[0055] The invention will be further described below in conjunction with specific embodiments.
[0056] Embodiment 1:
[0057] like Figure 1As shown, the present invention provides a method for building a marine biological target detection algorithm model, which mainly includes the following steps:
[0058] Step 1: obtain a dataset of several marine biological images, label the target species using labeling software, and divide the dataset into a training set, a validation set, and a test set;
[0059] Step 2: Build a deep learning target detection algorithm model based on the EfficientDet network model. The algorithm model includes three parts: a backbone feature extraction network, an enhanced feature extraction network, and a classification prediction network. The backbone feature extraction network is an EfficientNet network, which is used to perform preliminary extraction of image features. The enhanced feature extraction network is used to further extract feature information of the details, texture, and background of the target object. The classification prediction network is used for positioning and classification.
[0060] Step 3: Set the training parameters of the algorithm model, use the training set and validation set obtained in step 1 to train the model, and continuously optimize the model parameters during the training process;
[0061] Step 4: Input the images of the test set into the trained model and output the detection results for verification.
[0062] 1. Get the dataset.
[0063] The downloaded marine biological dataset was manually annotated and divided into training set, validation set and test set according to the ratio of 7:2:1.
[0064] First, we downloaded the marine life dataset from the information provided by the National Natural Science Foundation of China for the underwater robot competition and screened it to remove images without target objects. Finally, a total of 5543 valid images were retained.
[0065] Then, the target species are labeled using the labeling software labelImg. For example, this dataset can include four species: holothurian, echinus, starfish, and scallop. The image annotations must cover all parts to avoid affecting the recognition effect due to incomplete features. The label file is saved as an .xml file in PASCAL VOC format. The label file information includes the storage location, label type, and label size.
[0066] Then put the image file in the JPEGimages file, put the label file in the Annotation file, and create a txt file corresponding to the label category name, which contains the names of various classes in the dataset.
[0067] 2. Build a deep learning target detection model.
[0068] The deep learning target detection algorithm model is built based on the EfficientDet network model. The algorithm model includes three parts: backbone feature extraction network, enhanced feature extraction network and classification prediction network. The overall structure of the model is as follows: Figure 2 shown.
[0069] 1) Backbone feature extraction network:
[0070] The backbone feature extraction network performs preliminary extraction of image features. The EfficientDet network uses EfficientNet as the backbone network. EfficientNet is composed of 16 stacked blocks, and the 16 large blocks can be divided into 1, 2, 2, 3, 3, 4, and 1 blocks, such as Figure 3 As shown in Figure 2, the overall design idea of Block is to invert the residual structure.
[0071] The specific structure is:
[0072] a. Conv3×3, performs preliminary feature extraction on the input image and resizes the image to 512×512.
[0073] b. MBConv1, after a 3*3 convolutional network, the number of convolution kernels is 1, the number of output channels is 16, and then batch normalization is performed. The activation function Swish operation is performed, and the output is (256, 256, 16). This output is the P1 layer.
[0074] c. MB Conv 6, after two 3*3 convolutional networks, the number of convolution kernels is 6, the number of output channels is 24, and then batch normalization is performed, the activation function Swish operation is performed, and the output is (128, 128, 24). This output is the P2 layer.
[0075] d. MB Conv 6, after two 5*5 convolutional networks, the number of convolution kernels is 6, the number of output channels is 40, and then batch normalization is performed. The activation function Swish operation is performed, and the output is (64, 64, 40). This output is the P3 layer.
[0076] e. MB Conv 6, after three 3*3 convolutional networks, the number of convolution kernels is 6, the number of output channels is 80, and then batch normalization is performed, the activation function Swish operation is performed, and the output is (32,32,80). This output is the P4 layer.
[0077] f. MB Conv 6, after three 5*5 convolutional networks, the number of convolution kernels is 6, the number of output channels is 112, and then batch normalization is performed. The activation function Swish operation is performed, and the output is (32,32,112). This output is the P5 layer.
[0078] g. MB Conv 6, after four 5*5 convolutional networks, the number of convolution kernels is 6, the number of output channels is 192, and then batch normalization is performed. The activation function Swish operation is performed, and the output is (16, 16, 192). This output is the P6 layer.
[0079] h. MB Conv 6, after a 3*3 convolutional network, the number of convolution kernels is 6, the number of output channels is 320, and then batch normalization is performed. The activation function Swish operation is performed, and the output is (16,16,320). This output is the P7 layer.
[0080] Through the backbone feature extraction network, 7 common feature layers can be obtained, among which P3, P4, P5, P6, and P7 features enter the next step of processing.
[0081] 2) Strengthen the feature extraction network:
[0082] Strengthen the feature extraction network to further extract the feature information of the details, texture, and background of the target object. The specific structure is as follows Figure 4 shown.
[0083] The specific structure and processing process of the enhanced feature extraction network are as follows:
[0084] The intermediate feature layers P4, P5 and P6 of the backbone feature extraction network are used as input, and the P4, P5 and P6 layers correspond to the low-level, intermediate-level and high-level feature maps respectively; the P4 and P5 layers obtain the global information of the features through average pooling Avgpool and standard deviation pooling Stdpool, and the P6 layer obtains the global information of the features through maximum pooling Maxpool; the P4, P5 and P6 layers are respectively subjected to 1×1 convolutional layers to obtain the mutual dependence between channels; the P4 and P5 layers are standardized by batch normalization, and the Sigmoid function is taken to obtain the channel weights, and the weights are multiplied with the P4, P5 and P6 layers as new feature layers and input into the bidirectional feature pyramid network BiFPN;
[0085] The specific formula is as follows:
[0086] The P4 and P5 layers use average pooling Avgpool and standard deviation pooling Stdpool:
[0087]
[0088]
[0089] g(x)=[μ c ,ξ c ]
[0090] Among them, X i,j is the value of the input channel, μ c is the feature vector after average pooling, ξ c is the vector representation after standard deviation pooling of the feature layer, g(x) is the feature information after the concatenation of average pooling and standard deviation pooling, H and W are the width and height of the input image;
[0091] After obtaining the pooling information of the feature map, the weights of the feature maps of each channel are obtained:
[0092] S=σ(BN(C1D(g(x))))
[0093] In the formula, σ is the Sigmoid function, BN is used for standardization, C1D represents one-dimensional convolution, which is used to obtain the correlation between channels, and S represents the weight of the channel direction;
[0094] The original input X is recalibrated by the weight S, so the output formula of the P4 and P5 layers is:
[0095] Y 4,5 =SX
[0096] The P6 layer uses the maximum pooling Maxpool, the formula is as follows:
[0097]
[0098] S=σ(C1D(M c ))
[0099] M c is the feature vector after maximum pooling, X i,j is the value of the input channel, σ is the Sigmoid function, C1D represents one-dimensional convolution, which is used to obtain the correlation between channels, and S represents the weight in the channel direction. The output of the P6 layer is:
[0100] Y6=SX
[0101] Finally, the feature layers Y4, Y5, Y6 and P3 and P7 in the backbone feature extraction network are input into the bidirectional feature pyramid network BiFPN, and top-down and bottom-up bidirectional feature fusion is performed repeatedly.
[0102] The structure of the bidirectional feature pyramid network BiFPN is as follows Figure 5 As shown in the figure, the specific process of feature fusion of the bidirectional feature pyramid network BiFPN is as follows:
[0103] S1, feature layers P3, Y4, Y5, Y6 and P7 are used as new inputs P3_in, P4_in, P5_in, P6_in and P7_in, and two P4_in, P5_in and P6_in are constructed at the same time;
[0104] S2, after obtaining P3_in, P4_in_1, P4_in_2, P5_in_1, P5_in_2, P6_in_1, P6_in_2 and P7_in, upsample P7_in, and stack it with P6_in_1 after upsampling to obtain P6_td; then upsample P6_td, and stack it with P5_in_1 after upsampling to obtain P5_td; then upsample P5_td, and stack it with P4_in_1 after upsampling to obtain P4_td; then upsample P4_td, and stack it with P3_in after upsampling to obtain P3_out;
[0105] S3, after obtaining P3_out, P4_td, P4_in_2, P5_td, P5_in_2, P6_td, P6_in_2 and P7_in, downsample P3_out, and stack it with P4_td and P4_in_2 to obtain P4_out; then downsample P4_out, and stack it with P5_td and P5_in_2 to obtain P5_out; then downsample P5_out, and stack it with P6_td and P6_in_2 to obtain P6_out; then downsample P6_out, and stack it with P7_in to obtain P7_out;
[0106] S4, use the obtained P3_out, P4_out, P5_out, P6_out and P7_out as P3_in, P4_in, P5_in, P6_in, P7_in, and repeat steps S2 and S3 to stack them.
[0107] 3) Classification prediction network
[0108] The classification prediction network consists of the class prediction network ClassNet and the bounding box prediction network BoxNet, such as Figure 6 As shown, to generate object classes and bounding box locations respectively;
[0109] The class prediction network ClassNet consists of three 64-channel convolutions and one num_anchors x num_classes convolution, which is used to predict the category corresponding to each prediction box at each grid point on the feature layer. The num_anchors refers to the number of prior boxes owned by the feature layer, and the num_classes refers to the total number of target classes detected by the network;
[0110] The bounding box prediction network BoxNet consists of 3 64-channel convolutions and 1 num_anchors x 4 convolution, which is used to predict the changes of each prior box at each grid point on the feature layer. The num_anchors refers to the number of prior boxes owned by the feature layer, and the 4 refers to the adjustment of the prior box.
[0111] Among them, the ClassNet used in each feature layer is the same ClassNet; the BoxNet used in each feature layer is the same BoxNet.
[0112] 3. Train the completed algorithm model.
[0113] Training can be performed on a Linux system, and the required GPU version is NVIDIA GTX 1080Ti, including 16GRAM. The software platform includes Anaconda 3, Pytorch 1.2.0, CUDA 10.0, CUDNN 7.3.0, and Python 3.6.
[0114] First, run the label annotation file voc_annotation.py to obtain the training train.txt and val.txt, which contain the label information of the training set and the validation set.
[0115] Secondly, set the training parameters in the model training file train.py. The training epochs is set to 200; the batch size is set to 16; the initial learning rate is set to 0.01, and the learning rate is reduced tenfold when the epochs are 100 and 150.
[0116] Then, run the training file train.py to start training. After 200 epochs, the loss value reaches stability. At this time, the weight file of the trained model will be generated in the corresponding weight folder.
[0117] 4. Test the trained model.
[0118] The prediction file predict.py is needed for training result prediction. First, you need to modify the model path model_path and the category path classes_path. model_path points to the trained weight file, and classes_path points to the txt file corresponding to the detection category. After completing the above operations, you can run predict.py for detection. Enter the path of the test image to output the detection result.
[0119] like Figure 7 As shown in the figure, it is a comparison of the accuracy mAP values of the commonly used target detection algorithms and the algorithm of the present invention, (a) is the mAP of the present algorithm; (b) is the mAP of Faster R-CNN; (c) is the mAP of SSD; (d) is the mAP of YOLOv3; It can be seen from the comparison that the recognition rate of the algorithm model constructed by the present invention is significantly improved.
[0120] After the marine biological target detection algorithm model is built, the actual scene usage process includes the following steps: obtaining marine biological images; inputting the obtained images into the detection algorithm model built by the construction method as described above; and obtaining the types of marine biological targets after model processing.
[0121] Embodiment 2:
[0122] like Figure 8 As shown, the present invention also provides a device for identifying marine biological targets, the device includes at least one processor and at least one memory, and also includes a communication interface and an internal bus; the memory stores a program of the detection algorithm model built by the construction method described in Example 1; when the processor executes the execution program stored in the memory, the detection and identification of marine biological targets can be realized. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.
[0123] The device may be provided as a terminal, a server or other forms of devices.
[0124] Figure 8 is a block diagram of a device shown as an example. The device may include one or more of the following components: a processing component, a memory, a power component, a multimedia component, an audio component, an input / output (I / O) interface, a sensor component, and a communication component. The processing component generally controls the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component may include one or more processors to execute instructions to complete all or part of the steps of the above method. In addition, the processing component may include one or more modules to facilitate interaction between the processing component and other components. For example, the processing component may include a multimedia module to facilitate interaction between the multimedia component and the processing component.
[0125] The memory is configured to store various types of data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0126] The power supply assembly provides power to various components of the electronic device. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device. The multimedia assembly includes a screen that provides an output interface between the electronic device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia assembly includes a front camera and / or a rear camera. When the electronic device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0127] The audio component is configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. The I / O interface provides an interface between the processing component and the peripheral interface module, and the above-mentioned peripheral interface module can be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
[0128] The sensor assembly includes one or more sensors for providing various aspects of status assessment for the electronic device. For example, the sensor assembly can detect the on / off state of the electronic device, the relative positioning of components, such as the display and keypad of the electronic device, and the sensor assembly can also detect the position change of the electronic device or a component of the electronic device, the presence or absence of user contact with the electronic device, the orientation or acceleration / deceleration of the electronic device and the temperature change of the electronic device. The sensor assembly may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
[0129] The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0130] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.
[0131] Embodiment 3:
[0132] The present invention also provides a non-volatile computer-readable storage medium, in which a computer execution program of the detection algorithm model constructed by the construction method described in Example 1 is stored. When the computer execution program is executed by a processor, it is used to realize marine biological target detection and identification.
[0133] Specifically, a system, device or equipment equipped with a readable storage medium may be provided, on which a software program code for implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system, device or equipment reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the above-mentioned embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.
[0134] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk (such as CD-ROM, CD-R, CD-RW, DVD-20ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tape, etc. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.
[0135] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.
[0136] It should be understood that the storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in a terminal or server.
[0137] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0138] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0139] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
[0140] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A method for building a marine biological target detection algorithm model, characterized in that: The following steps are involved: Step 1: obtain a dataset of several marine biological images, label the target species using labeling software, and divide the dataset into a training set, a validation set, and a test set; Step 2: Build a deep learning target detection algorithm model based on the EfficientDet network model. The algorithm model includes three parts: a backbone feature extraction network, an enhanced feature extraction network, and a classification prediction network. The backbone feature extraction network is an EfficientNet network, which is used to perform preliminary extraction of image features. The enhanced feature extraction network is used to further extract feature information of the details, texture, and background of the target object. The classification prediction network is used for positioning and classification. Among them, the specific structure and processing process of the enhanced feature extraction network are as follows: The intermediate feature layers P4, P5 and P6 of the backbone feature extraction network are used as input, and the P4, P5 and P6 layers correspond to the low-level, intermediate-level and high-level feature maps respectively; the P4 and P5 layers obtain the global information of the features through average pooling Avgpool and standard deviation pooling Stdpool, and the P6 layer obtains the global information of the features through maximum pooling Maxpool; the P4, P5 and P6 layers are respectively subjected to 1×1 convolutional layers to obtain the mutual dependence between channels; the P4 and P5 layers are standardized by batch normalization, and the Sigmoid function is taken to obtain the channel weights, and the weights are multiplied with the P4, P5 and P6 layers as new feature layers and input into the bidirectional feature pyramid network BiFPN; The specific formula is as follows: The P4 and P5 layers use average pooling Avgpool and standard deviation pooling Stdpool: g(x)=[μ c ,x c ] Among them, X i,j is the value of the input channel, μ c is the feature vector after average pooling, ξ c is the vector representation after standard deviation pooling of the feature layer, g(x) is the feature information after the concatenation of average pooling and standard deviation pooling, H and W are the width and height of the input image; After obtaining the pooling information of the feature map, the weights of the feature maps of each channel are obtained: S=σ(BN(C1D(g(x)))) In the formula, σ is the Sigmoid function, BN is used for standardization, C1D represents one-dimensional convolution, which is used to obtain the correlation between channels, and S represents the weight of the channel direction; The original input X is recalibrated by the weight S, so the output formula of the P4 and P5 layers is: AND 4,5 =SX The P6 layer uses the maximum pooling Maxpool, the formula is as follows: S=σ(C1D(M c )) M c is the feature vector after maximum pooling, X i,j is the value of the input channel, σ is the Sigmoid function, C1D represents one-dimensional convolution, which is used to obtain the correlation between channels, and S represents the weight in the channel direction. The output of the P6 layer is: Y6=SX Finally, the outputs Y4, Y5, Y6 of the feature layer and P3 and P7 in the backbone feature extraction network are input into the bidirectional feature pyramid network BiFPN, and the top-down and bottom-up bidirectional feature fusion is repeated; Step 3: Set the training parameters of the algorithm model, use the training set and validation set obtained in step 1 to train the model, and continuously optimize the model parameters during the training process; Step 4: Input the images of the test set into the trained model and output the detection results for verification.
2. The method for building a marine biological target detection algorithm model according to claim 1, characterized in that: In step 3, the training parameters are set, the training cycle is set to 200, the batch size is set to 16, the initial learning rate is set to 0.01, and the learning rate is reduced tenfold when the cycle is 100 and 150.
3. A method for identifying marine biological targets, characterized in that: The following steps are involved: Acquire images of marine organisms; input the acquired images into the detection algorithm model constructed by the construction method as described in claim 1 or 2; After model processing, the types of marine biological targets are obtained.
4. A device for identifying marine biological targets, characterized in that: The device includes at least one processor and at least one memory; the memory stores a program of the detection algorithm model built by the building method as described in claim 1 or 2; when the processor executes the program stored in the memory, marine biological target detection and identification can be achieved.
5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer execution program of the detection algorithm model constructed by the construction method as described in claim 1 or 2, and when the computer execution program is executed by the processor, it can realize marine biological target detection and identification.
