Federal learning-based ship detection network training method and application method

Abstract

The invention provides a federated learning-based ship detection network training method and an application method, and belongs to the field of computer vision, and the training method comprises the steps: constructing a ship detection network and a local detection network; inputting ship image data into each local detection network, performing double-branch attention enhancement and feature fusion detection to obtain ship detection output, determining predicted total loss based on a dynamic non-monotonic focusing method, and updating local parameters of the local detection network according to the predicted total loss; and performing adaptive weighted aggregation on the local parameters to obtain global parameters of the ship detection network, updating the local parameters according to the global parameters, and iteratively updating the local parameters and the global parameters until the network performance is not improved any more. According to the method, a federal learning training mode is improved through double-branch attention enhancement, dynamic non-monotone focusing and local parameter adaptive weighted aggregation, so that the training data of each client can be reasonably utilized on the premise of protecting the data privacy of each client, and the accuracy of a ship detection network is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a ship detection network training method based on federated learning and an application method thereof. Background Art

[0002] Visual supervision of waterway traffic plays an important role in the modern maritime field. With the rapid development of the shipping industry, the demand for waterway traffic monitoring systems is increasing. An effective waterway traffic supervision system can not only improve the efficiency of ship traffic management, but also prevent accidents, ensure maritime safety, and support key tasks such as maritime law enforcement. With the rapid development of deep learning technology, image and video-based waterway traffic supervision has been significantly improved. The powerful feature extraction capability of deep learning algorithms makes it possible to automatically identify ships and other waterway traffic targets. However, in large-scale waters, traditional centralized machine learning training requires data from various departments. For the maritime field, centralized machine learning methods are difficult to meet the privacy and security needs of various maritime regulatory departments.

[0003] In order to ensure the data privacy security of various maritime regulatory departments (clients) during the training process, federated learning, as a distributed machine learning method, has gradually attracted the attention of researchers. Federated learning allows multiple waterway traffic regulatory departments to use their own data sets to train their own models and share the trained network parameters. Under the premise of ensuring data privacy, they can jointly train machine learning models. However, in the current federated learning, due to the uneven distribution of data from different sources and the uneven data quality, the existing federated learning methods are difficult to reasonably use these unevenly distributed and unevenly quality training data during the training process, which leads to reduced model accuracy.

[0004] Therefore, the existing federated learning methods have difficulty in making reasonable use of training data with uneven distribution and uneven quality during the training process, which leads to technical problems such as reduced model accuracy and needs to be improved. Summary of the invention

[0005] In view of this, it is necessary to provide a ship detection network training method and application method based on federated learning to solve the technical problem that the existing federated learning method is difficult to reasonably utilize the training data with unbalanced distribution and uneven quality during the training process, resulting in reduced model accuracy.

[0006] In order to solve the above technical problems, on the one hand, the present invention provides a ship detection network training method based on federated learning, comprising: Build a ship detection network and multiple local detection networks for multiple clients; The acquired ship image data is input into each local detection network, and the ship image data is subjected to dual-branch attention enhancement and feature fusion detection to obtain the ship detection output. The predicted total loss is determined based on the dynamic non-monotonic focusing method, and the local parameters of the local detection network are updated according to the predicted total loss. The local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network. The local parameters are updated according to the global parameters to obtain a new round of local detection networks. The local parameters and global parameters are iteratively updated until the network performance is no longer improved.

[0007] In a possible implementation, the ship image data includes local ship image data acquired by each client, and the local ship image data is respectively input into a local detection network of the corresponding client.

[0008] In a possible implementation, dual-branch attention enhancement and feature fusion detection are performed on ship image data to obtain ship detection output, including: Perform feature segmentation on the ship image data to obtain a number of input feature images; Perform dual-branch attention enhancement on the input feature image to obtain an enhanced feature image; Multi-scale feature extraction and prediction output are performed on the enhanced feature image to obtain the ship detection output.

[0009] In a possible implementation, the ship image data is feature segmented to obtain a number of input feature images, including: After convolution of the ship image data features, channel dimension feature segmentation is performed to obtain several input feature images.

[0010] In a possible implementation, performing dual-branch attention enhancement on the input feature image to obtain an enhanced feature image includes: Perform height average pooling and width average pooling on the input feature image to obtain a height feature map and a width feature map; The height feature map and the width feature map are sequentially subjected to 1×1 convolution, sigmoid activation function, weighted merging and group normalization to obtain the first branch feature; Perform 3×3 convolution on the input feature image to obtain the second branch feature; The first branch features are sequentially subjected to 2D average pooling and softmax activation function, and then matrix multiplication is performed to fuse the second branch features to obtain the first attention features. The second branch features are sequentially subjected to 2D average pooling and softmax activation function, and then matrix multiplication is performed to fuse the first branch features to obtain the second attention features. The first attention feature and the second attention feature are combined and then passed through the sigmoid activation function to obtain the cross-spatial attention weight. The input feature image is feature-weighted according to the cross-spatial attention weight to obtain the enhanced feature image.

[0011] In a possible implementation, determining the predicted total loss based on a dynamic non-monotonic focusing method includes: The initial bounding box loss is obtained by calculating the loss of the ship detection output based on the double-layer distance attention mechanism; Determine an exponential moving average according to the iteration rounds, determine the anchor box outlier according to the exponential moving average, determine a non-monotonic focusing coefficient according to the anchor box outlier, and determine a bounding box regression loss according to the non-monotonic focusing coefficient and the initial bounding box loss; Determine predicted probability loss and classification loss based on vessel detection output; The total prediction loss is determined based on the bounding box regression loss, prediction probability loss, and classification loss.

[0012] In a possible implementation, the local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network, including: Determine the aggregation weight according to the amount of valid data in the ship image data of each client; The global parameters of the ship detection network are obtained by adaptively weighting and aggregating local parameters according to the aggregation weights.

[0013] On the other hand, the present invention also provides a ship detection network application method, comprising: Get the image of the ship to be detected; Input the image of the ship to be detected into the well-trained ship detection network to obtain the ship detection result; Among them, the fully trained ship detection network is determined according to the above-mentioned ship detection network training method based on federated learning.

[0014] On the other hand, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned ship detection network training method based on federated learning and / or the above-mentioned ship detection network application method are implemented.

[0015] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned ship detection network training method based on federated learning and / or the above-mentioned ship detection network application method are implemented.

[0016] The beneficial effects of the present invention are as follows: the ship detection network training method based on federated learning provided by the embodiment of the present invention first constructs a ship detection network and multiple local detection networks of multiple clients; then the acquired ship image data is input into each local detection network, the ship image data is subjected to dual-branch attention enhancement and feature fusion detection to obtain the ship detection output, the predicted total loss is determined based on the dynamic non-monotonic focusing method, and the local parameters of the local detection network are updated according to the predicted total loss; finally, the local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network, the local parameters are updated according to the global parameters to obtain a new round of local detection networks, and the local parameters and the global parameters are iteratively updated until the network performance is no longer improved. In the process of federated learning, the present invention uses dual-branch attention enhancement to learn more comprehensive feature information, balances training data of different qualities through dynamic non-monotonic focusing, and balances training data of different distributions through adaptive weighted aggregation of local parameters, so that the training data of each client can be reasonably utilized under the premise of protecting the data privacy of each client, and the accuracy of the ship detection network obtained by federated learning training can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flow chart of an embodiment of a ship detection network training method based on federated learning provided by the present invention; Figure 2 A schematic diagram of the process flow of ship detection output according to an embodiment of the present invention; Figure 3 A schematic diagram of a process of dual-branch attention enhancement according to an embodiment of the present invention; Figure 4 This is a network structure diagram of a dual-branch attention according to an embodiment of the present invention; Figure 5 A schematic diagram of a loss calculation process according to an embodiment of the present invention; Figure 6 A schematic diagram of a process of adaptive weighted aggregation of network parameters according to an embodiment of the present invention; Figure 7 A schematic diagram of a flow chart of an embodiment of a ship detection network application method provided by the present invention; Figure 8 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise specified, "multiple" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone.

[0021] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0022] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] The present invention provides a ship detection network training method, application method, electronic device and medium based on federated learning, which are described below respectively.

[0024] Figure 1 A flow chart of an embodiment of a ship detection network training method based on federated learning provided by the present invention is as follows: Figure 1 As shown in the figure, the ship detection network training method based on federated learning includes: S101, constructing a ship detection network and multiple local detection networks of multiple clients; S102, inputting the acquired ship image data into each local detection network, performing dual-branch attention enhancement and feature fusion detection on the ship image data to obtain ship detection output, determining the predicted total loss based on the dynamic non-monotonic focusing method, and updating the local parameters of the local detection network according to the predicted total loss; S103, adaptively weighting and aggregating the local parameters of each local detection network to obtain the global parameters of the ship detection network, updating the local parameters according to the global parameters to obtain a new round of local detection networks, and iteratively updating the local parameters and the global parameters until the network performance is no longer improved.

[0025] Compared with the prior art, the ship detection network training method based on federated learning provided by the embodiment of the present invention first constructs a ship detection network and multiple local detection networks of multiple clients; then the acquired ship image data is input into each local detection network, the ship image data is subjected to dual-branch attention enhancement and feature fusion detection to obtain the ship detection output, the predicted total loss is determined based on the dynamic non-monotonic focusing method, and the local parameters of the local detection network are updated according to the predicted total loss; finally, the local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network, the local parameters are updated according to the global parameters to obtain a new round of local detection networks, and the local parameters and the global parameters are iteratively updated until the network performance is no longer improved. In the process of federated learning, the present invention uses dual-branch attention enhancement to learn more comprehensive feature information, balances training data of different qualities through dynamic non-monotonic focusing, and balances training data of different distributions through adaptive weighted aggregation of local parameters, so that the training data of each client can be reasonably utilized under the premise of protecting the data privacy of each client, and the accuracy of the ship detection network obtained by federated learning training can be effectively improved.

[0026] In some embodiments of the present invention, the ship image data includes local ship image data acquired by each client, and the local ship image data is respectively input into the local detection network of the corresponding client.

[0027] Specifically, the ship image data used for model training in the embodiment includes local ship image data of each maritime regulatory department client, which is obtained by photographing and collecting waterway ships through camera equipment installed on different clients. In order to ensure the data privacy security between clients during the training process. Each local ship image data is only used for local data feature learning in the local detection network corresponding to each client.

[0028] In some embodiments of the present invention, Figure 2 FIG. 1 is a flow chart of ship detection output according to an embodiment of the present invention. Figure 2 As shown, the ship image data is subjected to dual-branch attention enhancement and feature fusion detection to obtain the ship detection output, including: S201, performing feature segmentation on the ship image data to obtain a number of input feature images; S202, performing dual-branch attention enhancement on the input feature image to obtain an enhanced feature image; S203, performing multi-scale feature extraction and prediction output on the enhanced feature image to obtain a ship detection output.

[0029] Specifically, the embodiment constructs a cross-space detection network based on the detection framework of YOLOv8, and the cross-space detection network consists of four components, including input, backbone, neck and head.

[0030] Among them, the input component is used to process the input image to increase data and enhance the generalization ability of the model, such as cropping, transformation and scaling.

[0031] The backbone implementation scheme adds feature segmentation and dual-branch attention enhancement. Feature segmentation reshapes some channels into batch dimensions, and divides them into multiple sub-features according to the channel dimension, so that multi-semantic features are distributed in each group, which can effectively prevent the disappearance of details caused by convolutional dimensionality reduction, and can achieve local cross-channel interaction in parallel subnetworks without reducing channels. Dual-branch attention enhancement learns features through two parallel branches for each group feature. After the channel weights in each branch are updated, the outputs of the two parallel branches are aggregated using cross-space interactions to capture enhanced feature images containing pixel-level feature information.

[0032] The neck and head are based on the YOLOv8 framework. The neck performs multi-scale feature extraction and fuses features of different scales. Finally, the head of the network outputs the regression results and category probabilities.

[0033] In some embodiments of the present invention, the ship image data is subjected to feature segmentation to obtain a plurality of input feature images, including: After convolution of the ship image data features, channel dimension feature segmentation is performed to obtain several input feature images.

[0034] Specifically, after the ship image data is input into the local detection network, for each ship image, the embodiment first convolves the input ship image through a CBS convolution block to obtain the input feature , where the CBS convolution block consists of a convolution layer, a normalization layer, and an activation function layer. Then the input features are split into Sub-features , As the input feature image of the model.

[0035] In some embodiments of the present invention, Figure 3 Schematic diagram of the process of dual-branch attention enhancement according to an embodiment of the present invention. Figure 3 As shown, the input feature image is enhanced by dual-branch attention enhancement to obtain an enhanced feature image, including: S301, performing height average pooling and width average pooling on the input feature image to obtain a height feature map and a width feature map; S302, subjecting the height feature map and the width feature map to 1×1 convolution, sigmoid activation function, weighted merging and group normalization in sequence to obtain the first branch feature; S303, performing 3×3 convolution on the input feature image to obtain the second branch feature; S304, the first branch feature is subjected to 2D average pooling and softmax activation function, and then matrix multiplication is performed to fuse the second branch feature to obtain the first attention feature, and the second branch feature is subjected to 2D average pooling and softmax activation function, and then matrix multiplication is performed to fuse the first branch feature to obtain the second attention feature; S305, merging the first attention feature and the second attention feature and performing a sigmoid activation function to obtain a cross-spatial attention weight, and performing feature weighting on the input feature image according to the cross-spatial attention weight to obtain an enhanced feature image.

[0036] Specifically, in order to learn more comprehensive feature information, the embodiment adds a dual-branch attention module to the backbone of the local detection network. Figure 4 This is a network structure diagram of the dual-branch attention of an embodiment of the present invention, combined with Figure 4 In the dual-branch attention enhancement, the embodiment distributes the attention extraction in two branches, 1×1 and 3×3.

[0037] For the 1×1 branch, first input the feature image Divided into two directions, namely height and width, the distribution is used for global average pooling to obtain two feature maps. The formula is expressed as:

[0038]

[0039] in, express The height feature map, express Width feature map.

[0040] Then the height feature map and width feature map are connected to the global receptive field, and the result is sent to the shared 1×1 convolution, activated by the Sigmoid function, and the attention weights are obtained respectively. and , the formula is:

[0041]

[0042] in, Represents the activation of the Sigmoid function, Represents a 1×1 convolution operation.

[0043] Then, the input feature image is weighted according to the attention weight to obtain a feature map with attention weights in the height and width directions. The formula is expressed as:

[0044] The example is then normalized by group Instead of the conventional normalization method, the feature image is processed to obtain the first branch feature , to reduce the dependence on batch size, and then perform 2D average pooling to obtain global information , the formula is:

[0045]

[0046] Finally, the softmax activation function and matrix multiplication (MM) operation are applied to fuse the features of the 3×3 branches to obtain The first attention feature , the formula is:

[0047] in represents the second branch feature obtained from the 3×3 branch, Represents the softmax activation function.

[0048] For the 3×3 branch, the embodiment performs the input feature image Perform 3×3 convolution to obtain the second branch feature , the formula is:

[0049] in, Represents a 3×3 convolution operation.

[0050] Then perform 2D average pooling on the second branch features to obtain a dimension of The pooling result , the formula is:

[0051] Finally, the pooling result is passed through the softmax activation function and combined with the first branch feature of the 1×1 branch Perform matrix multiplication and get the dimension Second attention characteristics , the formula is:

[0052] After completing the attention extraction of the two branches, the embodiment converts the two branches and After the output results of are merged, the cross-space attention weight can be obtained through the sigmoid activation function. Finally, the input feature image is weighted according to the cross-space attention weight to obtain the final output enhanced feature image , the formula is:

[0053] In summary, in the dual-branch attention module of the present invention, attention extraction is performed from the 1×1 and 3×3 branches respectively, wherein the 1×1 branch performs global average pooling on the features from both the height and width directions, and after splicing, passes through a shared 1×1 convolution and a sigmoid activation function, weighted merging and group normalization to obtain the first branch feature, and the 3×3 branch performs a 3×3 convolution on the input feature image to obtain the second branch feature, and the features obtained from the two branches are averaged pooled and activated, and then fused with the other branch matrix multiplication to obtain the attention features of the two branches, thereby determining the cross-space attention weight of the input feature image, which can enable the detection network to learn more comprehensive feature information, effectively improving the training effect of the ship detection network.

[0054] In some embodiments of the present invention, Figure 5 FIG. 1 is a flow chart of loss calculation according to an embodiment of the present invention. Figure 5 As shown, the total prediction loss is determined based on the dynamic non-monotonic focusing method, including: S501, calculating the loss of the ship detection output based on the double-layer distance attention mechanism to obtain the initial bounding box loss; S502, determining an exponential moving average according to the iteration rounds, determining an anchor box outlier according to the exponential moving average, determining a non-monotonic focusing coefficient according to the anchor box outlier, and determining a bounding box regression loss according to the non-monotonic focusing coefficient and an initial bounding box loss; S503, determining the predicted probability loss and the classification loss according to the ship detection output; S504: Determine the total prediction loss according to the bounding box regression loss, the prediction probability loss, and the classification loss.

[0055] Specifically, the total prediction loss of the local detection network includes the bounding box regression loss , prediction probability loss and classification loss , predict the total loss It is obtained by the weighted sum of each loss, and the formula is expressed as:

[0056] in, , and Corresponding to the weight values ​​of the three losses respectively.

[0057] Considering the uneven quality of training data, in order to balance the training data of different qualities and improve the training effect of the local detection network, the embodiment optimizes the bounding box regression loss through the non-monotonic focusing coefficient. ,The embodiment first adopts a two-layer distance attention mechanism to obtain the initial bounding box loss, which is expressed as:

[0058] in, and Represents the horizontal and vertical coordinates of the center point of the prediction box, and Represents the horizontal and vertical coordinates of the center point of the real frame, and is the width and height of the minimum detection box, It is the intersection loss and the ratio loss.

[0059] Based on the initial bounding box loss, the embodiment uses the outlier degree of the anchor box to improve the detection accuracy. , and use it to construct the nonmonotonic focusing coefficient , the formula is:

[0060]

[0061] in, Has momentum The exponential moving average of is the gradient gain, determined by the hyperparameter and Controlling the degree of outliers and the gradient gain The mapping of The smaller the value, the higher the quality of the anchor box. In addition, in order to avoid ignoring low-quality anchor boxes in early training, the momentum is designed. To delay the exponential moving average The time to approach the true value. During training, when the batch size is When momentum for:

[0062] in, is the number of training iterations.

[0063] Then the bounding box regression loss It can be expressed as:

[0064] In this way, in the middle and late stages of training, Small gradient gains can be assigned to low-quality anchor boxes to reduce harmful gradients.

[0065] For the predicted probability loss , which can be expressed as:

[0066] in, is the model's predicted probability for the target category. If the sample is a positive sample, then , if the sample is a negative sample, then , is the balancing factor that adjusts the balance between positive and negative samples. represents the parameter for adjusting the balance between difficult and easy samples, It is a weighted term for samples that are difficult to distinguish, and its purpose is to increase the loss weight of samples that are difficult to distinguish.

[0067] For classification loss , which can be expressed as:

[0068] in, is the predicted IoU-aware classification score, is the target score, indicating a soft label related to IOU. When , there is no hyperparameter, that is, there is no attenuation, and when When , negative samples have hyperparameters. It will reduce the contribution of negative samples. It is a hyperparameter to prevent over-suppression, but overall it will reduce the contribution of negative samples.

[0069] In some embodiments of the present invention, Figure 6 FIG. 1 is a flow chart of adaptive weighted aggregation of network parameters according to an embodiment of the present invention. Figure 6 As shown, the local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network, including: S601, determining an aggregation weight according to the amount of valid data in the ship image data of each client; S602: Adaptively weight and aggregate local parameters according to aggregation weights to obtain global parameters of the ship detection network.

[0070] Specifically, considering the problem of unbalanced data distribution between different clients and non-independent data distribution, the embodiment proposes an adaptive weighted aggregation model in the process of aggregating local parameters of each local detection network to obtain global parameters. The adaptive weighted aggregation model sets weights according to the effective number in each client data set to ensure that clients with a large amount of effective data in the data set receive higher attention. For clients with smaller effective data sets, these data sets are considered less common in the real world, and the present invention reduces their impact on the global model. The formula for adaptive weighted aggregation is expressed as:

[0071] in, represents the total number of all local data sets, Represents the client The amount of valid data in the data set. Indicates that the aggregator distributes to the client The previous round parameters.

[0072] In the embodiment, local parameters are obtained by continuously training the local detection network locally, and then the global parameters of the ship detection network are obtained according to the adaptive weighted aggregation of the local parameters. The local parameters of each local detection network are then updated by the global parameters. The training process is iterated to achieve model training by reasonably utilizing the data of each client while ensuring the privacy of the data of each client, until the prediction effect of the trained model converges, and a fully trained ship detection network is obtained.

[0073] In summary, in order to reasonably utilize the training data with uneven distribution and uneven quality in the process of federated learning training and improve the accuracy of the model, the present invention first constructs a ship detection network and multiple local detection networks of multiple clients; then the acquired ship image data is input into each local detection network, the ship image data is subjected to dual-branch attention enhancement and feature fusion detection to obtain the ship detection output, the predicted total loss is determined based on the dynamic non-monotonic focusing method, and the local parameters of the local detection network are updated according to the predicted total loss; finally, the local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network, the local parameters are updated according to the global parameters to obtain a new round of local detection networks, and the local parameters and global parameters are iteratively updated until the network performance is no longer improved. In the process of federated learning, the present invention uses dual-branch attention enhancement to learn more comprehensive feature information, balances training data of different qualities through dynamic non-monotonic focusing, and balances training data of different distributions through adaptive weighted aggregation of local parameters, so that the training data of each client can be reasonably utilized under the premise of protecting the data privacy of each client, and the accuracy of the ship detection network obtained by federated learning training can be effectively improved.

[0074] The embodiment of the present invention also provides a ship detection network application method, combining Figure 7 Come and see, Figure 7 A flow chart of an embodiment of the ship detection network application method provided by the present invention is as follows: Figure 7 As shown, the ship detection network application method includes: S701, obtaining a picture of a ship to be detected; S702, inputting the image of the ship to be detected into a well-trained ship detection network to obtain a ship detection result; Among them, the fully trained ship detection network is determined according to the above-mentioned ship detection network training method based on federated learning.

[0075] In an embodiment of the present invention, firstly, the image of the ship to be detected is effectively acquired, and then the image of the ship to be detected is effectively detected using the above-mentioned fully trained ship detection network, and the ship detection result can be output.

[0076] like Figure 8 As shown, the present invention also provides an electronic device 800. The electronic device 800 includes a processor 801, a memory 802 and a display 803. Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0077] In some embodiments, the processor 801 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 802, such as the ship detection network training method based on federated learning and / or the ship detection network application method in the present invention.

[0078] In some embodiments, the processor 801 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 801 may be local or remote. In some embodiments, the processor 801 may be implemented in a cloud platform. In one embodiment, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination thereof.

[0079] In some embodiments, the memory 802 may be an internal storage unit of the electronic device 800, such as a hard disk or memory of the electronic device 800. In other embodiments, the memory 802 may also be an external storage device of the electronic device 800, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 800.

[0080] Furthermore, the memory 802 may include both an internal storage unit of the electronic device 800 and an external storage device. The memory 802 is used to store application software installed in the electronic device 800 and various data.

[0081] In some embodiments, the display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 803 is used to display information of the electronic device 800 and to display a visual user interface. The components 801-803 of the electronic device 800 communicate with each other via a system bus.

[0082] In one embodiment, when the processor 801 executes the ship detection network training program in the memory 802, the following steps may be implemented: Build a ship detection network and multiple local detection networks for multiple clients; The acquired ship image data is input into each local detection network, and the ship image data is subjected to dual-branch attention enhancement and feature fusion detection to obtain the ship detection output. The predicted total loss is determined based on the dynamic non-monotonic focusing method, and the local parameters of the local detection network are updated according to the predicted total loss. The local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network. The local parameters are updated according to the global parameters to obtain a new round of local detection networks. The local parameters and global parameters are iteratively updated until the network performance is no longer improved.

[0083] In one embodiment, when the processor 801 executes the ship detection network application in the memory 802, the following steps may be implemented: Get the image of the ship to be detected; The image of the ship to be detected is input into the well-trained ship detection network to obtain the ship detection result.

[0084] It should be understood that: when the processor 801 executes the ship detection network training program and / or the ship detection network application program in the memory 802, in addition to the above functions, other functions can also be implemented. For details, please refer to the description of the corresponding method embodiments above.

[0085] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 800 mentioned, and the electronic device 800 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 800 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0086] Accordingly, an embodiment of the present application also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, it can implement the steps or functions of the ship detection network training method based on federated learning and / or the ship detection network application method provided in the above-mentioned method embodiments.

[0087] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0088] The above is a detailed introduction to the ship detection network training method and application method based on federated learning provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A ship detection network training method based on federated learning, characterized in that: include: Build a ship detection network and multiple local detection networks for multiple clients; Inputting the acquired ship image data into each of the local detection networks, performing dual-branch attention enhancement and feature fusion detection on the ship image data to obtain a ship detection output, determining a predicted total loss based on a dynamic non-monotonic focusing method, and updating local parameters of the local detection network according to the predicted total loss; The local parameters of each local detection network are adaptively weighted and aggregated to obtain the global parameters of the ship detection network, the local parameters are updated according to the global parameters to obtain a new round of local detection networks, and the local parameters and the global parameters are iteratively updated until the network performance is no longer improved.

2. The ship detection network training method based on federated learning according to claim 1 is characterized in that: The ship image data includes local ship image data acquired by each client, and the local ship image data is respectively input into the local detection network corresponding to the client.

3. The ship detection network training method based on federated learning according to claim 1 is characterized in that: The performing dual-branch attention enhancement and feature fusion detection on the ship image data to obtain a ship detection output includes: Performing feature segmentation on the ship image data to obtain a plurality of input feature images; Performing dual-branch attention enhancement on the input feature image to obtain an enhanced feature image; Multi-scale feature extraction and prediction output are performed on the enhanced feature image to obtain a ship detection output.

4. The method for training a ship detection network based on federated learning according to claim 3 is characterized in that: The feature segmentation of the ship image data to obtain a plurality of input feature images includes: After the feature convolution of the ship image data, channel dimension feature segmentation is performed to obtain a plurality of input feature images.

5. The ship detection network training method based on federated learning according to claim 3 is characterized in that: The step of performing dual-branch attention enhancement on the input feature image to obtain an enhanced feature image includes: Performing height average pooling and width average pooling on the input feature image to obtain a height feature map and a width feature map; The height feature map and the width feature map are sequentially subjected to 1×1 convolution, sigmoid activation function, weighted merging and group normalization to obtain a first branch feature; Performing a 3×3 convolution on the input feature image to obtain a second branch feature; The first branch feature is sequentially subjected to 2D average pooling and softmax activation function, and then matrix multiplication is performed to fuse the second branch feature to obtain a first attention feature. The second branch feature is sequentially subjected to 2D average pooling and softmax activation function, and then matrix multiplication is performed to fuse the first branch feature to obtain a second attention feature. The first attention feature and the second attention feature are combined and then subjected to a sigmoid activation function to obtain a cross-spatial attention weight, and the input feature image is feature-weighted according to the cross-spatial attention weight to obtain an enhanced feature image.

6. The ship detection network training method based on federated learning according to claim 1 is characterized in that: The method of determining the predicted total loss based on the dynamic non-monotonic focusing method includes: Performing loss calculation on the ship detection output based on a double-layer distance attention mechanism to obtain an initial bounding box loss; Determine an exponential moving average according to the iteration rounds, determine an anchor box outlier according to the exponential moving average, determine a non-monotonic focusing coefficient according to the anchor box outlier, and determine a bounding box regression loss according to the non-monotonic focusing coefficient and the initial bounding box loss; determining a predicted probability loss and a classification loss based on the vessel detection output; A prediction total loss is determined according to the bounding box regression loss, the prediction probability loss, and the classification loss.

7. The method for training a ship detection network based on federated learning according to claim 1, characterized in that: The adaptive weighted aggregation of the local parameters of each local detection network to obtain the global parameters of the ship detection network includes: Determine the aggregation weight according to the amount of valid data in the ship image data of each client; The local parameters are adaptively weighted and aggregated according to the aggregation weights to obtain the global parameters of the ship detection network.

8. A ship detection network application method, characterized in that: include: Get the image of the ship to be detected; Inputting the image of the ship to be detected into a well-trained ship detection network to obtain a ship detection result; Wherein, the fully trained ship detection network is determined according to the ship detection network training method based on federated learning according to any one of claims 1 to 7.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the ship detection network training method based on federated learning according to any one of claims 1 to 7 and / or the ship detection network application method according to claim 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the ship detection network training method based on federated learning according to any one of claims 1 to 7 and / or the ship detection network application method according to claim 8 are implemented.

Images