Target detection model training method, target detection method, device and equipment

Abstract

The present disclosure provides a method for training a target detection model, a target detection method, an apparatus, and a device, comprising: obtaining a model training sample; training a preset detection model based on the model training sample to obtain an initial detection model; inputting the model training sample into a first branch of the initial detection model to obtain image extraction features; training a preset compression model based on the model training sample and the image extraction features to obtain an image compression model; inputting the model training sample into the image compression model to obtain image compression features; inputting the image compression features into a second branch of the initial detection model to obtain target detection results corresponding to historical image data; and fine-tuning parameters of the first and second branches based on the target detection results corresponding to the historical image data using a preset loss function to obtain a target detection model. This effectively improves the detection accuracy of the target detection model.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of model training technology, and in particular, to a training method, target detection method, apparatus, and device applicable to a target detection model. Background Art

[0002] When detecting a target, fine-grained and specific judgments are required, not only to determine whether the detection area contains the target object, but also to locate the specific position of each target object.

[0003] In related technologies, sea surface target detection is mainly achieved through radar detection. For example, the radar high-resolution range image can reflect the size and structure of the target along the radar radial direction. By framing the training data, a time domain or frequency domain template library of different targets is constructed, and the test samples are matched with the templates to achieve target classification and recognition, thereby obtaining the detection results of the sea surface targets.

[0004] However, the detection accuracy in existing methods is not high. Summary of the Invention

[0005] The embodiments described herein provide a method for training a target detection model, a target detection method, an apparatus, and an equipment to overcome the above-mentioned problems.

[0006] In a first aspect, according to the present disclosure, a method for training an object detection model is provided, comprising:

[0007] Obtaining a model training sample, wherein the model training sample is used to describe historical image data collected of an unmanned sea surface cluster;

[0008] Training a preset detection model based on the model training samples to obtain an initial detection model, wherein the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, wherein the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction;

[0009] Inputting the model training sample into the first branch of the initial detection model to obtain image extraction features;

[0010] Training a preset compression model based on the model training samples and the image extraction features to obtain an image compression model;

[0011] Inputting the model training sample into the image compression model to obtain image compression features;

[0012] Inputting the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data;

[0013] A preset loss function is used to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

[0014] In a second aspect, according to the present disclosure, a method for detecting a target of an unmanned sea surface cluster is provided, comprising:

[0015] Obtain the current image data of the collected unmanned sea surface cluster;

[0016] Inputting the current image data into a target detection model, and determining a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data;

[0017] The target detection model is trained based on the target detection model training method described in the first aspect.

[0018] In a third aspect, according to the present disclosure, a training device for a target detection model is provided, comprising:

[0019] A first acquisition module is used to acquire model training samples, where the model training samples are used to describe historical image data collected of unmanned sea surface clusters;

[0020] a training module, configured to train a preset detection model based on the model training samples to obtain an initial detection model, wherein the initial detection model is configured to detect the distribution position and category of a target object in image data, the initial detection model comprising a first branch and a second branch, wherein the first branch is configured to extract image features, and the second branch is configured to identify the distribution position and category of a target object in the image data based on a result of the image feature extraction;

[0021] A first determination module is configured to input the model training sample into the first branch of the initial detection model to obtain image extraction features;

[0022] A second determining module is used to train a preset compression model based on the model training sample and the image extraction feature to obtain an image compression model;

[0023] A third determining module is used to input the model training sample into the image compression model to obtain image compression features;

[0024] a fourth determination module, configured to input the image compression feature into the second branch of the initial detection model to obtain a target detection result corresponding to the historical image data;

[0025] A fine-tuning module is used to use a preset loss function to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

[0026] In a fourth aspect, according to the present disclosure, a target detection device for an unmanned sea surface swarm is provided, comprising:

[0027] The second acquisition module is used to acquire the current image data of the collected unmanned sea surface cluster;

[0028] a fifth determination module, configured to input the current image data into a target detection model, and determine a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data;

[0029] The target detection model is trained based on the target detection model training method described in the first aspect.

[0030] In a fifth aspect, a computer device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the training method of the target detection model in any of the above embodiments are implemented, or the steps of the target detection method of the unmanned sea cluster in any of the above embodiments are implemented.

[0031] In a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the training method of the target detection model in any of the above embodiments are implemented, or the steps of the target detection method of the unmanned sea cluster in any of the above embodiments are implemented.

[0032] The training method of the target detection model provided in the embodiment of the present application obtains model training samples, and the model training samples are used to describe historical image data collected of unmanned clusters on the sea surface; based on the model training samples, a preset detection model is trained to obtain an initial detection model, and the initial detection model is used to detect the distribution position and category of the target object in the image data. The initial detection model includes: a first branch and a second branch, the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the results of the image feature extraction; the model training samples are input into the first branch of the initial detection model to obtain image extraction features; based on the model training samples and the image extraction features, a preset compression model is trained to obtain an image compression model; the model training samples are input into the image compression model to obtain image compression features; the image compression features are input into the second branch of the initial detection model to obtain target detection results corresponding to the historical image data; a preset loss function is used to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain a target detection model. In this way, the non-feature extraction part of the initial detection model is trained through the image compression features output by the image compression model, and a target detection model that best matches the image compression model and has better performance is obtained, effectively improving the detection accuracy of the target detection model.

[0033] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to more clearly understand the technical means of the embodiments of the present application, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below. It should be noted that the drawings described below only relate to some embodiments of the present disclosure and are not intended to limit the present disclosure.

[0035] Figure 1 It is a flowchart of a method for training a target detection model provided by the present disclosure.

[0036] Figure 2 This is a flow chart of a target detection method for unmanned sea surface clusters provided by the present disclosure.

[0037] Figure 3 It is a structural diagram of a training device for a target detection model provided by the present disclosure.

[0038] Figure 4 This is a structural diagram of a target detection device for an unmanned sea surface cluster provided by the present invention.

[0039] Figure 5 It is a structural diagram of a computer device provided by the present disclosure.

[0040] It should be noted that the elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work also fall within the scope of protection of the present disclosure.

[0042] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the specification and the relevant art, and will not be interpreted in an idealized or overly formal manner unless otherwise explicitly defined herein. As used herein, a statement that two or more parts are "connected" or "coupled" together shall mean that the parts are joined together either directly or through one or more intermediate components.

[0043] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it necessarily refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] The term "and / or" in this document simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists, A and B exist simultaneously, and B exists. Additionally, the character " / " in this document generally indicates that the related objects are in an "or" relationship. Terms such as "first" and "second" are used solely to distinguish one component (or portion of a component) from another component (or portion of a component).

[0045] In the description of this application, unless otherwise specified, "plurality" means more than two (including two), and similarly, "multiple groups" means more than two (including two).

[0046] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0047] Figure 1 It is a flowchart of a method for training a target detection model provided by an embodiment of the present disclosure.

[0048] like Figure 1 As shown in Figure 2, the specific process of the target detection model training method includes:

[0049] S110 , obtaining model training samples, where the model training samples are used to describe historical image data collected of unmanned sea surface clusters.

[0050] When obtaining the model training samples, the unmanned sea cluster can be photographed by a designated camera (such as a camera) to obtain a natural light image. Accordingly, the historical image data of the unmanned sea cluster is the natural light image obtained by photographing the unmanned sea cluster.

[0051] S120 . Train a preset detection model based on the model training sample to obtain an initial detection model. The initial detection model includes: a first branch and a second branch.

[0052] Among them, the initial detection model is used to detect the distribution location and category of the target object in the image data.

[0053] The initial detection model can be YOLOv8. The distribution position of the target object in the image data is the three-dimensional coordinate of the target object in the world coordinate system. The category of the target in the image data indicates the category of the object to which the target belongs. The categories of the target object may include, but are not limited to, biological debris, floating devices, and abandoned items.

[0054] The first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction.

[0055] Training a preset detection model based on a model training sample to obtain an initial detection model may include: inputting the model training sample into the preset detection model, performing loss calculation based on the distribution position and category of the target object output by the preset detection model, and the distribution position and category of the target object corresponding to the model training sample; if the calculated loss value meets the preset threshold, it is determined that the initial detection model is obtained; if the calculated loss value does not meet the preset threshold, adjusting the model parameters of the preset detection model until the distribution position and category of the target object output by the preset detection model, and the loss calculation result of the distribution position and category of the target object corresponding to the model training sample meet the preset threshold, the training of the preset detection model is completed, and the initial detection model is obtained.

[0056] S130: Input the model training sample into the first branch of the initial detection model to obtain image extraction features.

[0057] The model training samples are input into the first branch of the initial detection model for feature extraction, and the image extraction features are determined based on the output results of the first branch.

[0058] Image extraction features describe the extraction results of the main features contained in the image corresponding to the model training sample. For example, if the image corresponding to the model training sample includes a target area and a background area, the image extraction features can be the corresponding data features extracted from the target area.

[0059] S140: Training a preset compression model based on the model training samples and image extraction features to obtain an image compression model.

[0060] Among them, the image compression model compresses the image features of the model training samples, so that the main feature information in the model training samples is more concentrated.

[0061] In some embodiments, a preset compression model is trained based on model training samples and image extraction features to obtain an image compression model, including:

[0062] Obtain a preset compression model, which includes: an autoencoder, a super-a priori encoder, a super-a priori decoder, and a converter.

[0063] The autoencoder consists of multiple convolutional layers and ReLu activation functions. The multiple convolutional layers in the autoencoder are connected in a cascade relationship, that is, the output of one convolutional layer is the input of another convolutional layer connected to it.

[0064] The model training samples are input into the autoencoder of the preset compression model to obtain pixel space representation data.

[0065] For example, the autoencoder of the preset compression model can process the image data and convert it into a potential representation in the pixel space, as shown in the following formula (1).

[0066] y=f e (x; φ) (1)

[0067] In formula (1), f e Represents an autoencoder. Each convolutional layer in the autoencoder takes the output of the previous layer as input and converts the input into more expressive features before outputting them. y represents the potential representation of the entire autoencoder (i.e., pixel space representation data), and φ represents the training parameters of the autoencoder.

[0068] The pixel space representation data is quantized to obtain pixel space quantized data.

[0069] Among them, during the training process of the image compression model, it is necessary to discretize the nearly continuous potential representation in the pixel space representation data.

[0070] In some embodiments, quantizing the pixel space representation data to obtain pixel space quantized data includes:

[0071] Acquire potential representation data that meets the data continuity condition in the pixel space representation data; and quantize the potential representation data that meets the data continuity condition in the pixel space representation data using additive uniform noise to obtain pixel space quantized data.

[0072] The data continuity condition can be used to describe a constraint on the difference between two pixel values ​​in the pixel space representation data, such as a constraint threshold for the difference between two pixel values ​​in the pixel space representation data. When the difference between two pixel values ​​in the pixel space representation data is less than the constraint threshold, the two pixel values ​​are considered to be potential representation data.

[0073] Quantization is a very important component when discretizing nearly continuous potential representations. However, the derivative of the quantization function is almost always zero. To optimize an image compression model end-to-end, the quantization operation must be approximated so that gradients can be effectively returned. Additive uniform noise can be used instead of quantization, as shown in the following formula (2).

[0074]

[0075] In formula (2), round(*) represents the quantization operation; represents uniformly distributed noise.

[0076] The pixel space quantization data is input into a super a priori encoder of a preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data.

[0077] Among them, when encoding, the super prior encoder regards the entire image as a whole, and obtains the conditional probability of the entire image by multiplying the conditional probability generated by each pixel value in the image, so as to encode the image data according to the conditional probability.

[0078] In some embodiments, inputting the pixel space quantized data into a super a priori encoder of a preset compression model to obtain super a priori encoded data corresponding to the pixel space quantized data includes:

[0079] Obtain a pixel conditional probability generated by each pixel value in the pixel space quantization data; perform a cumulative multiplication operation on the pixel conditional probability generated by each pixel value in the pixel space quantization data to obtain an image conditional probability corresponding to the pixel space quantization data; encode the pixel space quantization data based on the image conditional probability to obtain super-prior encoded data corresponding to the pixel space quantization data.

[0080] The image conditional probability corresponding to the pixel space quantization data is the product of the pixel conditional probabilities generated by multiple pixel values. The quantized latent representation is arithmetic entropy coded to obtain a bitstream.

[0081] Thus, the pixel space quantization data is encoded through the image conditional probability of the entire image, and the super-prior coded data corresponding to the pixel space quantization data is effectively obtained.

[0082] The super-prior encoded data is input into the super-prior decoder of the preset compression model to obtain super-prior feature information.

[0083] The super-prior feature information is obtained by extracting features from the pixel domain potential representation obtained by decoding the super-prior encoded data.

[0084] The super-a priori decoder reconstructs the latent representation in the pixel domain (i.e., pixel spatial quantization data) by performing arithmetic entropy decoding on the bitstream (i.e., super-a priori coded data) and extracts the corresponding super-a priori information (i.e., super-a priori feature information) from the latent representation, as shown in the following formula (3). The mean and variance of the latent representation are then calculated from the super-a priori information to assist entropy coding.

[0085]

[0086] In formula (3), z represents the super prior feature information; h a represents the super prior encoder; θ a represents the training parameters of the super prior encoder; h s represents the super-prior decoder; θ s represents the training parameters of the super-prior decoder.

[0087] The hyper-prior feature information is input into the converter of the preset compression model to obtain high-dimensional spatial data features.

[0088] Among them, the converter of the preset compression model consists of multiple convolutional layers and residual blocks. The super-prior feature information is processed by each layer in the converter to obtain a high-dimensional spatial feature (i.e., high-dimensional spatial data feature). The input of each layer is the feature output by the previous layer, and the output is a feature that is more suitable for target detection. The overall expression is shown in the following formula (4).

[0089]

[0090] In formula (4), F represents the high-dimensional spatial data features output by the converter, represents the conversion network, Represents the parameters of the transformation network.

[0091] Using a preset loss function, the autoencoder, super-prior encoder, super-prior decoder and converter of the preset compression model are optimized and trained based on high-dimensional spatial data features and image extraction features to obtain an image compression model.

[0092] Among them, a preset loss function is used to calculate the loss value of the high-dimensional space data feature F and the image extraction feature, so that the features reconstructed by the image compression model as the student network can approach the extracted features of the first branch of the initial detection model as the teacher network.

[0093] The preset loss function is shown in the following formula (5).

[0094]

[0095] In formula (5), F ^ =YOLOv8 back-end (x) is the first branch in the initial detection model; λ is an adjustable hyperparameter, and adjusting λ can adjust the relationship between bit rate and detection performance.

[0096] S150: Input the model training samples into the image compression model to obtain image compression features.

[0097] Among them, the image compression model includes: autoencoder, super prior encoder, super prior decoder and converter.

[0098] For example, the model training samples are input into the image compression model, and the pixel space representation data corresponding to the model training samples are obtained through the autoencoder; the pixel space representation data corresponding to the model training samples are quantized to obtain the corresponding pixel space quantization data; the corresponding pixel space quantization data is input into the super-prior encoder to obtain the corresponding super-prior encoded data; the corresponding super-prior encoded data is input into the super-prior decoder to obtain the corresponding super-prior feature information; the corresponding super-prior feature information is input into the converter to obtain the corresponding high-dimensional space data features, that is, the image compression features.

[0099] In some embodiments, after inputting the model training samples into the image compression model to obtain the image compression features, the method further includes:

[0100] The image compression features are input into the post-enhancement model to obtain image enhancement features. The post-enhancement model is used to perform feature enhancement processing on the input features; based on the feature distinction data of the image enhancement features and the image compression features, the image compression features are updated.

[0101] Among them, the post-enhancement model consists of multiple convolutional layers, each of which enhances the input features. The number of channels of the last convolutional layer is the same as the number of channels of the input image. The features obtained after processing become a three-channel feature map, thereby obtaining a feature map that is more robust and more suitable for detection tasks (i.e., image enhancement features).

[0102] The image enhancement feature is expressed as shown in the following formula (6).

[0103]

[0104] In formula (6), F1 represents the image enhancement feature; E represents the post-enhancement model; Represents the training parameters of the post-augmentation model.

[0105] In some embodiments, the training process of the post-enhancement model includes:

[0106] The high-dimensional spatial data features output by the converter of the preset compression model after training are input into the second branch of the initial detection model to obtain the feature enhancement results; the preset loss function is used to optimize the training of the preset enhancement model based on the feature enhancement results to obtain the post-enhancement model.

[0107] Among them, the high-dimensional spatial data features output by the converter of the preset compression model after training are input into YOLOv8 front-end YOLOv8 front-end It represents the rest of the initial detection model except the feature detector (i.e., the second branch), which is used to fix the weight of the initial detection model and use the performance of the initial detection model as an indicator to train the enhanced model so that it is closer to the characteristics of the initial detection model.

[0108] S160: Input the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data.

[0109] The target detection results corresponding to the historical image data may include: the distribution location of the target objects in the historical image data and the categories of the target objects. The categories of the target objects may include, but are not limited to, biological debris, floating devices, and abandoned items.

[0110] S170. Using a preset loss function, fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain a target detection model.

[0111] Among them, the preset loss function shown in formula (5) can be used to calculate the loss value between the target detection result corresponding to the historical image data and the preset detection result corresponding to the historical image data, so as to fine-tune the parameters of the first branch and the second branch of the initial detection model to obtain the target detection model that best matches the image compression model and has the best performance.

[0112] In this embodiment, a model training sample is obtained, and the model training sample is used to describe historical image data collected of unmanned sea surface clusters. A preset detection model is trained based on the model training sample to obtain an initial detection model. The initial detection model is used to detect the distribution location and category of targets in the image data. The initial detection model includes a first branch and a second branch, the first branch being used to extract image features, and the second branch being used to identify the distribution location and category of targets in the image data based on the image feature extraction results. The model training sample is input into the first branch of the initial detection model to obtain image extraction features. A preset compression model is trained based on the model training sample and the image extraction features to obtain an image compression model. The model training sample is input into the image compression model to obtain image compression features. The image compression features are input into the second branch of the initial detection model to obtain target detection results corresponding to the historical image data. A preset loss function is used to fine-tune the parameters of the first and second branches of the initial detection model based on the target detection results corresponding to the historical image data to obtain a target detection model. In this way, the non-feature extraction portion of the initial detection model is trained using the image compression features output by the image compression model, resulting in a target detection model that best matches the image compression model and has better performance, effectively improving the detection accuracy of the target detection model.

[0113] Figure 2 This is a flow chart of a target detection method for an unmanned sea surface cluster provided by an embodiment of the present disclosure. Figure 2 As shown in FIG, the specific process of the target detection method of the unmanned sea cluster includes:

[0114] S210: Acquire the collected current image data of the unmanned sea surface cluster.

[0115] When collecting the current image data of the unmanned sea surface cluster, the unmanned sea surface cluster can be photographed by a designated camera device (such as a camera) to obtain a natural light image. Accordingly, the current image data of the unmanned sea surface cluster is a natural light image obtained by photographing the unmanned sea surface cluster.

[0116] S220: Input the current image data into the target detection model, and determine the target detection result of the current image data based on the output result of the target detection model.

[0117] The target detection results for the current image data include the distribution location and category of the target objects in the current image data. The target detection model is trained based on the target detection model training method described in the above embodiment. This effectively improves the accuracy of sea surface target detection.

[0118] In addition, this embodiment also provides a test example of the target detection model. During the test, the test set is used to test the target detection model, and the target detection performance of the unmanned cluster image at sea under different compression rates is calculated, and a visual target detection result is given.

[0119] Example 1: With the hyperparameter λ set to 100, mAP (mean Average Precision, the average of the AP values ​​across categories) was used to measure object detection performance. For a test set of 900 images totaling 28.6MB, the mAP achieved for object detection on the original images was 85.2%. This embodiment's method compressed the total test set size to 6.7MB, achieving a mAP of 87.3%. Furthermore, the test set compressed using VVC (Versatile Video Coding) had a total size of 19.5MB and achieved a mAP of 84.9%. This comparison demonstrates that while maintaining object detection performance, this embodiment also provides superior compression performance.

[0120] Example 2: With the hyperparameter λ = 150, for a test set of 900 images with a total size of 28.6MB, the mAP achieved for object detection on the original images was 85.2%. This embodiment's method compressed the total size of the test set to 5.9MB, achieving a mAP of 85.6%. Furthermore, the test set compressed using VVC had a total size of 19.5MB and achieved a mAP of 84.9%. This comparison shows that this embodiment provides superior compression performance while maintaining object detection performance.

[0121] Example 3: With a hyperparameter of λ = 70, for a test set of 900 images with a total size of 28.6MB, the mAP achieved for object detection on the original images was 85.2%. This embodiment's method compressed the total size of the test set to 9.5MB, achieving a mAP of 87.9%. Furthermore, the test set compressed using VVC had a total size of 19.5MB and achieved a mAP of 84.9%. This comparison shows that this embodiment provides superior compression performance while maintaining object detection performance.

[0122] Example 4: With the hyperparameter λ = 50, for a test set of 900 images with a total size of 28.6MB, the mAP achieved for object detection on the original images was 85.2%. This embodiment's method compressed the total size of the test set to 10.1MB, achieving a mAP of 88.0%. Furthermore, the test set compressed using VVC had a total size of 19.5MB and achieved a mAP of 84.9%. This comparison shows that this embodiment provides superior compression performance while maintaining object detection performance.

[0123] In summary, this embodiment extracts a latent representation in pixel space by inputting the image to be used for cloud-based object detection into an encoder, quantizing the representation. Uniform noise is used during training to replace quantization, rounding is performed to directly discretize the representation, and the latent representation is entropy encoded using the distribution prior information obtained by the super-prior codec. The resulting latent representation is then input into a conversion module. Distillation learning is used to make the converted features approximate the features extracted by the teacher network, and these features are then input into the object detection model for object detection. Distillation learning results in a smaller model with smaller parameters, achieving higher detection performance at lower compression rates.

[0124] The first branch of the trained initial detection model is used as the teacher model, and the image compression model to be trained is used as the student model, so that the compressed features of the student model are close to the features extracted by the initial detection model, thereby achieving efficient image compression for target detection of unmanned sea clusters with low compression rate and high detection accuracy.

[0125] Figure 3 A structural diagram of a target detection model training device provided in this embodiment, the target detection model training device may include: a first acquisition module 310, a training module 320, a first determination module 330, a second determination module 340, a third determination module 350, a fourth determination module 360 ​​and a fine-tuning module 370.

[0126] The first acquisition module 310 is used to acquire model training samples, where the model training samples are used to describe historical image data collected of unmanned sea surface clusters.

[0127] The training module 320 is used to train the preset detection model based on the model training sample to obtain an initial detection model. The initial detection model is used to detect the distribution position and category of the target object in the image data. The initial detection model includes: a first branch and a second branch. The first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the results of the image feature extraction.

[0128] The first determination module 330 is used to input the model training sample into the first branch of the initial detection model to obtain image extraction features.

[0129] The second determination module 340 is configured to train a preset compression model based on the model training samples and the image extraction features to obtain an image compression model.

[0130] The third determination module 350 is used to input the model training samples into the image compression model to obtain image compression features.

[0131] The fourth determination module 360 ​​is configured to input the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data.

[0132] The fine-tuning module 370 is used to use a preset loss function to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

[0133] In this embodiment, optionally, the second determination module 340 includes: an acquisition unit, a first determination unit, a quantization unit, a second determination unit, a third determination unit, a fourth determination unit, and a training unit.

[0134] The acquisition unit is used to acquire a preset compression model, which includes: an autoencoder, a super-a priori encoder, a super-a priori decoder and a converter.

[0135] The first determining unit is used to input the model training sample into the autoencoder of the preset compression model to obtain pixel space representation data.

[0136] The quantization unit is used to perform quantization processing on the pixel space representation data to obtain pixel space quantized data.

[0137] The second determining unit is configured to input the pixel space quantization data into a super a priori encoder of a preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data.

[0138] The third determination unit is used to input the super-prior encoded data into the super-prior decoder of the preset compression model to obtain super-prior feature information, where the super-prior feature information is obtained by feature extraction of the pixel domain potential representation obtained by decoding the super-prior encoded data.

[0139] The fourth determining unit is used to input the super-prior feature information into a converter of a preset compression model to obtain high-dimensional spatial data features.

[0140] The training unit is used to optimize and train the autoencoder, super-prior encoder, super-prior decoder and converter of the preset compression model based on the high-dimensional spatial data features and image extraction features using a preset loss function to obtain an image compression model.

[0141] In this embodiment, optionally, it further includes: a sixth determination module and an update module.

[0142] The sixth determination module is used to input the image compression features into the post-enhancement model to obtain image enhancement features. The post-enhancement model is used to perform feature enhancement processing on the input features.

[0143] The updating module is used to update the image compression feature based on the feature distinction data of the image enhancement feature and the image compression feature.

[0144] In this embodiment, optionally, the training unit is also used to input the high-dimensional spatial data features output by the converter of the preset compression model after training into the second branch of the initial detection model to obtain feature enhancement results; and use a preset loss function to optimize the training of the preset enhancement model based on the feature enhancement results to obtain a post-enhancement model.

[0145] In this embodiment, optionally, the first determining unit is specifically configured to:

[0146] Acquire potential representation data that meets the data continuity condition in the pixel space representation data; and quantize the potential representation data that meets the data continuity condition in the pixel space representation data using additive uniform noise to obtain pixel space quantized data.

[0147] In this embodiment, optionally, the second determining unit is specifically configured to:

[0148] Obtain a pixel conditional probability generated by each pixel value in the pixel space quantization data; perform a cumulative multiplication operation on the pixel conditional probability generated by each pixel value in the pixel space quantization data to obtain an image conditional probability corresponding to the pixel space quantization data; encode the pixel space quantization data based on the image conditional probability to obtain super-prior encoded data corresponding to the pixel space quantization data.

[0149] The training device for the target detection model provided in the present disclosure can execute the above method embodiments. Its specific implementation principles and technical effects can be found in the above method embodiments, and the present disclosure will not repeat them here.

[0150] Figure 4 This is a schematic structural diagram of a target detection device for an unmanned sea surface cluster provided by this embodiment. The target detection device for an unmanned sea surface cluster may include: a second acquisition module 410 and a fifth determination module 420 .

[0151] The second acquisition module 410 is used to acquire the collected current image data of the unmanned sea surface cluster.

[0152] The fifth determination module 420 is used to input the current image data into the target detection model, and determine the target detection result of the current image data based on the output result of the target detection model. The target detection result of the current image data includes: the distribution position and category of the target object in the current image data.

[0153] The target detection model is trained using the target detection model training method in the above embodiment.

[0154] The target detection device for unmanned sea surface clusters provided by the present disclosure can execute the above method embodiments. Its specific implementation principles and technical effects can be found in the above method embodiments, and the present disclosure will not repeat them here.

[0155] The present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0156] The computer device includes a memory 510 and a processor 520 that are interconnected and communicate with each other via a system bus. It should be noted that the figure only shows a computer device with a memory 510 and a processor 520, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0157] Computer devices can be desktop computers, laptops, PDAs, cloud servers, etc. Computer devices can interact with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0158] The memory 510 includes at least one type of readable storage medium, including non-volatile memory or volatile memory, such as flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, optical disk, etc. RAM may include static RAM or dynamic RAM. In some embodiments, the memory 510 may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory 510 may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, or a Flash Card equipped on the computer device. Of course, the memory 510 may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory 510 is generally used to store the operating system and various application software installed on the computer device, such as the program code of the above-mentioned method. In addition, the memory 510 may also be used to temporarily store various types of data that have been output or are about to be output.

[0159] The processor 520 is generally used to perform the overall operation of the computer device. In this embodiment, the memory 510 is used to store program code or instructions, which include computer operating instructions. The processor 520 is used to execute the program code or instructions stored in the memory 510 or process data, such as the program code for running the above method.

[0160] In this document, a bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus system can be divided into address buses, data buses, and control buses. For ease of illustration, the figure uses only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0161] Another embodiment of the present application further provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads the computer-readable program code stored in the computer-readable medium, enabling the processor to execute the functional actions specified in each step or combination of steps in the above method, and to generate a device that implements the functional actions specified in each block or combination of blocks in the block diagram.

[0162] Computer-readable media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any appropriate combination of the foregoing, the memory is used to store program codes or instructions, the program codes include computer operating instructions, and the processor is used to execute the program codes or instructions of the above-mentioned methods stored in the memory.

[0163] For the definitions of memory and processor, please refer to the description of the aforementioned computer device embodiment and will not be repeated here.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0165] Each functional unit or module in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0167] In the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" described in the present application does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented with the aid of hardware comprising several different elements and with the aid of a suitably programmed computer. In a unit claim that lists several means, several units of these means may be embodied by the same hardware item. The use of first, second, and third etc. does not indicate any order and these words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

[0168] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for training a target detection model, characterized in that: include: Obtaining a model training sample, wherein the model training sample is used to describe historical image data collected of an unmanned sea surface cluster; Training a preset detection model based on the model training samples to obtain an initial detection model, wherein the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, wherein the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction; Inputting the model training sample into the first branch of the initial detection model to obtain image extraction features; Training a preset compression model based on the model training samples and the image extraction features to obtain an image compression model; Inputting the model training sample into the image compression model to obtain image compression features; Inputting the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data; A preset loss function is used to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

2. The method according to claim 1, characterized in that The step of training a preset compression model based on the model training sample and the image extraction features to obtain an image compression model includes: Obtaining a preset compression model, wherein the preset compression model includes: an autoencoder, a super-a priori encoder, a super-a priori decoder, and a converter; Inputting the model training sample into the autoencoder of the preset compression model to obtain pixel space representation data; quantizing the pixel space representation data to obtain pixel space quantized data; Inputting the pixel space quantization data into the super a priori encoder of the preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data; Inputting the super a priori coded data into the super a priori decoder of the preset compression model to obtain super a priori feature information, wherein the super a priori feature information is obtained by extracting features from a pixel domain latent representation obtained by decoding the super a priori coded data; Inputting the super-prior feature information into the converter of the preset compression model to obtain high-dimensional spatial data features; The preset loss function is used to optimize and train the autoencoder, the super-a priori encoder, the super-a priori decoder and the converter of the preset compression model based on the high-dimensional spatial data features and the image extraction features to obtain the image compression model.

3. The method according to claim 2, characterized in that After inputting the model training sample into the image compression model to obtain the image compression feature, the method further includes: Inputting the image compression features into a post-enhancement model to obtain image enhancement features, wherein the post-enhancement model is used to perform feature enhancement processing on the input features; The image compression feature is updated based on the feature distinction data of the image enhancement feature and the image compression feature.

4. The method according to claim 3, characterized in that The training process of the post-enhancement model includes: Inputting the high-dimensional spatial data features output by the converter of the preset compression model after training into the second branch of the initial detection model to obtain a feature enhancement result; A preset loss function is used to optimize and train the preset enhancement model based on the feature enhancement result to obtain the post-enhancement model.

5. The method according to claim 3, characterized in that The quantizing the pixel space representation data to obtain pixel space quantized data includes: Acquire potential representation data that satisfies a data continuity condition in the pixel space representation data; Additive uniform noise is used to quantize the potential representation data that meets the data continuity condition in the pixel space representation data to obtain the pixel space quantized data.

6. The method according to claim 3, characterized in that Inputting the pixel space quantization data into the super a priori encoder of the preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data includes: Obtaining a pixel conditional probability generated by each pixel value in the pixel space quantization data; Performing a cumulative multiplication operation on the pixel conditional probability generated by each pixel value in the pixel space quantization data to obtain an image conditional probability corresponding to the pixel space quantization data; Based on the image conditional probability, the pixel space quantization data is encoded to obtain the super-prior coded data corresponding to the pixel space quantization data.

7. A method for detecting unmanned cluster targets on the sea surface, characterized in that: include: Obtain the current image data of the collected unmanned sea surface cluster; Inputting the current image data into a target detection model, and determining a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data; The target detection model is trained based on the target detection model training method described in any one of claims 1 to 6.

8. A training device for a target detection model, characterized in that: include: A first acquisition module is used to acquire model training samples, where the model training samples are used to describe historical image data collected of unmanned sea surface clusters; a training module, configured to train a preset detection model based on the model training samples to obtain an initial detection model, wherein the initial detection model is configured to detect the distribution position and category of a target object in image data, the initial detection model comprising a first branch and a second branch, wherein the first branch is configured to extract image features, and the second branch is configured to identify the distribution position and category of a target object in the image data based on a result of the image feature extraction; A first determination module is configured to input the model training sample into the first branch of the initial detection model to obtain image extraction features; A second determining module is used to train a preset compression model based on the model training sample and the image extraction feature to obtain an image compression model; A third determining module is used to input the model training sample into the image compression model to obtain image compression features; a fourth determination module, configured to input the image compression feature into the second branch of the initial detection model to obtain a target detection result corresponding to the historical image data; A fine-tuning module is used to use a preset loss function to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

9. A target detection device for unmanned swarms on the sea, characterized in that: include: The second acquisition module is used to acquire the current image data of the collected unmanned sea surface cluster; a fifth determination module, configured to input the current image data into a target detection model, and determine a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data; The target detection model is trained based on the target detection model training method described in any one of claims 1 to 6.

10. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the training method of the target detection model as described in any one of claims 1 to 6 is implemented, or the target detection method of the unmanned sea surface cluster as described in claim 7 is implemented.

Images