Target detection model training method, target detection method, device and equipment

Abstract

The invention provides a target detection model training method, a target detection method, a target detection device and target detection equipment. The method comprises the following steps: acquiring a model training sample; training a preset detection model based on the model training sample to obtain an initial detection model; inputting the model training sample into a first branch of the initial detection model to obtain an image extraction feature; training a preset compression model based on the model training sample and the image extraction features to obtain an image compression model; inputting the model training sample into an image compression model to obtain image compression features; inputting the image compression feature into a second branch of the initial detection model to obtain a target detection result corresponding to the historical image data; and performing parameter fine tuning on the first branch and the second branch based on a target detection result corresponding to the historical image data by adopting a preset loss function to obtain a target detection model. Therefore, the detection accuracy of the target detection model is effectively improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of model training technology, and in particular, to a training method, target detection method, apparatus and device applicable to a target detection model. Background Art

[0002] When detecting a target, fine-grained and specific judgments are required, not only to determine whether the detection area contains the target object, but also to locate the specific position of each target object.

[0003] In the related art, when detecting sea surface targets, it is mainly achieved through radar detection. For example, the radar high-resolution range image can reflect the size and structure of the target along the radar radial direction. By framing the training data, constructing a time domain or frequency domain template library of different targets, matching the test samples with the templates to achieve target classification and recognition, the detection results of the sea surface targets are obtained.

[0004] However, in the existing method, the detection accuracy is not high. Summary of the invention

[0005] The embodiments described herein provide a method, apparatus, device and medium for processing pending tasks in a medical device system, which overcome the above-mentioned problems.

[0006] In a first aspect, according to the present disclosure, a method for training a target detection model is provided, comprising:

[0007] Acquire a model training sample, where the model training sample is used to describe historical image data collected of an unmanned sea surface cluster;

[0008] Based on the model training samples, a preset detection model is trained to obtain an initial detection model, wherein the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, wherein the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction;

[0009] Inputting the model training sample into the first branch of the initial detection model to obtain image extraction features;

[0010] Training a preset compression model based on the model training samples and the image extraction features to obtain an image compression model;

[0011] Inputting the model training samples into the image compression model to obtain image compression features;

[0012] Inputting the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data;

[0013] A preset loss function is used to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

[0014] In a second aspect, according to the content of the present disclosure, a target detection method of an unmanned sea surface cluster is provided, comprising:

[0015] Obtain the current image data of the collected unmanned sea surface cluster;

[0016] Inputting the current image data into a target detection model, and determining a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data;

[0017] The target detection model is trained based on the target detection model training method described in the first aspect.

[0018] In a third aspect, according to the present disclosure, a training device for a target detection model is provided, comprising:

[0019] A first acquisition module is used to acquire a model training sample, where the model training sample is used to describe historical image data collected of an unmanned sea surface cluster;

[0020] A training module, used to train a preset detection model based on the model training sample to obtain an initial detection model, wherein the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, wherein the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction;

[0021] A first determination module, used for inputting the model training sample into the first branch of the initial detection model to obtain image extraction features;

[0022] A second determination module is used to train a preset compression model based on the model training sample and the image extraction feature to obtain an image compression model;

[0023] A third determination module is used to input the model training sample into the image compression model to obtain image compression features;

[0024] a fourth determination module, configured to input the image compression feature into the second branch of the initial detection model to obtain a target detection result corresponding to the historical image data;

[0025] A fine-tuning module is used to use a preset loss function to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

[0026] In a fourth aspect, according to the content of the present disclosure, a target detection device for an unmanned sea surface cluster is provided, comprising:

[0027] The second acquisition module is used to acquire the current image data of the collected sea surface unmanned cluster;

[0028] a fifth determination module, configured to input the current image data into a target detection model, and determine a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data;

[0029] The target detection model is trained based on the target detection model training method described in the first aspect.

[0030] In a fifth aspect, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the training method of the target detection model in any of the above embodiments are implemented, or the steps of the target detection method of the unmanned cluster on the sea surface in any of the above embodiments are implemented.

[0031] In a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the training method of the target detection model in any of the above embodiments are implemented, or the steps of the target detection method of the unmanned cluster on the sea surface in any of the above embodiments are implemented.

[0032] The training method of the target detection model provided in the embodiment of the present application obtains a model training sample, and the model training sample is used to describe the historical image data collected of the unmanned cluster on the sea surface; based on the model training sample, a preset detection model is trained to obtain an initial detection model, and the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction; the model training sample is input into the first branch of the initial detection model to obtain the image extraction feature; based on the model training sample and the image extraction feature, a preset compression model is trained to obtain the image compression model; the model training sample is input into the image compression model to obtain the image compression feature; the image compression feature is input into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data; a preset loss function is used to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection result corresponding to the historical image data to obtain the target detection model. In this way, the non-feature extraction part of the initial detection model is trained through the image compression features output by the image compression model, so as to obtain a target detection model that best matches the image compression model and has better performance, thereby effectively improving the detection accuracy of the target detection model.

[0033] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below. It should be noted that the drawings described below only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure, wherein:

[0035] Figure 1 It is a flowchart of a training method for a target detection model provided in the present invention.

[0036] Figure 2 It is a flow chart of a target detection method for an unmanned sea surface cluster provided by the present invention.

[0037] Figure 3 It is a structural schematic diagram of a training device for a target detection model provided by the present invention.

[0038] Figure 4 It is a structural schematic diagram of a target detection device for an unmanned sea surface cluster provided by the present invention.

[0039] Figure 5 It is a structural schematic diagram of a computer device provided by the present disclosure.

[0040] It should be noted that the elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work also fall within the scope of protection of the present disclosure.

[0042] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by a person skilled in the art to which the subject matter of the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the specification and the relevant art, and will not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, a statement that two or more parts are "connected" or "coupled" together shall mean that the parts are joined together directly or through one or more intermediate components.

[0043] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiments" in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists, A and B exist at the same time, and B exists. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. Terms such as "first" and "second" are only used to distinguish one component (or a part of a component) from another component (or another part of a component).

[0045] In the description of the present application, unless otherwise specified, "plurality" means more than two (including two), and similarly, "multiple groups" means more than two groups (including two).

[0046] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0047] Figure 1 It is a flowchart of a method for training a target detection model provided in an embodiment of the present disclosure.

[0048] like Figure 1 As shown, the specific process of the training method of the target detection model includes:

[0049] S110, obtaining model training samples, where the model training samples are used to describe historical image data collected of unmanned sea surface clusters.

[0050] When obtaining the model training samples, the sea surface unmanned cluster can be photographed by a designated camera device (such as a camera) to obtain a natural light picture. Correspondingly, the historical image data of the sea surface unmanned cluster is the natural light picture of the sea surface unmanned cluster obtained by taking the picture.

[0051] S120. Train a preset detection model based on the model training sample to obtain an initial detection model, where the initial detection model includes: a first branch and a second branch.

[0052] Among them, the initial detection model is used to detect the distribution position and category of the target object in the image data.

[0053] The initial detection model may be YOLOv8. The distribution position of the target object in the image data is the three-dimensional coordinate of the target object in the world coordinate system. The category of the target in the image data indicates which category the target object belongs to. The category of the target object may include but is not limited to: biological debris, floating equipment, abandoned items, etc.

[0054] The first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction.

[0055] Training a preset detection model based on a model training sample to obtain an initial detection model may include: inputting the model training sample into the preset detection model, performing loss calculation based on the distribution position and category of the target object output by the preset detection model, and the distribution position and category of the target object corresponding to the model training sample; if the calculated loss value meets a preset threshold, it is determined that the initial detection model is obtained; if the calculated loss value does not meet the preset threshold, adjusting the model parameters of the preset detection model until the distribution position and category of the target object output by the preset detection model, and the loss calculation result of the distribution position and category of the target object corresponding to the model training sample meet the preset threshold, the preset detection model training is completed, and the initial detection model is obtained.

[0056] S130, inputting the model training sample into the first branch of the initial detection model to obtain image extraction features.

[0057] The model training samples are input into the first branch of the initial detection model for feature extraction, and the image extraction features are determined based on the output result of the first branch.

[0058] The image extraction feature is used to describe the extraction result of the main features contained in the image corresponding to the model training sample. If an image corresponding to the model training sample includes a target area and a background area, the image extraction feature can be the corresponding data feature extracted from the target area.

[0059] S140: training a preset compression model based on model training samples and image extraction features to obtain an image compression model.

[0060] Among them, the image compression model compresses the image features of the model training samples, so that the main feature information in the model training samples is more concentrated.

[0061] In some embodiments, a preset compression model is trained based on model training samples and image extraction features to obtain an image compression model, including:

[0062] Obtain a preset compression model, the preset compression model includes: an autoencoder, a super-a priori encoder, a super-a priori decoder and a converter.

[0063] The autoencoder is composed of multiple convolutional layers and activation functions ReLu functions. The multiple convolutional layers in the autoencoder are connected in a cascade relationship, that is, the output of one convolutional layer is the input of another convolutional layer connected to this convolutional layer.

[0064] The model training samples are input into the autoencoder of the preset compression model to obtain the pixel space representation data.

[0065] For example, the self-encoder of the preset compression model can process the image data and convert it into a potential representation in the pixel space, as shown in the following formula (1).

[0066] y=f e (x; φ) (1)

[0067] In formula (1), f e Represents an autoencoder. Each convolutional layer in the autoencoder takes the output of the previous layer as input and converts the input into more expressive features before outputting them. y represents the potential representation of the entire autoencoder after encoding (i.e., pixel space representation data), and φ represents the training parameters of the autoencoder.

[0068] The pixel space representation data is quantized to obtain pixel space quantized data.

[0069] Among them, in the training process of the image compression model, it is necessary to discretize the nearly continuous potential representation in the pixel space representation data.

[0070] In some embodiments, quantizing the pixel space representation data to obtain pixel space quantized data includes:

[0071] Acquire potential representation data that meets the data continuity condition in the pixel space representation data; use additive uniform noise to quantize the potential representation data that meets the data continuity condition in the pixel space representation data to obtain pixel space quantized data.

[0072] The data continuity condition can be used to describe the constraint of the difference between two pixel values ​​in the pixel space representation data, such as the constraint threshold of the difference between two pixel values ​​in the pixel space representation data. When the difference between two pixel values ​​in the pixel space representation data is less than the constraint threshold, the two pixel values ​​are considered to be potential representation data.

[0073] When discretizing nearly continuous potential representations, quantization is a very important component. However, the derivative of the quantization function is almost always zero. To optimize an image compression model end-to-end, the quantization operation must be approximated so that it can effectively perform gradient backpropagation. Additive uniform noise can be used instead of quantization, as shown in the following formula (2).

[0074]

[0075] In formula (2), round(*) represents the quantization operation; represents uniformly distributed noise.

[0076] The pixel space quantization data is input into a super a priori encoder of a preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data.

[0077] Among them, the super-prior encoder regards the entire image as a whole when encoding, and obtains the conditional probability of the entire image by multiplying the conditional probability generated by each pixel value in the image, so as to encode the image data according to the conditional probability.

[0078] In some embodiments, the pixel space quantization data is input into a super a priori encoder of a preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data, including:

[0079] Obtain the pixel conditional probability generated by each pixel value in the pixel space quantization data; perform cumulative multiplication operation on the pixel conditional probability generated by each pixel value in the pixel space quantization data to obtain the image conditional probability corresponding to the pixel space quantization data; based on the image conditional probability, encode the pixel space quantization data to obtain the super priori encoded data corresponding to the pixel space quantization data.

[0080] The image conditional probability corresponding to the pixel space quantization data is the product of the pixel conditional probabilities generated by multiple pixel values. The quantized potential representation is arithmetic entropy encoded to obtain a bit stream.

[0081] Thus, the pixel space quantization data is encoded through the image conditional probability of the entire image, and the super-prior coded data corresponding to the pixel space quantization data is effectively obtained.

[0082] The super-a priori encoded data is input into a super-a priori decoder of a preset compression model to obtain super-a priori feature information.

[0083] The super-prior feature information is obtained by extracting features from the pixel domain potential representation obtained by decoding the super-prior encoded data.

[0084] The super-prior decoder reconstructs the latent representation in the pixel domain (i.e., pixel space quantized data) by performing arithmetic entropy decoding on the bit stream (i.e., the super-prior coded data), and extracts the corresponding super-prior information (i.e., the super-prior feature information) from the latent representation, as shown in the following formula (3). It is convenient to calculate the mean and variance of the latent representation from the super-prior information to assist entropy coding.

[0085]

[0086] In formula (3), z represents the super prior feature information; h a represents the super prior encoder; θ a represents the training parameters of the super prior encoder; h s represents the super prior decoder; θ s represents the training parameters of the super-prior decoder.

[0087] The hyper-prior feature information is input into the converter of the preset compression model to obtain high-dimensional spatial data features.

[0088] Among them, the converter of the preset compression model is composed of multiple convolutional layers and residual blocks. The super-prior feature information is processed by each layer in the converter to obtain a high-dimensional spatial feature (i.e., high-dimensional spatial data feature). The input of each layer is the feature output by the previous layer, and the output is a feature that is more suitable for target detection. The overall expression is shown in the following formula (4).

[0089]

[0090] In formula (4), F represents the high-dimensional spatial data features output by the converter, represents the conversion network, Represents the parameters of the transformation network.

[0091] Using a preset loss function, the autoencoder, super prior encoder, super prior decoder and converter of the preset compression model are optimized and trained based on high-dimensional spatial data features and image extraction features to obtain an image compression model.

[0092] Among them, a preset loss function is used to calculate the loss value of the high-dimensional space data feature F and the image extraction feature, so that the features reconstructed by the image compression model as the student network can approach the extracted features of the first branch of the initial detection model as the teacher network.

[0093] The preset loss function is shown in the following formula (5).

[0094]

[0095] In formula (5), F ^ =YOLOv8 back-end (x) is the first branch in the initial detection model; λ is an adjustable hyperparameter, and adjusting λ can adjust the relationship between bit rate and detection performance.

[0096] S150: Input the model training samples into the image compression model to obtain image compression features.

[0097] Among them, the image compression model includes: autoencoder, super prior encoder, super prior decoder and converter.

[0098] For example, the model training samples are input into the image compression model, and the pixel space representation data corresponding to the model training samples are obtained through the autoencoder; the pixel space representation data corresponding to the model training samples are quantized to obtain the corresponding pixel space quantization data; the corresponding pixel space quantization data is input into the super a priori encoder to obtain the corresponding super a priori encoded data; the corresponding super a priori encoded data is input into the super a priori decoder to obtain the corresponding super a priori feature information; the corresponding super a priori feature information is input into the converter to obtain the corresponding high-dimensional space data features, i.e., the image compression features.

[0099] In some embodiments, after inputting the model training sample into the image compression model to obtain the image compression feature, the method further includes:

[0100] The image compression feature is input into the post-enhancement model to obtain the image enhancement feature. The post-enhancement model is used to perform feature enhancement processing on the input feature. Based on the feature distinction data of the image enhancement feature and the image compression feature, the image compression feature is updated.

[0101] Among them, the post-enhancement model is composed of multiple convolutional layers, each convolutional layer enhances the input features, and the number of channels of the last convolutional layer is the same as the number of channels of the input image. The features obtained after processing become a three-channel feature map, thereby obtaining a more robust feature map (i.e., image enhancement feature) that is more suitable for detection tasks.

[0102] The image enhancement feature is expressed as shown in the following formula (6).

[0103]

[0104] In formula (6), F1 represents the image enhancement feature; E represents the post-enhancement model; Represents the training parameters of the post-augmentation model.

[0105] In some embodiments, the training process of the post-enhancement model includes:

[0106] The high-dimensional spatial data features output by the converter of the preset compression model after training are input into the second branch of the initial detection model to obtain the feature enhancement result; the preset loss function is used to optimize the preset enhancement model based on the feature enhancement result to obtain the post-enhancement model.

[0107] Among them, the high-dimensional spatial data features output by the converter of the preset compression model after training are input into YOLOv8 front-end YOLOv8 front-end It represents the rest of the initial detection model except the feature detector (i.e., the second branch), which is used to fix the weight of the initial detection model. The performance of the initial detection model is used as an indicator to train the enhanced model, so as to be closer to the characteristics of the initial detection model.

[0108] S160: Input the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data.

[0109] The target detection results corresponding to the historical image data may include: the distribution location of the target in the historical image data and the category of the target. The category of the target may include but is not limited to: biological debris, floating equipment, abandoned items, etc.

[0110] S170. Using a preset loss function, fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain a target detection model.

[0111] Among them, a preset loss function as shown in formula (5) can be used to calculate the loss value between the target detection result corresponding to the historical image data and the preset detection result corresponding to the historical image data, so as to fine-tune the parameters of the first branch and the second branch of the initial detection model to obtain the target detection model that best matches the image compression model and has the best performance.

[0112] In this embodiment, a model training sample is obtained, and the model training sample is used to describe the historical image data collected of the unmanned sea surface cluster; based on the model training sample, a preset detection model is trained to obtain an initial detection model, and the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction; the model training sample is input into the first branch of the initial detection model to obtain the image extraction feature; based on the model training sample and the image extraction feature, a preset compression model is trained to obtain the image compression model; the model training sample is input into the image compression model to obtain the image compression feature; the image compression feature is input into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data; a preset loss function is used to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection result corresponding to the historical image data to obtain the target detection model. In this way, the non-feature extraction part of the initial detection model is trained by the image compression feature output by the image compression model, and a target detection model that best matches the image compression model and has better performance is obtained, which effectively improves the detection accuracy of the target detection model.

[0113] Figure 2 FIG. 1 is a flow chart of a method for detecting a target of an unmanned sea surface cluster provided by an embodiment of the present disclosure. Figure 2 As shown in FIG. 1 , the specific process of the target detection method of the sea surface unmanned cluster includes:

[0114] S210: Acquire the collected current image data of the unmanned sea surface cluster.

[0115] When collecting the current image data of the unmanned sea surface cluster, the unmanned sea surface cluster can be photographed by a designated camera device (such as a camera) to obtain a natural light picture. Accordingly, the current image data of the unmanned sea surface cluster is a natural light picture obtained by photographing the unmanned sea surface cluster.

[0116] S220: Input the current image data into the target detection model, and determine the target detection result of the current image data based on the output result of the target detection model.

[0117] The target detection result of the current image data includes: the distribution position and category of the target object in the current image data. The target detection model is trained based on the training method of the target detection model in the above embodiment. Thus, the accuracy of sea surface target detection is effectively improved.

[0118] In addition, this embodiment also provides a test example of the target detection model. During the test, the test set is used to test the target detection model, and the target detection performance of the unmanned marine cluster image under different compression rates is calculated, and a visual target detection result is given.

[0119] Example 1: With hyperparameter λ=100, mAP (mean Average Precision, the average value of AP in each category) is used to measure the performance of target detection. For 900 images in the test set with a total size of 28.6MB, the mAP achieved by target detection on the original image is 85.2%, while the method of this embodiment can compress the total size of the test set to 6.7MB, and achieve an mAP of 87.3%; and the total size of the test set images obtained after compression using VVC (Versatile Video Coding) is 19.5MB, and the mAP achieved is 84.9%. By comparison, it can be concluded that while ensuring the target detection performance, this embodiment also provides better compression performance.

[0120] Example 2: With hyperparameter λ=150, for 900 images in the test set with a total size of 28.6MB, the mAP achieved by object detection on the original image is 85.2%, while the method of this embodiment can compress the total size of the test set to 5.9MB, achieving an mAP of 85.6%; and the total size of the test set images obtained after VVC compression is 19.5MB, achieving an mAP of 84.9%. By comparison, it can be concluded that while ensuring the object detection performance, this embodiment also provides better compression performance.

[0121] Example 3: With hyperparameter λ=70, for 900 images in the test set with a total size of 28.6MB, the mAP achieved by object detection on the original image is 85.2%, while the method of this embodiment can compress the total size of the test set to 9.5MB, and achieve an mAP of 87.9%; while the total size of the test set images obtained after VVC compression is 19.5MB, and the mAP achieved is 84.9%. By comparison, it can be concluded that while ensuring the object detection performance, this embodiment also provides better compression performance.

[0122] Example 4: With hyperparameter λ=50, for 900 images in the test set with a total size of 28.6MB, the mAP achieved by object detection on the original image is 85.2%, while the method of this embodiment can compress the total size of the test set to 10.1MB, and achieve an mAP of 88.0%; while the total size of the test set images obtained after VVC compression is 19.5MB, and the mAP achieved is 84.9%. By comparison, it can be concluded that while ensuring the object detection performance, this embodiment also provides better compression performance.

[0123] In summary, this embodiment inputs the image to be detected in the cloud into the encoder, extracts the potential representation in the pixel space, quantizes the representation, uses uniform noise instead of quantization during training, performs rounding operations to directly discretize, uses the distribution prior information obtained by the super prior codec to entropy encode the potential representation, inputs the obtained potential representation into the conversion module, uses distillation learning to make the converted features close to the features extracted by the teacher network, and inputs it into the target detection model for target detection. Using distillation learning, the obtained model parameters are small, and higher detection performance can be achieved at a lower compression rate.

[0124] The first branch of the trained initial detection model is used as the teacher model, and the image compression model to be trained is used as the student model, so that the compressed features of the student model are close to the features extracted by the initial detection model, thereby achieving efficient image compression with low compression rate and high detection accuracy for target detection of unmanned sea surface clusters.

[0125] Figure 3 A structural schematic diagram of a target detection model training device provided in this embodiment, the target detection model training device may include: a first acquisition module 310, a training module 320, a first determination module 330, a second determination module 340, a third determination module 350, a fourth determination module 360 ​​and a fine-tuning module 370.

[0126] The first acquisition module 310 is used to acquire model training samples, where the model training samples are used to describe the historical image data collected of the unmanned sea surface cluster.

[0127] The training module 320 is used to train the preset detection model based on the model training samples to obtain an initial detection model. The initial detection model is used to detect the distribution position and category of the target object in the image data. The initial detection model includes: a first branch and a second branch. The first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction.

[0128] The first determination module 330 is used to input the model training sample into the first branch of the initial detection model to obtain image extraction features.

[0129] The second determination module 340 is used to train the preset compression model based on the model training samples and the image extraction features to obtain the image compression model.

[0130] The third determination module 350 is used to input the model training samples into the image compression model to obtain image compression features.

[0131] The fourth determination module 360 ​​is used to input the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data.

[0132] The fine-tuning module 370 is used to use a preset loss function to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

[0133] In this embodiment, optionally, the second determination module 340 includes: an acquisition unit, a first determination unit, a quantization unit, a second determination unit, a third determination unit, a fourth determination unit and a training unit.

[0134] The acquisition unit is used to acquire a preset compression model, which includes: an autoencoder, a super-a priori encoder, a super-a priori decoder and a converter.

[0135] The first determination unit is used to input the model training sample into the autoencoder of the preset compression model to obtain pixel space representation data.

[0136] The quantization unit is used to perform quantization processing on the pixel space representation data to obtain pixel space quantization data.

[0137] The second determination unit is used to input the pixel space quantization data into a super a priori encoder of a preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data.

[0138] The third determination unit is used to input the super a priori coded data into a super a priori decoder of a preset compression model to obtain super a priori feature information, where the super a priori feature information is obtained by extracting features from a pixel domain potential representation obtained by decoding the super a priori coded data.

[0139] The fourth determination unit is used to input the super-prior feature information into a converter of a preset compression model to obtain high-dimensional spatial data features.

[0140] The training unit is used to optimize the training of the autoencoder, super prior encoder, super prior decoder and converter of the preset compression model by using a preset loss function based on high-dimensional spatial data features and image extraction features to obtain an image compression model.

[0141] In this embodiment, optionally, it also includes: a sixth determination module and an update module.

[0142] The sixth determination module is used to input the image compression features into the post-enhancement model to obtain image enhancement features. The post-enhancement model is used to perform feature enhancement processing on the input features.

[0143] The updating module is used to update the image compression feature based on the feature distinction data of the image enhancement feature and the image compression feature.

[0144] In this embodiment, optionally, the training unit is also used to input the high-dimensional spatial data features output by the converter of the preset compression model after training into the second branch of the initial detection model to obtain feature enhancement results; use a preset loss function to optimize the preset enhancement model based on the feature enhancement results to obtain a post-enhancement model.

[0145] In this embodiment, optionally, the first determining unit is specifically configured to:

[0146] Acquire potential representation data that meets the data continuity condition in the pixel space representation data; use additive uniform noise to quantize the potential representation data that meets the data continuity condition in the pixel space representation data to obtain pixel space quantized data.

[0147] In this embodiment, optionally, the second determining unit is specifically configured to:

[0148] Obtain the pixel conditional probability generated by each pixel value in the pixel space quantization data; perform cumulative multiplication operation on the pixel conditional probability generated by each pixel value in the pixel space quantization data to obtain the image conditional probability corresponding to the pixel space quantization data; based on the image conditional probability, encode the pixel space quantization data to obtain the super priori encoded data corresponding to the pixel space quantization data.

[0149] The training device of the target detection model provided in the present disclosure can execute the above method embodiments. Its specific implementation principles and technical effects can be found in the above method embodiments, and the present disclosure will not repeat them here.

[0150] Figure 4 This is a schematic structural diagram of a target detection device for an unmanned sea surface cluster provided in this embodiment. The target detection device for an unmanned sea surface cluster may include: a second acquisition module 410 and a fifth determination module 420 .

[0151] The second acquisition module 410 is used to acquire the current image data of the unmanned sea surface cluster.

[0152] The fifth determination module 420 is used to input the current image data into the target detection model, and determine the target detection result of the current image data based on the output result of the target detection model. The target detection result of the current image data includes: the distribution position and category of the target object in the current image data.

[0153] The target detection model is trained by the target detection model training method in the above embodiment.

[0154] The target detection device for unmanned sea surface clusters provided in the present disclosure can execute the above method embodiments. Its specific implementation principles and technical effects can be found in the above method embodiments, and the present disclosure will not repeat them here.

[0155] The present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0156] The computer device includes a memory 510 and a processor 520 that are connected to each other through a system bus. It should be noted that the figure only shows a computer device with a memory 510 and a processor 520, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0157] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0158] The memory 510 includes at least one type of readable storage medium, and the readable storage medium includes a non-volatile memory or a volatile memory, for example, a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc., and the RAM may include a static RAM or a dynamic RAM. In some embodiments, the memory 510 may be an internal storage unit of a computer device, for example, a hard disk or a memory of the computer device. In other embodiments, the memory 510 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, or a flash card (FlashCard) equipped on the computer device. Of course, the memory 510 may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory 510 is generally used to store the operating system and various application software installed on the computer device, such as the program code of the above method. In addition, the memory 510 may also be used to temporarily store various data that have been output or are to be output.

[0159] The processor 520 is generally used to perform the overall operation of the computer device. In this embodiment, the memory 510 is used to store program codes or instructions, the program code includes computer operation instructions, and the processor 520 is used to execute the program codes or instructions stored in the memory 510 or process data, such as running the program code of the above method.

[0160] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus system can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0161] Another embodiment of the present application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads a computer-readable program code stored in the computer-readable medium, so that the processor can execute the functional actions specified in each step or a combination of steps in the above method; and generate a device for implementing the functional actions specified in each block or a combination of blocks in the block diagram.

[0162] Computer-readable media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any appropriate combination of the foregoing, the memory is used to store program codes or instructions, the program codes include computer operating instructions, and the processor is used to execute the program codes or instructions of the above methods stored in the memory.

[0163] For the definitions of memory and processor, please refer to the description of the aforementioned computer device embodiment and will not be repeated here.

[0164] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0165] Each functional unit or module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0166] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0167] In the claims, any reference symbols placed between brackets shall not be construed as limiting the claims. The word "comprising" described in the present application does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented with the aid of hardware comprising several different elements and with the aid of a suitably programmed computer. In a unit claim that lists a number of devices, several units of these devices may be embodied by the same hardware item. The use of first, second, and third, etc. does not indicate any order, and these words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

[0168] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for training a target detection model, characterized in that: include: Acquire a model training sample, where the model training sample is used to describe historical image data collected of an unmanned sea surface cluster; Based on the model training samples, a preset detection model is trained to obtain an initial detection model, wherein the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, wherein the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction; Inputting the model training sample into the first branch of the initial detection model to obtain image extraction features; Training a preset compression model based on the model training samples and the image extraction features to obtain an image compression model; Inputting the model training samples into the image compression model to obtain image compression features; Inputting the image compression feature into the second branch of the initial detection model to obtain the target detection result corresponding to the historical image data; A preset loss function is used to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

2. The method according to claim 1, characterized in that The step of training a preset compression model based on the model training sample and the image extraction features to obtain an image compression model includes: Obtaining a preset compression model, wherein the preset compression model includes: an autoencoder, a super-a priori encoder, a super-a priori decoder, and a converter; Inputting the model training sample into the autoencoder of the preset compression model to obtain pixel space representation data; Quantizing the pixel space representation data to obtain pixel space quantized data; Inputting the pixel space quantization data into the super a priori encoder of the preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data; Inputting the super a priori encoded data into the super a priori decoder of the preset compression model to obtain super a priori feature information, wherein the super a priori feature information is obtained by extracting features from a pixel domain potential representation obtained by decoding the super a priori encoded data; Inputting the super-prior feature information into the converter of the preset compression model to obtain high-dimensional spatial data features; The preset loss function is used to optimize and train the autoencoder, the super a priori encoder, the super a priori decoder and the converter of the preset compression model based on the high-dimensional spatial data features and the image extraction features to obtain the image compression model.

3. The method according to claim 2, characterized in that After inputting the model training sample into the image compression model to obtain the image compression feature, the method further includes: Inputting the image compression feature into a post-enhancement model to obtain an image enhancement feature, wherein the post-enhancement model is used to perform feature enhancement processing on the input feature; The image compression feature is updated based on the feature distinction data of the image enhancement feature and the image compression feature.

4. The method according to claim 3, characterized in that The training process of the post-enhancement model includes: Inputting the high-dimensional spatial data features output by the converter of the preset compression model after training into the second branch of the initial detection model to obtain a feature enhancement result; A preset loss function is used to optimize and train the preset enhancement model based on the feature enhancement result to obtain the post-enhancement model.

5. The method according to claim 3, characterized in that: The step of quantizing the pixel space representation data to obtain pixel space quantized data includes: Acquire potential representation data satisfying a data continuity condition in the pixel space representation data; Additive uniform noise is used to quantize the potential representation data satisfying the data continuity condition in the pixel space representation data to obtain the pixel space quantized data.

6. The method according to claim 3, characterized in that The step of inputting the pixel space quantization data into the super a priori encoder of the preset compression model to obtain super a priori encoded data corresponding to the pixel space quantization data includes: Obtaining a pixel conditional probability generated by each pixel value in the pixel space quantization data; Performing a cumulative multiplication operation on the pixel conditional probability generated by each pixel value in the pixel space quantization data to obtain an image conditional probability corresponding to the pixel space quantization data; Based on the image conditional probability, the pixel space quantization data is encoded to obtain the super a priori encoded data corresponding to the pixel space quantization data.

7. A method for detecting unmanned cluster targets on the sea surface, characterized in that: include: Obtain the current image data of the collected unmanned sea surface cluster; Inputting the current image data into a target detection model, and determining a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data; The target detection model is trained based on the target detection model training method described in any one of claims 1 to 6.

8. A training device for a target detection model, characterized in that: include: A first acquisition module is used to acquire a model training sample, where the model training sample is used to describe historical image data collected of an unmanned sea surface cluster; A training module, used to train a preset detection model based on the model training sample to obtain an initial detection model, wherein the initial detection model is used to detect the distribution position and category of the target object in the image data, and the initial detection model includes: a first branch and a second branch, wherein the first branch is used to extract image features, and the second branch is used to identify the distribution position and category of the target object in the image data based on the result of the image feature extraction; A first determination module, used for inputting the model training sample into the first branch of the initial detection model to obtain image extraction features; A second determination module is used to train a preset compression model based on the model training sample and the image extraction feature to obtain an image compression model; A third determination module is used to input the model training sample into the image compression model to obtain image compression features; a fourth determination module, configured to input the image compression feature into the second branch of the initial detection model to obtain a target detection result corresponding to the historical image data; A fine-tuning module is used to use a preset loss function to fine-tune the parameters of the first branch and the second branch of the initial detection model based on the target detection results corresponding to the historical image data to obtain the target detection model.

9. A target detection device for unmanned sea surface clusters, characterized in that: include: The second acquisition module is used to acquire the current image data of the collected sea surface unmanned cluster; a fifth determination module, configured to input the current image data into a target detection model, and determine a target detection result of the current image data based on an output result of the target detection model, wherein the target detection result of the current image data includes: a distribution position and a category of a target object in the current image data; The target detection model is trained based on the target detection model training method described in any one of claims 1 to 6.

10. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the training method of the target detection model as described in any one of claims 1 to 6 is implemented, or the target detection method of the unmanned sea surface cluster as described in claim 7 is implemented.

Images