Target detection model training method and apparatus, detection method and apparatus, device, and medium

Abstract

The present application provides a target detection model training method and apparatus, a detection method and apparatus, a device, and a medium. The target detection model training method comprises: preprocessing each sample image in an unsupervised training image data set, to obtain different types of preprocessed images; performing feature extraction on the different types of preprocessed images, to obtain a plurality of fused image features and target classification features; determining a loss value on the basis of the differences between the plurality of fused image features, and on the basis of the differences between the target classification features and predicted classification features corresponding to the fused image features, and updating model parameters of an image processing model on the basis of the loss value, to obtain a pre-trained image processing model; constructing a target detection model on the basis of the pre-trained image processing model; and using the supervised training image data set to perform supervised training on the target detection model, to obtain a trained target detection model. In this way, convenient and accurate model training is achieved on the basis of a small amount of labeled data in actual application scenarios.

Description

Training method and detection method of target detection model, device, equipment and medium

[0001] Cross-reference to related applications

[0002] The present application claims priority to the Chinese patent application No. 202410658649.6, filed on May 24, 2024, and entitled "Training method and detection method of target detection model, device, equipment and medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the field of computer vision, in particular to a training method and a detection method of a target detection model, a device, an equipment and a medium. BACKGROUND

[0004] In recent years, artificial intelligence is increasingly widely applied in the industrial field, and in actual application scenarios (such as industrial scenarios), data collection is difficult, data labeling is difficult, and sample differences are large. At present, the deep learning mode based on pre-training combined with transfer learning is widely used. This method usually uses the ImageNet dataset for pre-training, and migrates the training parameters to the task model, fine-tunes the task model with a small amount of labeled data in the application scenario, and obtains model parameters specific to the task.

[0005] In the existing transfer learning process, the pre-trained model (such as vision transformer, vit model) completes the unsupervised pre-training of the model in the guidance mode of image reconstruction as an auxiliary task. Since the features used for image reconstruction are often single, it is difficult to accurately reflect the rich semantic information in the image, resulting in unsatisfactory transfer learning effect, which is difficult to meet the task requirements of actual application scenarios.

[0006] Therefore, how to realize convenient and accurate model training based on a small amount of labeled data in the actual application scenario is a technical problem to be solved. SUMMARY

[0007] In view of the above problems, the embodiments of the present application provide a training method and a detection method of a target detection model, a device, an equipment and a medium, so as to overcome the above problems or at least partially solve the above problems.

[0008] In a first aspect, the embodiments of the present application disclose a training method of a target detection model, the method comprising:

[0009] Preprocessing each sample image in an unsupervised training image dataset to obtain different types of preprocessed images;

[0010] respectively, to obtain a plurality of fused image features and target classification features, the fused image features being obtained by fusing image features of different depths;

[0011] According to the differences between the plurality of fused image features, and according to the differences between the target classification features and the corresponding predicted classification features of the fused image features, a loss value is determined, and model parameters of an image processing model are updated according to the loss value to obtain a pre-trained image processing model, the pre-trained image processing model having feature extraction capability and feature classification capability learned based on the fused image features;

[0012] According to the pre-trained image processing model, a target detection model is constructed.

[0013] The target detection model is supervised trained by using a supervised training image dataset to obtain a trained target detection model, the target detection model being used for target detection on images.

[0014] A second aspect of the embodiments of the present application discloses a target detection method, the method comprising:

[0015] Obtaining a to-be-detected image and a target detection model;

[0016] Using the target detection model to perform target detection on the to-be-detected image to obtain a target detection result;

[0017] The target detection model is trained by using the training method of the target detection model of the first aspect of the embodiments of the present application.

[0018] A third aspect of the embodiments of the present application discloses a training device of a target detection model, the device comprising:

[0019] A preprocessing module is configured to preprocess each sample image in an unsupervised training image dataset to obtain different types of preprocessed images;

[0020] A feature extraction module is configured to perform feature extraction on the different types of preprocessed images respectively to obtain a plurality of fused image features and target classification features, the fused image features being obtained by fusing image features of different depths;

[0021] a parameter updating module, configured to determine a loss value according to differences between the plurality of fused image features and according to differences between the target classification feature and a predicted classification feature corresponding to the fused image features, and update model parameters of the image processing model according to the loss value to obtain a pre-trained image processing model, the pre-trained image processing model having a feature extraction capability and a feature classification capability learned based on the fused image features;

[0022] a model construction module, configured to construct a target detection model according to the pre-trained image processing model;

[0023] a model training module, configured to perform supervised training on the target detection model by using a supervised training image dataset to obtain a trained target detection model, the trained target detection model being used for target detection on images.

[0024] A fourth aspect of the embodiments of the present application discloses a target detection device, and the device comprises:

[0025] a obtaining module, configured to obtain a to-be-detected image and a target detection model;

[0026] a detecting module, configured to perform target detection on the to-be-detected image by using the target detection model to obtain a target detection result;

[0027] The target detection model is obtained by training the target detection model training method of the first aspect of the embodiments of the present application.

[0028] A fifth aspect of the embodiments of the present application discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the target detection model training method of the first aspect of the embodiments of the present application or the steps of the target detection method of the second aspect of the embodiments of the present application.

[0029] A sixth aspect of the embodiments of the present application discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the target detection model training method of the first aspect of the embodiments of the present application or the steps of the target detection method of the second aspect of the embodiments of the present application.

[0030] The embodiments of the present application have the following advantages:

[0031] In the embodiment of the present application, each sample image in the unsupervised training image dataset is preprocessed to obtain different types of preprocessed images; and feature extraction is performed on different types of preprocessed images to obtain a plurality of fusion image features and target classification features, the fusion image features being obtained by fusing image features of different depths; then, a loss value is determined according to the differences between the plurality of fusion image features, and according to the differences between the target classification features and the corresponding predicted classification features of the fusion image features, and the model parameters of the image processing model are updated according to the loss value to obtain a pre-trained image processing model, the pre-trained image processing model having feature extraction and classification capabilities learned based on the fusion image features; then, a target detection model is constructed according to the pre-trained image processing model; and the target detection model is supervised trained using a supervised training image dataset to obtain a trained target detection model, the target detection model being used for target detection of images.

[0032] Since the pre-trained image processing model is learned based on the fusion image features, and the fusion image features are obtained by fusing image features of different depths, the fusion image features contain rich semantic information, so that the unsupervised training guided by the fusion image features improves the training accuracy of the image processing model. Moreover, since the target detection model is constructed according to the pre-trained image processing model, only a small amount of labeled data in an application scenario is needed to fine-tune (i.e., supervised training) the target detection model to obtain the trained target detection model. In this way, convenient and accurate model training is achieved based on a small amount of labeled data in an actual application scenario. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0034] FIG. 1 is a step flowchart of a training method of a target detection model according to an embodiment of the present application;

[0035] FIG. 2 is a flowchart of an unsupervised training method of an image processing model according to an embodiment of the present application;

[0036] FIG. 3 is a structural schematic diagram of a first fusion model according to an embodiment of the present application;

[0037] FIG. 4 is a structural schematic diagram of another first fusion model according to an embodiment of the present application;

[0038] FIG. 5 is a structural schematic diagram of a linear decoding layer provided by an embodiment of the present application;

[0039] FIG. 6 is a structural schematic diagram of a target detection model provided by an embodiment of the present application;

[0040] FIG. 7 is a structural schematic diagram of a second target image processing network provided by an embodiment of the present application;

[0041] FIG. 8 is a structural schematic diagram of another target detection model provided by an embodiment of the present application;

[0042] FIG. 9 is a step flowchart of a target detection method provided by an embodiment of the present application;

[0043] FIG. 10 is a structural schematic diagram of a training device of a target detection model provided by an embodiment of the present application;

[0044] FIG. 11 is a structural schematic diagram of a target detection device provided by an embodiment of the present application;

[0045] FIG. 12 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0047] In the existing transfer learning process, the pre-training model generally adopts the mean square error (MAE) method, that is, the unsupervised pre-training of the model is completed in the guidance mode of the image reconstruction as an auxiliary task. Since the features used for image reconstruction are often single and difficult to accurately reflect the rich semantic information in the image, the transfer learning effect is not ideal, and it is difficult to meet the task requirements of the actual application scenarios. Therefore, in order to overcome the limitations of the related art, the applicant proposes the following technical concept: using the fusion image features of different deep image features to guide the unsupervised pre-training, so that the pre-training model (such as an image processing model) learns the feature extraction capability and feature classification capability based on the fusion image features, obtains accurate pre-training parameters, and improves the effect of transfer learning; thereby quickly constructing a specific task processing model (such as a target detection model) based on the pre-training parameters.

[0048] Based on the technical concept, the embodiment of the present application provides a target detection model training method. Referring to FIG. 1, FIG. 1 is a step flow chart of the target detection model training method provided by the embodiment of the present application. As shown in FIG. 1, the method can include steps S110 to S150.

[0049] Step S110: Preprocessing each sample image in the unsupervised training image dataset to obtain different types of preprocessed images.

[0050] In the embodiment of the present application, the unsupervised training image dataset includes image data carrying true value labels and image data not carrying true value labels, and the unsupervised training image dataset is constructed according to a target application scenario (such as an industrial scenario), that is, the unsupervised training image dataset includes a large amount of image data obtained in the target application scenario and / or a similar application scenario. In some embodiments, the unsupervised training image dataset includes image data with consistent information that can be obtained in the target scenario. In other embodiments, the unsupervised training image dataset includes image data with consistent information that can be obtained in the target scenario, and image data with background difference information that can be obtained in a similar scenario to the target scenario.

[0051] The different types of preprocessed images can be mask images, non-mask images, and enhanced images. By processing the sample images into different types of preprocessed images, the model can learn unsupervisedly based on different types of preprocessed images of the same sample image, enriching the feature information learned by the model from the sample image. Thus, the problem of poor adaptability of the target domain and the source domain in the transfer learning process is solved, that is, the problem of large difference between the pre-training images and the images of the actual application scenario to be transferred is solved.

[0052] Step S120: Extracting features from the different types of preprocessed images respectively to obtain a plurality of fusion image features and target classification features, the fusion image features being obtained by fusing image features of different depths.

[0053] In the embodiments of the present application, the fused image features are obtained by fusing image features of different depths, and the fused image features contain rich semantic information, so that subsequent accurate model training is realized based on multiple fused image features. Specifically, different depths of image features of the preprocessed image are extracted, and then the image features of different depths are fused to obtain fused image features. One fused image feature can be obtained for different types of preprocessed images. For example, if the preprocessed image is a mask image, the fused image feature corresponding to the mask image can be extracted, and if the preprocessed image is a non-mask image, the fused image feature corresponding to the non-mask image can be extracted. The target classification feature contains the category information of the image, and the model can learn the classification ability of the feature through the target classification feature.

[0054] Step S130: determining a loss value according to the difference between the multiple fused image features, and according to the difference between the target classification feature and the predicted classification feature corresponding to the fused image feature, and updating the model parameters of the image processing model according to the loss value to obtain a pre-trained image processing model, the pre-trained image processing model having feature extraction ability and feature classification ability learned based on the fused image feature.

[0055] In the embodiments of the present application, the image processing model refers to a network model having an image feature processing function, for example, a vit model. For the training of the image processing model, a text self-editor method is used to separate the fused image features, and an unsupervised training image dataset is used to pre-train the image processing model. In the unsupervised pre-training process, the image processing model learns the feature classification learning feature extraction ability and feature classification ability based on the fused feature image. Specifically, the feature extraction loss value is calculated through the difference between the multiple fused image features, and the feature classification loss value is calculated according to the difference between the target classification feature and the predicted classification feature corresponding to the fused image feature, so as to update the model parameters of the image processing model according to the feature extraction loss value and the feature classification loss value, so as to realize the unsupervised training of the image processing model.

[0056] In the embodiments of the present application, the image processing model is pre-trained unsupervisedly using an unsupervised training image dataset. Since the unsupervised training image dataset is constructed according to the target application scenario, the problem of poor adaptability of the target domain and the source domain in the transfer learning process is solved, that is, the problem of large difference between the pre-training image and the image of the actual application scenario to be transferred is solved, and a more reliable pre-training image processing model is provided for the target application scenario transfer learning, that is, more accurate pre-training parameters of the image processing model are provided.

[0057] Step S140: constructing a target detection model according to the pre-trained image processing model.

[0058] In the embodiments of the present application, the target detection model is used for target detection on images. According to different target application scenarios, the target detection task performed by the target detection model is also different. For example, in the field of automatic driving, the target detection model is used to detect vehicles, pedestrians and the like in the driving environment; in the field of quality management, the target detection model is used to detect defects and defective products of products.

[0059] Specifically, according to the pre-trained image processing model, the target detection model is constructed, including: according to the image processing model, an initial target detection model is constructed; model parameters of the pre-trained image processing model are loaded into the initial target detection model to obtain a target detection model. In this way, the target detection model construction is quickly completed according to the pre-trained image processing model, that is, the pre-training parameters are migrated to the target detection model, so that a trained target detection model can be obtained by subsequently fine-tuning the target detection model with a small amount of labeled data in the application scenario.

[0060] Step S150: performing supervised training on the target detection model using a supervised training image dataset to obtain a trained target detection model, the target detection model being used for target detection on images.

[0061] In the embodiments of the present application, the supervised training image dataset is constructed according to the unsupervised training image dataset. The supervised training image dataset can be a subset of the unsupervised training image dataset, and the supervised training image dataset includes all image data carrying true value labels or part of the image data carrying true value labels in the unsupervised training image dataset.

[0062] Specifically, performing supervised training on the target detection model using a supervised training image dataset to obtain a trained target detection model includes: constructing a supervised training image dataset according to the unsupervised training image dataset, each sample image in the supervised training image dataset carrying a true value label; and performing supervised training on the target detection model using the supervised training image dataset to obtain a trained target detection model.

[0063] In the embodiments of the present application, the supervised training image dataset is constructed according to the unsupervised training image dataset. The image data can be dynamically enhanced by using a mosaic data enhancement method, and the image can be balanced in class by sample class resampling, and finally the supervised training image dataset is obtained. Since the supervised training image dataset is constructed according to the unsupervised training image dataset, the problem of poor adaptability between the target domain and the source domain in the transfer learning process is solved, so that the target detection model can be fine-tuned using the supervised training image dataset to quickly obtain a trained target detection model.

[0064] In summary, in the embodiments of the present application, since the pre-trained image processing model is learned based on the fused image features, and the fused image features are obtained by fusing image features of different depths, the fused image features contain rich semantic information, and thus the training accuracy of the image processing model is improved based on the fused image features to guide unsupervised training. Moreover, since the target detection model is constructed according to the pre-trained image processing model, only a small amount of labeled data in an application scenario is needed to fine-tune (i.e., supervised training) the target detection model to obtain the trained target detection model. In this way, convenient and accurate model training is achieved based on a small amount of labeled data in an actual application scenario.

[0065] The present application will be described in detail below in combination with the above-mentioned implementation steps.

[0066] In the embodiments of the present application, in order to improve the adaptability of the image features to the migrated target application scenario, a text self-editor method is used to separate the fused image features, and an unsupervised training image dataset is used to perform unsupervised pre-training on the image processing model.

[0067] As shown in FIG. 2, the unsupervised pre-training of the image processing model using the unsupervised training image dataset includes steps A1 to A5:

[0068] Step A1: Preprocessing each sample image in the unsupervised training image dataset to obtain different types of preprocessed images, including mask images, non-mask images, and enhanced images.

[0069] In the embodiments of the present application, the mask image refers to an image obtained by masking part of the sample image, the non-mask image does not mask the sample image, and the enhanced image refers to an image obtained by enhancing the sample image. Specifically, the sample image is processed into a mask image and a non-mask image by randomly taking a mask for each sample image.

[0070] Step A2: inputting the enhanced image into a teacher model for processing to obtain target classification features.

[0071] The teacher model refers to a real application scenario image text model (DALL-E), which is used as a teacher model in the feature guidance process to make up for the lack of image-text sample pair information in the target application scenario (such as an industrial scenario).

[0072] Step A3: inputting the non-mask image into a first image processing model for feature extraction to obtain first image features of different depths, and inputting the first image features of different depths into a first fusion model for fusion processing to obtain first fused image features.

[0073] Step A4: inputting the mask image into a second image processing model to perform feature extraction, obtaining second image features of different depths, and inputting the second image features of different depths into a second fusion model to perform fusion processing, to obtain second fusion image features.

[0074] The second image processing model is a model that is the same as the first image processing model in structure and different from the first image processing model in model parameter updating manner; and the second fusion model is a model that is the same as the first fusion model in structure and different from the first fusion model in model parameter updating manner.

[0075] In the embodiments of the present application, the first fusion image features refer to features obtained by fusing image features of different depths, and the second fusion image features refer to features obtained by fusing image features of different depths. The first fusion image features and the second fusion image features both contain rich semantic information, and thus subsequent model learning based on the first fusion image features and the second fusion image features.

[0076] The first image processing model refers to a network model used for image processing, and the first image processing model can output multiple image features of different depths. The second image processing model is the same as the first image processing model in structure, but the model parameter updating of the second image processing model considers smooth constraint. The first image processing model and the second image processing model are different in model parameter updating manner.

[0077] The first fusion model refers to a multi-level feature map fusion network. The first image features fuse multiple image features of different depths, and the fused image features have semantic information of different levels (depths). The second fusion model is the same as the first fusion model in structure. Similarly, the model parameter updating of the second fusion model considers smooth constraint. The first fusion model and the second fusion model are different in model parameter updating manner.

[0078] Step A5: calculating a feature extraction loss value according to the difference between the first fusion image features and the second fusion image features, and calculating a feature classification loss value according to the difference between the target classification features and the predicted classification features corresponding to the first fusion image features, and updating the model parameters of the image processing model according to the feature extraction loss value and the feature classification loss value.

[0079] Specifically, the feature extraction loss value is calculated according to the difference between the first fusion image feature and the second fusion image feature, including: performing regression processing on the first fusion image feature to obtain a regression processing image feature, and performing mean square error calculation on the regression processing image feature and the second fusion image feature to obtain the feature extraction loss value. The feature classification loss value is calculated according to the difference between the target classification feature and a predicted classification feature corresponding to the first fusion image feature, including: performing regression processing on the first fusion image feature to obtain a regression processing image feature, and performing feature decoding on the regression processing image feature to obtain a decoded classification feature, and performing cross-entropy loss calculation on the decoded classification feature and the target classification feature to obtain the feature classification loss value.

[0080] In the embodiments of the present application, the image processing model learns feature extraction capability and feature classification energy based on the fusion image feature. Since the fusion image feature increases the semantic richness of the image feature, it contains semantic information of different scales (depths) at the same time. Moreover, the fusion image feature reduces the feature bias introduced by the auxiliary task (for example, the feature classification task guided for the DALL-E model in the text automatic encoder pre-training algorithm CAE) in the unsupervised pre-training process, so that guiding the unsupervised training based on the fusion image feature improves the training accuracy of the image processing model.

[0081] In the embodiments of the present application, the first image processing model includes a first target image processing model and a second target image processing model. The second target image processing model is an image processing model with different feature map resolution characteristics stacked by multiple small size convolutional layers compared with the first image processing model. The first target image processing model and the second target image processing model output image features in different representation manners, that is, the first target image processing model outputs image features in the form of vectors, and the second target image processing model outputs image features in the form of feature maps, so the fusion manner of the image features is also different.

[0082] The fusion manner of the image features output by the two image processing models will be described below.

[0083] (1) In the case where the first image processing model is the first target image processing model, the first image feature of different depths is a feature vector of different depths, and the first fusion image feature is a fusion feature vector.

[0084] Specifically, the first image features of different depths are input into the first fusion model for fusion processing to obtain first fusion image features, including: inputting the feature vectors of different depths into corresponding linear layers respectively for processing to obtain linear processing feature vectors; inputting each linear processing feature vector into a corresponding weight layer for processing to obtain a weight processing feature vector; and performing weighted summation on a plurality of the weight processing feature vectors to obtain a fusion feature vector.

[0085] In the embodiments of the present application, the first fusion model refers to a multi-level feature map fusion network, and the first image features fuse a plurality of image features of different depths, and the fused image features have semantic information of different levels (depths). At this time, the first fusion model includes D linear layers and D weight layers.

[0086] For example, FIG. 3 is a structural schematic diagram of a first fusion model provided by an embodiment of the present application, taking the fusion feature layer depth as (3, 5, 7, 12) as an example, and assuming that the image features of specified depths from shallow to deep are f0, f1, f2, and f3, the first fusion model is composed of four linear layers (layer0, layer1, layer2, layer3) and four weight parameters (w0, w1, w2, w3), the input dimension number and the output dimension number of each linear layer are consistent with the feature set dimension of the first target image processing model, and the initial value of each weight parameter is 1. The calculation method of the fusion feature vector is as follows:

[0087] wherein, w i represents the i-th weight layer, layer i represents the i-th linear layer, f i represents the i-th layer image feature.

[0088] In the embodiments of the present application, the image feature fusion is completed by the first fusion model without changing the feature dimension, and the linear layer parameters and the weight parameters are adaptively adjusted through network training, so that the feature separation and feature extraction capability of the image processing network in the pre-training process is further improved through the first fusion model.

[0089] (2) In the case where the first image processing model is a second target image processing model, the first image features of different depths are feature maps of different depths, and the first fusion image features are fusion feature maps.

[0090] Specifically, the first image features of different depths are input into the first fusion model for fusion processing to obtain first fusion image features, including: inputting the feature maps of different depths into corresponding deconvolution layers or direct connection layers respectively for processing to obtain deconvolution processing feature maps; inputting each deconvolution processing feature map into a corresponding weight layer for processing to obtain a weight processing feature map; and performing weighted summation on a plurality of the weight processing feature maps to obtain a fusion feature map.

[0091] In the embodiments of the present application, the first fusion model refers to a multi-level feature map fusion network, and the first image features fuse a plurality of image features of different depths, and the fused image features have semantic information of different levels (depths). At this time, the first fusion model includes C deconvolution layers, 1 direct connection layer, and (C+1) weight layers, wherein the direct connection layer corresponds to the last layer of feature maps of the second target image processing model, and the deconvolution layers convert a plurality of shallow feature maps into feature maps with the same size and dimension as the last layer of feature maps by setting different up-sampling rates.

[0092] For example, FIG. 4 is a structural schematic diagram of another first fusion model provided by the embodiments of the present application, taking the fusion feature layer depth as (3, 5, 7, 12) as an example, and assuming that the image features of specified depths are f0, f1, f2, and f3 from shallow to deep, the first fusion model is composed of 3 deconvolution layers, 1 direct connection layer, and 4 weight parameters, the input dimension number and the output dimension number of each deconvolution layer are consistent with the feature set dimension of the second target image processing model, and the initial value of each weight parameter is 1.

[0093] In the embodiments of the present application, the second fusion model completes image feature fusion without changing the feature dimension, and the deconvolution layer parameters and the weight parameters are adaptively adjusted through network training, so that the feature separation and feature extraction capability of the image processing network in the pre-training process is further improved through the first fusion model.

[0094] In the embodiments of the present application, after obtaining the pre-trained image processing model, a target detection model is constructed according to the pre-trained image processing model. Specifically, an initial target detection model is constructed according to the image processing model, and model parameters of the pre-trained image processing model are loaded into the initial target detection model to obtain a target detection model. Since the first image processing model includes the first target image processing model and the second target image processing model, the second target image processing model is an image processing model with different feature map depth resolution characteristics stacked by a plurality of small size convolution layers compared with the first image processing model. The structures of the first target image processing model and the second target image processing model are different, and correspondingly, the ways of constructing the initial target detection model based on the first target image processing model and the second target image processing model are also different.

[0095] The structures of the first target image processing model and the second target image processing model and the manner of constructing the initial target detection model are described below.

[0096] (1) In the case where the image processing model is the first target image processing model, the initial target detection model is constructed according to the image processing model, including steps B1 to B2:

[0097] Step B1: connecting the first target image processing model with a feature pyramid structure to obtain a backbone network of the initial target detection model, the first target image processing model being used to output image features of different depths, and the feature pyramid structure being used to make the image features of different depths have different resolutions.

[0098] Step B2: connecting the backbone network with a detection head structure to obtain the initial target detection model, the detection head structure being used to perform target detection according to the output features of the backbone network.

[0099] In the embodiments of the present application, the target detection model only relies on the first target image processing model as a feature extraction network. Considering that the resolutions of the image features of different depths output by the first target image processing model are the same, the first target image processing model is connected with a feature pyramid structure to serve as a backbone network of the target detection model, and the feature pyramid structure is used to make the image features of different depths output by the backbone network have different resolutions. In this way, the backbone network is constructed through the feature pyramid structure, and the problem of incompatibility between the first target image processing model and the traditional faster-rcnn is solved.

[0100] Specifically, the feature pyramid structure includes L model layers, the value of L being equal to the number of features output by the first target image processing model; each model layer includes 1 direct connection layer, K deconvolution layers, and (3-K) convolution layers, and the value of K is greater than or equal to 1.

[0101] For example, taking the first target image processing model with an output image feature number of 4 as an example, the output image features are set as f1, f2, f3, and f4 from shallow to deep, and the feature pyramid structure (backbone-fpn) is composed of 4 (i.e., L is 4) model layers, each of which includes 1 direct connection layer, K deconvolution layers, and (3-K) convolution layers; when the patchsize of the first target image processing model is greater than or equal to 14, K is 2, the direct connection layer corresponds to f2, the down-sampling rate of the convolution layer corresponding to f1 is 2, the up-sampling rate of the deconvolution layer corresponding to f3 is 2, and the up-sampling rate of the deconvolution layer corresponding to f4 is 4; when the patchcsize is less than 14, K is 1, the direct connection layer corresponds to f3, the down-sampling rate of the convolution layer corresponding to f1 is 4, the down-sampling rate of the convolution layer corresponding to f2 is 2, and the up-sampling rate of the deconvolution layer corresponding to f4 is 2.

[0102] Further, in order to improve the dimension of the image features, the method further includes step B3:

[0103] Step B3: A linear decoding layer is added between the feature pyramid structure and the detection head structure of the initial target detection model to obtain a first optimized initial target detection model, and the linear decoding layer is used to improve the dimension of the image features.

[0104] In the embodiments of the present application, in order to solve the problem of performance degradation of the target detection model caused by insufficient feature dimension of the small-size first target image processing model (such as the vit-tiny model and the vit-small model), a linear decoding layer is further added in the initial target detection model, the linear decoding layer is located in the backbone network of the initial target detection model, processes the feature vectors output by the feature pyramid structure, and outputs high-dimensional feature vectors. The linear decoding layer can improve the feature dimension, thereby increasing the expression ability of the features of the small-size model backbone network, improving the performance upper limit of the target detection model, and achieving the target detection task only relying on the feature extraction of the first target image processing model.

[0105] As shown in FIG. 5, the linear decoding layer includes L linear layers, and L is greater than or equal to 3; wherein the L linear layers include 1 encoding feature dimension, 1 decoding feature dimension, and (L-2) hidden layer feature dimensions, the decoding feature dimension is greater than the encoding feature dimension, the encoding feature dimension is used to convert the input dimension into the hidden layer dimension, and the decoding feature dimension is used to convert the hidden layer dimension into the output dimension.

[0106] Specifically, the linear decoding layer inputs low-dimensional image features, processes them through the L linear layers, and outputs high-dimensional image features.

[0107] For example, taking the last layer feature map (192 dimensions) of the first target image processing model as an image feature output as an example, the linear decoding layer decoder-linear is composed of multiple linear layers. If the number of linear layers is 3 (that is, the value of L is 3), the hidden layer feature dimension is 2048, the encoding feature dimension is 192, and the decoding feature dimension is 768. The structure of this linear decoding layer is represented as: [linear(192, 2048) -> linear(2048, 2048) -> linear(2048, 768)],

[0108] linear(192, 2048) represents the encoding feature dimension, linear(2048, 2048) represents the hidden layer feature dimension, and linear(2048, 768) represents the decoding feature dimension.

[0109] Correspondingly, loading the model parameters of the pre-trained image processing model into the initial target detection model to obtain a target detection model, including: loading the model parameters of the pre-trained image processing model into the first optimized initial target detection model to obtain a target detection model.

[0110] For example, FIG. 6 is a structural schematic diagram of a target detection model provided by an embodiment of the present application. FIG. 6 illustrates a target detection model constructed in the case where the first target image processing model is a vit model. Specifically, in the target detection model, the image features of four different depths output by the vit model are respectively output to a feature pyramid structure, the output of the feature pyramid structure is connected with a linear decoding layer, and the linear decoding layer is connected with a detection head structure.

[0111] (2) In the case where the image processing model is a second target image processing model, an initial target detection model is constructed according to the image processing model, including step C1:

[0112] Step C1: connecting the second target image processing model as a backbone network of the initial target detection model with a detection head structure to obtain an initial target detection model;

[0113] The second target image processing model is used to output image features of different depths, and the resolutions of the image features of different depths are different. The detection head structure is used to perform target detection according to the output features of the backbone network.

[0114] In the embodiment of the present application, the target detection model only relies on the second target image processing model as a feature extraction network. Since the resolutions of the image features of different depths output by the second target image processing model are different, the second target image processing model is directly connected with a detection head structure to obtain an initial target detection model.

[0115] Specifically, the second target image processing model comprises:

[0116] a convolution integration layer, which replaces a first layer convolution layer of the first target image processing model with two identical replacement convolution layers, a convolution kernel size, a step size, and a dimension of the replacement convolution layers are all smaller than those of the first layer convolution layer, and a down-sampling rate of the convolution integration layer is 1 / 4 of the first layer convolution layer;

[0117] N perception modules, which are split from M perception layers of the first target image processing model, each perception module comprising a perception layer and a down-sampling module, the down-sampling module comprising a plurality of convolution layers, and N is smaller than M.

[0118] For example, if a convolution kernel size of the first layer convolution layer of the first target image processing model is 16*16, a step size is 16, and a dimension is 192, the convolution integration layer replaces the first layer convolution layer with two convolution layers with a convolution kernel size of 3*3, a step size of 2, and a dimension of 96, that is, the convolution integration layer is composed of two convolution layers and the down-sampling rate is 4.

[0119] The 12 (M is 12) perception layers of the first target image processing model are split into 4 (N is 4) perception modules, each perception module being composed of different numbers of perception layers and down-sampling modules, wherein the down-sampling module is composed of three convolution layers, and the parameters of the three convolution layers are as follows: 1*1, 1, input is feature map dimension, output is target integration dimension, 3*3, 2, input and output are both target output dimensions, and 1*1, 1, input and output are both target output dimensions.

[0120] The parameters of the perception modules are as follows: the first module is composed of 2 perception layers and integrates 96-dimensional feature maps into 96-dimensional feature maps; the second module is composed of 2 perception layers and integrates 96-dimensional feature maps into 192-dimensional feature maps; the third module is composed of 6 perception layers and integrates 192-dimensional feature maps into 384-dimensional feature maps; and the fourth module is composed of 2 perception layers and integrates 384-dimensional feature maps into 576-dimensional feature maps. For example, FIG. 7 is a structural schematic diagram of a second target image processing network according to an embodiment of the present application.

[0121] Further, in order to improve the dimension of image features, the method further comprises step C2:

[0122] Step C2: a convolution decoding layer is added between the second target image processing model of the initial target detection model and the detection head structure, to obtain a second optimized initial target detection model, and the convolution decoding layer is used to improve the dimension of image features.

[0123] In the embodiments of the present application, in order to solve the problem of performance degradation of the target detection model caused by insufficient feature dimension of the small-size second target image processing model (such as the tiny-vit model), a convolutional decoding layer is further added in the initial target detection model, which is located in the backbone network of the initial target detection model, processes the feature map output by the second target image processing model, and outputs a high-dimensional feature map. The convolutional decoding layer can improve the feature dimension, thereby increasing the expression ability of the small-size model backbone network features and improving the performance upper limit of the target detection model.

[0124] Specifically, the convolutional decoding layer includes L convolutional layers, and L is greater than or equal to 3; wherein the L convolutional layers include 1 encoding feature dimension, 1 decoding feature dimension, and (L-2) hidden layer feature dimensions, the decoding feature dimension is greater than the encoding feature dimension, the encoding feature dimension is used to convert the input dimension to the hidden layer dimension, and the decoding feature dimension is used to convert the hidden layer dimension to the output dimension.

[0125] Correspondingly, the model parameters of the pre-trained image processing model are loaded into the initial target detection model to obtain a target detection model, including: loading the model parameters of the pre-trained image processing model into the second optimized initial target detection model to obtain a target detection model.

[0126] For example, FIG. 8 is a structural schematic diagram of another target detection model provided by the embodiments of the present application, and FIG. 8 illustrates a target detection model constructed in the case where the second target image processing model is a tiny-vit model. Specifically, in the target detection model, the image features of four different depths output by the tiny-vit model are respectively output to the convolutional decoding layer, and the convolutional decoding layer is connected with the detection head structure.

[0127] In order to better analyze the performance of the training method of the target detection model provided by the present application, the evaluation index of the performance of the target detection model and the ablation experiment are designed to analyze the performance.

[0128] 1. The evaluation index of the performance of the target detection model is determined as: the mean Average Precision (map) index obtained by supervised training based on the model parameters obtained by unsupervised pre-training in the embodiments of the present application, compared with the map index obtained by supervised training based on the open-source coco (coco refers to a data set for image recognition) pre-training weight, which specifically includes:

[0129] (1) Load the open-source coco pre-trained vit weight into the vitdet model (target detection model), perform supervised training of the target detection model, and test the corresponding map index on the test set;

[0130] (2) Pre-train the vit model based on the industrial image using the training method of the target detection model provided in the embodiments of the present application and obtain the weight, load the pre-training weight into the vitdet model for supervised fine-tuning of the target detection model, and test the corresponding map index on the test set.

[0131] 2. Analyze the performance through ablation tests of each module:

[0132] Ablation test of the CAE pre-training algorithm: In the ablation test of the CAE algorithm, the performance of the following four pre-training weights is compared under the condition of controlling the consistency of the remaining structure: pre-training weight based on the open-source coco dataset, weight obtained by using random initialization, CAE pre-training, and MAE pre-training method based on industrial data, respectively.

[0133] First fusion model MFF: In the ablation test of the MFF structure, the performance of the following two pre-training weights is compared under the condition of controlling the consistency of the remaining structure: weight obtained by using the last layer feature map and the fusion feature map of the MFF structure for feature classification pre-training based on industrial data and the CAE pre-training framework, respectively.

[0134] Linear decoding layer decoder-linear: In the ablation test of the decoder-linear, the performance of the following two detection models is compared under the condition of controlling the consistency of the remaining structure: detection index obtained by using decoder-linear and not using decoder-linear during the supervised fine-tuning process of the vitdet model.

[0135] Backbone structure: In the ablation test of the backbone structure, the performance of the following models is compared: detection index obtained by using different backbones under the condition of the same data set, the same training hyperparameters, and the training method of the target detection model provided in the embodiments of the present application.

[0136] The embodiments of the present application also provide a target detection method, as shown in FIG. 9, which is a step flow chart of a target detection method provided by the embodiments of the present application. As shown in FIG. 9, the method can include steps S910 to S920:

[0137] Step S910: obtaining a to-be-detected image and a target detection model;

[0138] Step S920: performing target detection on the to-be-detected image by using the target detection model to obtain a target detection result.

[0139] The target detection model is obtained by training the target detection model according to the training method of the target detection model provided in the embodiments of the present application.

[0140] In the embodiments of the present application, the to-be-detected image is an image obtained in an actual application scenario. In some embodiments, the to-be-detected image is an image collected in real time by an imaging device; in other embodiments, the to-be-detected image is an image collected in advance by an imaging device and stored in a storage device.

[0141] The target detection model is obtained by training the target detection model according to the training method of the target detection model provided in the embodiments of the present application.

[0142] The embodiments of the present application also provide a training device of a target detection model. Referring to FIG. 10, FIG. 10 is a structural schematic diagram of a training device of a target detection model according to an embodiment of the present application. The device comprises:

[0143] A preprocessing module 1010 is configured to perform preprocessing on each sample image in the unsupervised training image dataset to obtain different types of preprocessed images.

[0144] A feature extraction module 1020 is configured to perform feature extraction on the different types of preprocessed images respectively to obtain a plurality of fused image features and target classification features, wherein the fused image features are obtained by fusing image features at different depths.

[0145] A parameter updating module 1030 is configured to determine a loss value according to the differences between the plurality of fused image features, and according to the differences between the target classification features and the predicted classification features corresponding to the fused image features, and update the model parameters of the image processing model according to the loss value to obtain a pre-trained image processing model, wherein the pre-trained image processing model has feature extraction capability and feature classification capability learned based on the fused image features.

[0146] A model construction module 1040 is configured to construct a target detection model according to the pre-trained image processing model.

[0147] The model training module 1050 is configured to perform supervised training on the target detection model by using a supervised training image dataset, to obtain a trained target detection model, wherein the target detection model is configured to perform target detection on an image.

[0148] In the embodiments of the present application, since the pre-trained image processing model is learned based on the fused image features, and the fused image features are obtained by fusing image features of different depths, the fused image features contain rich semantic information, and thus the unsupervised training guided based on the fused image features improves the training accuracy of the image processing model. Moreover, since the target detection model is constructed according to the pre-trained image processing model, only a small amount of labeled data in an application scenario is needed to fine-tune (i.e., perform supervised training) the target detection model, so as to obtain the trained target detection model. In this way, the training device of the target detection model provided in the embodiments of the present application realizes convenient and accurate model training by using a small amount of labeled data in an actual application scenario.

[0149] In an optional embodiment, the different types of preprocessed images include mask images, non-mask images, and enhanced images; and the feature extraction module includes:

[0150] The first feature extraction module is configured to input the enhanced image into the teacher model for processing, to obtain target classification features;

[0151] The second feature extraction module is configured to input the non-mask image into the first image processing model for feature extraction, to obtain first image features of different depths, and input the first image features of different depths into the first fusion model for fusion processing, to obtain first fused image features;

[0152] The third feature extraction module is configured to input the mask image into the second image processing model for feature extraction, to obtain second image features of different depths, and input the second image features of different depths into the second fusion model for fusion processing, to obtain second fused image features;

[0153] The second image processing model is a model that has the same structure as the first image processing model but has a different model parameter updating manner; and the second fusion model is a model that has the same structure as the first fusion model but has a different model parameter updating manner.

[0154] In an optional embodiment, in a case where the first image processing model is a first target image processing model, the first image features of different depths are feature vectors of different depths, and the first fused image features are fused feature vectors; and the first feature module includes:

[0155] a linear processing module, configured to input the feature vectors of different depths into corresponding linear layers respectively for processing to obtain linear processing feature vectors;

[0156] a first weight processing module, configured to input each linear processing feature vector into a corresponding weight layer for processing to obtain a weight processing feature vector;

[0157] a first weighted sum module, configured to perform weighted sum on a plurality of the weight processing feature vectors to obtain a fused feature vector.

[0158] In an optional embodiment, when the first image processing model is a second target image processing model, the first image features of different depths are feature maps of different depths, and the first fused image feature is a fused feature map; the first feature module comprises:

[0159] a deconvolution processing module, configured to input the feature maps of different depths into corresponding deconvolution layers or direct connection layers respectively for processing to obtain deconvolution processing feature maps;

[0160] a second weight processing module, configured to input each deconvolution processing feature map into a corresponding weight layer for processing to obtain a weight processing feature map;

[0161] a second weighted sum module, configured to perform weighted sum on a plurality of the weight processing feature maps to obtain a fused feature map.

[0162] In an optional embodiment, the model construction module comprises:

[0163] an initial model construction module, configured to construct an initial target detection model according to the image processing model;

[0164] a parameter loading module, configured to load model parameters of the pre-trained image processing model into the initial target detection model to obtain a target detection model.

[0165] In an optional embodiment, when the image processing model is a first target image processing model, the initial model construction module comprises:

[0166] a first connection model, configured to connect the first target image processing model and a feature pyramid structure to obtain a backbone network of an initial target detection model, the first target image processing model being configured to output image features of different depths, and the feature pyramid structure being configured to make the image features of different depths have different resolutions;

[0167] a second connection module, configured to connect the backbone network and a detection head structure to obtain the initial target detection model, the detection head structure being configured to perform target detection according to output features of the backbone network.

[0168] In an optional embodiment, the initial model construction module further comprises:

[0169] a first increasing module, configured to increase a linear decoding layer between the feature pyramid structure and the detection head structure of the initial target detection model, to obtain a first optimized initial target detection model, the linear decoding layer being configured to improve image feature dimension;

[0170] The parameter loading module comprises:

[0171] a first parameter loading submodule, configured to load the model parameters of the pre-trained image processing model into the first optimized initial target detection model, to obtain a target detection model.

[0172] In an optional embodiment, in a case where the image processing model is a second target image processing model, the initial model construction module comprises:

[0173] a third connecting module, configured to connect the second target image processing model as a backbone network of the initial target detection model with a detection head structure, to obtain an initial target detection model;

[0174] The second target image processing model is configured to output image features of different depths, and resolutions of the image features of different depths are different, and the detection head structure is configured to perform target detection according to output features of the backbone network.

[0175] In an optional embodiment, the initial model construction module further comprises:

[0176] a second increasing module, configured to increase a convolution decoding layer between the second target image processing model and the detection head structure of the initial target detection model, to obtain a second optimized initial target detection model, the convolution decoding layer being configured to improve image feature dimension;

[0177] The parameter loading module comprises:

[0178] a second parameter loading submodule, configured to load the model parameters of the pre-trained image processing model into the second optimized initial target detection model, to obtain a target detection model.

[0179] In an optional embodiment, the second target image processing model comprises:

[0180] a convolution integration layer, configured to replace a first layer of convolution layers of a first target image processing model with two identical replacement convolution layers, a convolution kernel size, a step, and a dimension of the replacement convolution layers being smaller than those of the first layer of convolution layers, and a down-sampling rate of the convolution integration layer being 1 / 4 of that of the first layer of convolution layers.

[0181] N perception modules, the N perception modules being split from M perception layers of the first target image processing model, each perception module comprising a perception layer and a down-sampling module, the down-sampling module comprising a plurality of convolution layers, N being less than M.

[0182] In an optional embodiment, the feature pyramid structure comprises: L model layers, L being equal to a number of features output by the first target image processing model.

[0183] Each model layer comprises 1 direct connection layer, K deconvolution layers, and (3-K) convolution layers, K being greater than or equal to 1.

[0184] In an optional embodiment, the linear decoding layer comprises: L linear layers, L being greater than or equal to 3.

[0185] The L linear layers comprise 1 encoding feature dimension, 1 decoding feature dimension, and (L-2) hidden layer feature dimensions, the decoding feature dimension being greater than the encoding feature dimension, the encoding feature dimension being used to convert an input dimension into a hidden layer dimension, and the decoding feature dimension being used to convert a hidden layer dimension into an output dimension.

[0186] In an optional embodiment, the model training module comprises:

[0187] A data construction module configured to construct a supervised training image dataset according to the unsupervised training image dataset, each sample image in the supervised training image dataset carrying a true value label.

[0188] A supervised training module configured to perform supervised training on the target detection model by using the supervised training image dataset, to obtain a trained target detection model.

[0189] Embodiments of the present application also provide a target detection device. Referring to FIG. 11, FIG. 11 is a structural schematic diagram of a target detection device according to an embodiment of the present application. The device comprises:

[0190] An acquisition module 1110 configured to acquire a to-be-detected image and a target detection model.

[0191] A detection module 1120 configured to perform target detection on the to-be-detected image by using the target detection model, to obtain a target detection result.

[0192] The target detection model is trained by using the training method of the target detection model.

[0193] The embodiment of the present application further provides an electronic device. Referring to FIG. 12, FIG. 12 is a structural schematic diagram of an electronic device provided by the embodiment of the present application. As shown in FIG. 12, the electronic device 1200 comprises a memory 1210 and a processor 1220, the memory 1210 and the processor 1220 are in communication connection through a bus, and the memory 1210 stores a computer program, the computer program can run on the processor 1220, and then the steps of the training method of the target detection model provided by the embodiment of the present application or the steps of the target detection method provided by the embodiment of the present application are implemented.

[0194] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the training method of the target detection model provided by the embodiment of the present application or the steps of the target detection method provided by the embodiment of the present application.

[0195] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other.

[0196] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices and equipment according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal equipment generate the device for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0197] These computer program instructions can also be stored in a computer readable storage medium to guide the computer or other programmable data processing terminal equipment to work in a specific way, so that the instructions stored in the computer readable storage medium generate a product comprising instruction devices, which implement the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0198] These computer program instructions can also be loaded into a computer or other programmable data processing terminal equipment, so that a series of operation steps are performed on the computer or other programmable terminal equipment to generate a computer implemented process, so that the instructions executed on the computer or other programmable terminal equipment provide steps for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0199] While the preferred embodiments of the application have been described above, it will be recognized and understood that various modifications and changes can be made to the embodiments of the present application by those skilled in the art having the benefit of this detailed description without departing from the scope of the present application. It is therefore intended that the appended claims be construed to include all such embodiments and modifications as fall within the scope of the present application.

[0200] Finally, it should be noted that, in the specification, relational terms such as first and second, and the like, can be used solely to distinguish one entity or action from another entity or action without necessarily implying any actual relationship or order between such entities or actions. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0201] The above describes in detail the training method, detection method, device, equipment and medium of the target detection model provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for training a target detection model, the method comprising: The method comprises: preprocessing each sample image in an unsupervised training image dataset to obtain different types of preprocessed images; performing feature extraction on the different types of preprocessed images respectively to obtain a plurality of fused image features and target classification features, the fused image features being obtained by fusing image features at different depths; determining a loss value according to the differences between the plurality of fused image features, and according to the differences between the target classification features and predicted classification features corresponding to the fused image features, and updating model parameters of an image processing model according to the loss value to obtain a pre-trained image processing model, the pre-trained image processing model having feature extraction and classification capabilities learned based on the fused image features; constructing a target detection model according to the pre-trained image processing model; performing supervised training on the target detection model using a supervised training image dataset to obtain a trained target detection model, the target detection model being used for target detection on images.

2. The method of claim 1, wherein, The different types of preprocessed images include mask images, non-mask images, and enhanced images; performing feature extraction on the different types of preprocessed images respectively to obtain a plurality of fused image features and target classification features, comprising: inputting the enhanced images into a teacher model for processing to obtain target classification features; inputting the non-mask images into a first image processing model for feature extraction to obtain first image features at different depths, and inputting the first image features at different depths into a first fusion model for fusion processing to obtain first fused image features; inputting the mask images into a second image processing model for feature extraction to obtain second image features at different depths, and inputting the second image features at different depths into a second fusion model for fusion processing to obtain second fused image features; wherein the second image processing model is a model having the same structure as the first image processing model but different model parameter updating modes, and the second fusion model is a model having the same structure as the first fusion model but different model parameter updating modes.

3. The method of claim 2, wherein, In the case where the first image processing model is a first target image processing model, the first image features at different depths are feature vectors at different depths, and the first fused image features are fused feature vectors; inputting the first image features at different depths into the first fusion model for fusion processing to obtain the first fused image features, comprising: inputting the feature vectors at different depths into corresponding linear layers respectively for processing to obtain linearly processed feature vectors; inputting each linearly processed feature vector into a corresponding weight layer for processing to obtain a weight-processed feature vector; performing weighted summation on a plurality of the weight-processed feature vectors to obtain a fused feature vector.

4. The method of claim 2, wherein, In the case where the first image processing model is a second target image processing model, the first image features at different depths are feature maps at different depths, and the first fused image features are fused feature maps; The first image features of different depths are input into a first fusion model for fusion processing to obtain first fusion image features, including: The feature maps of different depths are respectively input into corresponding deconvolution layers or direct connection layers for processing to obtain deconvolution processing feature maps; Each deconvolution processing feature map is input into a corresponding weight layer for processing to obtain a weight processing feature map; The weight processing feature maps are weighted and summed to obtain a fusion feature map.

5. The method of claim 1, wherein, According to the pre-trained image processing model, an object detection model is constructed, including: According to the image processing model, an initial object detection model is constructed; The model parameters of the pre-trained image processing model are loaded into the initial object detection model to obtain an object detection model.

6. The method of claim 5, wherein, In the case where the image processing model is a first target image processing model, according to the image processing model, an initial object detection model is constructed, including: The first target image processing model is connected with a feature pyramid structure to obtain a backbone network of the initial object detection model, the first target image processing model is used to output image features of different depths, and the feature pyramid structure is used to make the image features of different depths have different resolutions; The backbone network is connected with a detection head structure to obtain an initial object detection model, and the detection head structure is used to perform object detection according to the output features of the backbone network.

7. The method of claim 6, wherein, The method further includes: A linear decoding layer is added between the feature pyramid structure and the detection head structure of the initial object detection model to obtain a first optimized initial object detection model, and the linear decoding layer is used to improve the dimension of image features; The model parameters of the pre-trained image processing model are loaded into the initial object detection model to obtain an object detection model, including: The model parameters of the pre-trained image processing model are loaded into the first optimized initial object detection model to obtain an object detection model.

8. The method of claim 5, wherein, In the case where the image processing model is a second target image processing model, according to the image processing model, an initial object detection model is constructed, including: The second target image processing model is connected with a detection head structure as a backbone network of the initial object detection model to obtain an initial object detection model; The second target image processing model is used to output image features of different depths, and the resolutions of the image features of different depths are different, and the detection head structure is used to perform object detection according to the output features of the backbone network.

9. The method of claim 8, wherein, The method further includes: A convolution decoding layer is added between the second target image processing model and the detection head structure of the initial object detection model to obtain a second optimized initial object detection model, and the convolution decoding layer is used to improve the dimension of image features; The model parameters of the pre-trained image processing model are loaded into the initial object detection model to obtain an object detection model, including: The model parameters of the pre-trained image processing model are loaded into the second optimized initial object detection model to obtain an object detection model.

10. The method of claim 8, wherein, The second target image processing model includes: a convolution integration layer, which replaces a first layer convolutional layer of the first target image processing model with two identical replacement convolutional layers, a convolution kernel size, a step, and a dimension of the replacement convolutional layers being smaller than those of the first layer convolutional layer, and a down-sampling rate of the convolution integration layer being 1 / 4 of the first layer convolutional layer; N perception modules, which are split from M perception layers of the first target image processing model, each perception module including a perception layer and a down-sampling module, the down-sampling module including a plurality of convolutional layers, and N being less than M.

11. The method of claim 6, wherein, the feature pyramid structure includes L model layers, and L is equal to a number of features output by the first target image processing model; each model layer includes 1 direct connection layer, K deconvolutional layers, and (3-K) convolutional layers, and K is greater than or equal to 1.

12. The method of claim 7 or 9, wherein, the linear decoding layer includes L linear layers, and L is greater than or equal to 3; the L linear layers include 1 encoding feature dimension, 1 decoding feature dimension, and (L-2) hidden layer feature dimensions, the decoding feature dimension is greater than the encoding feature dimension, the encoding feature dimension is used to convert an input dimension into a hidden layer dimension, and the decoding feature dimension is used to convert a hidden layer dimension into an output dimension.

13. The method of claim 1, wherein, performing supervised training on the target detection model using a supervised training image dataset to obtain a trained target detection model, including: constructing a supervised training image dataset according to the unsupervised training image dataset, each sample image in the supervised training image dataset carrying a true value label; performing supervised training on the target detection model using the supervised training image dataset to obtain a trained target detection model.

14. A target detection method characterized by, The method includes: obtaining a to-be-detected image and a target detection model; performing target detection on the to-be-detected image using the target detection model to obtain a target detection result; The target detection model is trained by the training method of the target detection model in any one of claims 1-13.

15. A training device for an object detection model, characterized in that, The device includes: a preprocessing module configured to preprocess each sample image in an unsupervised training image dataset to obtain different types of preprocessed images; a feature extraction module configured to extract features from the different types of preprocessed images to obtain a plurality of fused image features and a target classification feature, the fused image features being obtained by fusing image features at different depths; a parameter updating module configured to determine a loss value based on differences between the plurality of fused image features and based on differences between the target classification feature and a predicted classification feature corresponding to the fused image features, and update model parameters of an image processing model based on the loss value to obtain a pre-trained image processing model, the pre-trained image processing model having feature extraction and classification capabilities learned based on the fused image features; a model construction module configured to construct a target detection model based on the pre-trained image processing model. A model training module is configured to perform supervised training on the target detection model by using a supervised training image dataset, to obtain a trained target detection model, which is configured to perform target detection on an image.

16. A target detection apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire a to-be-detected image and a target detection model; A detection module is configured to perform target detection on the to-be-detected image by using the target detection model, to obtain a target detection result. The target detection model is trained by the training method of the target detection model according to any one of claims 1-13.

17. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the training method of the target detection model according to any one of claims 1-13, or the steps of the target detection method according to claim 14.

18. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the training method of the target detection model according to any one of claims 1-13, or the steps of the target detection method according to claim 14.

Images