Model training method and related product, image processing method and related product

Abstract

The application discloses a model training method and related products, an image processing method and related products. The method comprises the following steps: training a first target detection network based on a first image sample and a second image sample to obtain a second target detection network, wherein the second image sample is obtained by performing random mask processing on the first image sample; training the first target detection network based on the first image sample and a third image sample to obtain a third target detection network, wherein the third image sample is obtained by performing image region disorder processing on the first image sample; training the second target detection network based on a fourth image sample to obtain a fourth target detection network, wherein the fourth image sample comprises an image of a target domain of the second target detection network; and training the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a model training method and related products, an image processing method and related products. BACKGROUND

[0002] Thanks to the rapid development of deep learning technology, the performance of deep learning models is increasingly powerful, and therefore deep learning models are widely used in the field of image processing, including target detection using deep learning models. However, the robustness of deep learning models trained based on target-based model training methods is poor, which in turn leads to low target detection accuracy through deep learning models. SUMMARY

[0003] The present application provides an image processing method and device, an electronic device and a computer readable storage medium.

[0004] In a first aspect, a model training method is provided, which comprises:

[0005] training a first target detection network based on a first image sample and a second image sample to obtain a second target detection network, the second image sample being obtained by randomly masking the first image sample;

[0006] training the first target detection network based on the first image sample and a third image sample to obtain a third target detection network, the third image sample being obtained by randomly reordering image regions of the first image sample;

[0007] training the second target detection network based on a fourth image sample to obtain a fourth target detection network, the fourth image sample comprising images of a target domain of the second target detection network;

[0008] training the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network.

[0009] In a second aspect, an image processing method is also provided, which comprises:

[0010] obtaining a to-be-processed image;

[0011] performing target detection on the to-be-processed image using a sixth target detection network to obtain a target detection result of the to-be-processed image, the sixth target detection network being trained according to any one of the embodiments of the first aspect.

[0012] In a third aspect, a model training device is provided, which comprises:

[0013] The first processing unit is configured to train the first target detection network based on the first image sample and a second image sample to obtain a second target detection network, wherein the second image sample is obtained by performing random mask processing on the first image sample.

[0014] The second processing unit is configured to train the first target detection network based on the first image sample and a third image sample to obtain a third target detection network, wherein the third image sample is obtained by performing image region disorder processing on the first image sample.

[0015] The third processing unit is configured to train the second target detection network based on a fourth image sample to obtain a fourth target detection network, wherein the fourth image sample comprises images in a migration domain of the second target detection network.

[0016] The fourth processing unit is configured to train the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network.

[0017] In a fourth aspect, an image processing apparatus is provided, and the apparatus comprises:

[0018] The acquisition unit is configured to acquire a to-be-processed image.

[0019] The fifth processing unit is configured to perform target detection on the to-be-processed image by using a sixth target detection network to obtain a target detection result of the to-be-processed image, wherein the sixth target detection network is obtained by training according to any one of the embodiments of the first aspect.

[0020] In a fifth aspect, an electronic device is provided, and the electronic device comprises a processor and a memory, wherein the memory is configured to store computer program code, the computer program code comprises computer instructions, and the electronic device executes the method according to any one of the possible implementation manners of the first aspect when the processor executes the computer instructions.

[0021] In a sixth aspect, another electronic device is provided, and the electronic device comprises a processor, a sending apparatus, an input apparatus, an output apparatus and a memory, wherein the memory is configured to store computer program code, the computer program code comprises computer instructions, and the electronic device executes the method according to any one of the possible implementation manners of the first aspect when the processor executes the computer instructions.

[0022] In a seventh aspect, a computer readable storage medium is provided, the computer readable storage medium having stored therein a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method according to the first aspect as such and any possible implementation thereof.

[0023] In an eighth aspect, a computer program product is provided, the computer program product comprising a computer program or instructions that, when executed on a computer, cause the computer to perform the method according to the first aspect as such and any possible implementation thereof.

[0024] In a ninth aspect, an electronic device is provided, comprising a processor and a memory, the memory being configured to store computer program code comprising computer instructions that, when executed by the processor, cause the electronic device to perform the method according to the second aspect as such.

[0025] In a tenth aspect, another electronic device is provided, comprising a processor, a transmitting device, an input device, an output device and a memory, the memory being configured to store computer program code comprising computer instructions that, when executed by the processor, cause the electronic device to perform the method according to the second aspect as such.

[0026] In an eleventh aspect, a computer readable storage medium is provided, the computer readable storage medium having stored therein a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method according to the second aspect as such.

[0027] In a twelfth aspect, a computer program product is provided, the computer program product comprising a computer program or instructions that, when executed on a computer, cause the computer to perform the method according to the second aspect as such.

[0028] It should be understood that the general description above and the detailed description below are only exemplary and explanatory, rather than restrictive, of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the drawings needed to be used in the embodiments of the present application or the background art will be described below.

[0030] The drawings herein are incorporated into the description and form part of the description, which illustrate the embodiments consistent with the present application, and together with the description, serve to explain the technical solutions of the present application.

[0031] Figure 1 A flowchart of a model training method provided by an embodiment of the present application is shown in FIG. 1.

[0032] Figure 2a An image division schematic diagram provided by an embodiment of the present application is shown in FIG. 2.

[0033] Figure 2b A random mask processing schematic diagram provided by an embodiment of the present application is shown in FIG. 3.

[0034] Figure 2c Another random mask processing schematic diagram provided by an embodiment of the present application is shown in FIG. 4.

[0035] Figure 3 A disorder processing schematic diagram provided by an embodiment of the present application is shown in FIG. 5.

[0036] Figure 4 A training schematic diagram provided by an embodiment of the present application is shown in FIG. 6.

[0037] Figure 5 Another training schematic diagram provided by an embodiment of the present application is shown in FIG. 7.

[0038] Figure 6 Still another training schematic diagram provided by an embodiment of the present application is shown in FIG. 8.

[0039] Figure 7 A flowchart of an image processing method provided by an embodiment of the present application is shown in FIG. 9.

[0040] Figure 8 A structural schematic diagram of a model training device provided by an embodiment of the present application is shown in FIG. 10.

[0041] Figure 9 A structural schematic diagram of an image processing device provided by an embodiment of the present application is shown in FIG. 11.

[0042] Figure 10 A hardware structural schematic diagram of a model training device provided by an embodiment of the present application is shown in FIG. 12.

[0043] Figure 11 A hardware structural schematic diagram of an image processing device provided by an embodiment of the present application is shown in FIG. 13. DETAILED DESCRIPTION

[0044] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor fall within the scope of protection of the present application.

[0045] The terms "first", "second", and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. The terms "comprises", "comprising", "includes", "including" and the like are to be construed open-ended, allowing for instances where there are equivalents to the recited elements and steps that are not recited n the claims or appearing since the equivalent structure shows the qualitative performance of the brand.

[0046] For example, a process, a method, a system, a product or an apparatus that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units that are not listed, or optionally further includes other steps or units inherent to such processes, methods, products or apparatuses.

[0047] It should be understood that in the present application, "at least one" refers to one or more, "multiple" refers to two or more, "at least two" refers to two or three and more, and "and / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural.

[0048] The character " / " can represent that the associated objects before and after it are in an "or" relationship, which means any combination of these items, including any combination of single or multiple items.

[0049] For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0050] The character " / " can also represent the division sign in mathematical operations, for example, a / b=a divided by b; 6 / 3=2. "At least one of the following" or the like.

[0051] In this document, the term "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it mutually exclusive or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] The execution subject of the model training method in the embodiments of the present application is a model training device, wherein the model training device can be any kind of electronic device that can execute the technical solutions disclosed in the embodiments of the present application.

[0053] Optionally, the model training apparatus can be one of a mobile phone, a computer, a tablet computer, and a wearable smart device.

[0054] It should be understood that the model training method in the embodiments of the present application can also be implemented by a processor executing computer program code. The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0055] Please refer to Figure 1 , Figure 1 is a flowchart of a model training method provided by the embodiments of the present application.

[0056] 101, based on the first image sample and the second image sample, the first target detection network is trained to obtain the second target detection network.

[0057] In the embodiments of the present application, the first image sample is any image that can be used for target detection training. Specifically, the annotation data of the first image sample includes the position of the target object (i.e. the detection object of target detection), wherein the target object can be any object.

[0058] For example, the target object is a person, the target object is a cow, the target object is a car, the target object is a sheep, and the target object is a car.

[0059] In the embodiments of the present application, the second image sample is obtained by randomly masking the first image sample. Wherein, the random masking refers to erasing any one pixel block in the first image sample, and splicing the pixel blocks other than the erased pixel block.

[0060] In the embodiments of the present application, the first target detection network is a deep learning model used for target detection. For example, the target detection network is a neural network used to implement target detection.

[0061] Using the target detection network to perform target detection on the image can determine whether the image includes the target object and the position of the target object in the image.

[0062] For example, the target object is a cat. Using the target detection network to process the image a, it is determined that the image a includes a cat and the position of the cat in the image a.

[0063] For another example, the target object is an apple. Using the target detection network to process the image b, it is determined that the image b does not include an apple.

[0064] Based on the first image sample and the second image sample, the first target detection network is trained, which is equivalent to making the first target network perform the random mask position prediction self-supervised task (hereinafter referred to as the first self-supervised task) while performing target detection training.

[0065] In this way, the second target detection network can extract information in the image that is conducive to performing the first self-supervised task, and thus the accuracy of target detection can be improved by using the information when performing target detection.

[0066] Specifically, if there is a pixel block with missing image content in the image, the second target detection network can determine the position of the pixel block with missing image content in the image, and thus the robustness of the second target detection network can be improved, thereby improving the accuracy of target detection of the second target detection network.

[0067] 102. Train the first target detection network based on the first image sample and the third image sample to obtain a third target detection network.

[0068] In an embodiment of the present application, the third image sample is obtained by performing image region shuffling on the first image sample. The shuffling includes: in the case of dividing the first image sample into a second preset number of pixel blocks, randomly selecting two pixel blocks from the second preset number of pixel blocks for splicing.

[0069] Training the first target detection network based on the first image sample and the third image sample is equivalent to making the first target network perform a relative position prediction self-supervised task (hereinafter referred to as a second self-supervised task) while performing target detection training.

[0070] In this way, the third target detection network can extract information in the image that is conducive to performing the second self-supervised task, and thus the accuracy of target detection can be improved by using the information when performing target detection.

[0071] Specifically, in the process of performing target detection on the image, the third target detection network can determine the correlation between the pixel blocks according to the relative positions of the pixel blocks, and thus the accuracy of target detection can be improved according to the correlation.

[0072] 103. Train the second target detection network based on a fourth image sample to obtain a fourth target detection network.

[0073] In an embodiment of the present application, the fourth image sample includes images of a target domain of the second target detection network. By performing step 103, the target detection capability of the second target detection network can be transferred from a source domain to which the first image sample belongs to the target domain.

[0074] For example, the source domain is cattle and the target domain is sheep, i.e., the target object in the first image sample is cattle and the target object in the fourth image sample is sheep. By performing step 103, the second target detection network can learn the ability to perform target detection on sheep.

[0075] 104. training the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network.

[0076] By performing step 104, the fourth target detection network can learn the capability of the third target detection network to obtain the fifth target detection network.

[0077] In a possible implementation, the fourth target detection network is trained using the first image sample under the supervision of the third target detection network to obtain the fifth target detection network.

[0078] In the embodiments of the present application, because the information carried by the feature data extracted from the image by the third target detection network is beneficial to improving the accuracy of target detection and performing the second self-supervised task, the fourth target detection network is trained for target detection under the supervision of the third target detection network, so that the information carried by the feature data extracted from the image by the fourth target detection network is beneficial to improving the accuracy of target detection and performing the second self-supervised task.

[0079] Because the information carried by the feature data extracted from the image by the fourth target detection network is also beneficial to performing the first self-supervised task, the image processing apparatus obtains the fifth target detection network by training the fourth target detection network for target detection under the supervision of the third target detection network, so that the information carried by the feature data extracted from the image by the fifth target detection network is beneficial to improving the accuracy of target detection and performing the first self-supervised task.

[0080] The present application first trains the first target detection network under the supervision of the first self-supervised task to obtain the second target detection network capable of performing the first self-supervised task by performing step 101, and trains the first target detection network under the supervision of the second self-supervised task to obtain the third target detection network capable of performing the second self-supervised task by performing step 102.

[0081] Then, the fourth target detection network capable of performing the second self-supervised task and target detection on the target domain is obtained by performing step 103.

[0082] Finally, the fifth target detection network is obtained by making the second target detection network learn the capability of the fourth target detection network performing the second self-supervised task by performing step 104, so that the fifth target detection network has the capability of performing the first self-supervised task and the capability of performing the second self-supervised task.

[0083] Moreover, the optimization direction of the deep learning model under the supervision of the first self-supervised task (hereinafter referred to as a first optimization direction) is different from the optimization direction of the deep learning model under the supervision of the second self-supervised task (hereinafter referred to as a second optimization direction).

[0084] In the embodiments of the present application, since the step 101 and the step 102 are executed independently, the mutual interference between the first self-supervised task and the second self-supervised task can be reduced, that is, the mutual interference between the first optimization direction and the second optimization direction can be reduced.

[0085] Therefore, the effect of the target detection network in performing the first self-supervised task and the effect of the target detection network in performing the second self-supervised task can be improved, and the target detection accuracy of the target detection network can be improved.

[0086] As an optional implementation, the model training apparatus performs the following steps in the process of executing the step 101:

[0087] 201, processing the first image sample by using the first target detection network to obtain a first target detection loss.

[0088] In the embodiments of the present application, the target detection loss (including the first target detection loss, the second target detection loss, the third target detection loss, the fourth target detection loss, the fifth target detection loss, and the sixth target detection loss) is used to train the target detection capability of the deep learning model.

[0089] In a possible implementation, the model training apparatus performs feature extraction processing on the first image sample by using the first target detection network to obtain second feature data. Based on the first image sample and the second feature data, a second interest loss is obtained. Based on the first image sample and the second feature data, a second classification loss is obtained.

[0090] Based on the first image sample and the second feature data, a first regression loss is obtained. Based on the second interest loss, the first regression loss, and the second classification loss, the first target detection loss is obtained.

[0091] Optionally, the model training apparatus performs feature extraction processing on the first image sample by using the backbone of the first target detection network to obtain the second feature data.

[0092] Optionally, the model training apparatus obtains the second interest loss by performing the following steps: determining a first candidate region of the target object from the first image sample based on the second feature data. Then, a first prediction probability that the first candidate region is foreground is determined. According to the first prediction probability and a fourth annotation position of the target object in the first image sample, the second interest loss is obtained.

[0093] In the embodiments of the present application, the first candidate region is a pixel region including the target object determined by the first target detection network. That is, the first target detection network determines a pixel region including the target object from the first image sample according to the second feature data, to obtain the first candidate region of the target object.

[0094] Optionally, the first target detection network determines a candidate box of the target object from the first image sample according to the second feature data, and takes a pixel region surrounded by the candidate box as the first candidate region.

[0095] In the embodiments of the present application, the foreground includes the target object. For example, the target object is a cow, and the first prediction probability represents a probability of existence of the cow in the first candidate region.

[0096] Since the fourth annotation position is the position of the target object in the first image sample, the model training apparatus can determine the foreground region from the first image sample according to the fourth annotation position, and further obtain the second interest loss according to the first difference between the first prediction probability and the foreground region, wherein the second interest loss is positively correlated with the first difference.

[0097] Optionally, the model training apparatus obtains the second classification loss by performing the following steps: determining a first prediction class of the first image sample based on the second feature data. Obtaining the second classification loss according to a second difference between the first prediction class and an annotation class of the first image sample, wherein the second classification loss is positively correlated with the second difference.

[0098] Optionally, the model training apparatus obtains the first regression loss by performing the following steps: determining a third prediction position of the target object in the first image sample based on the second feature data. Obtaining the first regression loss according to a third difference between the third prediction position and the fourth annotation position, wherein the first regression loss is positively correlated with the third difference.

[0099] 202, processing the second image sample by using the first target detection network to obtain the first loss.

[0100] In the embodiments of the present application, the first loss includes the following losses: the first interest loss, the first classification loss and the random mask loss.

[0101] In a possible implementation manner, the model training apparatus performs feature extraction processing on the second image sample by using the first target detection network to obtain the first feature data.

[0102] The first interest loss is obtained based on the second image sample and the first feature data. The first classification loss is obtained based on the second image sample and the first feature data. The random mask loss is obtained based on the second image sample and the first feature data. The first detection loss is obtained based on the first interest loss, the random mask loss and the first classification loss.

[0103] Optionally, the model training apparatus performs feature extraction processing on the second image sample by using a backbone network of the first target detection network to obtain the first feature data.

[0104] Optionally, the model training apparatus obtains the first interest loss by performing the following steps: determining a second candidate region of the target object from the second image sample based on the first feature data. Then, determining a second prediction probability that the second candidate region is foreground. Obtaining the first interest loss according to the second prediction probability and a fifth annotation position of the target object in the second image sample.

[0105] In the embodiments of the present application, the second candidate region is a pixel region including the target object determined by the first target detection network. That is, the first target detection network determines a pixel region including the target object from the second image sample according to the first feature data to obtain the second candidate region of the target object.

[0106] Optionally, the first target detection network determines a candidate box of the target object from the second image sample according to the first feature data, and takes a pixel region surrounded by the candidate box as the second candidate region.

[0107] In the embodiments of the present application, the foreground includes the target object. For example, the target object is a cow, and the second prediction probability represents a probability that there is a cow in the second candidate region.

[0108] Since the fifth annotation position is the position of the target object in the second image sample, the model training apparatus can determine a foreground region from the second image sample according to the fifth annotation position, and then can obtain the first interest loss according to a fourth difference between the second prediction probability and the foreground region, wherein the first interest loss is positively correlated with the fourth difference.

[0109] Optionally, the model training apparatus obtains the first classification loss by performing the following steps: determining a second predicted class of the second image sample based on the first feature data. Obtaining the first classification loss according to a fifth difference between the second predicted class and an annotation class of the second image sample, wherein the first classification loss is positively correlated with the fifth difference.

[0110] 203、updating parameters of the first target detection network according to the first loss and the first target detection loss to obtain the second target detection network.

[0111] In a possible implementation, the model training apparatus obtains a first total loss by weighting and summing the first target detection loss and the first loss. According to the first total loss, the parameters of the first target detection network are updated until the first total loss converges, and the training of the first target detection network is completed to obtain the second target detection network.

[0112] In this implementation, in the process of making the first total loss converge by updating the parameters of the first target detection network, the first target detection network can learn the ability of target detection and the ability of performing the first self-supervised task.

[0113] As an optional implementation, the model training apparatus obtains the random mask loss by performing the following steps:

[0114] 301. According to the first feature data, a first predicted position of a pixel block processed by the random mask processing in the first image sample is obtained.

[0115] In the embodiments of the present application, the pixel block processed by the random mask processing is a pixel block erased from the first image sample by performing random mask processing on the first image sample.

[0116] For example, performing random mask processing on the first image sample to obtain the second image sample specifically includes: erasing the pixel block A in the first image sample, and splicing the pixel blocks in the first image sample except the pixel block A to obtain the second image sample. At this time, since the pixel block erased by the random mask processing is the pixel block A, the pixel block processed by the random mask processing is the pixel block A.

[0117] The first target detection network predicts the position of the pixel block processed by the random mask processing in the first image sample according to the first feature data, to obtain the first predicted position.

[0118] 302. According to the difference between the first predicted position and the first labeled position of the pixel block processed by the random mask processing in the second image sample, the random mask loss is obtained.

[0119] The difference between the first predicted position and the first labeled position of the pixel block processed by the random mask processing in the second image sample is referred to as a fourth difference. The random mask loss is positively correlated with the fourth difference.

[0120] In the process of training the first target detection network, the parameters of the first target detection network are updated according to the random mask loss, so that the first target detection network can learn the ability of performing the first self-supervised task.

[0121] As an optional implementation, the model training apparatus further performs the following steps:

[0122] 401. Crop a target object from the first image sample to obtain a fifth image sample.

[0123] 402. Perform random mask processing on the fifth image sample to obtain the second image sample.

[0124] Since the first image sample can include at least two target objects, the fifth image sample is obtained by cropping a target object from the first image sample, which can expand the training data set.

[0125] For example, the first image sample includes a cow and a sheep, and both the cow and the sheep are target objects. The model training apparatus obtains the fifth image sample by cropping a pixel block of the cow from the first image sample. The model training apparatus obtains the fifth image sample by cropping a pixel block of the sheep from the first image sample.

[0126] As an optional implementation, the model training apparatus performs the following steps in the process of performing step 402:

[0127] 501. Divide the fifth image sample into a first preset number of pixel blocks.

[0128] 502. Randomly select one pixel block from the first preset number of pixel blocks for erasing processing to obtain the second image sample.

[0129] In the embodiments of the present application, the first preset number is a positive integer. Optionally, the first preset number is 9.

[0130] In one possible implementation, the first preset number is 9, Figure 2a As shown, the fifth image sample is divided into 9 pixel blocks.

[0131] As shown, Figure 2a The fifth image sample is divided into a No. 0 pixel block, a No. 1 pixel block, a No. 2 pixel block, a No. 3 pixel block, a No. 4 pixel block, a No. 5 pixel block, a No. 6 pixel block, a No. 7 pixel block, and a No. 8 pixel block.

[0132] As shown, Figure 2b The No. 4 pixel block is erased. Then as shown, Figure 2c The No. 0 pixel block, the No. 1 pixel block, the No. 2 pixel block, the No. 3 pixel block, the No. 5 pixel block, the No. 6 pixel block, the No. 7 pixel block, and the No. 8 pixel block are spliced to obtain the second image sample.

[0133] As an optional implementation, the model training apparatus performs the following steps in the process of performing step 102:

[0134] 601、utilizing the first target detection network to process the first image sample to obtain a second target detection loss.

[0135] In a possible implementation, the model training apparatus performs feature extraction processing on the first image sample by using the first target detection network to obtain fourth feature data.

[0136] Based on the first image sample and the fourth feature data, a fourth interest loss is obtained. Based on the first image sample and the fourth feature data, a fourth classification loss is obtained. Based on the first image sample and the fourth feature data, a second regression loss is obtained.

[0137] Finally, the model training apparatus obtains a second target detection loss based on the fourth interest loss, the second regression loss, and the fourth classification loss.

[0138] Optionally, the model training apparatus performs feature extraction processing on the first image sample by using a backbone network of the first target detection network to obtain the fourth feature data.

[0139] Optionally, the model training apparatus obtains the fourth interest loss by performing the following steps: determining a third candidate region of the target object from the first image sample based on the fourth feature data. Then, determining a third prediction probability that the third candidate region is foreground. According to the third prediction probability and a fourth annotation position of the target object in the first image sample, the fourth interest loss is obtained.

[0140] In the embodiments of the present application, the third candidate region is a pixel region including the target object determined by the first target detection network. That is, the first target detection network determines a pixel region including the target object from the first image sample according to the fourth feature data to obtain the third candidate region of the target object.

[0141] Optionally, the first target detection network determines a bounding box of the target object from the first image sample according to the fourth feature data, and takes a pixel region surrounded by the bounding box as the third candidate region.

[0142] In the embodiments of the present application, the foreground includes the target object. For example, the target object is a cow, and the third prediction probability represents a probability that there is a cow in the third candidate region.

[0143] Since the fourth annotation position is the position of the target object in the first image sample, the model training apparatus can determine a foreground region from the first image sample according to the fourth annotation position, and then can obtain the fourth interest loss according to a fifth difference between the third prediction probability and the foreground region, wherein the fourth interest loss is positively correlated with the fifth difference.

[0144] Optionally, the model training apparatus obtains the second regression loss based on a sixth difference between the fourth predicted position and a fourth labeled position of the target object in the first image sample.

[0145] The second regression loss is positively correlated with the sixth difference.

[0146] Optionally, the model training apparatus obtains the fourth classification loss based on a third predicted class of the first image sample.

[0147] The fourth classification loss is positively correlated with a seventh difference between the third predicted class and a first labeled class of the first image sample.

[0148] 602、using the first target detection network to process the third image sample to obtain a second loss.

[0149] In the embodiments of the present application, the second loss includes a third interest loss, a third classification loss and a relative position prediction loss.

[0150] In one possible implementation, the model training apparatus uses the first target detection network to perform feature extraction processing on the third image sample to obtain third feature data.

[0151] Based on the third image sample and the third feature data, the third interest loss is obtained. Based on the third image sample and the third feature data, the third classification loss is obtained. Based on the third image sample and the third feature data, the relative position prediction loss is obtained.

[0152] Finally, the model training apparatus obtains the first detection loss based on the third interest loss, the relative position prediction loss and the third classification loss.

[0153] Optionally, the model training apparatus uses the backbone network of the first target detection network to perform feature extraction processing on the third image sample to obtain the third feature data.

[0154] Optionally, the model training apparatus obtains the third interest loss by performing the following steps: based on the third feature data, determining a fourth candidate region of the target object from the third image sample.

[0155] A fourth predicted probability that the fourth candidate region is foreground is determined. The third interest loss is obtained based on the fourth predicted probability and a first labeled position of the target object in the third image sample.

[0156] In an embodiment of the present application, the fourth candidate region is a pixel region including the target object determined by the first target detection network. That is, the first target detection network determines a pixel region including the target object from the third image sample according to the third feature data, to obtain the fourth candidate region of the target object.

[0157] Optionally, the first target detection network determines a candidate box of the target object from the third image sample according to the third feature data, and takes a pixel region surrounded by the candidate box as the fourth candidate region.

[0158] In an embodiment of the present application, the foreground includes the target object. For example, the target object is a cow, and the fourth prediction probability represents a probability that a cow exists in the fourth candidate region.

[0159] Since the first annotation position is the position of the target object in the third image sample, the model training apparatus can determine the foreground region from the third image sample according to the first annotation position, and further can obtain the third interest loss according to the eighth difference between the fourth prediction probability and the foreground region, wherein the third interest loss is positively correlated with the eighth difference.

[0160] Optionally, the model training apparatus obtains the third classification loss by performing the following steps: determining a fourth prediction class of the third image sample based on the third feature data.

[0161] According to a ninth difference between the fourth prediction class and a first annotation class of the third image sample, the third classification loss is obtained, wherein the third classification loss is positively correlated with the ninth difference.

[0162] 603、According to the above-mentioned second loss and the above-mentioned second target detection loss, the parameters of the above-mentioned first target detection network are updated to obtain the above-mentioned third target detection network.

[0163] In a possible implementation manner, the model training apparatus obtains a second total loss by weighting and summing the second target detection loss and the second loss.

[0164] According to the second total loss, the parameters of the first target detection network are updated until the second total loss converges, to complete the training of the first target detection network to obtain the third target detection network.

[0165] In this implementation manner, in the process of making the first total loss converge by updating the parameters of the first target detection network, the first target detection network can learn the ability of target detection and the ability of performing the second self-supervised task.

[0166] As an optional implementation manner, the third image sample includes a first pixel block and a second pixel block. The model training apparatus obtains the relative position prediction loss by performing the following steps:

[0167] 701、According to the third feature data, the relative positions of the first pixel block and the second pixel block are predicted to obtain a second predicted position.

[0168] Since the third image sample is obtained by performing the disordering processing on the first image sample, the first image sample also includes the first pixel block and the second pixel block.

[0169] In the embodiments of the present application, the second predicted position represents the relative positions of the first pixel block and the second pixel block in the first image sample.

[0170] 702、According to the difference between the second predicted position and a labeled relative position, a relative position prediction loss is obtained.

[0171] In the embodiments of the present application, the labeled relative position is the relative position of the first pixel block and the second pixel block determined based on the labeled position of the first pixel block in the third image sample and the labeled position of the second pixel block in the third image sample.

[0172] If the difference between the second predicted position and the labeled relative position is referred to as a tenth difference, the relative position prediction loss is positively correlated with the tenth difference.

[0173] In the process of training the first target detection network, the parameters of the first target detection network are updated according to the relative position prediction loss, so that the first target detection network learns the ability to perform the second self-supervised task.

[0174] As an optional implementation, the model training apparatus further performs the following steps:

[0175] 801、The target object is cut from the third image sample to obtain a sixth image sample.

[0176] 802、The sixth image sample is divided into a second preset number of pixel blocks.

[0177] In the embodiments of the present application, the second preset number is a positive integer. Optionally, the first preset number is equal to the second preset number. Optionally, the second preset number is 9.

[0178] 803、Two pixel blocks are randomly selected from the second preset number of pixel blocks to be spliced to obtain the third image sample.

[0179] In a possible implementation, the second preset number is 9, Figure 2a As shown in the figure, the sixth image sample is divided into 9 pixel blocks.

[0180] As shown in the figure, the sixth image sample is divided into 9 pixel blocks. Figure 2aAs shown, the sixth image sample is divided into a 0th pixel block, a 1st pixel block, a 2nd pixel block, a 3rd pixel block, a 4th pixel block, a 5th pixel block, a 6th pixel block, a 7th pixel block, and an 8th pixel block.

[0181] As shown, the 1st pixel block and the 5th pixel block are selected for splicing. At this time, the 1st pixel block is the first pixel block, and the 5th pixel block is the second pixel block. Figure 3

[0182] As an optional implementation, the model training apparatus performs the following steps in the process of performing step 103:

[0183] 901, The target object is cut from the fourth image sample to obtain a seventh image sample.

[0184] 902, The fourth image sample is processed by using the second target detection network to obtain fifth feature data.

[0185] In a possible implementation, the fourth image sample is processed by using the backbone network of the second target detection network to obtain the fifth feature data.

[0186] 903, The seventh image sample is processed by using the second target detection network to obtain sixth feature data.

[0187] In a possible implementation, the seventh image sample is processed by using the backbone network of the second target detection network to obtain the sixth feature data.

[0188] 904, A third target detection loss is obtained based on the fourth image sample and the fifth feature data.

[0189] In a possible implementation, a fifth predicted category of the fourth image sample is obtained according to the fifth feature data. A fifth classification loss is obtained according to an eleventh difference between the fifth predicted category and the category of the fourth image sample, wherein the fifth classification loss is positively correlated with the eleventh difference.

[0190] A fifth candidate region of the target object in the fourth image sample is determined according to the fifth feature data. A fifth predicted probability that the fifth candidate region is a foreground region is obtained. A fifth interest loss is obtained according to the fifth predicted probability and a position of the target object in the fourth image sample.

[0191] A fifth predicted position of the target object in the fourth image sample is determined according to the fifth feature data. A third regression loss is obtained according to a twelfth difference between the fifth predicted position and a position of the target object in the fourth image sample, wherein the third regression loss is positively correlated with the twelfth difference. ​

[0192] The third target detection loss is obtained by weighted sum of the fifth classification loss, the fifth interest loss and the third regression loss.

[0193] 905、Based on the seventh image sample and the sixth feature data, the fourth target detection loss is obtained.

[0194] In a possible implementation, the sixth predicted category of the seventh image sample is obtained according to the sixth feature data. The sixth classification loss is obtained according to a thirteenth difference between the sixth predicted category and the category of the seventh image sample, where the sixth classification loss is positively correlated with the thirteenth difference.

[0195] The sixth candidate region of the target object is determined from the seventh image sample according to the sixth feature data. The sixth predicted probability that the sixth candidate region is a foreground region is obtained. The sixth interest loss is obtained according to the sixth predicted probability and the position of the target object in the seventh image sample.

[0196] The sixth predicted position of the target object in the seventh image sample is determined according to the sixth feature data. The fourth regression loss is obtained according to a fourteenth difference between the sixth predicted position and the position of the target object in the seventh image sample, where the fourth regression loss is positively correlated with the fourteenth difference.

[0197] The fourth target detection loss is obtained by weighted sum of the sixth classification loss, the sixth interest loss and the fourth regression loss.

[0198] 906、According to the third target detection loss and the fourth target detection loss, the parameters of the second target detection network are updated to obtain the fourth target detection network.

[0199] As an optional implementation, the model training apparatus performs the following steps in the process of performing step 104:

[0200] 1001、The target object is cropped from the first image sample to obtain an eighth image sample.

[0201] 1002、The first image sample is subjected to feature extraction processing by using the fourth target detection network to obtain seventh feature data.

[0202] In a possible implementation, the first image sample is subjected to feature extraction processing by using the backbone network of the fourth target detection network to obtain the seventh feature data.

[0203] 1003、The eighth image sample is subjected to feature extraction processing by using the fourth target detection network to obtain eighth feature data.

[0204] In a possible implementation, the backbone network of the fourth target detection network is used to perform feature extraction processing on the eighth image sample, to obtain eighth feature data.

[0205] 1004、Based on the first image sample and the seventh feature data, a fifth target detection loss is obtained.

[0206] In a possible implementation, a seventh predicted category of the first image sample is obtained according to the seventh feature data. A seventh classification loss is obtained according to a fifteenth difference between the seventh predicted category and the category of the first image sample, where the seventh classification loss is positively correlated with the fifteenth difference.

[0207] A seventh candidate region of the target object in the first image sample is determined according to the seventh feature data. A seventh predicted probability that the seventh candidate region is a foreground region is determined, to obtain the seventh predicted probability. A seventh interest loss is obtained according to the seventh predicted probability and a position of the target object in the first image sample.

[0208] A seventh predicted position of the target object in the first image sample is determined according to the seventh feature data. A seventh regression loss is obtained according to a sixteenth difference between the seventh predicted position and the position of the target object in the first image sample, where the seventh regression loss is positively correlated with the sixteenth difference.

[0209] The seventh classification loss, the seventh interest loss, and the seventh regression loss are weighted and summed to obtain the fifth target detection loss.

[0210] 1005、Based on the eighth image sample and the eighth feature data, a sixth target detection loss is obtained.

[0211] In a possible implementation, an eighth predicted category of the first image sample is obtained according to the eighth feature data. An eighth classification loss is obtained according to a seventeenth difference between the eighth predicted category and the category of the first image sample, where the eighth classification loss is positively correlated with the seventeenth difference.

[0212] An eighth candidate region of the target object in the first image sample is determined according to the eighth feature data. An eighth predicted probability that the eighth candidate region is a foreground region is determined, to obtain the eighth predicted probability. An eighth interest loss is obtained according to the eighth predicted probability and a position of the target object in the first image sample.

[0213] An eighth predicted position of the target object in the first image sample is determined according to the eighth feature data. An eighth regression loss is obtained according to an eighteenth difference between the eighth predicted position and the position of the target object in the first image sample, where the eighth regression loss is positively correlated with the eighteenth difference.

[0214] The eighth classification loss, the eighth interest loss, and the eighth regression loss are weighted and summed to obtain a sixth target detection loss.

[0215] 1006、Based on the third target detection network and the eighth image sample, a third loss is obtained.

[0216] In a possible implementation manner, the third target detection network is used to perform feature extraction processing on the eighth image sample to obtain ninth feature data. A nineteenth difference between the eighth feature data and the ninth feature data is obtained, and the third loss is obtained in a positive correlation with the nineteenth difference.

[0217] In this implementation manner, the model training apparatus obtains the third loss according to the nineteenth difference between the eighth feature data and the ninth feature data. In this way, the parameters of the fourth target detection network are updated according to the third loss, so that the feature data extracted from the image by the fourth target detection network is close to the feature data extracted from the image by the third target detection network, in other words, the fourth target detection network is trained based on the third loss, and the fourth target detection network can learn the ability of the third target detection network to extract image features.

[0218] 1007、According to the third loss, the fifth target detection loss, and the sixth target detection loss, the parameters of the fourth target detection network are updated to obtain the fifth target detection network.

[0219] In the embodiments of the present application, the model training apparatus updates the parameters of the fourth target detection network according to the third loss, the fifth target detection loss, and the sixth target detection loss, which not only improves the target detection accuracy of the third target detection network, but also enables the fourth target detection network to learn the ability of the third target detection network to extract image features, that is, enables the fourth target detection network to learn the ability to perform the second self-supervised task.

[0220] Based on the technical solutions provided in the embodiments of the present application, the embodiments of the present application further provide a training method of a target detection network. The training method includes three stages, and the implementation process of the first stage can be referred to Figure 4 .

[0221] As shown in Figure 4 , the target object in the first image sample is intercepted to obtain the fifth image sample. Optionally, the target object in the first image sample is intercepted to obtain a first intercepted image. The size of the first intercepted image is changed to obtain at least one fifth image sample.

[0222] The fifth image sample is subjected to pixel block processing, specifically, the fifth image sample is subjected to random mask processing to obtain a second image sample.

[0223] The backbone network of the first target detection network extracts feature data of the first image sample to obtain second feature data. A region proposal network (RPN) obtains a first prediction probability that the first candidate region is foreground by processing the second feature data.

[0224] The feature data output by the RPN is processed through a full connection layer to determine a third prediction position of the target object in the first image sample and a first prediction class of the first image sample.

[0225] According to the first prediction probability and a fourth labeled position of the target object in the first image sample, a second interest loss is calculated by calculating an interest loss. According to a second difference between the first prediction class and a labeled class of the first image sample, a second classification loss is calculated by calculating a classification loss. According to a third difference between the third prediction position and the fourth labeled position, a first regression loss is calculated by calculating a regression loss.

[0226] In Figure 4 , a second image sample is obtained by performing pixel block processing on the fifth image sample. Specifically, the second image sample is obtained by performing random mask processing on the fifth image sample. The backbone network of the first target detection network extracts feature data of the second image sample to obtain first feature data.

[0227] The mask full connection layer processes the first feature data to predict a position of the pixel block processed by the random mask processing in the second image sample to obtain a first prediction position. According to a difference between the first prediction position and a first labeled position of the pixel block processed by the random mask processing in the second image sample, a random mask loss (i.e., a mask loss in Figure 4 ) is obtained. The first classification loss and the first interest loss can also be obtained according to the first feature data.

[0228] The first interest loss, the second interest loss, the first classification loss, the second classification loss, the random mask loss, and the first regression loss are weighted and summed to obtain a first total loss. According to the first total loss, the parameters of the first target detection network are updated until the first total loss converges to obtain a second target detection network. At this time, the backbone network of the second target detection network is the mask backbone network in Figure 4 .

[0229] As shown in Figure 4 , the sixth image sample can also be obtained by intercepting the target object in the first image sample. Optionally, the target object is intercepted from the first image sample to obtain a second intercepted image. At least one sixth image sample is obtained by performing a size transformation on the second intercepted image.

[0230] The sixth image sample is processed by pixel block processing, and specifically, the sixth image sample is processed by disordering to obtain a third image sample.

[0231] The backbone network of the first target detection network extracts feature data of the first image sample to obtain fourth feature data. The RPN processes the fourth feature data to obtain a third prediction probability that the third candidate region is foreground.

[0232] The feature data output by the RPN is processed by the full connection layer to determine a fourth predicted position of the target object in the first image sample and a third predicted class of the first image sample.

[0233] According to the third prediction probability and the fourth labeled position of the target object in the first image sample, a third interest loss is calculated based on the loss of interest. According to the seventh difference between the third predicted class and the labeled class of the first image sample, a fourth classification loss is calculated based on the classification loss. According to the sixth difference between the third predicted position and the fourth labeled position, a second regression loss is calculated based on the regression loss.

[0234] In Figure 4 , the sixth image sample is processed by pixel block processing to obtain a third image sample. Specifically, the sixth image sample is processed by disordering to obtain the third image sample. The backbone network of the first target detection network extracts feature data of the second image sample to obtain third feature data.

[0235] According to the third feature data, a relative position prediction loss, a third classification loss and a third interest loss are obtained.

[0236] The third interest loss, the fourth interest loss, the third classification loss, the fourth classification loss, the relative position prediction loss and the second regression loss are weighted and summed to obtain a second total loss. According to the second total loss, the parameters of the first target detection network are updated until the second total loss converges to obtain a third target detection network. At this time, the backbone network of the third target detection network is the disordering backbone network in Figure 4 .

[0237] The implementation process of the second stage training (Finetuning) can be referred to in Figure 5 . As shown in Figure 5 , a target object is extracted from the fourth image sample to obtain a seventh image sample. The feature data of the fourth image sample is extracted by the mask backbone network to obtain fifth feature data. The feature data of the seventh image sample is extracted by the mask backbone network to obtain sixth feature data.

[0238] Based on the fourth image sample and the fifth feature data, a third target detection loss is obtained. Based on the seventh image sample and the sixth feature data, a fourth target detection loss is obtained.

[0239] According to the third target detection loss and the fourth target detection loss, the parameters of the second target detection network are updated to obtain a fourth target detection network.

[0240] The implementation process of the third stage training can be referred to Figure 6 . As shown in Figure 6 , the target object is cropped from the first image sample to obtain an eighth image sample. The first image sample is processed for feature extraction by using the backbone network of the fourth target detection network (i.e., the disorder backbone network in Figure 6 , to obtain seventh feature data.

[0241] The eighth image sample is processed for feature extraction by using the backbone network of the fourth target detection network (i.e., the disorder backbone network in Figure 6 , to obtain eighth feature data. The eighth image sample is processed for feature extraction by using the backbone network of the third target detection network (i.e., the mask backbone network in Figure 6 , to obtain ninth feature data.

[0242] Based on the first image sample and the seventh feature data, a fifth target detection loss is obtained. Based on the eighth image sample and the eighth feature data, a sixth target detection loss is obtained. Based on the difference between the eighth feature data and the ninth feature data, a third loss is obtained.

[0243] According to the third loss and the fifth target detection loss, the parameters of the fourth target detection network are updated to obtain a fifth target detection network.

[0244] Based on the training method described above, the embodiments of the present application have also made several sets of comparative experiments in different combinations, and the experimental data are shown in Table 1.

[0245] As shown in Table 1, the experimental combinations can not include random mask processing, disorder processing, three-stage training processing, or include one or more of random mask processing, disorder processing, and three-stage training.

[0246] Figure 4 shows a training process of target detection training under the supervision of the first self-supervised task, i.e., the pixel block processing in Figure 4 is random mask processing.

[0247] Figure 4 or shows a training process of target detection training under the supervision of the second self-supervised task, i.e., the pixel block processing in Figure 4 is disorder processing.

[0248] The three-stage training refers to that the training process includes the training process shown in Figure 4 , and Figure 5the training process shown in Figure 6 the training process shown in

[0249] From Table 1, it can be seen that the comparative experiment has 7 experimental combinations. The 7 experimental combinations are as follows: the first experimental combination does not include random mask processing and disorder processing, that is, only the target detection network trained by target detection training of the deep learning model is obtained.

[0250]

[0251] Table 1

[0252] The second experimental combination includes random mask processing, that is, the target detection network trained by target detection training under the supervision of the first self-supervised task is obtained.

[0253] The third experimental combination includes disorder processing, that is, the target detection network trained by target detection training under the supervision of the second self-supervised task is obtained.

[0254] The fourth experimental combination includes random mask processing and disorder processing, that is, the target detection network trained by target detection training under the common supervision of the first self-supervised task and the second self-supervised task is obtained.

[0255] In the fourth experimental combination, the first experimental loss is obtained under the supervision of the first self-supervised task, the second experimental loss is obtained under the supervision of the second self-supervised task, and the third experimental loss is obtained by target detection training.

[0256] The fourth experimental loss is obtained by weighted summation of the first experimental loss, the second experimental loss and the third experimental loss. According to the fourth experimental loss, the parameters of the deep learning model are updated to obtain the trained target detection network.

[0257] The fifth experimental combination includes disorder processing and three-stage training, that is, under the condition that the pixel block processing of Figure 4 the training process shown in Figure 4 The second self-supervised target detection network is trained by the training process shown in

[0258] The target detection network trained by the second self-supervised training is trained by the training process shown in Figure 5

[0259] The sixth experimental combination includes random mask processing and three-stage training, that is, under the condition that the pixel block processing of Figure 4 the training process shown in Figure 4 ​The training process shown trains a first self-supervised object detection network, wherein the backbone network of the first self-supervised object detection network is a mask backbone network.

[0260] pass Figure 5 The training process shown trains the target detection network after the first self-supervised training to obtain a trained target detection network.

[0261] The seventh experimental combination includes random mask processing, shuffle processing and three-stage training, that is, Figure 4 When the pixel block is processed as a random mask, Figure 4 The training process shown trains a first self-supervised object detection network, wherein the backbone network of the first self-supervised object detection network is a mask backbone network.

[0262] exist Figure 4 When the pixel blocks are processed out of order, Figure 4 The training process shown trains a second self-supervised object detection network, wherein the backbone network of the second self-supervised object detection network is a disordered backbone network.

[0263] pass Figure 5 The training process shown trains the object detection network after the first self-supervised training to obtain a third self-supervised object detection network, wherein the backbone network of the third self-supervised object detection network is a mask backbone network.

[0264] Finally, through Figure 6 The training process shown is to train the second self-supervised object detection network to obtain the trained object detection network.

[0265] The training data for this experiment includes a base class training image set and a new class training image set, wherein the base class training image set has a large number of images of each category. Optionally, the base class training image set is the first image sample mentioned above.

[0266] The number of images of each category in the new class image set is relatively small. Optionally, the new class training image set is the fourth image sample mentioned above.

[0267] Training image set 1, training image set 2, and training image set 3 in Table 1 all represent new class image sets. Each training image set is divided into three cases: the case where the number of images in each category is 1, the case where the number of images in each category is 3, and the case where the number of images in each category is 10.

[0268] According to Table 1, using the same training image set and trained with different experimental combinations, the trained object detection network has different test results for object detection on the same test dataset.

[0269] For example, in the case that the number of images of each category in the training image set 1 is 1, the accuracy of the test results obtained by training the above seven experimental combinations using the training image set 1 is respectively: the accuracy of the test results obtained by the first experimental combination is 48.3%, the accuracy of the test results obtained by the second experimental combination is 50.6%, the accuracy of the test results obtained by the third experimental combination is 50.3%, the accuracy of the test results obtained by the fourth experimental combination is 46.6%, the accuracy of the test results obtained by the fifth experimental combination is 51.7%, the accuracy of the test results obtained by the sixth experimental combination is 54.1%, and the accuracy of the test results obtained by the seventh experimental combination is 54.6%.

[0270] According to the experimental data shown in Table 1, the following conclusions are also drawn: 1. The accuracy of the test results obtained by the fourth experimental combination is the lowest, that is, the training effect of the fourth experimental combination is the worst; 2. The accuracy of the test results obtained by the seventh experimental combination is the highest, that is, the training effect of the seventh experimental combination is the best.

[0271] The present application also provides an image processing method for processing images using the model trained based on the model training method provided in the foregoing. The execution subject of the image processing method is an image processing device, wherein the image processing device can be any electronic device capable of executing the technical solutions disclosed in the embodiments of the image processing method. Optionally, the image processing device can be one of the following: a mobile phone, a computer, a tablet computer, and a wearable smart device. Optionally, the image processing device is the same as the model training device.

[0272] Please refer to Figure 7 , Figure 7 is a flowchart of another image processing method provided in the embodiments of the present application.

[0273] 7001、obtaining an image to be processed.

[0274] In the embodiments of the present application, the image to be processed can contain any content. For example, the image to be processed includes a cat. For another example, the image to be processed includes a dog and a vehicle. The present application does not limit the content in the image to be processed.

[0275] In one implementation manner of obtaining the image to be processed, the image processing device receives the image to be processed input by a user through an input component.

[0276] Optionally, the input component is one of the following: a keyboard, a mouse, a touch screen, a touch pad, and an audio input device.

[0277] In another implementation of obtaining the image to be processed, the image processing apparatus receives the image to be processed sent by the terminal.

[0278] Optionally, the terminal is one of the following: a mobile phone, a computer, a tablet computer, a server, and a wearable device.

[0279] In yet another implementation of obtaining the image to be processed, the image processing apparatus is loaded with an image acquisition component, and the image processing apparatus acquires the image to be processed through the image acquisition component.

[0280] Optionally, the image acquisition component is a camera.

[0281] 7002. performing target detection on the image to be processed by using the sixth target detection network to obtain a target detection result of the image to be processed.

[0282] In the embodiments of the present application, the target detection result includes the position of the target object in the image to be processed. For example, the target object is an apple, and the target detection result includes the position of the apple in the image to be processed. For another example, the target object includes a cat and a dog, and the target detection result includes the position of the cat in the image to be processed and the position of the dog in the image to be processed.

[0283] In the embodiments of the present application, the sixth target detection network is trained by the model training method provided in the foregoing. Since the model is trained by the model training method provided in the foregoing, the target detection accuracy of the model can be improved, and the target detection result of the image to be processed can be improved by using the sixth target detection network to perform target detection on the image to be processed.

[0284] As an optional implementation, the target object includes a person, that is, the detection object of the sixth target detection network includes a person. The image to be processed is an image obtained by photographing a target scene. The target scene is any scene. For example, the target scene is an intersection. For another example, the target scene is a shopping mall.

[0285] After obtaining the target detection result, the image processing apparatus further performs the following step: determining the number of persons in the image to be processed as the number of persons in the target scene according to the target detection result.

[0286] In the case where the target object includes a person, the target detection result includes the position of the person in the image to be processed. Therefore, the image processing apparatus can determine how many persons are in the image to be processed according to the target detection result, and further determine the number of persons in the image to be processed.

[0287] Since the image to be processed is an image obtained by photographing a target scene, the number of persons in the image to be processed is the number of persons in the target scene.

[0288] In the embodiment of the present application, the image processing apparatus can realize pedestrian counting of the target scene according to the target detection result, and further determine the number of people in the target scene.

[0289] In one possible implementation scenario, the image processing apparatus is an edge device integrated in an indicator light. The edge device can determine the number of people at an intersection by using the sixth target detection network to perform target detection processing on an image of the intersection. Related personnel can further control the traffic at the intersection according to the number of people at the intersection.

[0290] In another possible implementation scenario, the image processing apparatus is a mobile phone of a shopping mall manager. The mobile phone is in communication connection with a camera installed in the shopping mall. The mobile phone obtains an image of the shopping mall through the communication connection. The number of people in the shopping mall can be determined by using the target detection network to perform target detection processing on the image to be processed.

[0291] According to the number of people in the shopping mall, it can be determined which goods in the shopping mall are more popular with customers and which goods are not favored by customers, which is conducive to the shopping mall manager to adjust the operation strategy of the shopping mall.

[0292] Those skilled in the art can understand that the sequence of writing each step in the above method of the specific implementation manner does not mean a strict execution sequence and does not constitute any limitation on the implementation process. The specific execution sequence of each step should be determined by its function and possible inherent logic.

[0293] If the technical solution of the present application involves personal information, the product applying the technical solution of the present application has informed the personal information processing rules before processing the personal information and has obtained the personal independent consent. If the technical solution of the present application involves sensitive personal information, the product applying the technical solution of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent".

[0294] For example, at the personal information collection device such as a camera, an explicit and prominent mark is set to inform that the personal information collection range has been entered and the personal information will be collected. If a person voluntarily enters the collection range, it is regarded as consent to collect the personal information. Or, at the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the person to upload his / her personal information. The personal information processing can include the personal information processor, the processing purpose, the processing method, and the type of personal information to be processed.

[0295] The above describes the method of the embodiment of the present application in detail. The device of the embodiment of the present application is provided below.

[0296] Please refer to Figure 8, Figure 8 A structural schematic diagram of a model training device provided by an embodiment of the present application is shown in the figure. The model training device 1 comprises a first processing unit 11, a second processing unit 12, a third processing unit 13, and a fourth processing unit 14. Specifically:

[0297] The first processing unit 11 is configured to train a first target detection network based on a first image sample and a second image sample, to obtain a second target detection network, wherein the second image sample is obtained by performing random mask processing on the first image sample.

[0298] The second processing unit 12 is configured to train the first target detection network based on the first image sample and a third image sample, to obtain a third target detection network, wherein the third image sample is obtained by performing image region disorder processing on the first image sample.

[0299] The third processing unit 13 is configured to train the second target detection network based on a fourth image sample, to obtain a fourth target detection network, wherein the fourth image sample comprises images of a migration domain of the second target detection network.

[0300] The fourth processing unit 14 is configured to train the fourth target detection network based on the first image sample and the third target detection network, to obtain a fifth target detection network.

[0301] According to any one of the embodiments of the present application, the first processing unit 11 is configured to:

[0302] process the first image sample by using the first target detection network to obtain a first target detection loss;

[0303] process the second image sample by using the first target detection network to obtain a first loss;

[0304] update parameters of the first target detection network according to the first loss and the first target detection loss, to obtain the second target detection network.

[0305] According to any one of the embodiments of the present application, the first processing unit 11 is configured to:

[0306] perform feature extraction processing on the second image sample by using the first target detection network to obtain first feature data;

[0307] obtain a first interest loss based on the second image sample and the first feature data;

[0308] obtain a random mask loss based on the second image sample and the first feature data;

[0309] obtain a first classification loss based on the second image sample and the first feature data;

[0310] obtain the first loss based on the first interest loss, the random mask loss and the first classification loss.

[0311] According to any one of the embodiments of the present application, the first processing unit 11 is configured to:

[0312] obtain a first predicted position of the pixel block processed by the random mask processing in the first image sample according to the first feature data;

[0313] obtain the random mask loss according to a difference between the first predicted position and a first labeled position of the pixel block processed by the random mask processing in the second image sample.

[0314] According to any one of the embodiments of the present application, the first processing unit 11 is further configured to:

[0315] obtain a fifth image sample by cutting a target object from the first image sample;

[0316] obtain the second image sample by performing random mask processing on the fifth image sample.

[0317] According to any one of the embodiments of the present application, the first processing unit 11 is configured to:

[0318] divide the fifth image sample into a first preset number of pixel blocks;

[0319] obtain the second image sample by randomly selecting one pixel block from the first preset number of pixel blocks for erasing processing.

[0320] According to any one of the embodiments of the present application, the first processing unit 11 is configured to:

[0321] obtain second feature data by performing feature extraction processing on the first image sample using the first target detection network;

[0322] obtain a second classification loss based on the first image sample and the second feature data;

[0323] obtain a first regression loss based on the first image sample and the second feature data;

[0324] obtain a second interest loss based on the first image sample and the second feature data;

[0325] obtain the first target detection loss based on the second classification loss, the first regression loss and the second interest loss.

[0326] According to any one of the embodiments of the present application, the second processing unit 12 is configured to:

[0327] processing the first image sample by using the first target detection network to obtain a second target detection loss;

[0328] processing the third image sample by using the first target detection network to obtain a second loss;

[0329] updating parameters of the first target detection network according to the second loss and the second target detection loss to obtain the third target detection network.

[0330] According to any one of the embodiments of the present application, the second processing unit 12 is configured to:

[0331] processing the third image sample by using the first target detection network to obtain third feature data;

[0332] obtaining a third interest loss based on the third image sample and the third feature data;

[0333] obtaining a relative position prediction loss based on the third image sample and the third feature data;

[0334] obtaining a third classification loss based on the third image sample and the third feature data;

[0335] obtaining the second loss based on the third interest loss and the relative position prediction loss.

[0336] According to any one of the embodiments of the present application, the third image sample includes a first pixel block and a second pixel block.

[0337] The second processing unit 12 is configured to:

[0338] predicting a relative position of the first pixel block and the second pixel block according to the third feature data to obtain a second predicted position;

[0339] obtaining the relative position prediction loss according to a difference between the second predicted position and a labeled relative position; the labeled relative position is a relative position of the first pixel block and the second pixel block determined based on a second labeled position of the first pixel block in the third image sample and a third labeled position of the second pixel block in the third image sample.

[0340] According to any one of the embodiments of the present application, the second processing unit 12 is further configured to:

[0341] cutting a target object from the third image sample to obtain a sixth image sample;

[0342] dividing the sixth image sample into a second preset number of pixel blocks;

[0343] randomly selecting two pixel blocks from the second preset number of pixel blocks to splice to obtain the third image sample.

[0344] According to any one of the embodiments of the present application, the second processing unit 12 is configured to:

[0345] performing feature extraction processing on the first image sample by using the first target detection network to obtain fourth feature data;

[0346] obtaining a fourth classification loss based on the first image sample and the fourth feature data;

[0347] obtaining a second regression loss based on the first image sample and the fourth feature data;

[0348] obtaining a fourth interest loss based on the first image sample and the fourth feature data;

[0349] obtaining the second target detection loss based on the fourth classification loss, the second regression loss and the fourth interest loss.

[0350] According to any one of the embodiments of the present application, the third processing unit 13 is configured to:

[0351] cutting a target object from the fourth image sample to obtain a seventh image sample;

[0352] performing processing on the fourth image sample by using the second target detection network to obtain fifth feature data;

[0353] performing processing on the seventh image sample by using the second target detection network to obtain sixth feature data;

[0354] obtaining a third target detection loss based on the fourth image sample and the fifth feature data;

[0355] obtaining a fourth target detection loss based on the seventh image sample and the sixth feature data;

[0356] updating parameters of the second target detection network according to the third target detection loss and the fourth target detection loss to obtain the fourth target detection network.

[0357] According to any one of the embodiments of the present application, the fourth processing unit 14 is configured to:

[0358] obtaining an eighth image sample by cropping the target object from the first image sample;

[0359] performing feature extraction processing on the first image sample by using the fourth target detection network to obtain seventh feature data;

[0360] performing feature extraction processing on the eighth image sample by using the fourth target detection network to obtain eighth feature data;

[0361] obtaining a fifth target detection loss based on the first image sample and the seventh feature data;

[0362] obtaining a sixth target detection loss based on the eighth image sample and the eighth feature data;

[0363] obtaining a third loss based on the third target detection network and the eighth image sample;

[0364] updating parameters of the fourth target detection network according to the third loss, the fifth target detection loss and the sixth target detection loss to obtain the fifth target detection network.

[0365] According to any one of the embodiments of the present application, the fourth processing unit 14 is configured to:

[0366] performing feature extraction processing on the eighth image sample by using the third target detection network to obtain ninth feature data;

[0367] obtaining the third loss based on a difference between the eighth feature data and the ninth feature data.

[0368] Please refer to Figure 9 , Figure 9 A structural schematic diagram of an image processing device provided by the embodiments of the present application is shown in FIG. 2. The image processing device 2 includes an acquisition unit 21 and a fifth processing unit 22. Specifically,

[0369] The acquisition unit 21 is configured to acquire a to-be-processed image.

[0370] The fifth processing unit 22 is configured to perform target detection on the to-be-processed image by using a sixth target detection network to obtain a target detection result of the to-be-processed image, wherein the sixth target detection network is trained by any one of the embodiments of the first aspect.

[0371] In some embodiments, the device provided by the embodiments of the present application has functions or includes modules which can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, the description will not be repeated here.

[0372] Figure 10A hardware structure schematic diagram of a model training device provided by an embodiment of the present application. The model training device 3 comprises a processor 31, a memory 32, an input device 33, and an output device 34. The processor 31, the memory 32, the input device 33, and the output device 34 are coupled through a connector, which comprises various interfaces, transmission lines, buses, and the like, and the present application is not limited in this regard. It should be understood that in various embodiments of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, various interfaces, transmission lines, buses, and the like.

[0373] The processor 31 can be one or more graphics processing units (GPUs). In the case where the processor 31 is a GPU, the GPU can be a single-core GPU or a multi-core GPU.

[0374] Optionally, the processor 31 can be a processor group composed of multiple GPUs, and the multiple processors are coupled to each other through one or more buses.

[0375] Optionally, the processor can also be other types of processors, and the present application is not limited in this regard.

[0376] The memory 32 can be used to store computer program instructions and various computer program codes for executing the present application. Optionally, the memory includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0377] The input device 33 is used to input data and / or signals, and the output device 34 is used to output data and / or signals. The input device 33 and the output device 34 can be independent devices or an integral device.

[0378] It can be understood that in the present application, the memory 32 can be used to store not only related instructions but also related data. For example, the memory 32 can be used to store a to-be-processed image obtained through the input device 33, or the memory 32 can be used to store a detection result obtained through the processor 31, and the present application is not limited in this regard.

[0379] It can be understood that, Figure 10 Only a simplified design of a model training apparatus is shown. In actual applications, the model training apparatus can also respectively include necessary other elements, including but not limited to any number of input / output apparatuses, processors, memories, etc., and all model training apparatuses that can implement the embodiments of the present application are within the protection scope of the present application.

[0380] Figure 11 A hardware structure schematic diagram of an image processing apparatus provided by the embodiments of the present application is shown. The image processing apparatus 4 includes a processor 41, a memory 42, an input apparatus 43, and an output apparatus 44.

[0381] The processor 41, the memory 42, the input apparatus 43, and the output apparatus 44 are coupled through a connector, which includes various interfaces, transmission lines, or buses, etc., and the embodiments of the present application do not limit this.

[0382] It should be understood that in various embodiments of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, connection through various interfaces, transmission lines, buses, etc.

[0383] The processor 41 can be one or more graphics processing units (GPUs). In the case where the processor 41 is a GPU, the GPU can be a single-core GPU or a multi-core GPU.

[0384] Optionally, the processor 41 can be a processor group composed of multiple GPUs, and the multiple processors are coupled to each other through one or more buses.

[0385] Optionally, the processor can also be other types of processors, etc., and the embodiments of the present application do not limit this.

[0386] The memory 42 can be used to store computer program instructions and various computer program codes for executing the schemes of the present application.

[0387] Optionally, the memory includes but is not limited to a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0388] The input device 43 is configured to input data and / or signals, and the output device 44 is configured to output data and / or signals. The input device 43 and the output device 44 can be independent devices, or can be integrated into one device.

[0389] It can be understood that, in the embodiments of the present application, the memory 42 can be used to store not only related instructions, but also related data. For example, the memory 42 can be used to store a to-be-processed image acquired through the input device 43, or the memory 42 can be used to store a detection result obtained through the processor 41, and the like. The embodiments of the present application do not limit the data stored in the memory.

[0390] It can be understood that, Figure 11 Only a simplified design of an image processing device is shown. In actual applications, the image processing device can also include other necessary elements, including but not limited to any number of input / output devices, processors, memories, and the like, and all image processing devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0391] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0392] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here. Those skilled in the art can also clearly understand that each embodiment of the present application describes different focuses, and for the convenience and brevity of description, the same or similar parts can not be described in different embodiments. Therefore, the parts not described or not described in detail in a certain embodiment can refer to the description in other embodiments.

[0393] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways.

[0394] For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0395] In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0396] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0397] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0398] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of them can be realized in the form of a computer program product. The computer program product includes one or more computer instructions.

[0399] When the computer program instructions are loaded and executed on the computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0400] The computer instructions can be stored in or transmitted by the computer readable storage medium.

[0401] The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode.

[0402] The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media.

[0403] The available medium can be a magnetic medium (such as floppy disk, hard disk, magnetic tape), an optical medium (such as digital versatile disc (DVD)) or a semiconductor medium (such as solid state disk (SSD)) etc.

[0404] Those skilled in the art can understand that all or part of the processes in the above-mentioned method embodiments can be implemented by a computer program instructing the relevant hardware to complete, and the program can be stored in a computer-readable storage medium. When the program is executed, the processes of the above-mentioned method embodiments can be included. The aforementioned storage medium includes a read-only memory (ROM) or a random access memory (RAM), a magnetic disc or an optical disc, and various media capable of storing program codes.

Claims

1. A model training method, characterized in that, The method comprises: training a first target detection network based on a first image sample and a second image sample to obtain a second target detection network, the second image sample being obtained by performing random mask processing on the first image sample, the random mask processing comprising: performing erasing processing on any one pixel block in the first image sample, and splicing pixel blocks other than the erased pixel block; training the first target detection network based on the first image sample and a third image sample to obtain a third target detection network, the third image sample being obtained by performing image region disorder processing on the first image sample, the disorder processing comprising: in the case of dividing the first image sample into a second preset number of pixel blocks, randomly selecting two pixel blocks from the second preset number of pixel blocks for splicing; training the second target detection network based on a fourth image sample to obtain a fourth target detection network, the fourth image sample comprising an image of a target domain of the second target detection network; training the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network, the training the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network comprising: training the fourth target detection network using the first image sample under the supervision of the third target detection network to obtain the fifth target detection network.

2. The method of claim 1, wherein, The training a first target detection network based on a first image sample and a second image sample to obtain a second target detection network comprises: processing the first image sample using the first target detection network to obtain a first target detection loss; processing the second image sample using the first target detection network to obtain a first loss; updating parameters of the first target detection network according to the first loss and the first target detection loss to obtain the second target detection network.

3. The method of claim 2, wherein, The processing the second image sample using the first target detection network to obtain a first loss comprises: performing feature extraction processing on the second image sample using the first target detection network to obtain first feature data; obtaining a first interest loss based on the second image sample and the first feature data; obtaining a random mask loss based on the second image sample and the first feature data; obtaining a first classification loss based on the second image sample and the first feature data; obtaining the first loss based on the first interest loss, the random mask loss, and the first classification loss.

4. The method of claim 3, wherein, The obtaining a random mask loss based on the second image sample and the first feature data comprises: obtaining a first predicted position of a pixel block processed by the random mask processing in the first image sample according to the first feature data; obtaining the random mask loss according to a difference between the first predicted position and a first labeled position of the pixel block processed by the random mask processing in the second image sample.

5. The method according to claim 3 or 4, characterized in that, Before the feature extraction processing of the second image sample by the first target detection network is performed, the method further comprises: cutting a target object from the first image sample to obtain a fifth image sample; performing random mask processing on the fifth image sample to obtain the second image sample.

6. The method of claim 5, wherein, The random mask processing on the fifth image sample to obtain the second image sample comprises: dividing the fifth image sample into a first preset number of pixel blocks; randomly selecting one pixel block from the first preset number of pixel blocks for erasing processing to obtain the second image sample.

7. The method according to any one of claims 2 to 4, characterized in that, The processing of the first image sample by the first target detection network to obtain a first target detection loss comprises: performing feature extraction processing on the first image sample by the first target detection network to obtain second feature data; obtaining a second classification loss based on the first image sample and the second feature data; obtaining a first regression loss based on the first image sample and the second feature data; obtaining a second interest loss based on the first image sample and the second feature data; obtaining the first target detection loss based on the second classification loss, the first regression loss and the second interest loss.

8. The method according to any one of claims 1 to 4, characterized in that, The training of the first target detection network based on the first image sample and the third image sample to obtain a third target detection network comprises: processing the first image sample by the first target detection network to obtain a second target detection loss; processing the third image sample by the first target detection network to obtain a second loss; updating parameters of the first target detection network according to the second loss and the second target detection loss to obtain the third target detection network.

9. The method of claim 8, wherein, The processing of the third image sample by the first target detection network to obtain a second loss comprises: performing feature extraction processing on the third image sample by the first target detection network to obtain third feature data; obtaining a third interest loss based on the third image sample and the third feature data; obtaining a relative position prediction loss based on the third image sample and the third feature data; obtaining a third classification loss based on the third image sample and the third feature data; obtaining the second loss based on the third interest loss and the relative position prediction loss.

10. The method of claim 9, wherein, The third image sample comprises a first pixel block and a second pixel block. The obtaining of the relative position prediction loss based on the third image sample and the third feature data comprises: predicting a relative position of the first pixel block and the second pixel block according to the third feature data to obtain a second predicted position; According to a difference between the second predicted position and a labeled relative position, a relative position prediction loss is obtained; the labeled relative position is a relative position of the first pixel block and the second pixel block determined based on a second labeled position of the first pixel block in the third image sample and a third labeled position of the second pixel block in the third image sample.

11. The method of claim 9, wherein, Before the feature extraction processing of the third image sample by the first target detection network is performed to obtain third feature data, the method further comprises: a target object is intercepted from the third image sample to obtain a sixth image sample; the sixth image sample is divided into a second preset number of pixel blocks; two pixel blocks are randomly selected from the second preset number of pixel blocks for splicing to obtain the third image sample.

12. The method of claim 8, wherein, The processing of the first image sample by the first target detection network to obtain a second target detection loss comprises: feature extraction processing of the first image sample by the first target detection network is performed to obtain fourth feature data; a fourth classification loss is obtained based on the first image sample and the fourth feature data; a second regression loss is obtained based on the first image sample and the fourth feature data; a fourth interest loss is obtained based on the first image sample and the fourth feature data; the second target detection loss is obtained based on the fourth classification loss, the second regression loss and the fourth interest loss.

13. The method according to any one of claims 1 to 4, characterized in that, The training of the second target detection network based on the fourth image sample to obtain a fourth target detection network comprises: a target object is intercepted from the fourth image sample to obtain a seventh image sample; processing of the fourth image sample by the second target detection network is performed to obtain fifth feature data; processing of the seventh image sample by the second target detection network is performed to obtain sixth feature data; a third target detection loss is obtained based on the fourth image sample and the fifth feature data; a fourth target detection loss is obtained based on the seventh image sample and the sixth feature data; parameters of the second target detection network are updated according to the third target detection loss and the fourth target detection loss to obtain the fourth target detection network.

14. The method according to any one of claims 1 to 4, characterized in that, The training of the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network comprises: a target object is intercepted from the first image sample to obtain an eighth image sample; feature extraction processing of the first image sample by the fourth target detection network is performed to obtain seventh feature data; feature extraction processing of the eighth image sample by the fourth target detection network is performed to obtain eighth feature data; a fifth target detection loss is obtained based on the first image sample and the seventh feature data; a sixth target detection loss is obtained based on the eighth image sample and the eighth feature data; a third loss is obtained based on the third target detection network and the eighth image sample; According to the third loss, the fifth target detection loss and the sixth target detection loss, the parameters of the fourth target detection network are updated to obtain the fifth target detection network.

15. The method of claim 14, wherein, The third loss is obtained based on the third target detection network and the eighth image sample, and the third loss comprises: The third target detection network is used for feature extraction processing on the eighth image sample to obtain ninth feature data. The third loss is obtained based on the difference between the eighth feature data and the ninth feature data.

16. An image processing method, characterized by, The method comprises: An image to be processed is obtained. A sixth target detection network is used for target detection on the image to be processed to obtain a target detection result of the image to be processed, and the sixth target detection network is trained by the method in any one of claims 1 to 15.

17. A model training apparatus, comprising: The device comprises: A first processing unit is configured to train a first target detection network based on a first image sample and a second image sample to obtain a second target detection network, the second image sample is obtained by performing random mask processing on the first image sample, and the random mask processing comprises erasing any pixel block in the first image sample and splicing pixel blocks other than the erased pixel block. A second processing unit is configured to train the first target detection network based on the first image sample and a third image sample to obtain a third target detection network, the third image sample is obtained by performing image region disorder processing on the first image sample, and the disorder processing comprises randomly selecting two pixel blocks from the second preset number of pixel blocks for splicing under the condition that the first image sample is divided into the second preset number of pixel blocks. A third processing unit is configured to train the second target detection network based on a fourth image sample to obtain a fourth target detection network, and the fourth image sample comprises images in a migration domain of the second target detection network. A fourth processing unit is configured to train the fourth target detection network based on the first image sample and the third target detection network to obtain a fifth target detection network, and the training of the fourth target detection network based on the first image sample and the third target detection network to obtain the fifth target detection network comprises training the fourth target detection network using the first image sample under the supervision of the third target detection network to obtain the fifth target detection network.

18. An image processing apparatus characterized by comprising: The device comprises: An acquisition unit is configured to obtain an image to be processed. A fifth processing unit is configured to use a sixth target detection network to perform target detection on the image to be processed to obtain a target detection result of the image to be processed, and the sixth target detection network is trained by the method in any one of claims 1 to 15.

19. An electronic device, comprising: The electronic device comprises: A processor and a memory, the memory is configured to store computer program code, the computer program code comprises computer instructions, and the electronic device executes the method in any one of claims 1 to 15 when the processor executes the computer instructions.

20. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program includes program instructions. The program instructions, when executed by a processor, cause the processor to perform the method in any one of claims 1 to 15.

Images