Model training method and apparatus, and electronic device

Abstract

The present disclosure provides a model training method and device and electronic equipment, the method comprising: obtaining a first sample image, the first sample image comprising an unoccluded first image block and an occluded second image block; processing the first sample image through a first model to obtain a first image feature corresponding to the first image block; reconstructing the second image block according to the first image feature to obtain a first image, and determining a fusion prediction feature of the first image block and the second image block according to the first image feature; obtaining a target image feature in a target image, the target image being an image after preprocessing of the first sample image; and updating model parameters of the first model according to the first image, the second image block, the fusion prediction feature and the target image feature. The effect of model training is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a model training method, apparatus, and electronic device. Background Technology

[0002] Self-supervised learning can learn a feature extractor from unlabeled data, and then obtain features through the feature extractor. This eliminates the need to label training samples, reducing the cost of model training.

[0003] Currently, self-supervised learning of models can be achieved through contrastive learning. For example, a model can compare the current input image with other images to construct the relationship between the graphics of the current input image and other images, and obtain a feature extractor by learning this relationship. However, models trained using the above method do not learn the correlation between different regions within an image, resulting in poor model training performance. Summary of the Invention

[0004] This disclosure provides a model training method, apparatus, and electronic device to solve the technical problem of poor model training performance in the prior art.

[0005] Firstly, this disclosure provides a model training method, which includes:

[0006] Obtain a first sample image, which includes an unoccluded first image block and an occluded second image block;

[0007] The first sample image is processed by the first model to obtain the first image feature corresponding to the first image block;

[0008] Based on the first image features, the second image block is reconstructed to obtain the first image, and based on the first image features, the fusion prediction features of the first image block and the second image block are determined;

[0009] The target image features are obtained from the target image, which is the image after preprocessing the first sample image.

[0010] The model parameters of the first model are updated based on the first image, the second image patch, the fused prediction features, and the target image features.

[0011] Secondly, this disclosure provides an image processing method, the method comprising:

[0012] Multiple images are processed by a first model to obtain multiple image features corresponding to the multiple images. The first model is a model trained by combining reconstructed image contrastive learning with predicted feature contrastive learning. Optionally, the first model can be the first model described in the first aspect above.

[0013] The multiple images are classified based on the multiple image features.

[0014] Thirdly, this disclosure provides a model training apparatus, including a first acquisition module, a processing module, a reconstruction module, a determination module, a second acquisition module, and an update module, wherein:

[0015] The first acquisition module is used to acquire a first sample image, wherein the first sample image includes an unoccluded first image block and an occluded second image block;

[0016] The processing module is used to process the first sample image through the first model to obtain the first image feature corresponding to the first image block;

[0017] The reconstruction module is used to reconstruct the second image block based on the first image features to obtain the first image;

[0018] The determining module is used to determine the fusion prediction features of the first image block and the second image block based on the first image features;

[0019] The second acquisition module is used to acquire target image features in the target image, wherein the target image is the image after preprocessing the first sample image;

[0020] The update module is used to update the model parameters of the first model based on the first image, the second image block, the fusion prediction features, and the target image features.

[0021] Fourthly, this disclosure provides an image processing apparatus, including a processing module and a classification module, wherein:

[0022] The processing module is used to process multiple images through a first model to obtain multiple image features corresponding to the multiple images. The first model is a model trained by combining reconstructed image contrast learning with predicted feature contrast learning.

[0023] The classification module is used to classify the multiple images based on the multiple image features.

[0024] Fifthly, embodiments of this disclosure provide an electronic device, including: a processor and a memory;

[0025] The memory stores computer-executed instructions;

[0026] The processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the model training method as described in the first aspect above and various possible methods involved in the first aspect, or the image processing method as described in the second aspect above and various possible methods involved in the second aspect.

[0027] In a sixth aspect, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the model training method described in the first aspect and various possible embodiments thereof, or the image processing method described in the second aspect and various possible embodiments thereof.

[0028] In a seventh aspect, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the model training method described in the first aspect above and various possible aspects thereof, or the image processing method described in the second aspect above and various possible aspects thereof.

[0029] This disclosure provides a model training method, apparatus, and electronic device. The electronic device acquires a first sample image, which includes an unoccluded first image patch and an occluded second image patch. A first model processes the first sample image to obtain first image features corresponding to the first image patch. Based on the first image features, the second image patch is reconstructed to obtain a first image. Based on the first image features, a fusion prediction feature of the first and second image patches is determined. Target image features are acquired in a target image, which is the preprocessed image of the first sample image. The model parameters of the first model are updated based on the first image, the second image patch, the fusion prediction feature, and the target image feature. According to the above method, the electronic device can obtain the correlation between regions within the sample first image by reconstructing the first image from the second image patch and the occluded first image patch. By fusing the prediction feature and the target image feature, the correlation between images can be obtained. Therefore, when training the first model, the first model can learn not only the correlation between regions within an image but also the correlation between images, thereby improving the model training effect. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0032] Figure 2 A schematic flowchart of a model training method provided in an embodiment of this disclosure;

[0033] Figure 3 A schematic diagram of a first sample image provided in an embodiment of this disclosure;

[0034] Figure 4 This is a schematic diagram illustrating a process for acquiring a target image, provided by an embodiment of the present disclosure.

[0035] Figure 5 This is a schematic diagram illustrating a process for acquiring target image features according to an embodiment of the present disclosure;

[0036] Figure 6 This is a schematic flowchart of a method for updating model parameters provided in an embodiment of the present disclosure;

[0037] Figure 7 This is a schematic diagram illustrating the process of a model training method provided in an embodiment of the present disclosure;

[0038] Figure 8 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure;

[0039] Figure 9 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;

[0040] Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure;

[0041] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0043] For ease of understanding, the concepts involved in the embodiments of this disclosure will be explained below.

[0044] Electronic device: A device with wireless transceiver capabilities. Electronic devices can be deployed on land, including indoors or outdoors, handheld, wearable, or vehicle-mounted; they can also be deployed on water (such as on ships). These electronic devices can be mobile phones, tablets, computers with wireless transceiver capabilities, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, vehicle-mounted electronic devices, wireless terminals in self-driving vehicles, wireless electronic devices in remote medical care, wireless electronic devices in smart grids, wireless electronic devices in transportation safety, wireless electronic devices in smart cities, wireless electronic devices in smart homes, wearable electronic devices, etc. The electronic devices involved in the embodiments of this disclosure can also be referred to as terminals, user equipment (UE), access electronic devices, vehicle-mounted terminals, industrial control terminals, UE units, UE stations, mobile stations, mobile stations, remote stations, remote electronic devices, mobile devices, UE electronic devices, wireless communication devices, UE agents, or UE devices, etc. Electronic devices can be fixed or mobile.

[0045] In related technologies, many model training scenarios lack sufficient labeled data (e.g., in medical image recognition), and manual annotation of labeled data is time-consuming. Self-supervised learning, however, can learn a feature extractor from unlabeled image data, thereby extracting image features and effectively reducing training costs. Currently, self-supervised learning can be achieved through contrastive learning. For example, the input image and its transformed versions (e.g., changing brightness, size, color, but not the graphic itself) are used as positive samples, while the input image and other images are used as negative samples for self-supervised training. This allows the model to learn the relationships between images. However, relationships also exist between regions within an image. The model trained using the above methods fails to learn these relationships, resulting in poor training performance.

[0046] To address the technical problems in related technologies, this disclosure provides a model training method. The method involves acquiring a first sample image, which includes an unoccluded first image patch and an occluded second image patch. A first model processes the first sample image to obtain first image features corresponding to the first image patch. Based on the first image features, the second image patch is reconstructed to obtain the first image. Furthermore, based on the first image features, a fusion prediction feature of the first and second image patches is determined. The first sample image is then processed through image enhancement and contrast improvement to obtain a target image. Target image features corresponding to a portion of the image are then obtained from the target image. A first loss function is obtained based on the first and second image patches. A second loss function is obtained by fusing the prediction feature and the target image features. The model parameters of the first model are updated using the first and second loss functions. In this way, the first model can learn the relationships between different regions within an image through the first and second image patches, and it can also learn the relationships between images by fusing the prediction feature and the target image features, thereby improving the model training effect.

[0047] Below, in conjunction with Figure 1 The application scenarios of the embodiments of this disclosure will be described.

[0048] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this disclosure. Please refer to [link / reference]. Figure 1 This includes image A, image B, image C, image D, and a classification model. The classification model includes a first model as described in this embodiment. Figure 1 (Not shown in the image). Inputting images A, B, C, and D into the classification model, the model can acquire the image features corresponding to each image and classify images A and C into one class, and images B and D into another. Thus, because the first model in the classification model learns the relationships between images and the relationships within image regions, the model training effect is better. Therefore, the classification model can accurately obtain the image features corresponding to each image, thereby improving the accuracy of image classification. It should be noted that... Figure 1 This is merely an illustrative example of one application scenario of the embodiments of this disclosure and is not intended to limit the application scenario.

[0049] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0050] Figure 2This is a schematic flowchart illustrating a model training method provided in an embodiment of this disclosure. Please refer to [link / reference]. Figure 2 The method may include:

[0051] S201. Obtain the first sample image.

[0052] The execution subject of this disclosure can be an electronic device or a model training device installed in an electronic device. Optionally, the model training device can be implemented by software, or it can be implemented by a combination of software and hardware.

[0053] Optionally, the first sample image is used to train the first model. Optionally, the first sample image includes multiple image blocks. For example, if the size of the first sample image is 224*224, the first sample image can be divided into 14*14 (196) image blocks without overlapping regions, or the first sample image can also be divided into image blocks of any size without overlapping regions. This disclosure does not limit this.

[0054] Optionally, each image block in the first sample image includes a corresponding identifier. For example, the first sample image includes 196 non-overlapping image blocks, each with a unique serial number. The electronic device can determine the position of the image block in the first sample image by using the serial number corresponding to the image block.

[0055] Optionally, the multiple image blocks may include an unoccluded first image block and an occluded second image block. For example, the first sample image may include image block A, image block B, and image block C. If image block A is occluded, then the first image block consists of image block B and image block C, and the second image block consists of image block A.

[0056] Optionally, the electronic device can occlude multiple image blocks in the first sample image using an occlusion area. For example, if the preset occlusion ratio is 0.5 and the first sample image includes 100 image blocks, the size of the occlusion mask can be set to the size of 50 image blocks, thereby occluding the first sample image using the occlusion mask. For example, if the first sample image includes 100 non-overlapping image blocks and the preset occlusion ratio is 0.5, 50 masks with the same size as the image blocks can be set, thereby randomly covering 50 image blocks with 50 masks. Optionally, the occlusion ratio in the first sample image can be any ratio, and this embodiment of the present disclosure does not limit this.

[0057] Below, in conjunction with Figure 3 The first sample image is described.

[0058] Figure 3 This is a schematic diagram of a first sample image provided in an embodiment of this disclosure. Please refer to [link / reference]. Figure 3The image includes image blocks A, B, C, and D. These image blocks do not overlap. If the contents of image blocks A and B are occluded, then the first image block includes image blocks C and D, and the second image block includes image blocks A and B.

[0059] S202. The first sample image is processed by the first model to obtain the first image feature corresponding to the first image block.

[0060] Optionally, the first model can be a neural network model, which can acquire image features. For example, the first model can be a Convolutional Neural Network (CNN), and by adjusting the parameters in the CNN, the accuracy of feature extraction can be improved.

[0061] Optionally, since the first image patch is an unoccluded image patch in the first sample image, the first image feature corresponding to the first image patch can be obtained according to the following feasible implementation: perform feature extraction processing on the first image patch in the first sample image to obtain the first image feature corresponding to the first image patch. For example, since the first image patch is an unoccluded image patch, the first model can directly recognize the image content in the first image patch, and thus obtain the first image feature corresponding to the first image patch. For example, the first model includes an online encoder. When the online encoder receives a recognizable first image patch, it can map the first image patch to a feature space, and then obtain the encoded features of the first image patch.

[0062] Optionally, the first image feature can be a vector feature. For example, the first image feature can be a 768-dimensional feature vector. If the first sample image includes 100 image patches, then after processing the first sample image through the first model, each image patch will correspond to a 768-dimensional feature vector.

[0063] S203. Based on the features of the first image, reconstruct the second image block to obtain the first image.

[0064] Optionally, the electronic device can reconstruct the image content in the occluded second image block using the first image features corresponding to the unoccluded first image block, thereby obtaining the first image. For example, the electronic device may include an online decoder. When the online decoder receives the first image features corresponding to the first image block, it can input the first image features into a pixel decoder (the features of the occluded second image block can be replaced by placeholders). The pixel decoder can then predict the first image corresponding to the unoccluded second image block based on the first image features.

[0065] S204. Based on the first image features, determine the fusion prediction features of the first image block and the second image block.

[0066] Optionally, the fusion prediction feature is used to indicate the fusion features of the first image block and the second image block. For example, the first image block and the second image block can form an image, and the second image block is an occluded image block. The electronic device can predict the fusion prediction feature after the first image block and the second image block are fused by the first image feature of the first image block.

[0067] Optionally, the electronic device can determine the fusion prediction features of the first image block and the second image block according to the following feasible implementation: obtaining a preset first vector. Optionally, the first vector can be the vector corresponding to the predicted second image block. For example, the electronic device may include an online decoder. When the online decoder receives the first image features corresponding to the first image block, it can predict the first vector corresponding to the second image block based on the first image features.

[0068] The first vector and the first image features are fused to obtain a fused vector. For example, when the electronic device obtains the first image features corresponding to the first image block, the electronic device can fill in the features corresponding to the second image block with placeholders based on the index of the second image block in the first sample image. When the online decoder receives the first image features, it can predict the feature vector corresponding to the second image block and replace the placeholders with the feature vector to obtain the fused vector.

[0069] The fusion prediction features are obtained from the fusion vector. For example, an online decoder includes a feature decoder. When the online decoder in an electronic device obtains the fusion vector, it can input the fusion vector into the feature decoder. The feature decoder can then obtain the fusion prediction features of the first image block and the second image block based on the fusion vector.

[0070] S205. Obtain the target image features from the target image.

[0071] Optionally, the target image is the image after preprocessing the first sample image. Optionally, preprocessing can be a method that changes image attributes. For example, preprocessing may include adjusting image brightness, adjusting image color, adjusting image contrast, adjusting image grayscale, adjusting image size, or other image processing methods, which are not limited in this embodiment. For example, enhancing the display of the first sample image to obtain the target image; increasing the brightness of the first sample image to obtain the target image; cropping a portion of the first sample image to obtain the target image, etc.

[0072] Optionally, the target image may include multiple image blocks. For example, if the first sample image includes 100 image blocks, and the target image is obtained by enhancing the display of the first sample image, then the target image may also include 100 image blocks. Optionally, all image blocks in the target image are unoccluded first image blocks.

[0073] Below, in conjunction with Figure 4 The process of acquiring the target image is explained.

[0074] Figure 4 This is a schematic diagram illustrating a process for acquiring a target image according to an embodiment of this disclosure. Please refer to [link / reference]. Figure 4 The dataset includes a first sample image and a target image. The first sample image comprises image blocks A, B, C, and D. The grayscale values ​​in the first sample image are adjusted to obtain the target image, which includes image blocks 1, 2, 3, and 4. Image block 1 is the image block after grayscale value adjustment of image block A; image block 2 is the image block after grayscale value adjustment of image block B; image block 3 is the image block after grayscale value adjustment of image block C; and image block 4 is the image block after grayscale value adjustment of image block D.

[0075] Optionally, the electronic device acquires target image features from the target image in the following two ways:

[0076] Case 1: The image features of the target image are determined as the target image features.

[0077] Optionally, the electronic device can process the target image to obtain the target image features. For example, the electronic device can process the target image using an encoder to obtain the feature map corresponding to the target image.

[0078] Case 2: Determine the image features of a portion of the target image as the target image features.

[0079] Optionally, the electronic device can obtain the target image features by determining a first region in the target image based on the position of the second image block in the first sample image. For example, if the second image block is in the middle region of the first sample image, then the middle region in the target image is determined as the first region; if the second image block is in the upper region of the first sample image, then the upper region in the target image is determined as the first region. Optionally, the size of the first region can be the same as the sum of the sizes of the multiple second image blocks. For example, if the first sample image includes 10 second image blocks, each second image block being 10 pixels long and 10 pixels wide, then the first region is 100 pixels long and 100 pixels wide. The size of the first region can be other sizes, which are not limited in this embodiment of the present disclosure.

[0080] The first region is offset to obtain the second region. For example, an electronic device can offset the first region in the target image by any distance in any direction to obtain the second region. In this way, the image content in the second region is similar to the image content in the first region, thereby improving the model training effect.

[0081] The image features corresponding to image blocks within a second region of the target image are acquired and identified as target image features. For example, if the target image includes multiple image blocks, after the electronic device determines the second region, it can identify the image features corresponding to the image blocks within the second region as target image features.

[0082] Optionally, the electronic device may randomly select a portion of the image region in the target image and determine the features corresponding to the image within the image region as the target image features. For example, the electronic device may arbitrarily acquire an image of a preset size in the target image and determine the image features corresponding to that image as the target image features. The preset size may be the same as or different from the size of the occluded area in the first sample image. This embodiment of the present disclosure does not limit this.

[0083] Optionally, the electronic device can process the target image using a target encoder to obtain the target image features corresponding to the target image. Optionally, the target encoder is used to acquire image features in the target image. For example, the electronic device includes an inline encoder, which can acquire first image features of multiple image blocks in the first sample image. The electronic device can also obtain second image features corresponding to the target image using the target encoder.

[0084] Below, in conjunction with Figure 5 The process of obtaining target image features from the target image is explained.

[0085] Figure 5This is a schematic diagram illustrating a process for acquiring target image features according to an embodiment of this disclosure. Please refer to [link / reference]. Figure 5 The target image includes image blocks A, B, C, and D. A first region of the target image includes image blocks A and B. The first region is shifted downwards to obtain a second region, which includes image blocks C and D. The image features corresponding to image blocks C and D are then identified as the target image features.

[0086] S206. Update the model parameters of the first model based on the first image, the second image patch, the fused prediction features, and the target image features.

[0087] Optionally, the model parameters of the first model can be updated according to the following feasible implementation: a first loss function of the first model is determined based on the first image and the second image patch. For example, the electronic device determines the first loss function of the first model based on the difference between the first image and the second image patch, and then updates the first model using the first loss function.

[0088] Alternatively, the first loss function can be determined according to the following formula:

[0089]

[0090] Where L1 is the first loss function; M is the number of second image patches; n is the number of image patches in the first sample image; x i For the first image; y i is the second image block; i is the sequence number of the second image block.

[0091] A second loss function is determined for the first model based on the fused prediction features and the target image features. For example, the electronic device obtains the second loss function by relating the fused prediction features of the first and second image patches to the target image features in the target image, and updates the model parameters in the first model using the second loss function.

[0092] The model parameters of the first model are updated based on the first loss function and the second loss function. For example, an electronic device can update the first model using the first loss function between the first image and the second image patch, and the second loss function between the fused predicted features and the target image features. In this way, the first model can not only learn the correlation between images, but also the correlation between different regions within an image, thereby improving the training effect of the model.

[0093] This disclosure provides a model training method, which involves acquiring a first sample image, processing an unoccluded first image patch in the first sample image using a first model to obtain a first image feature corresponding to the first image patch, reconstructing an occluded second image patch in the first sample image based on the first image feature to obtain a first image, determining a fusion prediction feature between the first image patch and the second image patch based on the first image feature, processing the first sample image through image enhancement, contrast improvement, etc., to obtain a target image, determining the target image features required for contrast learning in the target image, obtaining a first loss function based on the first image and the second image patch, obtaining a second loss function by fusing the prediction feature and the target image feature, and updating the model parameters of the first model using the first loss function and the second loss function. Since the first loss function can indicate the correlation between regions within an image, and the second loss function can indicate the correlation between images, the first model can learn features of regions within an image and features between images, thereby improving the model training effect.

[0094] The above model processing method also includes a method for updating the model parameters of the first model. Figure 2 Based on the embodiments shown, the following, in conjunction with Figure 6 The method for updating the model parameters of the first model is explained.

[0095] Figure 6 This is a schematic flowchart illustrating a method for updating model parameters according to an embodiment of this disclosure. Please refer to... Figure 6 ,include:

[0096] S601. Determine the first loss function based on the first image and the second image block.

[0097] It should be noted that the execution process of step S601 can refer to step S205, and this embodiment will not be described in detail here.

[0098] S602. Determine the second loss function based on the fused prediction features and the target image features.

[0099] Optionally, the second loss function can be determined by acquiring a second sample image and determining the second image features of the second sample image. Optionally, the second sample image can be any image other than the first sample image. For example, during the self-supervised learning process of the first model, the training set of the first model may include multiple training sample images; when learning from one of the training sample images, the other training sample images are all considered second sample images.

[0100] Optionally, the second sample image can be encoded using an encoder to obtain the second image features corresponding to the second sample image. For example, if the second sample image is a contrast sample (negative sample) in self-supervised learning, the electronic device can encode the image features of the second sample image using a target encoder to obtain the second image features.

[0101] A second loss function is determined based on the fused prediction features, the target image features, and the second image features. Optionally, the second loss function can be determined by obtaining a first similarity between the fused prediction features and the target image features. For example, the first similarity can be cosine similarity, Euclidean distance, etc., which is not limited in this embodiment. For example, each image patch in the target image corresponds to a 768-dimensional vector space. Before obtaining the first similarity, the electronic device can merge the multiple 768-dimensional vector spaces corresponding to multiple image patches into a single feature vector, and then determine the cosine similarity between this feature vector and the fused prediction features, and determine the cosine similarity as the first similarity.

[0102] Obtain the second similarity between the fused predicted features and the second image features. For example, merge the feature vectors corresponding to each image patch in the second sample image into a single feature vector, then determine the cosine similarity between the merged feature vector and the fused predicted features, and finally determine the cosine similarity as the second similarity.

[0103] The second loss function is determined based on the first and second similarities. Optionally, the second parameter can be determined using the following formula:

[0104]

[0105] Where L2 is the second parameter; s + The cosine similarity (first similarity) between positive samples; s - τ is the cosine similarity (second similarity) between positive and negative samples; τ is a coefficient that controls the learning difficulty (it can be any value, such as 0.07).

[0106] S603. Update the model parameters of the first model according to the first loss function and the second loss function.

[0107] Optionally, the model parameters of the first model can be updated according to the following feasible implementation: obtain the first weight corresponding to the first loss function and the second weight corresponding to the second loss function; update the model parameters of the first model based on the first parameters, the first loss function, the second parameters, and the second loss function. Optionally, the first weight and the second weight can be any pre-set values, and the sum of the first weight and the second weight is 1. For example, if the first weight is 0.3 and the second weight is 0.7, the electronic device can adjust the first weight and the second weight according to requirements, thereby adjusting the learning focus of the first model (e.g., favoring the correlation within images, or favoring the correlation between images), thereby improving the flexibility and effectiveness of model training.

[0108] This disclosure provides a method for updating model parameters. Based on a first image and a first image patch, a first loss function is determined; based on fused prediction features and target image features, a second loss function is determined; and based on the first and second loss functions, the model parameters of a first model are updated. In this way, an electronic device can accurately train the first model using the first and second loss functions, improving the accuracy of model training. Furthermore, since the first weights corresponding to the first loss function and the second weights corresponding to the second loss function can flexibly adjust the learning focus of the first model, the flexibility of model training can be improved, thereby enhancing the model training effect.

[0109] Based on any of the above embodiments, the following, in conjunction with Figure 7 The process of training the above model will be explained.

[0110] Figure 7 This is a schematic diagram illustrating a model training method provided in an embodiment of this disclosure. Please refer to [link / reference]. Figure 7 The system comprises a first model, an online decoder, and a target encoder. The first model includes a first sample image, which contains occluded regions. The first model processes the unoccluded regions in the first sample image to obtain three first image features corresponding to the unoccluded regions. The online encoder acquires these three first image features and predicts the features of the occluded regions in the first sample image, then adds the predicted features to the three first image features. The pixel decoder processes the added features to reconstruct the image of the occluded regions, obtaining a first image. A first loss function is then derived using the first image and the occluded image from the first sample image.

[0111] Please see Figure 7The target encoder acquires the target image, which is the enhanced version of the first sample image. The target encoder generates target image features corresponding to the target image. The feature decoder processes the first image features after feature addition to obtain fused prediction features for the occluded and unoccluded regions. By fusing the prediction features and the target image features, a second loss function is obtained. It should be noted that... Figure 7 In the illustrated embodiment, negative samples (second sample images) can also be combined. Figure 7 (Not shown in the image) Determine the second loss function.

[0112] Please see Figure 7 The online decoder and the first model are backpropagated using a first loss function and a second loss function, thereby updating the model parameters in the first model. For example, during the training of the first model, the parameters of the online decoder can also be updated using the first and second loss functions (as the first model continues to train, the parameters of the online decoder are also continuously updated). In practical use, only image features need to be extracted through the first model, eliminating the need for an online decoder and a target encoder.

[0113] Optionally, during the training of the first model, the parameters in the target encoder can also be updated using the following formula:

[0114]

[0115] in, Let the network weights of the target encoder be defined at step k. Let be the network weights of the online encoder at step k; α is a hyperparameter (e.g., α can be 0.9). In this way, by using the network weights of the online encoder and the network weights of the target encoder at the time of the last update, the parameters in the target encoder can be updated, thereby improving the training performance of the first model.

[0116] According to the above method, the first model can learn the relationship between regions within the first sample image by reconstructing the first image of the occluded region and the image of the occluded region in the first sample image. The first model can also learn the relationship between images by fusing the predicted features and the target image features, thereby improving the training effect of the first model.

[0117] This disclosure also includes an image processing method, which will be described below in conjunction with... Figure 8 The process of image processing methods is explained.

[0118] Figure 8 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this disclosure. Please refer to [link / reference]. Figure 8The method process includes:

[0119] S801. The first model is used to process multiple images to obtain multiple image features corresponding to the multiple images.

[0120] Optionally, the first model is a model trained by combining reconstructed image contrastive learning with predicted feature contrastive learning. Optionally, the first model can be any of the first models described in the above embodiments. For example, the first model can be... Figure 2 , Figure 6 The first model in the illustrated embodiment.

[0121] S802. Classify the multiple images according to the multiple image features.

[0122] Optionally, multiple images can be classified according to the following feasible implementation: The similarity between two image features corresponding to any two images is obtained, and the multiple images are classified based on this similarity to obtain the image classification result. For example, if the similarity between the image features corresponding to two images is greater than or equal to a first threshold, the two images are determined to be of the same type; if the similarity between the image features corresponding to two images is less than the first threshold, the two images are determined to be of different types. For example, if the image set includes images A, B, C, and D, and the similarity between the image features of image A and image B is greater than the first threshold, and the similarity between the image features of image C and image D is greater than the first threshold, while the similarity between the image features of image A or image B and the image features of image C or image D is less than the first threshold, then the electronic device will classify images A and B into one type of image and images C and image D into another type of image.

[0123] This disclosure provides an image processing method that processes multiple images using a first model to obtain multiple image features corresponding to the multiple images, and then classifies the multiple images based on these multiple image features. Since the first model is trained by combining reconstructed image contrastive learning with predicted feature contrastive learning, it can learn the relationships between different regions within an image and the relationships between images, thereby improving the training effect of the first model and resulting in higher accuracy of the image features output by the first model, thus improving the accuracy of image classification.

[0124] Figure 9 This is a schematic diagram of a model training device provided in an embodiment of this disclosure. Please refer to [link / reference]. Figure 9 The model training device 10 includes a first acquisition module 11, a processing module 12, a reconstruction module 13, a determination module 14, a second acquisition module 15, and an update module 16, wherein:

[0125] The first acquisition module 11 is used to acquire a first sample image, wherein the first sample image includes an unoccluded first image block and an occluded second image block;

[0126] The processing module 12 is used to process the first sample image through the first model to obtain the first image feature corresponding to the first image block;

[0127] The reconstruction module 13 is used to reconstruct the second image block based on the first image features to obtain the first image;

[0128] The determining module 14 is used to determine the fusion prediction features of the first image block and the second image block based on the first image features;

[0129] The second acquisition module 15 is used to acquire target image features in the target image, wherein the target image is the image after preprocessing the first sample image;

[0130] The update module 16 is used to update the model parameters of the first model based on the first image, the second image block, the fusion prediction features, and the target image features.

[0131] In one possible implementation, the update module 16 is specifically used for:

[0132] Based on the first image and the second image patch, determine the first loss function of the first model;

[0133] Based on the fused prediction features and the target image features, determine the second loss function of the first model;

[0134] The model parameters of the first model are updated based on the first loss function and the second loss function.

[0135] In one possible implementation, the update module 16 is specifically used for:

[0136] Acquire a second sample image and determine the second image features of the second sample image, wherein the second sample image is an image other than the first sample image;

[0137] The second loss function is determined based on the fused prediction features, the target image features, and the second image features.

[0138] In one possible implementation, the update module 16 is specifically used for:

[0139] Obtain the first similarity between the fused predicted features and the target image features;

[0140] Obtain the second similarity between the fused predicted features and the second image features;

[0141] The second loss function is determined based on the first similarity and the second similarity.

[0142] In one possible implementation, the update module 16 is specifically used for:

[0143] Obtain the first weight corresponding to the first loss function and the second weight corresponding to the second loss function;

[0144] The model parameters of the first model are updated based on the first loss function, the first weight, the second loss function, and the second weight.

[0145] In one possible implementation, the determining module 14 is specifically used for:

[0146] Obtain the first vector;

[0147] The first vector and the first image feature are fused to obtain a fused vector;

[0148] The fusion prediction feature is obtained based on the fusion vector.

[0149] In one possible implementation, the second acquisition module 15 is specifically used for:

[0150] Based on the position of the second image block in the first sample image, a first region is determined in the target image;

[0151] The first region is offset to obtain the second region;

[0152] The image features corresponding to the image blocks in the second region of the target image are determined as the target image features.

[0153] The model training apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0154] Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure. Please refer to [link / reference]. Figure 10 The image processing device 20 includes a processing module 21 and a classification module 22, wherein:

[0155] The processing module 21 is used to process multiple images through a first model to obtain multiple image features corresponding to the multiple images. The first model is a model trained by combining reconstructed image contrast learning with predicted feature contrast learning.

[0156] The classification module 22 is used to classify the multiple images based on the multiple image features.

[0157] The image processing apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0158] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Please refer to [link / reference]. Figure 11 The diagram illustrates a structural schematic of an electronic device 1100 suitable for implementing embodiments of the present disclosure. The electronic device 1100 can be a terminal device or an electronic device. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0159] like Figure 11 As shown, electronic device 1100 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1102 or a program loaded from storage device 1108 into random access memory (RAM) 1103. RAM 1103 also stores various programs and data required for the operation of electronic device 1100. The processing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0160] Typically, the following devices can be connected to I / O interface 1105: input devices 1106 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1107 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1108 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1109. Communication device 1109 allows electronic device 1100 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 An electronic device 1100 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0161] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1109, or installed from storage device 1108, or installed from ROM 1102. When the computer program is executed by processing device 1101, it performs the functions defined in the methods of embodiments of this disclosure.

[0162] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0163] The aforementioned computer-readable medium may be included in the aforementioned server; or it may exist independently and not assembled into the server.

[0164] The aforementioned computer-readable medium carries one or more programs, which, when executed by the server, cause the server to perform the method shown in the above embodiments.

[0165] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0166] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0167] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0168] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0169] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0170] In a first aspect, this disclosure provides one or more embodiments of a model training method, the method comprising:

[0171] Obtain a first sample image, which includes an unoccluded first image block and an occluded second image block;

[0172] The first sample image is processed by the first model to obtain the first image feature corresponding to the first image block;

[0173] Based on the first image features, the second image block is reconstructed to obtain the first image, and based on the first image features, the fusion prediction features of the first image block and the second image block are determined;

[0174] The target image features are obtained from the target image, which is the image after preprocessing the first sample image.

[0175] The model parameters of the first model are updated based on the first image, the second image patch, the fused prediction features, and the target image features.

[0176] According to one or more embodiments of this disclosure, updating the model parameters of the first model based on the first image, the second image patch, the fused prediction features, and the target image features includes:

[0177] Based on the first image and the second image patch, determine the first loss function of the first model;

[0178] Based on the fused prediction features and the target image features, determine the second loss function of the first model;

[0179] The model parameters of the first model are updated based on the first loss function and the second loss function.

[0180] According to one or more embodiments of this disclosure, determining a second loss function for the first model based on the fused prediction features and the target image features includes:

[0181] Acquire a second sample image and determine the second image features of the second sample image, wherein the second sample image is an image other than the first sample image;

[0182] The second loss function is determined based on the fused prediction features, the target image features, and the second image features.

[0183] According to one or more embodiments of this disclosure, determining the second loss function based on the fused prediction features, the target image features, and the second image features includes:

[0184] Obtain the first similarity between the fused predicted features and the target image features;

[0185] Obtain the second similarity between the fused predicted features and the second image features;

[0186] The second loss function is determined based on the first similarity and the second similarity.

[0187] According to one or more embodiments of this disclosure, updating the model parameters of the first model based on the first loss function and the second loss function includes:

[0188] Obtain the first weight corresponding to the first loss function and the second weight corresponding to the second loss function;

[0189] The model parameters of the first model are updated based on the first loss function, the first weight, the second loss function, and the second weight.

[0190] According to one or more embodiments of this disclosure, determining the fusion prediction features of the first image block and the second image block based on the first image features includes:

[0191] Obtain the first vector;

[0192] The first vector and the first image features are fused to obtain a fused vector;

[0193] The fusion prediction feature is obtained based on the fusion vector.

[0194] According to one or more embodiments of this disclosure, obtaining target image features in a target image includes:

[0195] Based on the position of the second image block in the first sample image, a first region is determined in the target image;

[0196] The first region is offset to obtain the second region;

[0197] The image features corresponding to the image blocks in the second region of the target image are determined as the target image features.

[0198] Secondly, one or more embodiments of this disclosure provide an image processing method, the method comprising:

[0199] Multiple images are processed by a first model to obtain multiple image features corresponding to the multiple images. The first model is a model trained by combining reconstructed image contrastive learning with predicted feature contrastive learning. Optionally, the first model is the first model described in any one of the first aspects.

[0200] The multiple images are classified based on the multiple image features.

[0201] Thirdly, one or more embodiments of this disclosure provide a model training apparatus, which includes a first acquisition module, a processing module, a reconstruction module, a determination module, a second acquisition module, and an update module, wherein:

[0202] The first acquisition module is used to acquire a first sample image, wherein the first sample image includes an unoccluded first image block and an occluded second image block;

[0203] The processing module is used to process the first sample image through the first model to obtain the first image feature corresponding to the first image block;

[0204] The reconstruction module is used to reconstruct the second image block based on the first image features to obtain the first image;

[0205] The determining module is used to determine the fusion prediction features of the first image block and the second image block based on the first image features;

[0206] The second acquisition module is used to acquire target image features in the target image, wherein the target image is the image after preprocessing the first sample image;

[0207] The update module is used to update the model parameters of the first model based on the first image, the second image block, the fusion prediction features, and the target image features.

[0208] In one possible implementation, the update module is specifically used for:

[0209] Based on the first image and the second image patch, determine the first loss function of the first model;

[0210] Based on the fused prediction features and the target image features, determine the second loss function of the first model;

[0211] The model parameters of the first model are updated based on the first loss function and the second loss function.

[0212] In one possible implementation, the update module is specifically used for:

[0213] Acquire a second sample image and determine the second image features of the second sample image, wherein the second sample image is an image other than the first sample image;

[0214] The second loss function is determined based on the fused prediction features, the target image features, and the second image features.

[0215] In one possible implementation, the update module is specifically used for:

[0216] Obtain the first similarity between the fused predicted features and the target image features;

[0217] Obtain the second similarity between the fused predicted features and the second image features;

[0218] The second loss function is determined based on the first similarity and the second similarity.

[0219] In one possible implementation, the update module is specifically used for:

[0220] Obtain the first weight corresponding to the first loss function and the second weight corresponding to the second loss function;

[0221] The model parameters of the first model are updated based on the first loss function, the first weight, the second loss function, and the second weight.

[0222] In one possible implementation, the determining module is specifically used for:

[0223] Obtain the first vector;

[0224] The first vector and the first image features are fused to obtain a fused vector;

[0225] The fusion prediction feature is obtained based on the fusion vector.

[0226] In one possible implementation, the second acquisition module is specifically used for:

[0227] Based on the position of the second image block in the first sample image, a first region is determined in the target image;

[0228] The first region is offset to obtain the second region;

[0229] The image features corresponding to the image blocks in the second region of the target image are determined as the target image features.

[0230] Fourthly, according to one or more embodiments of this disclosure, an image processing apparatus is provided, the image processing apparatus including a processing module and a classification module, wherein:

[0231] The processing module is used to process multiple images through a first model to obtain multiple image features corresponding to the multiple images. The first model is a model trained by combining reconstructed image contrast learning with predicted feature contrast learning.

[0232] The classification module is used to classify the multiple images based on the multiple image features.

[0233] Fifthly, embodiments of this disclosure provide an electronic device, including: a processor and a memory;

[0234] The memory stores computer-executed instructions;

[0235] The processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the model training method as described in the first aspect above and various possible methods involved in the first aspect, or the image processing method as described in the second aspect above and various possible methods involved in the second aspect.

[0236] In a sixth aspect, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the model training method described in the first aspect and various possible embodiments thereof, or the image processing method described in the second aspect and various possible embodiments thereof.

[0237] In a seventh aspect, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the model training method described in the first aspect above and various possible aspects thereof, or the image processing method described in the second aspect above and various possible aspects thereof.

[0238] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0239] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0240] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A model training method, characterized in that, include: Obtain a first sample image, which includes an unoccluded first image block and an occluded second image block; The first sample image is processed by the first model to obtain the first image feature corresponding to the first image block; Based on the first image features, the second image block is reconstructed to obtain the first image, and based on the first image features, the fusion prediction features of the first image block and the occluded second image block are determined; The target image features are obtained from the target image, which is the image after preprocessing the first sample image. Based on the first image and the second image patch, determine the first loss function of the first model; Acquire a second sample image and determine the second image features of the second sample image, wherein the second sample image is an image other than the first sample image; Based on the fused prediction features, the target image features, and the second image features, a second loss function for the first model is determined; The model parameters of the first model are updated based on the first loss function and the second loss function.

2. The method according to claim 1, characterized in that, The second loss function is determined based on the fused prediction features, the target image features, and the second image features, including: Obtain the first similarity between the fused predicted features and the target image features; Obtain the second similarity between the fused predicted features and the second image features; The second loss function is determined based on the first similarity and the second similarity.

3. The method according to claim 1, characterized in that, Based on the first loss function and the second loss function, update the model parameters of the first model, including: Obtain the first weight corresponding to the first loss function and the second weight corresponding to the second loss function; The model parameters of the first model are updated based on the first loss function, the first weight, the second loss function, and the second weight.

4. The method according to claim 1, characterized in that, Based on the first image features, determine the fusion prediction features of the first image patch and the second image patch, including: Obtain the first vector; The first vector and the first image features are fused to obtain a fused vector; The fusion prediction feature is obtained based on the fusion vector.

5. The method according to claim 1, characterized in that, Obtain target image features from the target image, including: Based on the position of the second image block in the first sample image, a first region is determined in the target image; The first region is offset to obtain the second region; The image features corresponding to the image blocks in the second region of the target image are determined as the target image features.

6. An image processing method, characterized in that, include: Multiple images are processed by a first model to obtain multiple image features corresponding to the multiple images. The first model is a model trained by any one of the model training methods of claims 1-5. The multiple images are classified based on the multiple image features.

7. A model training device, characterized in that, It includes a first acquisition module, a processing module, a reconstruction module, a determination module, a second acquisition module, and an update module, wherein: The first acquisition module is used to acquire a first sample image, wherein the first sample image includes an unoccluded first image block and an occluded second image block; The processing module is used to process the first sample image through the first model to obtain the first image feature corresponding to the first image block; The reconstruction module is used to reconstruct the second image block based on the first image features to obtain the first image; The determining module is used to determine the fusion prediction features of the first image block and the occluded second image block based on the first image features; The second acquisition module is used to acquire target image features in the target image, wherein the target image is the image after preprocessing the first sample image; The update module is used to determine the first loss function of the first model based on the first image and the second image patch; Acquire a second sample image and determine the second image features of the second sample image, wherein the second sample image is an image other than the first sample image; Based on the fused prediction features, the target image features, and the second image features, a second loss function for the first model is determined; The model parameters of the first model are updated based on the first loss function and the second loss function.

8. An image processing apparatus, characterized in that, It includes a processing module and a classification module, wherein: The processing module is used to process multiple images using a first model to obtain multiple image features corresponding to the multiple images, wherein the first model is a model trained using any one of the model training methods of claims 1-5; The classification module is used to classify the multiple images based on the multiple image features.

9. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the model training method as described in any one of claims 1 to 5, or the image processing method as described in claim 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the model training method as described in any one of claims 1 to 5, or the image processing method as described in claim 6.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the model training method as described in any one of claims 1 to 5, or the image processing method as described in claim 6.

Images