Model processing method for picture, picture processing method and related equipment

Abstract

The invention discloses a model processing method, a picture processing method and related equipment for pictures, and the method comprises the steps: obtaining an object region picture of a target training picture in a training data set, and enabling the object region picture to be determined according to a part of image regions in the target training picture; inputting the object region picture into a to-be-trained feature extraction model to obtain an object feature vector of the object region picture; the object feature vector is input into n image processors GPUs, a loss function value is obtained based on the object feature vector and a sub-linear matrix configured in each GPU, the sub-linear matrix configured in each GPU is determined after a linear matrix of a linear layer in a to-be-trained feature extraction model is divided, and n is a positive integer; and training a to-be-trained feature extraction model according to the loss function value to obtain a feature extraction model which is used for extracting object features corresponding to the to-be-processed picture. In this way, the image processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, specifically to the field of computer technology, and in particular to a model processing method for images, an image processing method and related equipment. Background Art

[0002] With the continuous development of image processing technology, image processing technology is often used in scenarios such as image search, image recognition, image classification, object detection and scene understanding. In the existing image processing process, a large amount of data processing is usually involved, including operations such as reading, modifying, rendering, and outputting image data. These operations require the support of memory and / or video memory. However, the current image processing efficiency is relatively low. Summary of the invention

[0003] The embodiments of the present application provide a model processing method for images, an image processing method and related devices, which can improve image processing efficiency.

[0004] In a first aspect, an embodiment of the present application provides a model processing method for an image, comprising:

[0005] Obtaining an object region image of a target training image in a training data set, wherein the object region image is determined based on a partial image region in the target training image;

[0006] Inputting the object area picture into the feature extraction model to be trained to obtain an object feature vector corresponding to the object area picture;

[0007] Input the object feature vector into n image processing units (GPUs), and obtain a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU, wherein the sub-linear matrix configured in each GPU is determined by dividing the linear matrix of the linear layer in the feature extraction model to be trained, and n is a positive integer;

[0008] The feature extraction model to be trained is trained according to the loss function value to obtain a feature extraction model, and the feature extraction model is used to extract object features corresponding to the image to be processed.

[0009] In a second aspect, an embodiment of the present application provides a method for processing an image, including:

[0010] Obtaining an object area image of an image to be processed sent by a terminal device;

[0011] Inputting the object area image of the image to be processed into a feature extraction model to generate object features corresponding to the object area image of the image to be processed;

[0012] Processing the image to be processed according to object features corresponding to the object area image of the image to be processed;

[0013] Among them, the processing of the picture to be processed includes: image search processing initiated according to object features corresponding to the object area picture of the picture to be processed, image classification processing according to object features corresponding to the object area picture of the picture to be processed, image detection processing according to object features corresponding to the object area picture of the picture to be processed, and image recognition processing according to object features corresponding to the object area picture of the picture to be processed, any one or more of them.

[0014] In a third aspect, an embodiment of the present application provides a model processing device for an image, including:

[0015] An acquisition unit, used to acquire an object region image of a target training image in a training data set, wherein the object region image is determined based on a partial image region in the target training image;

[0016] An extraction unit, used for inputting the object region picture into a feature extraction model to be trained to obtain an object feature vector corresponding to the object region picture;

[0017] A determination unit, used for inputting the object feature vector into n image processing units (GPUs), and obtaining a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU, wherein the sub-linear matrix configured in each GPU is determined by dividing the linear matrix of the linear layer in the feature extraction model to be trained, and n is a positive integer;

[0018] A training unit is used to train the feature extraction model to be trained according to the loss function value to obtain a feature extraction model, and the feature extraction model is used to extract object features corresponding to the image to be processed.

[0019] In a fourth aspect, an embodiment of the present application provides an image processing device, including:

[0020] An acquisition unit, used for acquiring an object area image of a to-be-processed image sent by a terminal device;

[0021] An extraction unit, used for inputting the object region image of the image to be processed into a feature extraction model to generate object features corresponding to the object region image of the image to be processed;

[0022] A processing unit, configured to process the image to be processed according to object features corresponding to the object area image of the image to be processed;

[0023] Among them, the processing of the picture to be processed includes: image search processing initiated according to object features corresponding to the object area picture of the picture to be processed, image classification processing according to object features corresponding to the object area picture of the picture to be processed, image detection processing according to object features corresponding to the object area picture of the picture to be processed, and image recognition processing according to object features corresponding to the object area picture of the picture to be processed, any one or more of them.

[0024] In a fifth aspect, an embodiment of the present application provides a computing device, comprising a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, and the processor is configured to call the computer program to execute the method described in the first aspect or the second aspect above.

[0025] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, the method described in the first aspect or the second aspect above is implemented.

[0026] In a sixth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0027] In the embodiment of the present application, after inputting the object area image of the target training image in the acquired training data set into the feature extraction model to be trained and obtaining the object feature vector corresponding to the object area image, the object feature vector is input into n GPUs, and based on the object feature vector and the sub-linear matrix configured in each GPU, the loss function value is calculated in each GPU to improve the calculation efficiency. Furthermore, the feature extraction model to be trained is trained according to the loss function value obtained by parallel calculation to obtain the feature extraction model, which can improve the feature extraction efficiency of the feature extraction model, and further improve the image processing efficiency when the feature extraction model is used to process the image to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 It is a structural schematic diagram of a model processing solution for images provided in an embodiment of the present application;

[0030] Figure 2It is a structural schematic diagram of another model processing solution for images provided in an embodiment of the present application;

[0031] Figure 3 It is a structural schematic diagram of another model processing solution for images provided in an embodiment of the present application;

[0032] Figure 4 It is a structural diagram of a model processing system for images provided in an embodiment of the present application;

[0033] Figure 5 It is a flowchart of a model processing method for an image provided in an embodiment of the present application;

[0034] Figure 6 is a schematic diagram of constructing a training data set provided in an embodiment of the present application;

[0035] Figure 7 This is a schematic diagram of a structure for determining a feature label of an object provided in an embodiment of the present application;

[0036] Figure 8a It is a structural schematic diagram of a feature extraction model to be trained provided in an embodiment of the present application;

[0037] Figure 8b It is a structural diagram of a model processing solution for pictures provided in an embodiment of the present application;

[0038] Fig. 9 It is a flowchart of another model processing method for pictures provided in an embodiment of the present application;

[0039] Fig.10 It is a structural schematic diagram of a pre-training process provided in an embodiment of the present application;

[0040] Fig.11 It is a flowchart of another model processing method for pictures provided in an embodiment of the present application;

[0041] Fig.12 It is a structural schematic diagram of a training target feature extraction model provided in an embodiment of the present application;

[0042] Fig.13 It is a flowchart of an interactive method for image processing provided by an embodiment of the present application;

[0043] Fig.14 It is a structural schematic diagram of a model processing device for images provided in an embodiment of the present application;

[0044] Fig.15 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0045] Fig.16 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0048] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0049] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0050] In a specific implementation, the computing device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with a touch surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch surface (e.g., a touch screen display and / or a touch pad).

[0051] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] The embodiment of the present application proposes a model processing scheme for images. The model processing scheme for images can be based on object area pictures of training pictures, and use multiple graphics processing units (GPUs) to calculate loss function values ​​according to sub-linear matrices configured in each GPU and object feature vectors extracted from object area pictures to improve computing efficiency. A feature extraction model for extracting object features of the picture is trained based on the loss function value, which can improve the efficiency of the feature extraction model in extracting object features, and further improve image processing efficiency when the feature extraction model is used to perform image processing on the picture to be processed.

[0053] The model processing scheme for the image may include a classification fine-tuning stage, and the structural diagram of the classification fine-tuning stage is as follows: Figure 1 As shown, Figure 1 Schematic diagram of a model processing solution for an image provided by an embodiment of the present application. Figure 1 As shown, a training dataset with object feature labels (i.e., cluster labels) added can be used. Based on the object region images and object feature labels of the training images in the training dataset, multiple GPUs can be used for parallel processing to train a feature extraction model for extracting object features of images. This large-scale classification task can be used to encourage the model to learn the feature of “similar items are close, and non-similar items are far away”, so that the trained feature extraction model has fine-grained ability to distinguish similar items.

[0054] The model processing solution for images proposed in the embodiment of the present application mainly involves deep learning technology, and specifically may involve one or more technologies in deep learning technology, such as supervised learning, contrastive learning, and knowledge distillation learning (such as student model and teacher model), and obtains a visual feature model (i.e., feature extraction model) based on deep learning technology training.

[0055] Among them, the so-called visual feature model is a computational model designed to capture various features commonly found in images. This model is usually used in the field of computer vision, including tasks such as image search, image recognition, image classification, object detection, and scene understanding. The function of the visual feature model is to map any image to a low-dimensional feature. This feature needs to have the characteristic of "similar ones are close and different ones are far", that is, the distance between different features reflects the similarity between the images. The two features mapped to two more similar images should have a smaller distance, and the two features mapped to two less similar images should have a larger distance.

[0056] Visual feature models have important applications in search, recommendation, and advertising. For example, in search engines that search by image, due to the high dimensionality of the image itself, the original image cannot be directly used for storage and search, so the visual feature model is a key link. In real applications, all image resources will first be extracted using the visual feature model to establish a feature base. When a user uploads an image to initiate a search, the image will first be extracted using the visual feature model, then retrieved from the feature base, recalled features with high similarity, and the corresponding image content will be returned to the user. Visual feature models are usually trained using metric learning methods, usually relying on labeled image data of the same type. After the model extracts image features, it is trained through contrast loss for feature learning. Among them, contrast loss in metric learning is mainly divided into two branches: proxy-based and pair-based. The former proxy-based is that for each cluster (i.e., object feature category), one or more proxies will be used to represent the features of this cluster, and then the distance between all samples in this cluster and the proxy will be optimized. Similar algorithms include softmax-loss and its variants SphereFace, CosFace, ArcFace, etc. The latter pair-based optimizes the distance between a pair of data, requiring that the distance between positive pairs (belonging to the same cluster) features should be smaller than the distance between negative pairs (belonging to different clusters). Similar algorithms include but are not limited to Contrastive Loss, Triplet Loss, etc.

[0057] Specifically, the principle of the model processing scheme for pictures is as follows: obtain the object area picture of the target training picture in the training data set, input the object area picture into the feature extraction model to be trained, and obtain the object feature vector corresponding to the object area picture; input the object feature vector into n image processors GPU, and obtain the loss function value based on the object feature vector and the sub-linear matrix configured in each GPU, where n is a positive integer; train the feature extraction model to be trained according to the loss function value to obtain the feature extraction model. Among them, the feature extraction model is used to extract the object features corresponding to the picture to be processed, so as to perform image search, image classification, image recognition, object detection and other processing on the picture to be processed based on the object features corresponding to the picture to be processed. By using the sub-linear matrix configured in each GPU and the object feature vector of the extracted object area picture to calculate the loss function value, the consumption of memory and video memory is reduced, the calculation efficiency is improved, and the feature extraction model for extracting the object features of the picture is obtained based on the loss function value to improve the efficiency of the feature extraction model in extracting the object features, and further improve the image processing efficiency when the feature extraction model is used to process the picture to be processed.

[0058] Optionally, the model processing scheme for the image may also include a pre-training stage and a classification fine-tuning stage, as follows: Figure 2 As shown, Figure 2 is a schematic diagram of another model processing solution for images provided in an embodiment of the present application. Figure 2 As shown, before the classification fine-tuning stage, a pre-training stage can be added. In the pre-training stage, the pre-training images in the pre-training data set can be used to pre-train the feature extraction model to be trained through knowledge distillation learning. By adding a pre-training stage to pre-train the feature extraction model to be trained, the pre-trained feature extraction model to be trained obtains basic feature extraction capabilities, which helps to further improve the feature extraction capabilities of the feature extraction model trained in the classification fine-tuning stage and enhance the reliability and accuracy of the feature extraction model.

[0059] Further optionally, the model processing scheme for the image may also include a pre-training stage, a classification fine-tuning stage and a comparison fine-tuning stage. The specific structural diagram is as follows: Figure 3 As shown, Figure 3 FIG. 1 is a schematic diagram of another model processing solution for an image provided by an embodiment of the present application. Figure 3 As shown, in order to increase the adaptability of the feature extraction model to actual application scenarios, the actual user behavior can be introduced to optimize the feature extraction model. Specifically, on the basis of adding a pre-training stage before the classification fine-tuning stage, a comparison fine-tuning stage can be added after the classification fine-tuning stage. The feature extraction model obtained in the classification fine-tuning stage is trained by comparative learning using a click training data set (including query images, clicked images of the same type, and unclicked similar images) to obtain a target feature extraction model, so as to increase the model's ability to distinguish between similar images and images of the same type, and output more fine-grained object features, so as to output images that meet user preferences in different application scenarios, which helps to further improve the accuracy of image processing and classification capabilities.

[0060] In a specific implementation, the model processing scheme for images can be applied to image processing schemes in application scenarios such as image search, image classification, image detection, and image recognition. The image processing scheme can be: obtaining an object area picture of the image to be processed sent by a terminal device; inputting the object area picture of the image to be processed into a feature extraction model to generate object features corresponding to the object area picture of the image to be processed; processing the image to be processed according to the object features corresponding to the object area picture of the image to be processed; wherein the processing of the image to be processed includes: image search processing initiated according to the object features corresponding to the object area picture of the image to be processed, image classification processing according to the object features corresponding to the object area picture of the image to be processed, image detection processing according to the object features corresponding to the object area picture of the image to be processed, image recognition processing according to the object features corresponding to the object area picture of the image to be processed, and the like, any one or more of the above.

[0061] In a specific implementation, the above-mentioned model processing solution for images can be executed by a computing device, which can be a terminal device or a server. Alternatively, the above-mentioned model processing solution for images can also be executed by the terminal device and the server together. For example, see Figure 4 As shown, Figure 4 It is a structural schematic diagram of a model processing system for images provided by an embodiment of the present application. The system can obtain the image to be processed by the terminal device, and send the image to be processed to the server. The server obtains the object area image of the image to be processed sent by the terminal device, and inputs the object area image of the image to be processed into the feature extraction model to generate the object features corresponding to the object area image of the image to be processed, and processes the image to be processed according to the object features corresponding to the object area image of the image to be processed. Among them, the processing of the image to be processed includes: image search processing initiated according to the object features corresponding to the object area image of the image to be processed, image classification processing according to the object features corresponding to the object area image of the image to be processed, image detection processing according to the object features corresponding to the object area image of the image to be processed, and image recognition processing according to the object features corresponding to the object area image of the image to be processed. Any one or more of them. It can be understood that here only exemplarily represents the two implementation logics of the terminal device and the server jointly executing the above-mentioned model processing scheme for images, and is not exhaustive.

[0062] Among them, the terminal devices mentioned above can be smart phones, computers (such as tablets, laptops, desktop computers, etc.), smart wearable devices (such as smart watches, smart glasses), smart voice interaction devices, smart home appliances (such as smart TVs), vehicle terminals or aircraft, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, etc. Further, the terminal devices and servers can be located inside or outside the blockchain network, without limitation; further, the terminal devices and servers can also upload any data stored inside to the blockchain network for storage to prevent the data stored inside from being tampered with and improve data security.

[0063] Based on the above description, the embodiment of the present application proposes a model processing method for images. The model processing method for images can be executed by a computing device (i.e., the terminal device or server mentioned above), or by a terminal device and a server together; for ease of explanation, the embodiment of the present application takes the computing device executing the model processing method for images as an example. Please refer to Figure 5 As shown, Figure 5 is a flow chart of a model processing method for an image provided by an embodiment of the present application. The model processing method for an image may include the following steps S501-S504:

[0064] S501: Obtain an object region image of a target training image in a training data set.

[0065] The object area picture is determined based on a partial image area in the target training image, and the partial image area includes an object, which may include but is not limited to any one or more of a person, an object, a plant, an animal, etc. Specifically, when the computing device obtains the object area picture of the target training image in the training data set, it may obtain the object detection frame coordinates of the target training image in the training data set; based on the target training image and the object detection frame coordinates, extract the image area corresponding to the object detection frame coordinates from the target training image, and determine the image area as the object area picture of the target training image. Optionally, the object detection frame coordinates of the target training image in the training data set may be obtained by determining the object detection frame coordinates corresponding to the pre-annotated object detection frame according to the coordinate system established according to the target training image based on the pre-annotated object detection frame.

[0066] It should be noted that the training data set may be pre-constructed, and the training data set includes multiple training data, each of which includes a training image, the object detection frame coordinates of the training image, and the object feature label of the training image. In some embodiments, the object feature label may include but is not limited to any one or more characters such as text, numbers, letters, etc., which are used to indicate the feature category of the object in the training image, such as a hat, clothes, pants, etc.

[0067] In an optional implementation, when constructing a training data set, multiple training images may be obtained, and the object detection frame coordinates of each training image may be determined, and an object feature label may be added to each training image; the multiple training images, the object detection frame coordinates of each training image, and the object feature label of each training image may be constructed into a training data set. Figure 6 Take the example to illustrate the process of building a training data set. Figure 6 is a schematic diagram of constructing a training data set provided in an embodiment of the present application, such as Figure 6 As shown, by acquiring a training picture as raw data 61, detecting objects in the training picture through a detection module 62, and filtering out the objects in the training picture through a classification model and a quality model of an attribute module 63, finally extracting object features corresponding to the object through a feature extraction module 64, and constructing a feature library 65 of the training picture based on the object features, so as to add corresponding object feature labels to each training picture based on the feature library 65. Among them, the detection module 62, the classification model and the quality model of the attribute module 63, and the feature extraction module 64 can be modules in a preset feature extraction model, and the preset feature extraction model can include but is not limited to deep learning, a neural network model, etc., and the embodiment of the present application does not specifically limit the preset feature extraction model.

[0068] In an optional implementation, when adding an object feature label to each training image, each training image can be clustered by a first clustering method to obtain multiple initial category image sets, each initial category image set corresponds to a category, and different initial category image sets correspond to different categories; each training image in each initial category image set is deduplicated according to the fingerprint of each training image, and each training image in each initial category image set after deduplication is denoised to obtain multiple target category image sets; each target category image set is clustered by a second clustering method to obtain multiple target subcategory image sets corresponding to each target category image set, and the feature similarity between each training image in each subcategory image set is greater than a preset similarity threshold; based on multiple target category image sets and multiple target subcategory image sets corresponding to each target category image set, an object feature label is added to each training image. Wherein, the first clustering method may include but is not limited to K-means clustering. It is understandable that the embodiment of the present application does not specifically limit the first clustering method.

[0069] Further, when clustering each target category picture set by the second clustering method to obtain multiple target subcategory picture sets corresponding to each target category picture set, each training picture in each target category picture set can be classified using a preset classification algorithm to obtain a first subcategory picture set and a second subcategory picture set corresponding to each target category picture set; based on the second subcategory picture set, image data crawler processing is performed to obtain a third subcategory picture set corresponding to each training picture in the second subcategory picture set; based on the first subcategory picture set and the third subcategory picture set, multiple target subcategory picture sets corresponding to each target category picture set are obtained. The preset classification algorithm may include but is not limited to a K-NearestNeighbor (KNN) classification algorithm.

[0070] Specific can Figure 7 Take the example to explain the process of adding object feature labels to each training image. Figure 7 is a schematic diagram of a structure for determining an object feature label provided by an embodiment of the present application, such as Figure 7As shown, each training image is clustered by K-means clustering 71 to obtain multiple initial category image sets, and each training image in each initial category image set is deduplicated according to the fingerprint of each training image 72, and each training image in each initial category image set after deduplication is denoised according to the category consistency 73 to obtain multiple target category image sets 74. For example, each target category image set is a hat category image set, a glasses category image set, a scarf category image set, etc., and the categories of each training image in each category image set are the same. For each target category image set, the KNN algorithm 75 is used to further classify each training image in each target category image set, and a sparse matrix 76 is constructed by the k data recalled by the KNN algorithm, and then DBS clustering 77 is performed based on the sparse matrix 76 to obtain the first subcategory image set 78 corresponding to each target category image set. For each target category picture set, the second subcategory picture set that has not been successfully clustered is subjected to picture data crawler processing 79, and the result graph 710 is crawled to obtain the result graph 710, and the result graph 710 is screened and clustered 711 to obtain the third subcategory picture set 712 corresponding to each training picture in the second subcategory picture set. Further, based on the first subcategory picture set and the third subcategory picture set, multiple target subcategory picture sets corresponding to each target category picture set can be obtained, and based on the multiple target category picture sets and the multiple target subcategory picture sets corresponding to each target category picture set, an object feature label is added to each training picture to distinguish the category to which each training picture in each target subcategory picture set belongs. For example, the hat category picture set is clustered to obtain multiple hat category picture sets of the same style (i.e., the target subcategory picture set), and an object feature label indicating the category to which each training picture belongs is added to each training picture.

[0071] The training data set constructed in this way can include training images of objects in the same category, which helps to better train the feature extraction model in the subsequent training.

[0072] S502: Input the object region image into the feature extraction model to be trained to obtain an object feature vector corresponding to the object region image.

[0073] Among them, the feature extraction model to be trained can be a feature model of a convolutional neural network (CNN) framework (such as deep residual network resnet, convnextv2), or a feature model of a transformer framework (such as vit, swin-transformer). It is understandable that the embodiment of the present application does not specifically limit the feature extraction model to be trained.

[0074] In an optional implementation, before inputting the object region image into the feature extraction model to be trained, the object region image may be subjected to random enhancement processing, and the object region image obtained after the random enhancement processing is input into the feature extraction model to be trained. The random enhancement processing method may include but is not limited to flipping, random noise, Gamma transformation, Randomized Quantization, etc., and the embodiment of the present application does not specifically limit the random enhancement processing method.

[0075] In one example, you can Figure 8a Take the structure of the feature extraction model to be trained as an example to illustrate schematically. Figure 8a is a schematic diagram of a structure of a feature extraction model to be trained provided in an embodiment of the present application, such as Figure 8a As shown, each training image in the training data set is input into the feature extraction model to be trained to obtain the object feature vector corresponding to each training image, and the object feature vector corresponding to each training image is input into the linear layer to obtain the feature classification probability corresponding to each training image. Among them, the object feature vector corresponding to each training image can be input into n image processors GPU through the linear layer, and the feature classification probability corresponding to each training image is calculated based on the object feature vector and the sub-linear matrix configured in each GPU, and the loss function value is further determined based on the feature classification probability, and the feature extraction model to be trained is trained according to the loss function value to obtain the feature extraction model. It should be noted that the linear layer is a structural layer of the feature extraction model to be trained, and the linear layer is configured with corresponding parameters, namely, a linear matrix, and each row of the linear matrix corresponds to parameter data of the category indicated by different object feature labels.

[0076] S503: Input the object feature vector into n image processing units (GPUs), and obtain a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU.

[0077] Among them, the sub-linear matrix configured in each GPU is determined after dividing the linear matrix of the linear layer in the feature extraction model to be trained, and n is a positive integer. The specific division method of the linear matrix of the linear layer in the feature extraction model to be trained is as follows: first determine the number of GPUs, and determine the quotient between the number of rows of the linear matrix and the number of GPUs as the number of matrix rows; and split the linear matrix based on the determined number of matrix rows to obtain n sub-linear matrices. For example, assuming that the number of GPUs is determined to be 8, and the linear matrix is ​​a matrix of N*C dimensions, it can be determined that N / 8 is used as the number of rows of each sub-linear matrix to obtain 8 sub-linear matrices, and each sub-linear matrix is ​​a matrix of N / 8*C dimensions. It should be noted that the data of each row of the linear matrix and the sub-linear matrix is ​​used to indicate a category of data, and different rows of data are used to indicate different categories of data. The category indicated by the data of each row is a pre-defined fixed category.

[0078] Further, after dividing the linear matrix of the linear layer in the feature extraction model to be trained, it is also necessary to assign each sub-linear matrix obtained by the division to each GPU, wherein the manner of assigning each sub-linear matrix obtained by the division to each GPU can be: sorting each sub-linear matrix according to the order in which the rows included in each sub-linear matrix are located in the linear matrix, and sequentially assigning each sorted sub-linear matrix to the corresponding GPU according to a preset GPU order (such as the order of GPU numbers, a custom order, etc.). It is understandable that the embodiment of the present application does not specifically limit the manner in which each sub-linear matrix obtained by the division is assigned to each GPU.

[0079] For example, suppose there are 4 GPUs, the preset GPU order is GPU1, GPU2, GPU3, GPU4, the linear matrix is ​​M, and the 4 sub-linear matrices are A, B, C, D. If the rows of sub-linear matrix A correspond to the 1st, ..., tth rows of M, the rows of B correspond to the t+1th, ..., 2t+1th rows of M, the rows of C correspond to the 2t+2th, ..., 3t+2th rows of M, and the rows of D correspond to the 3t+3th, ..., 4t+3th rows of M, where 4t+3 is the last row of M, then it can be determined that A is assigned to GPU1, B is assigned to GPU2, C is assigned to GPU3, and D is assigned to GPU4.

[0080] In one embodiment, when the computing device obtains the loss function value based on the object feature vector and the sub-linear matrix configured in each GPU, it can determine the feature classification probability of the object feature vector calculated in each GPU based on the product of the object feature vector and the sub-linear matrix configured in each GPU; obtain the object feature labels of the target training images in the training data set, and determine the loss function parameters calculated by each GPU based on the feature classification probability and object feature labels calculated by the determined object feature vector in each GPU; and calculate the loss function value based on the loss function parameters calculated by each GPU. In an optional embodiment, the method for calculating the loss function value may include but is not limited to any method such as SphereFace, ArcFace, CosFace, etc., wherein SphereFace is a loss function based on spherical measurement, which maps the object feature vector onto a sphere in the feature space to calculate the loss function value; ArcFace is a loss function based on angle cosine, which introduces the concept of angle interval in the feature control to calculate the loss function value; CosFace is a loss function based on cosine similarity, which introduces the concept of cosine angle in the feature space to calculate the loss function value; the embodiment of the present application does not specifically limit the method for calculating the loss function value.

[0081] In an optional embodiment, when the computing device determines the feature classification probability of the object feature vector calculated in each GPU based on the product of the object feature vector and the sub-linear matrix configured in each GPU, it can determine that the product of the object feature vector calculated by each GPU and the corresponding configured sub-linear matrix is ​​the product matrix calculated by each GPU; determine each classification probability of the object feature vector calculated in each GPU based on the data of each row in the product matrix calculated by each GPU; and determine the classification probability with the largest median value of the classification probabilities calculated by the object feature vector in each GPU as the feature classification probability of the object feature vector calculated in each GPU.

[0082] For example, assuming that there are two GPUs, the sub-linear matrices corresponding to the two GPUs are X and Y, and the object feature vector is A, then the product matrix of the object feature vector A and the sub-linear matrix X calculated by one GPU can be determined to be A*X, and the product matrix of the object feature vector A and the sub-linear matrix Y calculated by another GPU can be determined to be A*Y. If, based on the data of each row in the product matrix A*X, it is determined that each classification probability calculated by the object feature vector in one GPU includes a and b, and based on the data of each row in the product matrix A*Y, it is determined that each classification probability calculated by the object feature vector in another GPU includes c and d; if c is the largest among a, b, c, and d, then the classification probability c with the largest value among the classification probabilities a, b, c, and d calculated by the object feature vector in each GPU is determined as the feature classification probability calculated by the object feature vector in each GPU.

[0083] The embodiment of the present application utilizes multiple GPUs to perform parallel calculations based on the object feature vector and the sub-linear matrices configured in each GPU, so that the feature classification probability corresponding to the object feature vector can be determined more quickly and effectively.

[0084] S504: Train the feature extraction model to be trained according to the loss function value to obtain a feature extraction model, where the feature extraction model is used to extract object features corresponding to the image to be processed.

[0085] Among them, when the feature extraction model to be trained is trained according to the loss function value to obtain the feature extraction model, the loss function value can be compared with the preset threshold value. When the loss function value is greater than the preset threshold value, the loss function value is back-propagated to the feature extraction model to be trained to obtain the gradient of each model parameter in the feature extraction model to be trained, so as to determine the model parameters involved in the calculation in each GPU, and the gradient of each model parameter is distributed to the corresponding GPU, so that each GPU adjusts the corresponding model parameter based on the gradient of the distributed model parameter. After adjusting the model parameters, the object feature vector is input into n GPUs, and the loss function value is recalculated based on the object feature vector and the sub-linear matrix configured in each GPU, until the loss function value is less than the preset threshold value, it is determined that the feature extraction model is obtained.

[0086] Optional, can be combined with Figure 8b The model processing scheme for the image is explained. Figure 8b is a structural diagram of a model processing solution for images provided in an embodiment of the present application, such as Figure 8bAs shown, firstly, the target training picture 81, the object detection frame 82 and the object feature label 83 of the target training picture in the training data set are obtained, and the object area picture 84 of the target training picture is determined based on the target training picture 81 and the object detection frame 82. The object area picture 84 can be subjected to random enhancement processing 85, and the object area picture after the random enhancement processing 85 is input into the feature extraction model to be trained 86 to obtain the object feature vector 87, and the object feature vector 87 is mapped to the classification probability through the linear layer fc88, and the loss function value 89 is calculated based on the classification probability and the object feature label 83, so as to train the feature extraction model to be trained according to the loss function value to obtain the feature extraction model.

[0087] The embodiment of the present application utilizes multiple GPUs to train the feature extraction model to be trained according to the loss function value, which can reduce the consumption of memory and video memory in the model training process, and help improve the efficiency of feature extraction of the trained feature extraction model.

[0088] In a specific implementation, after training the feature extraction model, the computing device can obtain the object area picture of the picture to be processed sent by the terminal device; input the object area picture of the picture to be processed into the feature extraction model to generate object features corresponding to the object area picture of the picture to be processed; and process the picture to be processed according to the object features corresponding to the object area picture of the picture to be processed; wherein the processing of the picture to be processed includes: any one or more of image search processing initiated according to the object features corresponding to the object area picture of the picture to be processed, image classification processing performed according to the object features corresponding to the object area picture of the picture to be processed, image detection processing performed according to the object features corresponding to the object area picture of the picture to be processed, and image recognition processing performed according to the object features corresponding to the object area picture of the picture to be processed.

[0089] The embodiment of the present application can efficiently and accurately extract the object features of the image to be processed by utilizing the trained feature extraction model, which helps to further perform any one or more image processing such as image search, image classification, image detection, etc. on the image to be processed based on the object features of the image to be processed, thereby improving the efficiency and accuracy of image processing.

[0090] In the embodiment of the present application, the object region image of the target training image in the acquired training data set is input into the feature extraction model to be trained, the object feature vector corresponding to the object region image is obtained, the object feature vector is input into n GPUs, and based on the object feature vector and the sub-linear matrix configured in each GPU, the loss function value is calculated in each GPU to improve the calculation efficiency. Furthermore, the feature extraction model to be trained is trained according to the loss function value obtained by parallel calculation to obtain the feature extraction model, which can improve the feature extraction efficiency of the feature extraction model, and further improve the image processing efficiency when the feature extraction model is used to process the image to be processed.

[0091] The embodiment of the present application also proposes a model processing method for an image. In the embodiment of the present application, the computing device executing the model processing method for an image is still used as an example for explanation. Figure 5 The difference between the embodiments shown is that, before the feature extraction model to be trained is trained to obtain the feature extraction model, the embodiment of the present application adds a process of pre-training to obtain the feature extraction model to be trained. Fig. 9 As shown, Fig. 9 is a flow chart of another model processing method for an image provided by an embodiment of the present application. The model processing method for an image may include the following steps S901-S906:

[0092] S901: Obtain a target pre-training image in a pre-training dataset.

[0093] S902: Use the target pre-training image to train the preset student model to obtain a feature extraction model to be trained.

[0094] In a specific implementation, the computing device can use a first random enhancement method to randomly enhance the target pre-trained image to obtain a first enhanced image, perform image segmentation on the first enhanced image to obtain multiple first segmented image blocks, and randomly mask each first segmented image block to obtain multiple first image blocks; use a second random enhancement method to randomly enhance the target pre-trained image to obtain a second enhanced image, and perform image segmentation on the second enhanced image to obtain multiple second image blocks; use multiple first image blocks and multiple second image blocks to train a preset student model to obtain a feature extraction model to be trained.

[0095] Furthermore, when the computing device uses multiple first image blocks and multiple second image blocks to train a preset student model to obtain a feature extraction model to be trained, each first image block can be input into the preset student model to obtain a first global feature probability and a first local feature probability, and each second image block can be input into the preset teacher model to obtain a second global feature probability and a second local feature probability; a global loss function value is calculated based on the first global feature probability and the second global feature probability, and a local loss function value is calculated based on the first local feature probability and the second local feature probability; model parameters of the preset student model are adjusted based on the global loss function value and the local loss function value to obtain the feature extraction model to be trained.

[0096] Specific can Fig.10 Take this as an example for schematic explanation. Fig.10 is a schematic diagram of a pre-training process provided in an embodiment of the present application, such as Fig.10 As shown, first, a target pre-training image 101 is obtained, and then a first random enhancement method T1 is used to randomly enhance the target pre-training image 101 to obtain a first enhanced image 102, and image segmentation is performed on the first enhanced image to obtain multiple first segmented image blocks 1021, and each of the first segmented image blocks is randomly masked 1022, and the multiple first image blocks obtained by the random mask processing 1022 are input into a preset student model 1023, and the first global feature probability 10241 is extracted by the global projection head 1024, and the first local feature probability 10251 is extracted by the local projection head 1025. On the other hand, the target pre-training image 101 is randomly enhanced using the second random enhancement method T2 to obtain a second enhanced image 103, and the second enhanced image 103 is subjected to image segmentation to obtain a plurality of second image blocks 1031. Each second image block 1031 is input into a preset teacher model 1032, and a second global feature probability 10331 is extracted through a global projection head 1033, and a second local feature probability 10341 is extracted through a local projection head 1034. Further, a global loss function value 104 is calculated based on the first global feature probability 10241 and the second global feature probability 10331, and a local loss function value 105 is calculated based on the first local feature probability 10251 and the second local feature probability 10341.

[0097] It should be noted that the teacher model is the exponential moving average (Exponential Moving Average) result of the student model. Both features are extracted through the global projection head and the local projection head to extract global features and local features respectively. On the one hand, the global loss function value is calculated based on the two global feature probabilities as the cross entropy loss between the classification probability distribution of the global feature prediction output by the student model (i.e., the global feature probability) and the classification probability distribution of the global feature prediction output by the teacher model (i.e., the local feature probability). The global features extracted by the student model are encouraged to be as close as possible to the global features of the teacher model, and image-level self-supervised learning is performed. On the other hand, the probability distribution of the local feature prediction output by the student model and the probability distribution of the local feature prediction output by the teacher model will also calculate the cross entropy loss, and the student model is encouraged to reconstruct the masked image through MIM (Masked Image Modeling).

[0098] The embodiment of the present application obtains a feature extraction model to be trained by pre-training a preset student model using a target pre-training image, so that the feature extraction model to be trained has basic feature extraction capabilities, which helps to improve the accuracy of the feature extraction model obtained by subsequent training of the feature extraction model to be trained.

[0099] S903: Obtain an object region image of a target training image in a training data set.

[0100] S904: Input the object region image into the feature extraction model to be trained to obtain an object feature vector corresponding to the object region image.

[0101] S905: Input the object feature vector into n image processing units (GPUs), and obtain a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU.

[0102] S906: Train the feature extraction model to be trained according to the loss function value to obtain a feature extraction model, where the feature extraction model is used to extract object features corresponding to the image to be processed.

[0103] The process of training the feature extraction model to be trained to obtain the feature extraction model in the embodiment of the present application is as described above. Figure 5 The embodiments shown in the figure are described in detail and will not be described again here.

[0104] The embodiment of the present application can pre-train the feature extraction model to be trained by adding a pre-training stage, so that the feature extraction model to be trained obtained by pre-training can obtain basic feature extraction capabilities, which helps to further improve the reliability and accuracy of the feature extraction model trained in the classification fine-tuning stage.

[0105] The embodiment of the present application also proposes a model processing method for an image. In the embodiment of the present application, the computing device executing the model processing method for an image is still used as an example for explanation. Figure 5 , Fig. 9 The difference between the embodiments shown is that, before the feature extraction model to be trained is trained to obtain the feature extraction model, the embodiment of the present application adds a process of pre-training to obtain the feature extraction model to be trained, and then introduces user click data to train the feature extraction model after the feature extraction model is trained to obtain the target feature extraction model, so as to increase the fine-grained differentiation ability of the model and further improve the accuracy and reliability of the model. Fig.11 As shown, Fig.11 1 is a flow chart of another model processing method for an image provided by an embodiment of the present application. The model processing method for an image may include the following steps S1101-S1108:

[0106] S1101: Obtain a target pre-training image in a pre-training dataset.

[0107] S1102: Use the target pre-training image to train the preset student model to obtain a feature extraction model to be trained.

[0108] S1103: Obtain an object region image of a target training image in a training data set.

[0109] S1104: Input the object region image into the feature extraction model to be trained to obtain an object feature vector corresponding to the object region image.

[0110] S1105: Input the object feature vector into n image processing units (GPUs), and obtain a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU.

[0111] S1106: Train the feature extraction model to be trained according to the loss function value to obtain a feature extraction model, where the feature extraction model is used to extract object features corresponding to the image to be processed.

[0112] S1107: Obtain a click training data set.

[0113] The click training data set includes multiple click training data, and each click training data includes a query image, a clicked image of the same type, and a non-clicked similar image.

[0114] In a specific implementation, when obtaining a click training data set, the computing device may obtain a query image, and search the search engine based on the query image to obtain various search images corresponding to the query image; detect whether each search image has been clicked; if the detection result is yes, add a click tag to the search image that has been clicked to determine that the search image with the click tag is a clicked image of the same type as the query image; if the detection result is no, add a not-clicked tag to the search image that has not been clicked to determine that the search image with the not-clicked tag is a not-clicked similar image of the query image; determine the query image, the clicked image of the same type and the not-clicked similar image as one click training data, and determine multiple click training data as a click training data set.

[0115] S1108: Use the click training data set to train the feature extraction model to obtain a target feature extraction model.

[0116] In a specific implementation, the computing device can input the query image, the clicked same-item image, and the unclicked similar image in each click training data into a feature extraction model to obtain query image features, clicked same-item image features, and unclicked similar image features; based on the query image features and the clicked same-item image features, calculate the first distance between the query image features and the clicked same-item image features; based on the query image features and the unclicked similar image features, calculate the second distance between the query image features and the unclicked similar image features; train the feature extraction model according to the first distance and the second distance to obtain a target feature extraction model, wherein during the training process, the difference between the first distance and the second distance is less than a preset distance threshold. The target feature extraction model is used to extract object features, same-item object features, and similar object features corresponding to the image to be processed.

[0117] During the specific training process, if the difference between the first distance and the second distance is greater than or equal to the preset distance threshold, the model parameters of the feature extraction model are adjusted according to the first distance and the second distance, and the query image, clicked same image and unclicked similar image are input into the feature extraction model after the model parameters are adjusted. The training is repeated until the difference between the first distance and the second distance obtained through training is less than the preset distance threshold, the training is stopped, and it is determined that the target feature extraction model is obtained.

[0118] Below Fig.12 Take as an example the process of training the feature extraction model using a set of click training data. Fig.12 is a structural diagram of a training target feature extraction model provided in an embodiment of the present application, such as Fig.12As shown, the query image 121, the clicked same picture 122 and the unclicked similar picture 123 in the click training data are input into the feature extraction model 124 to obtain the query image feature 125, the clicked same picture feature 126 and the unclicked similar picture feature 127; the feature extraction model is trained by calculating the first distance between the query image feature 125 and the clicked same picture feature 126 and the second distance between the query image feature 125 and the unclicked similar picture feature 127 to obtain the target feature extraction model 128.

[0119] In practical applications, an object area image of a to-be-processed image sent by a terminal device can be obtained; the object area image of the to-be-processed image is input into a target feature extraction model to generate object features, same-item object features, and similar object features corresponding to the object area image of the to-be-processed image; the to-be-processed image is processed according to the object features, same-item object features, and similar object features corresponding to the object area image of the to-be-processed image. The processing of the to-be-processed image includes: image search processing initiated according to the object features, same-item object features, and similar object features corresponding to the object area image of the to-be-processed image, image classification processing according to the object features, same-item object features, and similar object features corresponding to the object area image of the to-be-processed image, image detection processing according to the object features, same-item object features, and similar object features corresponding to the object area image of the to-be-processed image, and image recognition processing according to the object features, same-item object features, and similar object features corresponding to the object area image of the to-be-processed image.

[0120] Furthermore, according to the image search processing initiated based on the object features, same-item object features and similar-item features corresponding to the object area picture of the picture to be processed, the search picture corresponding to the object features, the same-item picture corresponding to the same-item object features and the similar picture corresponding to the similar-item features can be searched and obtained. It is understandable that the image classification processing based on the object features, same-item object features and similar-item features corresponding to the object area picture of the picture to be processed, the image detection processing based on the object features, same-item object features and similar-item features corresponding to the object area picture of the picture to be processed, and the image recognition processing based on the object features, same-item object features and similar-item features corresponding to the object area picture of the picture to be processed are similar to the image search processing process and will not be described in detail here.

[0121] In an embodiment of the present application, by introducing clicked same pictures and unclicked similar pictures to train the feature extraction model, it is possible to introduce actual user behavior to optimize the feature extraction model, so that the optimized target feature extraction model has the ability to more accurately distinguish between the same pictures and similar pictures, thereby improving the accuracy of image processing (such as classification) of the target feature extraction model.

[0122] See also Fig.13 , Fig.13 : is a flowchart of an interactive method for image processing provided by an embodiment of the present application. The interactive method for image processing can be jointly executed by a terminal device and a server. The interactive method for image processing may include the following steps:

[0123] S1301: The terminal device obtains the image to be processed.

[0124] The user may input the image to be processed through the terminal device; the user may also send the image to be processed to the terminal device through other terminal devices; or the user may send an image processing request to the terminal device through other terminal devices, wherein the image processing request carries the image to be processed. It is understandable that the embodiment of the present application does not specifically limit the manner in which the terminal device obtains the image to be processed.

[0125] S1302: The terminal device sends the image to be processed to the server.

[0126] S1303: The server obtains the object area image of the image to be processed.

[0127] S1304: The server inputs the object region image of the image to be processed into a feature extraction model to generate object features corresponding to the object region image of the image to be processed.

[0128] S1305: The server processes the image to be processed according to the object features corresponding to the object area image of the image to be processed.

[0129] Among them, the processing of the picture to be processed includes: image search processing initiated according to object features corresponding to the object area picture of the picture to be processed, image classification processing according to object features corresponding to the object area picture of the picture to be processed, image detection processing according to object features corresponding to the object area picture of the picture to be processed, and image recognition processing according to object features corresponding to the object area picture of the picture to be processed, any one or more of them.

[0130] S1306: The server sends the processing result to the terminal device.

[0131] In the embodiment of the present application, the terminal device sends the acquired image to be processed to the server, so that the server obtains the object area image of the image to be processed, and inputs the object area image of the image to be processed into the feature extraction model, generates the object features corresponding to the object area image of the image to be processed, and processes the image to be processed according to the object features corresponding to the object area image of the image to be processed, and sends the processing result obtained after the processing to the terminal device. By using the feature extraction model obtained by training, the object features of the image to be processed can be extracted efficiently and accurately, and the image to be processed is processed based on the object features of the image to be processed, which helps to improve the accuracy and efficiency of image processing.

[0132] Based on the description of the above-mentioned model processing method embodiment for pictures, the present application embodiment also discloses a model processing device for pictures; the model processing device for pictures can be a computer program (including one or more instructions) running in a computing device, and the model processing device for pictures can execute each step in the above-mentioned related method flow. Fig.14 , Fig.14 : is a schematic diagram of a structure of a model processing device for an image provided by an embodiment of the present application. The model processing device for an image can run the following units:

[0133] An acquiring unit 1401 is used to acquire an object region image of a target training image in a training data set, wherein the object region image is determined according to a partial image region in the target training image;

[0134] An extraction unit 1402 is used to input the object region picture into a feature extraction model to be trained to obtain an object feature vector corresponding to the object region picture;

[0135] A determination unit 1403 is used to input the object feature vector into n image processing units (GPUs), and obtain a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU, wherein the sub-linear matrix configured in each GPU is determined by dividing the linear matrix of the linear layer in the feature extraction model to be trained, and n is a positive integer;

[0136] The training unit 1404 is used to train the feature extraction model to be trained according to the loss function value to obtain a feature extraction model, and the feature extraction model is used to extract object features corresponding to the image to be processed.

[0137] Further, the determining unit 1403 is further configured to:

[0138] Determining the number of the GPUs;

[0139] Determine the quotient between the number of rows of the linear matrix and the number of GPUs as the number of matrix rows;

[0140] The linear matrix is ​​split based on the determined number of matrix rows to obtain n sub-linear matrices.

[0141] Further, when the determination unit 1403 obtains the loss function value based on the object feature vector and the sub-linear matrix configured in each GPU, it can be specifically used to:

[0142] Determine, based on the product of the object feature vector and the sub-linear matrix configured in each GPU, a feature classification probability of the object feature vector calculated in each GPU;

[0143] Obtaining object feature labels of the target training images in the training data set, and determining loss function parameters calculated by each GPU according to the feature classification probabilities and the object feature labels calculated by the determined object feature vectors in each GPU;

[0144] The loss function value is calculated based on the loss function parameters calculated by the GPUs.

[0145] Further, when determining the feature classification probability of the object feature vector calculated in each GPU based on the product of the object feature vector and the sub-linear matrix configured in each GPU, the determining unit 1403 can be specifically used to:

[0146] Determine that the product of the object feature vector calculated by each GPU and the corresponding configured sub-linear matrix is ​​the product matrix calculated by each GPU;

[0147] Determine, based on the data of each row in the product matrix calculated by each GPU, each classification probability obtained by calculating the object feature vector in each GPU;

[0148] The classification probability with the largest median value among the classification probabilities calculated by the object feature vector in the respective GPUs is determined as the feature classification probability calculated by the object feature vector in the respective GPUs.

[0149] Furthermore, before acquiring the object region image of the target training image in the training data set, the acquiring unit 1401 is further configured to:

[0150] Obtain multiple training images, determine the coordinates of the object detection box of each training image, and add object feature labels to each training image;

[0151] Constructing the plurality of training pictures, the object detection frame coordinates of the respective training pictures, and the object feature label of each training picture into the training data set;

[0152] Wherein, when adding an object feature label to each training image, the acquisition unit 1401 is specifically used to:

[0153] Clustering the training images in a first clustering manner to obtain a plurality of initial category image sets, each initial category image set corresponding to a category, and different initial category image sets corresponding to different categories;

[0154] Deduplication processing is performed on each training image in each initial category image set according to the fingerprint of each training image, and denoising is performed on each training image in each initial category image set after deduplication processing, so as to obtain multiple target category image sets;

[0155] Clustering each target category picture set by a second clustering method to obtain a plurality of target subcategory picture sets corresponding to each target category picture set, wherein the feature similarity between each training picture in each subcategory picture set is greater than a preset similarity threshold;

[0156] Based on the multiple target category picture sets and the multiple target subcategory picture sets corresponding to each target category picture set, the object feature label is added to each training picture.

[0157] Further, when the acquisition unit 1401 clusters each target category picture set by the second clustering method to obtain a plurality of target subcategory picture sets corresponding to each target category picture set, it can be specifically used to:

[0158] Using a preset classification algorithm to classify each training picture in each target category picture set, to obtain a first subcategory picture set and a second subcategory picture set corresponding to each target category picture set;

[0159] Performing image data crawling processing based on the second subcategory image set to obtain a third subcategory image set corresponding to each training image in the second subcategory image set;

[0160] Based on the first subcategory picture set and the third subcategory picture set, a plurality of target subcategory picture sets corresponding to each target category picture set are obtained.

[0161] Furthermore, after training the feature extraction model to be trained according to the loss function value to obtain the feature extraction model, the training unit 1404 is further used to:

[0162] Obtain a click training data set, wherein the click training data set includes a plurality of click training data, each of which includes a query image, a clicked image of the same type, and a non-clicked similar image;

[0163] The feature extraction model is trained using the click training data set to obtain a target feature extraction model.

[0164] Furthermore, when acquiring the click training data set, the training unit 1404 may be specifically used to:

[0165] Obtaining a query image, and searching a search engine based on the query image to obtain various search images corresponding to the query image;

[0166] Detecting whether each of the search images has been clicked;

[0167] If the detection result is yes, a click tag is added to the search image that has been clicked, so as to determine that the search image to which the click tag is added is the same clicked image as the query image;

[0168] If the detection result is no, then adding a not-clicked label to the search image that has not been clicked, so as to determine that the search image with the not-clicked label is an unclicked similar image of the query image;

[0169] The query image, the clicked same image and the unclicked similar image are determined as one click training data, and a plurality of the click training data are determined as the click training data set.

[0170] Further, the training unit 1404 trains the feature extraction model using the click training data set to obtain a target feature extraction model, which can be specifically used for:

[0171] Input the query image, the clicked same-item image, and the unclicked similar image in each click training data into the feature extraction model to obtain query image features, clicked same-item image features, and unclicked similar image features;

[0172] Based on the query image feature and the clicked same-item image feature, a first distance between the query image feature and the clicked same-item image feature is calculated;

[0173] Based on the query image feature and the unclicked similar image feature, a second distance between the query image feature and the unclicked similar image feature is calculated;

[0174] The feature extraction model is trained according to the first distance and the second distance to obtain the target feature extraction model, wherein during the training process, the difference between the first distance and the second distance is less than a preset distance threshold.

[0175] Furthermore, before inputting the object area image into the feature extraction model to be trained, the acquisition unit 1401 may also be used to:

[0176] Obtain the target pre-training image in the pre-training dataset;

[0177] Using a first random enhancement method to perform random enhancement processing on the target pre-trained image to obtain a first enhanced image, performing image segmentation processing on the first enhanced image to obtain a plurality of first segmented image blocks, and performing random mask processing on each of the first segmented image blocks to obtain a plurality of first image blocks;

[0178] Using a second random enhancement method to perform random enhancement processing on the target pre-trained image to obtain a second enhanced image, and performing image segmentation processing on the second enhanced image to obtain a plurality of second image blocks;

[0179] The preset student model is trained using the multiple first image blocks and the multiple second image blocks to obtain the feature extraction model to be trained.

[0180] Further, when the acquisition unit 1401 uses the multiple first image blocks and the multiple second image blocks to train the preset student model to obtain the feature extraction model to be trained, it can be specifically used to:

[0181] Input each first image block into the preset student model to obtain a first global feature probability and a first local feature probability, and input each second image block into the preset teacher model to obtain a second global feature probability and a second local feature probability;

[0182] A global loss function value is calculated based on the first global feature probability and the second global feature probability, and a local loss function value is calculated based on the first local feature probability and the second local feature probability;

[0183] According to the global loss function value and the local loss function value, the model parameters of the preset student model are adjusted to obtain the feature extraction model to be trained.

[0184] Further, when acquiring the object area image of the target training image in the training data set, the acquiring unit 1401 may be specifically used to:

[0185] Obtaining the object detection frame coordinates of the target training image in the training data set;

[0186] Based on the target training picture and the object detection frame coordinates, an image area corresponding to the object detection frame coordinates is extracted from the target training picture, and the image area is determined to be an object area picture of the target training picture.

[0187] Fig.14 The various units in the model processing device for pictures shown can be combined into one or several other units separately or in whole, or one (some) of the units can be further divided into multiple smaller units in function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, other units may also be included based on the model processing device for pictures. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0188] The above method can be constructed by running a computer program (including one or more instructions) capable of executing each step involved in any of the above method embodiments on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM) and other processing elements and storage elements. Fig.14 The model processing device for pictures shown in the embodiment of the present application is used to implement the model processing method for pictures. The computer program can be recorded on a computer-readable storage medium, for example, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.

[0189] It is worth noting that in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can contain a part of the overall module or unit of the module or unit function.

[0190] In the embodiment of the present application, the object region image of the target training image in the acquired training data set is input into the feature extraction model to be trained, the object feature vector corresponding to the object region image is obtained, the object feature vector is input into n GPUs, and based on the object feature vector and the sub-linear matrix configured in each GPU, the loss function value is calculated in each GPU to improve the calculation efficiency. Furthermore, the feature extraction model to be trained is trained according to the loss function value obtained by parallel calculation to obtain the feature extraction model, which can improve the feature extraction efficiency of the feature extraction model, and further improve the image processing efficiency when the feature extraction model is used to process the image to be processed.

[0191] Based on the description of the above picture processing method embodiment, the present application embodiment also discloses a picture processing device; the picture processing device can be a computer program (including one or more instructions) running in a computing device, and the picture processing device can execute each step in the above related method flow. Fig.15 , Fig.15 : is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application, wherein the image processing device can run the following units:

[0192] An acquiring unit 1501 is used to acquire an object area picture of a to-be-processed picture sent by a terminal device;

[0193] An extraction unit 1502 is used to input the object region image of the image to be processed into a feature extraction model to generate object features corresponding to the object region image of the image to be processed;

[0194] A processing unit 1503 is used to process the image to be processed according to the object features corresponding to the object area image of the image to be processed;

[0195] Among them, the processing of the picture to be processed includes: image search processing initiated according to object features corresponding to the object area picture of the picture to be processed, image classification processing according to object features corresponding to the object area picture of the picture to be processed, image detection processing according to object features corresponding to the object area picture of the picture to be processed, and image recognition processing according to object features corresponding to the object area picture of the picture to be processed, any one or more of them.

[0196] The embodiment of the present application obtains an object region image of a to-be-processed image, inputs the object region image of the to-be-processed image into a feature extraction model, generates object features corresponding to the object region image of the to-be-processed image, and processes the to-be-processed image according to the object features corresponding to the object region image of the to-be-processed image. The object features of the to-be-processed image can be efficiently and accurately extracted, and the to-be-processed image can be processed based on the object features of the to-be-processed image, which helps to improve the accuracy and efficiency of image processing.

[0197] Based on the description of the above method embodiment and device embodiment, the present application embodiment also provides a computing device. Fig.16 , Fig.16 is a schematic diagram of a computing device provided in an embodiment of the present application, such as Fig.16 As shown, the computing device at least includes a processor 1601, an input interface 1602, an output interface 1603, and a computer-readable storage medium 1604. The processor 1601, the input interface 1602, the output interface 1603, and the computer-readable storage medium 1604 in the computing device can be connected via a bus or other means. The computer-readable storage medium 1604 can be stored in the memory of the computing device, and the computer-readable storage medium 1604 is used to store a computer program, and the computer program includes one or more instructions, and the processor 1601 is used to execute one or more instructions in the computer program stored in the computer-readable storage medium 1604. The processor 1601 (or CPU (Central Processing Unit)) is the computing core and control core of the computing device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function.

[0198] The processor 1601 described in the embodiment of the present application may be used to perform the following steps:

[0199] Obtaining an object region image of a target training image in a training data set, wherein the object region image is determined based on a partial image region in the target training image;

[0200] Inputting the object area picture into the feature extraction model to be trained to obtain an object feature vector corresponding to the object area picture;

[0201] Input the object feature vector into n image processing units (GPUs), and obtain a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU, wherein the sub-linear matrix configured in each GPU is determined by dividing the linear matrix of the linear layer in the feature extraction model to be trained, and n is a positive integer;

[0202] The feature extraction model to be trained is trained according to the loss function value to obtain a feature extraction model, and the feature extraction model is used to extract object features corresponding to the image to be processed.

[0203] Furthermore, the processor 1601 is further configured to:

[0204] Determining the number of the GPUs;

[0205] Determine the quotient between the number of rows of the linear matrix and the number of GPUs as the number of matrix rows;

[0206] The linear matrix is ​​split based on the determined number of matrix rows to obtain n sub-linear matrices.

[0207] Further, when the processor 1601 obtains the loss function value based on the object feature vector and the sub-linear matrix configured in each GPU, it can be specifically used to:

[0208] Determine, based on the product of the object feature vector and the sub-linear matrix configured in each GPU, a feature classification probability of the object feature vector calculated in each GPU;

[0209] Obtaining object feature labels of the target training images in the training data set, and determining loss function parameters calculated by each GPU according to the feature classification probabilities and the object feature labels calculated by the determined object feature vectors in each GPU;

[0210] The loss function value is calculated based on the loss function parameters calculated by the GPUs.

[0211] Further, when the processor 1601 determines the feature classification probability of the object feature vector calculated in each GPU based on the product of the object feature vector and the sub-linear matrix configured in each GPU, it can be specifically used to:

[0212] Determine that the product of the object feature vector calculated by each GPU and the corresponding configured sub-linear matrix is ​​the product matrix calculated by each GPU;

[0213] Determine, based on the data of each row in the product matrix calculated by each GPU, each classification probability obtained by calculating the object feature vector in each GPU;

[0214] The classification probability with the largest median value among the classification probabilities calculated by the object feature vector in the respective GPUs is determined as the feature classification probability calculated by the object feature vector in the respective GPUs.

[0215] Furthermore, before acquiring the object area image of the target training image in the training data set, the processor 1601 is further configured to:

[0216] Obtain multiple training images, determine the coordinates of the object detection box of each training image, and add object feature labels to each training image;

[0217] Constructing the plurality of training pictures, the object detection frame coordinates of the respective training pictures, and the object feature label of each training picture into the training data set;

[0218] Wherein, when adding an object feature label to each training image, the processor 1601 is specifically used to:

[0219] Clustering the training images in a first clustering manner to obtain a plurality of initial category image sets, each initial category image set corresponding to a category, and different initial category image sets corresponding to different categories;

[0220] Deduplication processing is performed on each training image in each initial category image set according to the fingerprint of each training image, and denoising is performed on each training image in each initial category image set after deduplication processing, so as to obtain multiple target category image sets;

[0221] Clustering each target category picture set by a second clustering method to obtain a plurality of target subcategory picture sets corresponding to each target category picture set, wherein the feature similarity between each training picture in each subcategory picture set is greater than a preset similarity threshold;

[0222] Based on the multiple target category picture sets and the multiple target subcategory picture sets corresponding to each target category picture set, the object feature label is added to each training picture.

[0223] Further, when the processor 1601 clusters each target category picture set by the second clustering method to obtain a plurality of target subcategory picture sets corresponding to each target category picture set, it can be specifically used to:

[0224] Using a preset classification algorithm to classify each training picture in each target category picture set, to obtain a first subcategory picture set and a second subcategory picture set corresponding to each target category picture set;

[0225] Performing image data crawling processing based on the second subcategory image set to obtain a third subcategory image set corresponding to each training image in the second subcategory image set;

[0226] Based on the first subcategory picture set and the third subcategory picture set, a plurality of target subcategory picture sets corresponding to each target category picture set are obtained.

[0227] Further, after training the feature extraction model to be trained according to the loss function value to obtain the feature extraction model, the processor 1601 is further configured to:

[0228] Obtain a click training data set, wherein the click training data set includes a plurality of click training data, each of which includes a query image, a clicked image of the same type, and a non-clicked similar image;

[0229] The feature extraction model is trained using the click training data set to obtain a target feature extraction model.

[0230] Furthermore, when acquiring the click training data set, the processor 1601 may be specifically configured to:

[0231] Obtaining a query image, and searching a search engine based on the query image to obtain various search images corresponding to the query image;

[0232] Detecting whether each of the search images has been clicked;

[0233] If the detection result is yes, a click tag is added to the search image that has been clicked, so as to determine that the search image to which the click tag is added is the same clicked image as the query image;

[0234] If the detection result is no, then adding a not-clicked label to the search image that has not been clicked, so as to determine that the search image with the not-clicked label is an unclicked similar image of the query image;

[0235] The query image, the clicked same image and the unclicked similar image are determined as one click training data, and a plurality of the click training data are determined as the click training data set.

[0236] Further, when the processor 1601 trains the feature extraction model using the click training data set to obtain the target feature extraction model, it can be specifically used to:

[0237] Input the query image, the clicked same-item image, and the unclicked similar image in each click training data into the feature extraction model to obtain query image features, clicked same-item image features, and unclicked similar image features;

[0238] Based on the query image feature and the clicked same-item image feature, a first distance between the query image feature and the clicked same-item image feature is calculated;

[0239] Based on the query image feature and the unclicked similar image feature, a second distance between the query image feature and the unclicked similar image feature is calculated;

[0240] The feature extraction model is trained according to the first distance and the second distance to obtain the target feature extraction model, wherein during the training process, the difference between the first distance and the second distance is less than a preset distance threshold.

[0241] Furthermore, before inputting the object area image into the feature extraction model to be trained, the processor 1601 may also be used to:

[0242] Obtain the target pre-training image in the pre-training dataset;

[0243] Using a first random enhancement method to perform random enhancement processing on the target pre-trained image to obtain a first enhanced image, performing image segmentation processing on the first enhanced image to obtain a plurality of first segmented image blocks, and performing random mask processing on each of the first segmented image blocks to obtain a plurality of first image blocks;

[0244] Using a second random enhancement method to perform random enhancement processing on the target pre-trained image to obtain a second enhanced image, and performing image segmentation processing on the second enhanced image to obtain a plurality of second image blocks;

[0245] The preset student model is trained using the multiple first image blocks and the multiple second image blocks to obtain the feature extraction model to be trained.

[0246] Further, when the processor 1601 uses the multiple first image blocks and the multiple second image blocks to train the preset student model to obtain the feature extraction model to be trained, it can be specifically used to:

[0247] Input each first image block into the preset student model to obtain a first global feature probability and a first local feature probability, and input each second image block into the preset teacher model to obtain a second global feature probability and a second local feature probability;

[0248] A global loss function value is calculated based on the first global feature probability and the second global feature probability, and a local loss function value is calculated based on the first local feature probability and the second local feature probability;

[0249] According to the global loss function value and the local loss function value, the model parameters of the preset student model are adjusted to obtain the feature extraction model to be trained.

[0250] Further, when the processor 1601 obtains the object area image of the target training image in the training data set, it can be specifically used to:

[0251] Obtaining the object detection frame coordinates of the target training image in the training data set;

[0252] Based on the target training picture and the object detection frame coordinates, an image area corresponding to the object detection frame coordinates is extracted from the target training picture, and the image area is determined to be an object area picture of the target training picture.

[0253] In the embodiment of the present application, the object region image of the target training image in the acquired training data set is input into the feature extraction model to be trained, the object feature vector corresponding to the object region image is obtained, the object feature vector is input into n GPUs, and based on the object feature vector and the sub-linear matrix configured in each GPU, the loss function value is calculated in each GPU to improve the calculation efficiency. Furthermore, the feature extraction model to be trained is trained according to the loss function value obtained by parallel calculation to obtain the feature extraction model, which can improve the feature extraction efficiency of the feature extraction model, and further improve the image processing efficiency when the feature extraction model is used to process the image to be processed.

[0254] It should be understood that in the embodiment of the present application, the processor 1601 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0255] In a specific implementation, the processor 1601 described in the embodiment of the present application can execute the implementation method described in the data processing method or the text data processing method provided in the embodiment of the present application, and can also execute the implementation method of the model processing device or the image processing device for the image described in the embodiment of the present application, which will not be repeated here.

[0256] An embodiment of the present application also provides a computer-readable storage medium, in which program instructions are stored, and when the program instructions are executed, any of the data processing methods described above is implemented.

[0257] The computer-readable storage medium may be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory of the device. The computer-readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the mobile device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0258] The embodiments of the present application also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes the methods provided in the above various embodiments.

[0259] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0260] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the terminals and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

Claims

1. A model processing method for an image, characterized in that: include: Obtaining an object region image of a target training image in a training data set, wherein the object region image is determined based on a partial image region in the target training image; Inputting the object area picture into the feature extraction model to be trained to obtain an object feature vector corresponding to the object area picture; Input the object feature vector into n image processing units (GPUs), and obtain a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU, wherein the sub-linear matrix configured in each GPU is determined by dividing the linear matrix of the linear layer in the feature extraction model to be trained, and n is a positive integer; The feature extraction model to be trained is trained according to the loss function value to obtain a feature extraction model, and the feature extraction model is used to extract object features corresponding to the image to be processed.

2. The method according to claim 1, characterized in that The method further comprises: Determining the number of the GPUs; Determine the quotient between the number of rows of the linear matrix and the number of GPUs as the number of matrix rows; The linear matrix is ​​split based on the determined number of matrix rows to obtain n sub-linear matrices.

3. The method according to claim 1, characterized in that The step of obtaining a loss function value based on the object feature vector and the sub-linear matrices configured in each GPU includes: Determine, based on the product of the object feature vector and the sub-linear matrix configured in each GPU, a feature classification probability of the object feature vector calculated in each GPU; Obtaining object feature labels of the target training images in the training data set, and determining loss function parameters calculated by each GPU according to the feature classification probabilities and the object feature labels calculated by the determined object feature vectors in each GPU; The loss function value is calculated based on the loss function parameters calculated by the GPUs.

4. The method according to claim 3, characterized in that The determining, based on the product of the object feature vector and the sub-linear matrix configured in each GPU, the feature classification probability calculated by the object feature vector in each GPU includes: Determine that the product of the object feature vector calculated by each GPU and the corresponding configured sub-linear matrix is ​​the product matrix calculated by each GPU; Determine, based on the data of each row in the product matrix calculated by each GPU, each classification probability obtained by calculating the object feature vector in each GPU; The classification probability with the largest median value among the classification probabilities calculated by the object feature vector in the respective GPUs is determined as the feature classification probability calculated by the object feature vector in the respective GPUs.

5. The method according to claim 1, characterized in that Before obtaining the object area image of the target training image in the training data set, the method further includes: Obtain multiple training images, determine the coordinates of the object detection box of each training image, and add object feature labels to each training image; Constructing the plurality of training pictures, the object detection frame coordinates of the respective training pictures, and the object feature label of each training picture into the training data set; The step of adding an object feature label to each training image includes: Clustering the training images in a first clustering manner to obtain a plurality of initial category image sets, each initial category image set corresponding to a category, and different initial category image sets corresponding to different categories; Deduplication processing is performed on each training image in each initial category image set according to the fingerprint of each training image, and denoising is performed on each training image in each initial category image set after deduplication processing, so as to obtain multiple target category image sets; Clustering each target category picture set by a second clustering method to obtain a plurality of target subcategory picture sets corresponding to each target category picture set, wherein the feature similarity between each training picture in each subcategory picture set is greater than a preset similarity threshold; Based on the multiple target category picture sets and the multiple target subcategory picture sets corresponding to each target category picture set, the object feature label is added to each training picture.

6. The method according to claim 5, characterized in that The clustering of each target category picture set by the second clustering method to obtain a plurality of target subcategory picture sets corresponding to each target category picture set includes: Using a preset classification algorithm to classify each training picture in each target category picture set, to obtain a first subcategory picture set and a second subcategory picture set corresponding to each target category picture set; Performing image data crawling processing based on the second subcategory image set to obtain a third subcategory image set corresponding to each training image in the second subcategory image set; Based on the first subcategory picture set and the third subcategory picture set, a plurality of target subcategory picture sets corresponding to each target category picture set are obtained.

7. The method according to claim 1, characterized in that After the feature extraction model to be trained is trained according to the loss function value to obtain the feature extraction model, the method further includes: Obtain a click training data set, wherein the click training data set includes a plurality of click training data, each of which includes a query image, a clicked image of the same type, and a non-clicked similar image; The feature extraction model is trained using the click training data set to obtain a target feature extraction model.

8. The method according to claim 7, characterized in that The step of obtaining a click training data set includes: Obtaining a query image, and searching a search engine based on the query image to obtain various search images corresponding to the query image; Detecting whether each of the search images has been clicked; If the detection result is yes, a click tag is added to the search image that has been clicked, so as to determine that the search image to which the click tag is added is the same clicked image as the query image; If the detection result is no, then adding a not-clicked label to the search image that has not been clicked, so as to determine that the search image with the not-clicked label is an unclicked similar image of the query image; The query image, the clicked same image and the unclicked similar image are determined as one click training data, and a plurality of the click training data are determined as the click training data set.

9. The method according to claim 7, characterized in that The step of training the feature extraction model using the click training data set to obtain a target feature extraction model includes: Input the query image, the clicked same-item image, and the unclicked similar image in each click training data into the feature extraction model to obtain query image features, clicked same-item image features, and unclicked similar image features; Based on the query image feature and the clicked same-item image feature, a first distance between the query image feature and the clicked same-item image feature is calculated; Based on the query image feature and the unclicked similar image feature, a second distance between the query image feature and the unclicked similar image feature is calculated; The feature extraction model is trained according to the first distance and the second distance to obtain the target feature extraction model, wherein during the training process, the difference between the first distance and the second distance is less than a preset distance threshold.

10. The method according to claim 1, characterized in that Before inputting the object area picture into the feature extraction model to be trained, the method further includes: Obtain the target pre-training image in the pre-training dataset; Using a first random enhancement method to perform random enhancement processing on the target pre-trained image to obtain a first enhanced image, performing image segmentation processing on the first enhanced image to obtain a plurality of first segmented image blocks, and performing random mask processing on each of the first segmented image blocks to obtain a plurality of first image blocks; Using a second random enhancement method to perform random enhancement processing on the target pre-trained image to obtain a second enhanced image, and performing image segmentation processing on the second enhanced image to obtain a plurality of second image blocks; The preset student model is trained using the multiple first image blocks and the multiple second image blocks to obtain the feature extraction model to be trained.

11. The method according to claim 10, characterized in that The method of using the plurality of first image blocks and the plurality of second image blocks to train a preset student model to obtain the feature extraction model to be trained includes: Input each first image block into the preset student model to obtain a first global feature probability and a first local feature probability, and input each second image block into the preset teacher model to obtain a second global feature probability and a second local feature probability; A global loss function value is calculated based on the first global feature probability and the second global feature probability, and a local loss function value is calculated based on the first local feature probability and the second local feature probability; According to the global loss function value and the local loss function value, the model parameters of the preset student model are adjusted to obtain the feature extraction model to be trained.

12. The method according to claim 1, characterized in that The step of obtaining an object region image of a target training image in a training data set includes: Obtaining the object detection frame coordinates of the target training image in the training data set; Based on the target training picture and the object detection frame coordinates, an image area corresponding to the object detection frame coordinates is extracted from the target training picture, and the image area is determined to be an object area picture of the target training picture.

13. A method for processing an image, characterized in that: include: Obtaining an object area image of an image to be processed sent by a terminal device; Inputting the object area image of the image to be processed into a feature extraction model to generate object features corresponding to the object area image of the image to be processed; Processing the image to be processed according to object features corresponding to the object area image of the image to be processed; Among them, the processing of the picture to be processed includes: image search processing initiated according to object features corresponding to the object area picture of the picture to be processed, image classification processing according to object features corresponding to the object area picture of the picture to be processed, image detection processing according to object features corresponding to the object area picture of the picture to be processed, and image recognition processing according to object features corresponding to the object area picture of the picture to be processed, any one or more of them.

14. A model processing device for an image, characterized in that: include: An acquisition unit, used to acquire an object region image of a target training image in a training data set, wherein the object region image is determined according to a partial image region in the target training image; An extraction unit, used for inputting the object region picture into a feature extraction model to be trained to obtain an object feature vector corresponding to the object region picture; A determination unit, used for inputting the object feature vector into n image processing units (GPUs), and obtaining a loss function value based on the object feature vector and a sub-linear matrix configured in each GPU, wherein the sub-linear matrix configured in each GPU is determined by dividing the linear matrix of the linear layer in the feature extraction model to be trained, and n is a positive integer; A training unit is used to train the feature extraction model to be trained according to the loss function value to obtain a feature extraction model, and the feature extraction model is used to extract object features corresponding to the image to be processed.

15. A picture processing device, characterized in that: include: An acquisition unit, used for acquiring an object area image of a to-be-processed image sent by a terminal device; An extraction unit, used for inputting the object region image of the image to be processed into a feature extraction model to generate object features corresponding to the object region image of the image to be processed; A processing unit, configured to process the image to be processed according to object features corresponding to the object area image of the image to be processed; Among them, the processing of the picture to be processed includes: image search processing initiated according to object features corresponding to the object area picture of the picture to be processed, image classification processing according to object features corresponding to the object area picture of the picture to be processed, image detection processing according to object features corresponding to the object area picture of the picture to be processed, and image recognition processing according to object features corresponding to the object area picture of the picture to be processed, any one or more of them.

16. A computing device, characterized in that The method comprises a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, and the processor is configured to call the computer program to execute the method according to any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, which, when executed, implement the method according to any one of claims 1 to 13.

18. A computer program product, characterized in that The computer program product comprises program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images