Image processing method, device, apparatus and storage medium

Abstract

The application discloses an image processing method, device and equipment and a storage medium, and belongs to the technical field of artificial intelligence. In the embodiment of the application, a pseudo classification label is used to label a second image when training an image classification model, and a corresponding loss value is designed based on the pseudo classification label to indicate whether it is accurate, based on which the model parameters are updated, and the pseudo classification label is also updated in the updating process. In this way, only part of the images have corresponding classification labels, pseudo classification labels are generated for other images in the model training process, that is, all images do not need to have corresponding classification labels, which can greatly reduce the labor cost brought by manual labeling and improve the training efficiency. Moreover, the pseudo classification label is continuously updated in the model training process, and finally a pseudo classification label with almost the same accuracy as the classification label can be determined, so as to improve the number of training samples and the accuracy of the image classification model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to an image processing method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, more and more fields use machine learning to train models to help people complete complex calculations.

[0003] Among them, in the field of image processing, an image classification model can be trained to classify human tissue images and determine the attribute type in the human tissue images. At present, the image processing method usually manually annotates human tissue images, and trains an image classification model based on images and annotation data. However, the manual annotation process consumes a large amount of manual cost, is low in efficiency and prone to errors. SUMMARY

[0004] The embodiments of the present application provide an image processing method, device, equipment and storage medium, which can improve the processing efficiency and accuracy. The technical solution is as follows:

[0005] In one aspect, an image processing method is provided, which includes:

[0006] Based on the image classification model, the prediction classification results of at least two first images and at least two second images are obtained, the first images carry corresponding classification labels, the second images carry corresponding pseudo-classification labels, the first images and the second images are human tissue images, and the prediction classification results, the classification labels and the pseudo-classification labels are used to indicate the attribute type in the images;

[0007] Based on the prediction classification results of the at least two first images and the corresponding classification labels, and the at least two second images and the corresponding pseudo-classification labels, a first loss value is obtained, the first loss value is used to indicate the accuracy of the prediction classification results of the images;

[0008] Based on the prediction classification results of the at least two second images and the corresponding pseudo-classification labels, a second loss value is obtained, the second loss value is used to indicate the accuracy of the pseudo-classification labels;

[0009] Based on the first loss value and the second loss value, the model parameters of the image classification model are updated, the corresponding pseudo-classification labels are updated based on the prediction classification results of the at least two second images, and the classification and loss value obtaining steps are continued to be executed until a target condition is met, and a target image classification model is obtained.

[0010] In one aspect, an image processing device is provided, which includes:

[0011] an acquisition module configured to acquire, based on the image classification model, predicted classification results of at least two first images and at least two second images, the first images carrying corresponding classification labels, the second images carrying corresponding pseudo classification labels, the first images and the second images being human tissue images, the predicted classification results, the classification labels, and the pseudo classification labels being used to indicate attribute types in the images;

[0012] The acquisition module is further configured to acquire, based on the predicted classification results of the at least two first images and the corresponding classification labels, and the at least two second images and the corresponding pseudo classification labels, a first loss value, the first loss value being used to indicate accuracy of the predicted classification results of the images.

[0013] The acquisition module is further configured to acquire, based on the predicted classification results of the at least two second images and the corresponding pseudo classification labels, a second loss value, the second loss value being used to indicate accuracy of the pseudo classification labels.

[0014] an update module configured to update model parameters of the image classification model based on the first loss value and the second loss value, continue to perform the classification and the loss value acquisition based on the predicted classification results of the at least two second images and the updated pseudo classification labels, and obtain a target image classification model when a target condition is met.

[0015] In some embodiments, the acquisition module is configured to:

[0016] acquire, according to distances between the at least two second images and different cluster centers, probabilities corresponding to the at least two images, the probabilities being probabilities that the pseudo classification labels are correct classification labels;

[0017] acquire, based on the probabilities corresponding to the at least two images, weight values of the at least two images.

[0018] In some embodiments, the acquisition module is configured to:

[0019] in response to the probability corresponding to any second image being greater than or equal to a probability threshold value, acquire the probability as a weight value of the second image;

[0020] in response to the probability corresponding to any second image being less than the probability threshold value, acquire a zero value as a weight value of the second image.

[0021] In some embodiments, the update module is configured to acquire, according to the predicted classification results of the at least two second images and clustering results, pseudo classification labels corresponding to the predicted classification results.

[0022] In some embodiments, the acquisition module is configured to:

[0023] The at least two second images, the predicted classification results of the at least two second images, and the corresponding pseudo classification labels are processed based on a Gaussian mixture model to obtain the second loss value.

[0024] The updating module is further configured to update model parameters of the Gaussian mixture model based on the second loss value.

[0025] In some embodiments, the obtaining module is further configured to obtain a predicted type of the at least two first images and the at least two second images according to the at least two first images and the at least two second images and the corresponding predicted classification results, the predicted type being used to indicate whether an image is a first image or a second image.

[0026] The obtaining module is further configured to obtain a third loss value according to the predicted type of the at least two first images and the at least two second images, the third loss value being used to indicate accuracy of the predicted type.

[0027] In some embodiments, the updating module is configured to update model parameters of the image classification model based on the first loss value, the second loss value, and the third loss value.

[0028] In some embodiments, the obtaining module is configured to determine a type of the at least two first images and the at least two second images based on a discriminative network according to the at least two first images and the at least two second images and the corresponding predicted classification results to obtain the predicted type of the at least two first images and the at least two second images.

[0029] The updating module is further configured to update network parameters of the discriminative network according to the third loss value.

[0030] In some embodiments, the obtaining module is further configured to obtain a third image in response to an image processing instruction, the third image being the human tissue image.

[0031] The apparatus further includes:

[0032] a classification module configured to input the third image into the target image classification model, extract features of the third image by the target image classification model, classify the third image based on the extracted image features, and obtain an attribute type in the third image.

[0033] In an aspect, an electronic device is provided, which includes one or more processors and one or more memories having at least one computer program stored therein, the at least one computer program is loaded and executed by the one or more processors to implement various optional implementations of the above image processing method.

[0034] In an aspect, a computer readable storage medium is provided, which stores at least one computer program, the at least one computer program is loaded and executed by a processor to implement various optional implementations of the above image processing method.

[0035] In an aspect, a computer program product or a computer program is provided, which includes one or more program codes stored in a computer readable storage medium. One or more processors of an electronic device can read the one or more program codes from the computer readable storage medium, and execute the one or more program codes, so that the electronic device can perform the image processing method of any possible implementation described above.

[0036] The embodiments of the present application adopt a pseudo classification label to label the second image when training the image classification model, and design a corresponding loss value based on the pseudo classification label to indicate whether it is accurate, update the model parameters based on this, and the pseudo classification label is also updated in the updating process. In this way, only part of the images correspond to the classification label, and the pseudo classification label is generated for other images in the model training process, that is, all images do not need to correspond to the classification label, which can greatly reduce the labor cost brought by manual labeling and improve the training efficiency. And the pseudo classification label is updated in the model training process, and finally the pseudo classification label with almost the same accuracy as the classification label can be determined to improve the number of training samples and improve the accuracy of the image classification model. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 is a schematic diagram of an implementation environment of an image processing method provided by the embodiments of the present application;

[0039] Figure 2 is a flowchart of an image processing method provided by the embodiments of the present application;

[0040] Figure 3 is a flowchart of an image processing method provided by an embodiment of the present application;

[0041] Figure 4 is a schematic diagram of a training system framework provided by an embodiment of the present application;

[0042] Figure 5 is a flowchart of an image processing method provided by an embodiment of the present application;

[0043] Figure 6 is a structural schematic diagram of an image processing apparatus provided by an embodiment of the present application;

[0044] Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0045] Figure 8 is a structural block diagram of a terminal provided by an embodiment of the present application;

[0046] Figure 9 is a structural schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0048] The terms "first", "second", and the like are used in the present application to distinguish between elements having substantially the same function and similar items, and it should be understood that there is no logical or chronological dependency between "first", "second", and "nth", and the number and execution order are not limited. It should also be understood that although the following description uses the terms first, second, and the like to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. The first image and the second image can both be images, and in some cases, can be separate and distinct images.

[0049] In the present application, the term "at least one" means one or more, and the term "multiple" in the present application means two or more, for example, multiple data packets refer to two or more data packets.

[0050] It should be understood that the terms used in the description of various examples herein are only for the purpose of describing specific examples and are not intended to be limiting.

[0051] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0052] It should also be understood that determining B based on A does not mean determining B solely based on A; it is also possible to determine B based on A and / or other information.

[0053] The following is an explanation of the terms used in this application.

[0054] Pseudo-labels: These are labels that approximate the classification of unlabeled data without requiring manual labeling. Instead, they are derived from labeled data to provide an approximate classification label. In other words, for unlabeled data (such as human tissue images), a classification label is determined based on other labeled data. This label is not manually assigned but calculated using technical means, hence the term "pseudo-label." Pseudo-label algorithms are a type of self-learning method and are widely used in computer vision for various classification tasks.

[0055] Human tissue images, also known as medical images, are images of human tissue. By processing these images, we can analyze their attributes. For example, we can analyze whether there are lesions, their location, and their type.

[0056] Lesion: A diseased part of the body, a localized diseased tissue containing pathogenic microorganisms. For example, if a part of the lung is destroyed by tuberculosis bacteria, this part is a pulmonary tuberculosis lesion. Another example is the ability to analyze otoscopy data to determine whether there are lesions in the tympanic membrane and what type of lesion they are.

[0057] Otitis media is a common ear disease. If it is not detected and diagnosed in time, and the patient does not receive timely treatment, it may leave serious sequelae (such as hearing loss) or even deafness. Otitis media can be treated by acquiring images of the middle ear using an otoscope. These images, also known as otoscope data, can be processed to determine the location of the tympanic membrane and analyze its type, such as normal or pathological tympanic membranes. Pathological tympanic membranes may include tympanic membrane sclerosis and tympanitis.

[0058] This application relates to artificial intelligence technology, which trains an image classification model to enable the model to process human tissue images, thereby replacing manual calculations. The artificial intelligence is described below.

[0059] Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computer or machine controlled by digital computer to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making.

[0060] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0061] Computer Vision (CV) is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further process graphics so that the computer processing becomes more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0062] Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0063] The scheme provided by the embodiments of the present application relates to image processing, machine learning and other technologies in computer vision technology of artificial intelligence, and is specifically explained through the following embodiments.

[0064] The implementation environment of the present application is described below.

[0065] Figure 1 is a schematic diagram of an implementation environment of an image processing method provided by the embodiments of the present application. The implementation environment includes a terminal 101, or the implementation environment includes the terminal 101 and an image processing platform 102. The terminal 101 is connected to the image processing platform 102 through a wireless network or a wired network.

[0066] The terminal 101 can be at least one of a smart phone, a game console, a desktop computer, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, or a laptop computer. The terminal 101 is installed and runs an application supporting image processing, for example, the application can be a system application, an image processing application, etc.

[0067] Exemplarily, the terminal 101 can have an image acquisition function and an image processing function, can process the acquired image, and execute corresponding functions according to the processing result. For example, the terminal 101 can be an otoscope device. In some embodiments, the terminal 101 can be a portable otoscope device. The terminal 101 can independently complete the work, or can be provided with data services by the image processing platform 102. The embodiments of the present application do not limit this.

[0068] The image processing platform 102 includes at least one of a server, multiple servers, a cloud computing platform, and a virtualization center. The image processing platform 102 is used to provide background services for the application supporting image processing. Optionally, the image processing platform 102 undertakes the main processing work, and the terminal 101 undertakes the secondary processing work; or the image processing platform 102 undertakes the secondary processing work, and the terminal 101 undertakes the main processing work; or the image processing platform 102 or the terminal 101 can undertake the processing work independently. Alternatively, the image processing platform 102 and the terminal 101 adopt a distributed computing architecture for collaborative computing.

[0069] Optionally, the image processing platform 102 comprises at least one server 1021 and a database 1022 for storing data, in the embodiment of the present application, the database 1022 can store sample human tissue images or trained image classification models, and provide data services for the at least one server 1021.

[0070] In some embodiments, the server 1021 in the image processing platform 102 can be a node device in a blockchain system, and the server 1021 can store sample human tissue images or image classification models trained based on images on the blockchain of the blockchain system. In some embodiments, the blockchain system can provide image processing services, and when an electronic device needs to perform image classification, it can send an image classification request to the blockchain system, and the node device of the blockchain system can respond to the image classification request and classify the image.

[0071] The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms, etc. The terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0072] Those skilled in the art can know that the number of the terminal 101 and the server 1021 can be more or less. For example, the terminal 101 and the server 1021 can be only one, or the terminal 101 and the server 1021 can be dozens or hundreds, or more, and the number and type of the terminal or the server are not limited in the embodiment of the present application.

[0073] Figure 2 is a flowchart of an image processing method provided by the embodiment of the present application, the method is applied in an electronic device, which is a terminal or a server, referring to Figure 2 Taking the case that the method is applied in a terminal, the method comprises the following steps.

[0074] 201, the terminal obtains the predicted classification results of at least two first images and at least two second images based on an image classification model, the first image carries a corresponding classification label, the second image carries a corresponding pseudo-classification label, the first image and the second image are human tissue images, and the predicted classification result, the classification label and the pseudo-classification label are used to indicate the attribute type in the image.

[0075] The classification label is used to represent the correct classification result or the true classification result of the image. The classification label can be obtained by manual labeling. For example, the number of classification categories can be two or more than two. For example, there are five classification categories, and five classification labels of 0, 1, 2, 3, and 4 can be set to identify the five classification categories. Each classification label corresponds to a classification category, and through the classification label, the correct classification result of the image carrying the classification label can be known, that is, the image should belong to which classification category after classification.

[0076] The pseudo classification label is not obtained by manual labeling, but is a label similar to the classification label by processing the image.

[0077] In the embodiment of the present application, the pseudo classification label can be generated in the training process of the image classification model, and the model parameters of the image classification model in the training process are constantly optimized, so the pseudo classification label is also constantly optimized in the training process.

[0078] The first image and the second image are sample images, which can also be referred to as sample human tissue images. The predicted classification result is a prediction result obtained by the image classification model, and the classification label and the pseudo classification label are used to indicate the true value, indicating the correct or true result. The sample images are processed by the image classification model, and based on the predicted classification result obtained by the image classification model and the classification label or the pseudo classification label carried thereby, whether the predicted classification result obtained by the image classification model is accurate is analyzed, so as to optimize the model parameters of the image classification model and improve the processing performance of the image classification model.

[0079] The predicted classification result, the classification label, and the pseudo classification label are used to indicate the attribute type in the image. The attribute type can be the type of some attributes in the image. For example, the attribute can be a lesion in the image, and the human tissue image can be an ear image, which can also be referred to as an otoscope image. The predicted classification result, the classification label, and the pseudo classification label can be used to indicate the type of the tympanic membrane in the ear image. The type can be whether it is normal. If it is normal, the normal type can further include complete health, health, or cured recovery. If it is not normal, the abnormal type can further include otitis media, tympanosclerosis, etc. For example, the attribute can also be other attributes in the image, such as image clarity, image acquisition distance, color tone, image style, etc. This is only an exemplary description, and those skilled in the art can set the above types according to the needs, and the embodiment of the present application is not limited in this regard.

[0080] 202、the terminal obtains a first loss value based on the predicted classification result of the at least two first images and the corresponding classification label, and the at least two second images and the corresponding pseudo classification label, the first loss value being used to indicate the accuracy of the predicted classification result of the image.

[0081] After the terminal determines the predicted classification result, the terminal can evaluate whether the predicted classification result is accurate according to the label carried by the image, so as to measure the processing performance of the image classification model.

[0082] Here, the first loss value is used as a measurement index of the processing performance. By comparing the predicted result (predicted classification result) and the correct result (classification label or pseudo classification label), the accuracy of the predicted result can be determined. It can be understood that the more accurate the predicted classification result is, the better the processing performance (such as accuracy) of the image classification model is, and the purpose of model training is to improve the processing performance of the image classification model.

[0083] 203、the terminal obtains a second loss value based on the predicted classification result of the at least two second images and the corresponding pseudo classification label, the second loss value being used to indicate the accuracy of the pseudo classification label.

[0084] In the embodiments of the present application, not only the accuracy of the predicted classification result is determined by the predicted classification result and the classification label or the pseudo classification label to improve the classification accuracy of the model, but also the accuracy of the pseudo classification label is estimated to further determine the classification accuracy of the image classification model. Thus, the pseudo classification label will be updated based on the predicted classification result in the next stage of training, and through the training process of multiple stages, the pseudo classification label becomes more and more accurate, and the classification of the image classification model also becomes more and more accurate.

[0085] 204、the terminal updates the model parameters of the image classification model based on the first loss value and the second loss value, and continues to perform the classification and loss value obtaining steps after the corresponding pseudo classification label is updated based on the predicted classification result of the at least two second images, until a target condition is met, and a target image classification model is obtained.

[0086] The above step 201 is an iteration process, after the terminal obtains the required loss value, the model parameters can be updated based on the loss value, and then the pseudo classification label used in the next stage can be updated. Through multiple iteration processes, the classification accuracy of the image classification model is gradually improved, and a target image classification model meeting the condition can be obtained. The target image classification model is a trained model.

[0087] The embodiment of the present application adopts a pseudo classification label to label the second image when training the image classification model, and designs a corresponding loss value based on the pseudo classification label to indicate whether it is accurate, updates the model parameters based on this, and the pseudo classification label is also updated during the updating process. In this way, only part of the images correspond to the classification label, and the pseudo classification label is generated for other images during the model training process, that is, all images do not need to correspond to the classification label, which can greatly reduce the labor cost brought by manual labeling and improve the training efficiency. And the pseudo classification label is updated during the model training process, and finally the pseudo classification label with almost the same accuracy as the classification label can be determined to improve the number of training samples and improve the accuracy of the image classification model.

[0088] Figure 3 is a flowchart of an image processing method provided by the embodiment of the present application, see Figure 3 The method comprises the following steps.

[0089] 301. The terminal acquires at least two first images and at least two second images, the first image carries a corresponding classification label, and the first image and the second image are human tissue images.

[0090] In the embodiment of the present application, two kinds of images are included: first images and second images. The first images and the second images are both sample images used in the training of the image classification model. The two kinds of images are classified according to whether there is a classification label. The one with a classification label is called a first image, and the one without a classification label is called a second image. The classification label is also a true classification label.

[0091] 302. The terminal acquires the predicted classification results of the at least two first images and the at least two second images based on the image classification model.

[0092] The terminal can call the image classification model, which can be an initial model, and the model parameters of the image classification model need to be optimized.

[0093] When the terminal acquires the predicted classification results based on the image classification model, the image can be first feature extracted, and then the image is classified based on the image features. That is, in step 302, the electronic device can input the at least two first images and the at least two second images into the image classification model, extract features from the at least two first images and the at least two second images based on the image classification model, obtain image features of the at least two first images and the at least two second images, and classify the first images and the second images based on the image features of the at least two first images and the at least two second images, respectively, to obtain the predicted classification results of the first images and the second images.

[0094] In this stage, the first image carries the corresponding classification label, while the second image does not carry the corresponding classification label, and thus, a pseudo classification label of the second image needs to be obtained according to the predicted classification result in this stage. In this way, in the next stage, the second image can carry the pseudo classification label and can participate in the classification loss.

[0095] The system framework of the method provided in the present application is described below through a specific example. The training system framework of the image classification model is as shown in FIG. 1. Figure 4 As shown in FIG. 1, the system framework can mainly consist of three modules: a feature extraction module (Feature Extractor, F) 401, a Gaussian mixture module (Gaussian Mixture, G) 402 and a classifier module (Classifier, C) 403.

[0096] The feature extraction module 401 is used to extract a learnable deep feature representation type, and the representation type is input into the classifier to learn the prediction result corresponding to the input data. That is, the feature extraction module 401 is used to extract image features of an image.

[0097] For at least two first images and at least two second images, the at least two first images and the classification labels corresponding to the first images can be referred to as a source domain data set, denoted as (X S , Y S ). The at least two second images and the pseudo classification labels corresponding to the second images can be referred to as a target domain data set, denoted as (X T , ).

[0098] For the source domain data set and the target domain data set, first, a feature representation type (i.e., image features) is extracted from the source domain and the target domain data by using the feature extraction module 401 (which can be a feature extraction network (F)), and then the image features (F S , F T ) of the source domain and the target domain are input into the classifier module 403 (which can be a classifier (C)), and the prediction results (P S , P T), i.e. the predicted classification result. The predicted result of the target domain is input into a Gaussian mixture module 402 (which can be a Gaussian mixture model) to calculate the weight (Weights) of the predicted result and its estimated pseudo-classification label (Pseudo-label). The Gaussian mixture module 402 is used to calculate the weight. The weight will participate in the calculation of the robust pseudo-label loss function together with the pseudo-classification label, so as to learn a more accurate target set test result. That is, the process of obtaining the second loss value described above. For a description of the Gaussian mixture module 402, see step 307 below. Only the process of feature extraction and classification of the image classification model is described here.

[0099] wherein, for the feature extraction module 401 and the classifier module 403, they can be any deep learning network structure, such as the feature extraction module and the classifier module can adopt ResNet50 (Residual Network), or other network structures such as EfficientNet (Efficient Network), MobileNet (Mobile Network), DenseNet (Dense Network), etc. In some embodiments, the feature extraction module 401 described above can adopt the network before the FC (Fully Connection) layer of ResNet50, and the classifier module 403 can adopt the last FC layer. The pseudo-classification label is obtained based on the predicted result of the target domain output by the FC layer.

[0100] 303. The terminal obtains a first loss value based on the classification label carried by the first image and the predicted classification result of the first image.

[0101] After the terminal predicts the predicted classification result of the first image and the second image by the image classification model, the predicted classification result is a predicted value. Since the first image also carries a true label, i.e. a true value, the terminal can use the error between the predicted value and the true value as a basis for model training. The first loss value can be used to indicate the accuracy of the predicted classification result of the first image. The first loss value is used to determine the accuracy of the classification result of the image classification model, which can be referred to as a classification loss. The classification process of the human tissue image can also be understood as a segmentation process, so the classification process can also be referred to as a segmentation process, and the classification loss can also be referred to as a segmentation loss.

[0102] In some embodiments, the first loss value can be a value of a cross-entropy loss function. The error between the classification label and the predicted classification result can be calculated by the cross-entropy loss function to measure the classification ability of the image classification model. It can be understood that if the first loss value is a cross-entropy loss value, the greater the first loss value, the greater the error between the predicted classification result and the classification label, and the worse the classification ability of the image classification model. The smaller the first loss value, the smaller the error between the predicted classification result and the classification label, and the better the classification ability of the image classification model.

[0103] Of course, the first loss value can also be obtained by using other loss functions, for example, L1 or L2, etc., and the embodiments of the present application are not limited thereto.

[0104] 304. The terminal updates the model parameters of the image classification model based on the first loss value.

[0105] After the terminal obtains the first loss value, the model parameters can be updated based on the first loss value. The model parameter updating process can be realized by any model updating algorithm. In some embodiments, the updating process can be realized by a stochastic gradient descent (SGD) algorithm. The embodiments of the present application are not limited thereto.

[0106] 305. The terminal obtains the pseudo classification label corresponding to the at least two second images based on the predicted classification result of the at least two second images.

[0107] The second image does not originally carry a corresponding classification label, and the corresponding pseudo classification label needs to be calculated based on the predicted classification result. Thus, in the next stage, the second image can carry a corresponding pseudo classification label, and then participate in the calculation of the classification loss, so as to expand the sample images and further improve the accuracy of the image classification model. And the process can also calculate the pseudo classification label with good accuracy for the second image.

[0108] In some embodiments, when obtaining the pseudo classification label, the terminal can cluster the images and determine the pseudo classification label based on the clustering result. Specifically, the terminal can cluster the at least two first images and the at least two second images to obtain at least two clustering centers, and then obtain the pseudo classification label corresponding to the predicted classification result according to the predicted classification result of the at least two second images and the clustering result.

[0109] For example, in a specific example, the pseudo classification label (Pseudo-label) can be realized by the following formula:

[0110]

[0111] wherein, argmax for representing pseudo-classification labels k is a function, is a function of a parameter (set) of a function, refers to x when C(F(X T ) obtains the maximum value k . [·] k represents using cosine similarity (i.e. the k-th clustering center calculated. C() is a classification process, F(X T ) represents feature extraction on the second image X T .

[0112] 306. The terminal obtains, based on the image classification model, a predicted classification result of at least two first images and at least two second images, the first image carrying a corresponding classification label, and the second image carrying a corresponding pseudo-classification label.

[0113] The step 306 is the same as the step 302 described above, and will not be described in detail here.

[0114] 307. The terminal obtains, based on the predicted classification result of the at least two first images and the corresponding classification label, and the at least two second images and the corresponding pseudo-classification label, a first loss value, the first loss value being used to indicate the accuracy of the predicted classification result of the image.

[0115] In this phase, the first image and the second image both correspond to a classification label or a pseudo-classification label, so both kinds of images can participate in the calculation of the classification loss or the segmentation loss, and thus the terminal can calculate the first loss value based on the predicted classification result, the classification label or the pseudo-classification label of the two kinds of images.

[0116] The step 307 is the same as the step 303 described above, except that in the step 307, the second image corresponds to a pseudo-classification label, so it can also participate in the calculation of the classification loss or the segmentation loss, and thus when the terminal obtains the first loss value, it also takes into account the error between the predicted classification result of the second image and the pseudo-classification label. The embodiments of the present application will not be described in detail here.

[0117] 308. The terminal obtains, based on the predicted classification result of the at least two second images and the corresponding pseudo-classification label, a second loss value, the second loss value being used to indicate the accuracy of the pseudo-classification label.

[0118] For the pseudo-classification label, the terminal can also evaluate whether the pseudo-classification label is accurate, whether it is a real classification label, or whether it is close to the real classification label. Thus, the terminal can also obtain a second loss value according to the predicted classification result of the second image and the pseudo-classification label, to determine whether the pseudo-classification label is accurate.

[0119] Specifically, the terminal can obtain errors between prediction classification results of the at least two second images and corresponding pseudo classification labels, and then weight errors corresponding to the at least two second images according to weights of the at least two second images to obtain the second loss value. By setting weights for each second image, the weights of the second image are adaptively adjusted according to whether the pseudo classification label is accurate, thereby ensuring the accuracy of the pseudo classification label.

[0120] In some embodiments, when the weights are obtained, the clustering method can be used. For images of the same category, the distribution of image features is more similar, so by clustering the images, the attribute type in the second image can be analyzed according to the distance from different clustering centers, and the possibility that the pseudo classification label is the correct classification label can be naturally reflected. For images with more accurate classification, a higher weight can be set; for images with less accurate classification, a lower weight can be set. That is, the weight can correspond to whether the pseudo classification label of the second image is the correct classification label, or the weight can correspond to the probability that the pseudo classification label of the second image is the correct classification label.

[0121] Specifically, the terminal can cluster the at least two first images and the at least two second images to obtain at least two clustering centers, and obtain weights of the at least two second images according to distances between the at least two second images and different clustering centers.

[0122] In some embodiments, the above clustering is used to cluster images with similar image features together, so as to determine the distribution of different types of images from the image feature level, and model the feature distance of the category center in the feature space to obtain accurate weights. Specifically, the terminal can obtain probabilities corresponding to the at least two images according to distances between the at least two second images and different clustering centers, the probability being the probability that the pseudo classification label is the correct classification label, and then obtain weights of the at least two images based on the probabilities corresponding to the at least two images.

[0123] In some embodiments, for the relationship between the probability and the weight, the terminal can obtain the probability as the weight of any second image in response to the probability corresponding to the second image being greater than or equal to a probability threshold. The terminal can obtain zero as the weight of any second image in response to the probability corresponding to the second image being less than the probability threshold.

[0124] The probability threshold can be set by a skilled person according to requirements, for example, the probability threshold can be 0.5, and embodiments of the present application do not limit this. Through the above method, a higher weight value is set for an image with accurate classification, and a lower weight value is set for an image with inaccurate classification, thereby further improving the accuracy of the weight value and improving the accuracy of the pseudo classification label.

[0125] In some embodiments, the step 308 can be implemented by a Gaussian mixture model. Specifically, based on the Gaussian mixture model, the at least two second images, the predicted classification results of the at least two second images, and the corresponding pseudo classification labels are processed to obtain the second loss value. That is, the electronic device can input the predicted classification results of the second images and the pseudo classification labels into the Gaussian mixture model, process the input data by the Gaussian mixture model, and output the second loss value.

[0126] The Gaussian mixture model is a model that accurately quantifies things by using a Gaussian probability density function (normal distribution curve), and it is a model that decomposes things into several models formed based on a Gaussian probability density function (normal distribution curve).

[0127] In some embodiments, the electronic device can also update the model parameters of the Gaussian mixture model based on the second loss value.

[0128] For example, for the source domain or the target domain image, the output of the classification network and respectively represent the predicted classification results of the source domain and the target domain data, represent the pseudo classification label of the target domain data, where K represents the number of classes of classification, and N S ,N T respectively represent the number of source domain images and target domain images. The weight value can be implemented by the following formula two:

[0129]

[0130] wherein, represents the probability that the pseudo classification label estimated by the model is the correct classification label, and z j ∈{0,1} is an intermediate variable introduced for the pseudo classification label of each target domain sample, which indicates whether the predicted pseudo classification label is correct (z j =1) or incorrect (z j= 0). From Equation Two, if the estimated pseudo-classification label is less than 0.5 in probability of being the correct classification label, the pseudo-classification label will be removed. The above uses the Gaussian Mixture Model and uses the distance between the data feature and the center of different categories to calculate the probability of the pseudo-classification label being the correct classification label.

[0131] 309、The terminal updates the model parameters of the image classification model based on the first loss value and the second loss value, and continues to perform the classification and loss value obtaining steps based on the updated pseudo-classification labels of the prediction classification results of the at least two second images until a target condition is met, to obtain a target image classification model.

[0132] The terminal updates the model parameters by synthesizing the two loss values. When synthesizing, a target loss value can be obtained based on the first loss value and the second loss value, and the model parameters are updated based on the target loss value.

[0133] In some embodiments, the terminal can weight the first loss value and the second loss value to obtain a target loss value. When the weight is 1, the terminal can take the sum of the first loss value and the second loss value as the target loss value, for example, use to represent, as shown in Equation Three below.

[0134]

[0135] wherein (X S , Y S ) represents source domain data, X S refers to a first image, and Y S refers to a classification label carried by the first image. represents target domain data, X T refers to a second image, refers to a pseudo-classification label carried by the second image. is the cross-entropy loss function of the first image X S and the classification label Y S carried by the first image in the source domain data, is the robust loss function of the second image and the predicted pseudo-classification label thereof in the target domain data, that is, the second loss value. w is the weight value of the pseudo-classification label being the correct classification label, MAE(·) represents the mean absolute error (Mean Absolute Error). F refers to a feature extraction module, and C refers to a classifier module. The output of the classification network and respectively represent the prediction classification results of the source domain data and the target domain data, wherein K represents the number of categories of classification, N S ,N Trespectively represent the number of the first images and the second images.

[0136] In this example, the first loss value only includes the cross-entropy loss function calculated from the first images, and does not include the cross-entropy loss function calculated from the second images. It should be noted that the terminal can determine whether to include the cross-entropy loss function calculated from the second images in the first loss value according to the setting. In the most initial iteration, the second image has not obtained the pseudo classification label, and the cross-entropy loss function cannot be calculated, so the first loss value can not be included. If the second image carries the pseudo classification label, the calculated cross-entropy loss function can also be included in the first loss value.

[0137] In some embodiments, the terminal can also obtain the predicted types of the at least two first images and the at least two second images according to the at least two first images and the at least two second images and the corresponding predicted classification results, the predicted types being used to indicate whether the images are first images or second images, and obtain a third loss value according to the predicted types of the at least two first images and the at least two second images, the third loss value being used to indicate the accuracy of the predicted types. Accordingly, in step 309, the terminal can update the model parameters of the image classification model based on the first loss value, the second loss value and the third loss value.

[0138] In some embodiments, the terminal can distinguish the types of the at least two first images and the at least two second images according to the at least two first images and the at least two second images and the corresponding predicted classification results based on the discrimination network, obtain the predicted types of the at least two first images and the at least two second images, and then update the network parameters of the discrimination network according to the third loss value.

[0139] That is, in the training process, a phased training method is adopted. The pseudo classification label generated by the training result of the previous phase is applied to the segmentation network training process in the current phase. In each phase, the segmentation network and the discrimination network are trained in an alternating manner.

[0140] In the training process, first, the image data is input into the segmentation network, and the real classification label of the source domain data and the pseudo classification label of the target domain data are used to calculate the segmentation loss That is, the above classification loss or the first loss value. Then the parameters of the segmentation network are updated by minimizing the segmentation loss. After segmentation, the discrimination network can be used to distinguish the source domain and the target domain data, and the data classification of the two domains is made more similar through discrimination. Specifically, the segmentation result P S and P T are input into the discrimination network at the same time, and P TThe generated information entropy result is used to calculate the adversarial loss Adversarial loss That is, the third loss value of the predicted type obtained according to the discrimination. Then the parameters of the discrimination network can be updated by maximizing the adversarial loss. Subsequently, the error generated by the adversarial loss function is also back-propagated to the segmentation network, and the parameters of the segmentation network are updated by minimizing the adversarial loss, so that the segmentation results predicted by the segmentation network for the source domain image and the target domain image become more and more similar, and domain adaptation is achieved. In the process of optimizing the network parameters, the Stochastic Gradient Descent (SGD) algorithm can be used to optimize and train the segmentation network (image classification model), and the Adam algorithm can be used to optimize and train the discrimination network.

[0141] For the above target condition, the loss value can converge, or the number of iterations can reach the target number. The target condition can also be related to the learning rate. In a specific example, the initial learning rates of the segmentation network and the discrimination network are 2.5x10 -4 and 1x10 -4 , respectively. The learning rate can affect the convergence speed of the model.

[0142] After the terminal trains the image classification model, the terminal can apply the image classification model to an image classification scenario. For example, the image classification scenario can be ear mirror data segmentation. The terminal can obtain a third image in response to an image processing instruction, the third image being the human tissue image, input the third image into the target image classification model, perform feature extraction on the third image by the target image classification model, classify the third image based on the extracted image features, and obtain an attribute type in the third image. Similarly, the attribute type can be determined according to the image classification scenario. For example, in the ear mirror data segmentation scenario, the attribute type can be the type of eardrum. For another example, in the brain image segmentation scenario, the attribute type can be whether the human tissue is a brain tumor or brain cancer, etc. For another example, in the image acquisition distance classification scenario, the attribute type can be distance far or distance near, etc. The process can also be considered as a test process of the model, in which the trained model is used to accurately classify the obtained image. For example, as shown in FIG. 5, for a test (Test) set, the test set can be input into the trained image classification model, the image classification model including a feature extraction module 501 and a classifier module 502, and finally the classifier module 502 outputs a prediction result of the test set, that is, a test prediction (Test Prediction). Figure 5

[0143] ​The embodiment of the present application adopts a pseudo classification label to label the second image when training the image classification model, and designs a corresponding loss value based on the pseudo classification label to indicate whether it is accurate, updates the model parameters based on this, and the pseudo classification label is also updated in the updating process. In this way, only part of the images correspond to the classification label, and the pseudo classification label is generated for other images in the model training process, that is, all images do not need to correspond to the classification label, which can greatly reduce the labor cost brought by manual labeling and improve the training efficiency. And the pseudo classification label is updated in the model training process, and finally the pseudo classification label with almost the same accuracy as the classification label can be determined to improve the number of training samples and improve the accuracy of the image classification model.

[0144] All the optional technical solutions described above can be combined to form optional embodiments of the present application, which will not be described one by one here.

[0145] Figure 6 is a structural schematic diagram of an image processing device provided by the embodiment of the present application, referring to Figure 6 The device comprises:

[0146] The acquisition module 601 is configured to acquire, based on an image classification model, a predicted classification result of at least two first images and at least two second images, the first image carrying a corresponding classification label, and the second image carrying a corresponding pseudo classification label, the first image and the second image being human tissue images, and the predicted classification result, the classification label and the pseudo classification label being used to indicate an attribute type in the image.

[0147] The acquisition module 601 is further configured to acquire, based on the predicted classification result of the at least two first images and the corresponding classification label, and the at least two second images and the corresponding pseudo classification label, a first loss value, the first loss value being used to indicate the accuracy of the predicted classification result of the image.

[0148] The acquisition module 601 is further configured to acquire, based on the at least two second images, the predicted classification result of the at least two second images and the corresponding pseudo classification label, a second loss value, the second loss value being used to indicate the accuracy of the pseudo classification label.

[0149] The update module 602 is configured to update model parameters of the image classification model based on the first loss value and the second loss value, continue to perform the classification and loss value acquisition steps after updating the corresponding pseudo classification label based on the predicted classification result of the at least two second images, and obtain a target image classification model when a target condition is met.

[0150] In some embodiments, the acquisition module 601 is configured to:

[0151] obtaining errors between the predicted classification results of the at least two second images and corresponding pseudo classification labels;

[0152] weighting the errors corresponding to the at least two second images according to the weights of the at least two second images to obtain the second loss value.

[0153] In some embodiments, the obtaining module 601 is configured to:

[0154] clustering the at least two first images and the at least two second images to obtain at least two cluster centers;

[0155] obtaining weights of the at least two second images according to distances between the at least two second images and different cluster centers.

[0156] In some embodiments, the obtaining module 601 is configured to:

[0157] obtaining probabilities corresponding to the at least two images according to distances between the at least two images and different cluster centers, the probability being a probability that the pseudo classification label is a correct classification label;

[0158] obtaining weights of the at least two images based on the probabilities corresponding to the at least two images.

[0159] In some embodiments, the obtaining module 601 is configured to:

[0160] in response to the probability corresponding to any second image being greater than or equal to a probability threshold, obtaining the probability as the weight of the second image;

[0161] in response to the probability corresponding to any second image being less than the probability threshold, obtaining zero as the weight of the second image.

[0162] In some embodiments, the updating module 602 is configured to obtain a pseudo classification label corresponding to the predicted classification result according to the predicted classification result of the at least two second images and the clustering result.

[0163] In some embodiments, the obtaining module 601 is configured to:

[0164] processing the at least two second images, the predicted classification results of the at least two second images and the corresponding pseudo classification labels based on a Gaussian mixture model to obtain the second loss value;

[0165] The updating module 602 is further configured to update model parameters of the Gaussian mixture model based on the second loss value.

[0166] In some embodiments, the acquisition module 601 is further configured to acquire, according to the at least two first images and the at least two second images and the corresponding predicted classification results, a predicted type of the at least two first images and the at least two second images, the predicted type being used to indicate whether an image is a first image or a second image.

[0167] The acquisition module 601 is further configured to acquire, according to the predicted types of the at least two first images and the at least two second images, a third loss value, the third loss value being used to indicate accuracy of the predicted types.

[0168] In some embodiments, the update module 602 is configured to update model parameters of the image classification model based on the first loss value, the second loss value and the third loss value.

[0169] In some embodiments, the acquisition module 601 is configured to determine, based on a discriminative network, types of the at least two first images and the at least two second images according to the at least two first images and the at least two second images and the corresponding predicted classification results, to obtain the predicted types of the at least two first images and the at least two second images.

[0170] The update module 602 is further configured to update network parameters of the discriminative network according to the third loss value.

[0171] In some embodiments, the acquisition module 601 is further configured to acquire, in response to an image processing instruction, a third image, the third image being the human tissue image.

[0172] The apparatus further includes:

[0173] The classification module is configured to input the third image into the target image classification model, perform feature extraction on the third image by the target image classification model, classify the third image based on the extracted image features, and obtain a type of the lesion in the third image.

[0174] The apparatus provided by the embodiments of the present application adopts a pseudo classification label to label the second image when training the image classification model, and designs a corresponding loss value based on the pseudo classification label to indicate whether it is accurate, updates the model parameters based on this, and the pseudo classification label is also updated in the updating process. Thus, only part of the images correspond to the classification label, pseudo classification labels are generated for other images in the model training process, that is, all images do not need to correspond to the classification label, which can greatly reduce the labor cost brought by manual labeling and improve the training efficiency. Moreover, the pseudo classification label is updated in the model training process, and finally a pseudo classification label with similar accuracy to the classification label can be determined to improve the number of training samples and further improve the accuracy of the image classification model.

[0175] It should be noted that the image processing apparatus provided in the above embodiments is only used as an example to illustrate the division of the above functional modules when the image processing apparatus performs image processing based on artificial intelligence. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the image processing apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and the image processing method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0176] Figure 7 FIG. 7 is a structural schematic diagram of an electronic device provided in an embodiment of the present application. The electronic device 700 can have great differences due to different configurations or performances, and can include one or more processors (Central Processing Units, CPUs) 701 and one or more memories 702. The memory 702 stores at least one computer program, which is loaded and executed by the processor 701 to implement the image processing method provided in each method embodiment. The electronic device can also include other components for realizing device functions, for example, the electronic device can also have a wired or wireless network interface and an input and output interface, etc., so as to perform input and output. The embodiments of the present application will not be repeated here.

[0177] The electronic device in the above method embodiments can be implemented as a terminal. For example, Figure 8 FIG. 8 is a structural block diagram of a terminal provided in an embodiment of the present application. The terminal 800 can be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a notebook computer, a desktop computer, an otoscope device. The terminal 800 can also be referred to as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, and other names.

[0178] Generally, the terminal 800 includes a processor 801 and a memory 802.

[0179] The processor 801 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 801 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 801 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 801 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 801 can further include an AI (Artificial Intelligence) processor for processing machine learning related computing operations.

[0180] The memory 802 can include one or more computer-readable storage media that can be non-transitory. The memory 802 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction for being executed by the processor 801 to implement the image processing method provided by the method embodiments in the present application.

[0181] In some embodiments, the terminal 800 can also optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802, and the peripheral device interface 803 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 803 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, and a power supply 809.

[0182] The peripheral interface 803 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802 and the peripheral interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802 and the peripheral interface 803 can be implemented on a separate chip or circuit board, and the present embodiments are not limited in this regard.

[0183] The radio frequency circuit 804 is used to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 can also include NFC (Near Field Communication) related circuitry, and the present application is not limited in this regard.

[0184] The display screen 805 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 is further configured to capture touch signals on or above the surface of the display screen 805. The touch signals can be input to the processor 801 as control signals for processing. In this case, the display screen 805 can also be configured to provide virtual buttons and / or virtual keyboard, also known as soft buttons and / or soft keyboard. In some embodiments, the display screen 805 can be one, disposed on the front panel of the terminal 800; in other embodiments, the display screen 805 can be at least two, respectively disposed on different surfaces of the terminal 800 or in a folding design; in other embodiments, the display screen 805 can be a flexible display screen, disposed on a curved surface or a folding surface of the terminal 800. Even, the display screen 805 can also be disposed in an irregular shape other than a rectangle, i.e., a special-shaped screen. The display screen 805 can be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0185] The camera assembly 806 is configured to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, to realize the background blur function by fusing the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function by fusing the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera assembly 806 can further include a flash. The flash can be a single-color-temperature flash or a dual-color-temperature flash. The dual-color-temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0186] The audio circuit 807 can include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor 801 for processing, or input to the radio frequency circuit 804 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, respectively arranged at different parts of the terminal 800. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker can be a conventional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can it convert electrical signals into sound waves that humans can hear, but it can also convert electrical signals into sound waves that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 807 can also include a headphone jack.

[0187] The power supply 809 is used to supply power to each component in the terminal 800. The power supply 809 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 809 includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery charged through a wired line, and the wireless charging battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0188] In some embodiments, the terminal 800 further includes one or more sensors 810. The one or more sensors 810 include, but are not limited to, an acceleration sensor 811, a gyroscope sensor 812, a pressure sensor 813, an optical sensor 815, and a proximity sensor 816.

[0189] The acceleration sensor 811 can detect the acceleration in three coordinate axes of the coordinate system established by the terminal 800. For example, the acceleration sensor 811 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 811. The acceleration sensor 811 can also be used for game or user motion data collection.

[0190] The gyroscope sensor 812 can detect the body direction and rotation angle of the terminal 800, and the gyroscope sensor 812 can collect 3D actions of the user on the terminal 800 in cooperation with the acceleration sensor 811. The processor 801 can realize the following functions according to the data collected by the gyroscope sensor 812: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0191] The pressure sensor 813 can be disposed on the side bezel of the terminal 800 and / or on the lower layer of the display screen 805. When the pressure sensor 813 is disposed on the side bezel of the terminal 800, it can detect the user's grip signal on the terminal 800, and the processor 801 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 813. When the pressure sensor 813 is disposed on the lower layer of the display screen 805, the processor 801 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 805. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0192] An optical sensor 815 is used to collect ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 based on the ambient light intensity collected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is decreased. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 based on the ambient light intensity collected by the optical sensor 815.

[0193] The proximity sensor 816, also known as a distance sensor, is typically located on the front panel of the terminal 800. The proximity sensor 816 is used to detect the distance between the user and the front of the terminal 800. In one embodiment, when the proximity sensor 816 detects that the distance between the user and the front of the terminal 800 is gradually decreasing, the processor 801 controls the display screen 805 to switch from a screen-on state to a screen-off state; when the proximity sensor 816 detects that the distance between the user and the front of the terminal 800 is gradually increasing, the processor 801 controls the display screen 805 to switch from a screen-off state to a screen-on state.

[0194] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on terminal 800 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0195] The electronic device in the above method embodiments can be implemented as a server. For example, Figure 9Fig. 9 is a schematic diagram of a structure of a server provided in an embodiment of the present application. The server 900 can have a large difference due to different configurations or performances, and can include one or more processors (Central Processing Units, CPUs) 901 and one or more memories 902. The memory 902 stores at least one computer program, which is loaded and executed by the processor 901 to implement the image processing method provided in each of the above-mentioned method embodiments. Of course, the server can also have a wired or wireless network interface and an input and output interface, and other components for implementing device functions, which are not described herein.

[0196] In an example embodiment, a computer-readable storage medium is also provided, for example, a memory including at least one computer program, which is executed by a processor to complete the image processing method in the above-mentioned embodiments. For example, the computer-readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0197] In an example embodiment, a computer program product or a computer program is also provided, which includes one or more program codes stored in a computer-readable storage medium. One or more processors of an electronic device can read the one or more program codes from the computer-readable storage medium, and execute the one or more program codes, so that the electronic device can perform the above-mentioned image processing method.

[0198] It should be understood that, in various embodiments of the present application, the size of the sequence number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0199] It should be understood that, according to A, B is determined, which does not mean that B is determined only according to A, but also can be determined according to A and / or other information.

[0200] It should be understood by those of ordinary skill in the art that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a Read-Only Memory, a magnetic disk or an optical disk, etc.

[0201] The above description is only optional embodiments of the present application, and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that, The method includes: Based on an image classification model, predicted classification results for at least two first images and at least two second images are obtained. The first images carry corresponding classification labels, and the second images carry corresponding pseudo-classification labels. The first and second images are human tissue images. The predicted classification results, classification labels, and pseudo-classification labels are used to indicate the attribute types in the images. The pseudo-classification label of the second image is determined based on the predicted classification results of the second image during the previous training stage. Based on the predicted classification results and corresponding classification labels of the at least two first images, and the at least two second images and corresponding pseudo-classification labels, a first loss value is obtained, which is used to indicate the accuracy of the predicted classification results of the images. Based on the predicted classification results of the at least two second images and the corresponding pseudo-classification labels, a second loss value is obtained. The second loss value is used to indicate the accuracy of the pseudo-classification label. The accuracy refers to whether the pseudo-classification label is a real classification label or whether it is close to the real classification label. Based on the first loss value and the second loss value, the model parameters of the image classification model are updated. Based on the predicted classification results of the at least two second images, the corresponding pseudo-classification labels are updated, and the classification and loss value acquisition steps are continued until the target conditions are met, thus obtaining the target image classification model.

2. The method according to claim 1, characterized in that, The predicted classification results of the at least two second images and their corresponding pseudo-classification labels are used to obtain a second loss value, including: Obtain the error between the predicted classification results of the at least two second images and their corresponding pseudo-classification labels; The second loss value is obtained by weighting the errors corresponding to the at least two second images based on their weights.

3. The method according to claim 2, characterized in that, The step of weighting the errors corresponding to the at least two second images according to their weights to obtain the second loss value includes: Cluster the at least two first images and the at least two second images to obtain at least two cluster centers; The weights of the at least two second images are obtained based on the distances between the at least two second images and different cluster centers.

4. The method according to claim 3, characterized in that, The step of obtaining the weights of the at least two second images based on the distances between the at least two second images and different cluster centers includes: Based on the distance between the at least two second images and different cluster centers, the probability corresponding to the at least two second images is obtained, where the probability is the probability that the pseudo-classification label is the correct classification label; The weights of the at least two second images are obtained based on the probabilities corresponding to the at least two second images.

5. The method according to claim 4, characterized in that, The step of obtaining the weights of the at least two second images based on the probabilities corresponding to the at least two second images includes: In response to the probability corresponding to any second image being greater than or equal to a probability threshold, the probability is obtained as the weight of the second image; In response to the probability corresponding to any second image being less than the probability threshold, a zero value is obtained as the weight of the second image.

6. The method according to claim 3, characterized in that, The step of updating the corresponding pseudo-classification labels based on the predicted classification results of the at least two second images includes: Based on the predicted classification results and clustering results of the at least two second images, obtain the pseudo-classification label corresponding to the predicted classification result.

7. The method according to claim 1, characterized in that, The predicted classification results of the at least two second images and their corresponding pseudo-classification labels are used to obtain a second loss value, including: Based on the Gaussian mixture model, the predicted classification results of the at least two second images and their corresponding pseudo-classification labels are processed to obtain the second loss value; The method further includes: Based on the second loss value, the model parameters of the Gaussian mixture model are updated.

8. The method according to claim 1, characterized in that, The method further includes: Based on the at least two first images and at least two second images, and the corresponding prediction classification results, the prediction type of the at least two first images and the at least two second images is obtained, and the prediction type is used to indicate whether the image is a first image or a second image; A third loss value is obtained based on the prediction types of the at least two first images and the at least two second images, the third loss value being used to indicate the accuracy of the prediction type; The step of updating the model parameters of the image classification model based on the first loss value and the second loss value includes: The model parameters of the image classification model are updated based on the first loss value, the second loss value, and the third loss value.

9. The method according to claim 8, characterized in that, The step of obtaining the predicted types of the at least two first images and the at least two second images, and the corresponding prediction classification results, includes: Based on the discriminant network, the types of the at least two first images and the at least two second images are determined according to the at least two first images and the at least two second images and the corresponding prediction classification results, so as to obtain the predicted types of the at least two first images and the at least two second images; The method further includes: The network parameters of the discriminant network are updated based on the third loss value.

10. The method according to claim 1, characterized in that, The method further includes: In response to an image processing instruction, a third image is acquired, wherein the third image is the human tissue image; The third image is input into the target image classification model, which extracts features from the third image and classifies the third image based on the extracted image features to obtain the attribute type in the third image.

11. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire predicted classification results of at least two first images and at least two second images based on an image classification model. The first images carry corresponding classification labels, and the second images carry corresponding pseudo-classification labels. The first and second images are human tissue images. The predicted classification results, classification labels, and pseudo-classification labels are used to indicate the attribute types in the images. The pseudo-classification label of the second image is determined based on the predicted classification results of the second image during the previous training stage. The acquisition module is further configured to acquire a first loss value based on the predicted classification results and corresponding classification labels of the at least two first images, and the at least two second images and corresponding pseudo-classification labels. The first loss value is used to indicate the accuracy of the predicted classification results of the images. The acquisition module is further configured to acquire a second loss value based on the predicted classification results of the at least two second images and the corresponding pseudo-classification labels. The second loss value is used to indicate the accuracy of the pseudo-classification labels. The accuracy refers to whether the pseudo-classification labels are true classification labels or whether they are close to the true classification labels. The update module is used to update the model parameters of the image classification model based on the first loss value and the second loss value, update the corresponding pseudo-classification labels based on the predicted classification results of the at least two second images, and continue to execute the classification and loss value acquisition steps until the target conditions are met, thereby obtaining the target image classification model.

12. The apparatus according to claim 11, characterized in that, The acquisition module is used for: Obtain the error between the predicted classification results of the at least two second images and their corresponding pseudo-classification labels; The second loss value is obtained by weighting the errors corresponding to the at least two second images based on their weights.

13. The apparatus according to claim 12, characterized in that, The acquisition module is used for: Cluster the at least two first images and the at least two second images to obtain at least two cluster centers; The weights of the at least two second images are obtained based on the distances between the at least two second images and different cluster centers.

14. The apparatus according to claim 13, characterized in that, The acquisition module is used for: Based on the distance between the at least two second images and different cluster centers, the probability corresponding to the at least two second images is obtained, where the probability is the probability that the pseudo-classification label is the correct classification label; The weights of the at least two second images are obtained based on the probabilities corresponding to the at least two second images.

15. The apparatus according to claim 14, characterized in that, The acquisition module is used for: In response to the probability corresponding to any second image being greater than or equal to a probability threshold, the probability is obtained as the weight of the second image; In response to the probability corresponding to any second image being less than the probability threshold, a zero value is obtained as the weight of the second image.

16. The apparatus according to claim 13, characterized in that, The update module is used to obtain the pseudo-classification label corresponding to the predicted classification result based on the predicted classification result and clustering result of the at least two second images.

17. The apparatus according to claim 11, characterized in that, The acquisition module is used to process the predicted classification results and corresponding pseudo-classification labels of the at least two second images based on a Gaussian mixture model to obtain the second loss value. The update module is further configured to update the model parameters of the Gaussian mixture model based on the second loss value.

18. The apparatus according to claim 11, characterized in that, The acquisition module is further configured to acquire the prediction type of the at least two first images and the at least two second images based on the at least two first images and the at least two second images, and the corresponding prediction classification results, wherein the prediction type is used to indicate whether an image is a first image or a second image; The acquisition module is further configured to acquire a third loss value based on the prediction types of the at least two first images and the at least two second images, wherein the third loss value is used to indicate the accuracy of the prediction type; The update module is used to update the model parameters of the image classification model based on the first loss value, the second loss value, and the third loss value.

19. The apparatus according to claim 18, characterized in that, The acquisition module is used to determine the type of the at least two first images and the at least two second images based on a discriminant network, according to the at least two first images and the at least two second images and the corresponding prediction classification results, so as to obtain the predicted type of the at least two first images and the at least two second images; The update module is further configured to update the network parameters of the discrimination network based on the third loss value.

20. The apparatus according to claim 11, characterized in that, The device further includes: The acquisition module is further configured to acquire a third image in response to an image processing instruction, wherein the third image is the human tissue image; The classification module is used to input the third image into the target image classification model, whereby the target image classification model extracts features from the third image and classifies the third image based on the extracted image features to obtain the attribute type in the third image.

21. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the image processing method as described in any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer program, which is loaded and executed by a processor to implement the image processing method as described in any one of claims 1 to 10.

23. A computer program product comprising one or more lines of program code stored in a computer-readable storage medium, wherein one or more processors of an electronic device are capable of reading the one or more lines of program code from the computer-readable storage medium, and wherein the one or more processors execute the one or more lines of program code such that the electronic device is capable of performing an image processing method as described in any one of claims 1 to 10.

Images