Image processing methods, apparatuses, storage media, electronic devices and products

Abstract

This application discloses an image processing method, apparatus, storage medium, electronic device, and product, which can be applied to various scenarios such as cloud technology, artificial intelligence, and intelligent transportation. The image processing method acquires erased sample labeled images from a sample image set, then inputs these images into an image processing model. The model performs image recognition and image restoration on the labeled images, obtaining image recognition and restoration results. A target loss value is then calculated based on these results, and the parameters of the image processing model are adjusted accordingly. This improves the model training effect by avoiding inaccurate training due to noise in the sample image labels, while still achieving the image recognition processing capability required for training the model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to an image processing method, an image processing apparatus, a computer-readable storage medium, an electronic device, and a computer program product. Background Technology

[0002] With the continuous and vigorous development of computer technology in recent years, neural network models have achieved very good performance in many computer vision tasks, making image processing technology increasingly attract people's attention.

[0003] In existing technologies, when training neural network models, sample images are typically labeled manually or by machine so that the neural network model can learn from these labels and thus acquire some image processing capabilities. However, since the accuracy of the labeled sample images cannot be guaranteed, these labels often contain noise. Training the model with such noisy labels will affect its performance. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide an image processing method, apparatus, computer-readable storage medium, electronic device, and computer program product.

[0005] According to one aspect of the embodiments of this application, an image processing method is provided, comprising: acquiring a sample image set, the sample image set including erased sample labeled images; inputting the sample labeled images contained in the sample image set into an image processing model to perform image recognition processing and image restoration processing on the sample labeled images through the image processing model, thereby obtaining image recognition results and image restoration results of the sample labeled images; calculating a target loss value based on the image recognition results and image restoration results; and adjusting the parameters of the image processing model based on the target loss value.

[0006] According to one aspect of the embodiments of this application, an image processing apparatus is provided, comprising: a sample acquisition device configured to acquire a sample image set, the sample image set including erased sample annotation images; a recognition and restoration device configured to input the sample annotation images contained in the sample image set into an image processing model, so as to perform image recognition processing and image restoration processing on the sample annotation images through the image processing model, and obtain image recognition results and image restoration results of the sample annotation images; a loss value calculation device configured to calculate a target loss value based on the image recognition results and image restoration results; and a parameter adjustment device configured to adjust the parameters of the image processing model based on the target loss value.

[0007] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the image processing method as described above.

[0008] According to one aspect of the embodiments of this application, an electronic device is provided, including: a processor; and a memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the image processing method as described above.

[0009] According to one aspect of the embodiments of this application, a computer program product is also provided, the computer program product including computer instructions, which, when executed by a processor, are used to implement the image processing method described above.

[0010] In the technical solution provided by the embodiments of this application, the sample labeled images included in the sample image set after erasure are obtained, and then the sample labeled images are input into the image processing model to perform image recognition processing and image restoration processing on the sample labeled images through the image processing model, so as to obtain the image recognition result and the image restoration result of the sample labeled images. Then, the target loss value is calculated based on the image recognition result and the image restoration result. The target loss value can not only reflect the image recognition processing capability of the image processing model through the image recognition result, but also avoid the noise of the sample image labels due to the combination of the image restoration result. Finally, the parameters of the image processing model are adjusted according to the target loss value, so as to avoid the inaccurate model training due to the noise of the sample image labels while achieving the image recognition processing capability of the image processing model, thereby improving the model training effect.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0013] Figure 1 This is a schematic diagram of the implementation environment involved in this application;

[0014] Figure 2 This is a flowchart illustrating an exemplary embodiment of the image processing method of this application;

[0015] Figure 3 This is a schematic diagram illustrating the erasure process in an exemplary embodiment of this application;

[0016] Figure 4 This is a flowchart illustrating an image processing method in another exemplary embodiment of this application;

[0017] Figure 5 This is a schematic diagram illustrating image processing model training as shown in an exemplary embodiment of this application;

[0018] Figure 6 This is a schematic diagram illustrating an image processing model as shown in an exemplary embodiment of this application;

[0019] Figure 7 This is a schematic diagram illustrating an exemplary embodiment of the present application of a target image processing model;

[0020] Figure 8 This is a schematic diagram illustrating image processing as shown in an exemplary embodiment of this application;

[0021] Figure 9 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of this application;

[0022] Figure 10 This is a schematic diagram of the structure of a computer system suitable for implementing the electronic devices of the present application embodiments. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0026] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0027] First, a brief introduction to the terms used in the embodiments of this application will be given.

[0028] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0029] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, training techniques for large image recognition models, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0030] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0031] In related technologies, one approach is to extract features from the input image, including gradient and texture features. Then, based on these extracted features, a classifier (such as a Support Vector Machine (SVM) or Random Forest) is trained to classify the input image and determine its category. However, this method has at least two problems: first, the extracted features have poor generalization ability, often resulting in the extraction of harmful features that confuse the subsequent classifier for diverse video data; second, feature extraction and classifier training are performed independently, leading to high training costs. Another approach is to process images using Convolutional Neural Networks (CNNs). This involves feeding the input image into a CNN network structure for feature extraction, followed by classification using fully connected layers. Model training is an end-to-end process based on a loss function. This process does not require specifying the features to be extracted; the model automatically learns the most suitable features for the current classification task during training. Furthermore, the entire process is end-to-end, eliminating the need for separate training for feature extraction and classification. However, this CNN-based image processing method still has some problems. For example, the sample image labels corresponding to the sample images used for model training are generally manually labeled, so they may be highly subjective. Furthermore, the content that the sample image labels can express is limited, which leads to the sample image labels being noisy. Training the model with such noisy labels will affect the model's performance, making the model's generalization ability worse and reducing the reliability of the model's output results.

[0032] Based on this, in order to improve the training effect of the model and enhance its image processing capabilities, embodiments of this application propose an image processing method, an image processing apparatus, a computer-readable storage medium, an electronic device, and a computer program product.

[0033] The image processing method provided in the embodiments of this application will be described below.

[0034] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment involved in the image processing method of this application. For example... Figure 1 As shown, the implementation environment includes a terminal 110 and a server 120, which can be directly or indirectly connected via wired or wireless communication.

[0035] Terminal 110 can be a mobile phone, computer, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, etc., but is not limited to these. Terminal 110 can refer to one of multiple terminals; this embodiment only uses terminal 110 as an example. Those skilled in the art will understand that the number of terminals can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. In this case, the implementation environment of the image processing method may also include other terminals. This application embodiment does not limit the number or type of terminals.

[0036] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or it can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and content delivery networks (CDN). ) Cloud servers that provide basic cloud computing services such as big data and artificial intelligence platforms.

[0037] For example, in Figure 1 In the implementation environment shown, for the model training phase:

[0038] The model training phase can be performed by server 120. Server 120 can obtain a set of sample images and then iteratively train the image processing model based on the set of sample images to obtain a trained target image processing model.

[0039] exist Figure 1 In the implementation environment shown, for the model application phase:

[0040] The model application phase can be executed by terminal 110. That is, the trained image processing model can be deployed on terminal 110. When there is an image to be recognized on terminal 110, terminal 110 can call the trained target image processing model to perform image recognition processing on the image to be recognized, and obtain the image recognition result output by the target image processing model.

[0041] Alternatively, the model application phase can be executed interactively by server 120 and terminal 110. The trained target image processing model can be deployed on server 120. When there is an image to be recognized in terminal 110, terminal 110 can send the image to be recognized to server 120. Server 120 can call the trained target image processing model to perform image recognition processing on the image to be recognized, obtain the image recognition result output by the target image processing model, and then server 120 can send the image recognition result to terminal 110.

[0042] The image processing method provided in this application can be applied to multiple fields, such as industrial production image processing, traffic image processing, and home video processing, etc., and this application does not limit it to these fields.

[0043] Let's take the field of industrial production image processing as an example. For instance, a trained target image processing model has the ability to inspect product quality. It acquires product images of industrial products, inputs these images into the target image processing model, and obtains the quality inspection result (defective or normal) corresponding to the product image output by the target image processing model.

[0044] Let's take traffic image processing as an example. For instance, a trained target image processing model has the ability to recognize traffic scenes. It acquires traffic images during vehicle movement, inputs these images into the target image processing model, and obtains the traffic scene recognition result (indicating traffic accidents, congestion, or normal driving) corresponding to the traffic images output by the target image processing model.

[0045] Let's take home video processing as an example. For instance, a trained target image processing model has the ability to recognize faces. It can acquire facial images contained in home videos, input the facial images into the target image processing model, and obtain the facial recognition results (user information corresponding to the facial image, emotion corresponding to the facial image, etc.) output by the target image processing model.

[0046] It is worth noting that the above application scenarios are merely illustrative examples. The image processing method provided in this embodiment can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. This application embodiment does not limit these applications.

[0047] It is understood that in the specific implementation of this application, data such as user information and images to be identified are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0048] Please see Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of the image processing method of this application. This image processing method can be applied to… Figure 1 The implementation environment shown is specifically executed by a server within that implementation environment. It should be understood that this method can also be applied to other exemplary implementation environments and executed by devices in other implementation environments; this embodiment does not limit the implementation environment to which the method is applicable.

[0049] The image processing method proposed in this application embodiment will be described in detail below, taking the server as the specific execution subject.

[0050] like Figure 2 As shown, in an exemplary embodiment, the image processing method includes at least steps S210 to S240, which are described in detail below:

[0051] Step S210: Obtain a sample image set, which includes sample labeled images that have been erased.

[0052] The sample image set includes sample images labeled with sample image tags, which are used to indicate the inclusion of target image content in the labeled sample images.

[0053] For example, when the sample labeled image is implemented as a product image of an industrial product, the sample image label is used to indicate the quality status of the industrial product corresponding to the product image, such as whether the industrial product has defects. If there are defects, the sample image label is also used to indicate the type of defect corresponding to the product image.

[0054] When the sample labeled image is a traffic capture image, the sample image label is used to indicate the presence of vehicles in the traffic capture image. Furthermore, when the traffic capture image includes vehicles, the sample image label is also used to indicate the identification of the vehicles in the traffic capture image, such as the license plate number of a vehicle in the traffic capture image. Here, a traffic capture image refers to an image captured by traffic camera equipment.

[0055] When the sample labeled image is a home video image, the sample image label is used to indicate the presence of living organisms in the home video image. When the home video image includes living organisms, the sample image label is also used to indicate information such as the type of living organism in the home video image, such as: the home video image includes pets (cats).

[0056] In some embodiments, the sample image set is a set of images obtained from a public dataset, or the sample image set is a set of images composed of image data that has been authorized and uploaded by the user. This application embodiment does not limit this.

[0057] The sample image labels can be annotated in the following ways: after the sample collector collects the sample image, he distinguishes the target image content in the sample image and labels the sample image; or, the image is input into a pre-trained recognition model and the output prediction result is used as the sample image label. In this case, the sample image label is implemented as a pseudo label.

[0058] The sample image set includes labeled images that have undergone erasing processing. Erasing processing refers to covering a portion of the image area without changing the image characteristics of the uncovered portion. The shape and size of the image area to be erased can be randomly selected, and the image areas to be randomly erased in each image can be the same or different; this application embodiment does not limit this.

[0059] Step S220: Input the sample labeled images contained in the sample image set into the image processing model, so as to perform image recognition processing and image restoration processing on the sample labeled images through the image processing model, and obtain the image recognition results and image restoration results of the sample labeled images.

[0060] The specific recognition purpose of image recognition processing can be flexibly selected according to the actual application, and this application does not limit it. For example, in the application scenario of product quality inspection, the purpose of image recognition processing can be to identify whether the products contained in the input sample labeled image have defects, and if the products contained in the sample labeled image have defects, the image recognition processing is also used to identify the type of defect and the corresponding probability value of the products contained in the sample labeled image. In the application scenario of traffic scene recognition, the purpose of image recognition processing can be to identify the presence of vehicles in the input sample labeled image, and when the sample labeled image includes vehicles, the image recognition processing is also used to identify information such as the identification of vehicles in the sample labeled image. It is understood that the specific recognition purpose of image recognition processing will also be different depending on the training objective of the image processing model.

[0061] Image restoration processing refers to the restoration of erased image regions in a sample labeled image to obtain a complete sample labeled image.

[0062] The image processing model is used to perform image recognition and image restoration processing on the sample labeled images to obtain the image recognition results and image restoration results of the sample labeled images output by the image processing model.

[0063] Step S230: Calculate the target loss value based on the image recognition result and the image restoration result.

[0064] The difference between the image recognition result output by the image processing model and the sample image label corresponding to the sample labeled image reflects the image recognition processing capability of the image processing model. The image processing model can be trained based on the image recognition result output by the image processing model. However, if the image processing model is trained only based on the difference between the image recognition result and the sample image label, the training effect of the model will be reduced due to the noise carried by the sample image label itself.

[0065] The difference between the image restoration result and the original sample image corresponding to the sample labeled image reflects the image restoration processing capability of the image processing model. In the process of training the image restoration processing capability of the image processing model, the image processing model is trained based on the difference between the image restoration result and the original sample image, and the sample image label is not involved in the training process.

[0066] Therefore, the target loss value is calculated by combining the image recognition results and the image restoration results. This target loss value can not only reflect the image recognition processing capability of the image processing model through the image recognition results, but also avoid the noise of the sample image labels due to the combination of the image restoration results. This can improve the model training effect by avoiding the inaccuracy of model training due to the noise of the sample image labels while achieving the image recognition processing capability of the image processing model.

[0067] Step S240: Adjust the parameters of the image processing model based on the target loss value.

[0068] The parameters of the image processing model can be adjusted by the target loss value. For example, after determining the target loss value, the model parameters of the image processing model to be trained can be adjusted according to the target loss value. For example, the model parameters of the image processing model can be adjusted based on the backpropagation algorithm. Then, return to step S210 and repeat the above steps. Iterate multiple times until the training termination condition is met to obtain the trained target image processing model.

[0069] The training termination condition refers to the conditions under which the target image processing model is successfully trained. These conditions include, but are not limited to, the training duration reaching the preset duration, the number of training iterations reaching the maximum number of iterations, the model parameters no longer changing, and the model loss information reaching the preset threshold.

[0070] Specifically, the server calculates the target loss value based on the image recognition and image restoration results. Then, the server first determines whether the training termination condition has been met. If not, the target loss value is used to back-update the model parameters in the image processing model to be trained, resulting in an updated image processing model. Gradient descent algorithms, such as stochastic gradient descent, batch gradient descent, mini-batch gradient descent, and full gradient descent, can be used to back-update the model parameters. The updated image processing model is then used as the image processing model to be trained, and the step of obtaining the sample image set is iteratively executed until the training termination condition is met, resulting in the trained target image processing model.

[0071] Therefore, the image processing method provided in this application obtains erased sample labeled images from a sample image set, then inputs the sample labeled images into an image processing model to perform image recognition and image restoration processing on the sample labeled images, obtaining image recognition results and image restoration results of the sample labeled images. Based on the image recognition results and image restoration results, a target loss value is calculated. This target loss value, through the image recognition results, not only reflects the image recognition processing capability of the image processing model, but also avoids noise from the sample image labels due to the combination of image restoration results. Finally, the parameters of the image processing model are adjusted according to the target loss value, so as to achieve the image recognition processing capability of the image processing model while avoiding inaccurate model training due to noise from the sample image labels, thereby improving the model training effect.

[0072] The following describes some implementation methods for obtaining sample image sets.

[0073] In some implementations, obtaining a set of sample images includes: obtaining an original sample image; erasing the area to be erased in the original sample image to obtain an erased sample image; annotating the erased sample image to obtain an annotated sample image; and obtaining a set of sample images based on the annotated sample image.

[0074] It should be noted that the original sample image refers to the image that has not undergone erasure processing, and the area to be erased refers to the area that needs to be erased.

[0075] The erased sample image is obtained by erasing the area to be erased in the original sample image. The erased sample image is then labeled to obtain a labeled sample image, which forms the basis of the sample image set. There is no specific order between the erasing and labeling steps; alternatively, the original sample image can be labeled first to obtain a labeled original sample image, and then the area to be erased in the labeled original sample image can be erased to obtain the erased labeled sample image.

[0076] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating an erasure process as shown in an exemplary embodiment of this application. Figure 3As shown, n original sample images are obtained. For each of these n original sample images, a region to be erased is randomly selected; the region to be erased can be different for each original sample image. Then, the regions to be erased in each of these n original sample images are erased, resulting in n erased sample images. Further, a sample image label is obtained for each erased sample image. Based on the sample image label, each erased sample image is annotated to obtain an annotated sample image, thus resulting in a sample image set containing n annotated sample images.

[0077] The original sample images are erased to facilitate the subsequent training of the image processing model on the input sample labeled images for image restoration, and the erased sample images are labeled to facilitate the subsequent training of the image processing model on the input sample labeled images for image recognition.

[0078] In some implementations, erasing the area to be erased in the original sample image to obtain an erased sample image includes: obtaining an erasure ratio corresponding to the original sample image; wherein the erasure ratio represents the ratio of the erased portion to the unerased portion of the original sample image; randomly selecting an area to be erased from the original sample image based on the erasure ratio; and erasing the area to be erased to obtain an erased sample image.

[0079] The erase ratio represents the ratio of the erased portion to the unerased portion of the original sample image. The larger the erase ratio, the larger the erased portion is in the original sample image, and vice versa.

[0080] The erasure ratio for each original sample image can be the same or different. For example, by obtaining a threshold range for the erasure ratio, and randomly selecting a value from that range for the current original sample image to be erased, the erasure ratio for that original sample image may be different for each original sample image. Alternatively, by obtaining a preset erasure ratio and using that preset ratio as the erasure ratio for each original sample image, the erasure ratio for each original sample image will be the same.

[0081] The threshold range or preset erase ratio can be manually set or flexibly selected based on the current training status of the image processing model. For example, the threshold range or preset erase ratio can be calculated based on one or more of the following data: the current number of iterations of the image processing model, the target loss value of the previous iteration, and the image restoration loss value between the image restoration result of the previous iteration and the original sample image. The erase ratio corresponding to each original sample image can then be determined based on the threshold range or preset erase ratio.

[0082] Based on the erasure ratio corresponding to the original sample image, the area to be erased is randomly selected from the original sample image, or... Figure 3 As shown, a region to be erased is randomly selected from the original sample image 1, and the resulting region to be erased is region 1. Then, region 1 in the original sample image 1 is erased to obtain erased sample image 1.

[0083] By randomly selecting regions to be erased from the original sample images using an erasure ratio, the model training effect can be reduced by avoiding fixed regions to be erased. Randomly selecting regions to be erased can ensure the image processing capability of the image processing model and improve the robustness of the model.

[0084] In some implementations, erasing the area to be erased in the original sample image to obtain an erased sample image includes: obtaining a pixel matrix corresponding to the original sample image; determining the pixel values ​​to be modified corresponding to the area to be erased based on the pixel matrix; modifying the pixel values ​​to be modified in the pixel matrix to preset pixel values ​​to obtain a modified pixel matrix; and obtaining an erased sample image based on the modified pixel matrix.

[0085] The original sample image can be viewed as a two-dimensional function, with each pixel in the original sample image corresponding to a function value, thus obtaining the pixel matrix of the original sample image. The function value here can have different meanings for different image representation models. For example, in the Red-Green-Blue (RGB) model based on the three primary colors, the function value can include the intensity values ​​of the pixel in the three RGB channels; similarly, in the Hue-Saturation-Value (HSV) model, the function value can include the chroma, saturation, and luminance values ​​of the pixel in the three HSV channels. Therefore, the same image will have different pixel matrices under different image representation models. This application does not limit the image representation model used for the pixel matrix of the original sample image.

[0086] Then, the pixel values ​​to be modified corresponding to the area to be erased in the pixel matrix are obtained. The pixel values ​​to be modified refer to the original pixel values ​​in the pixel matrix located in the area to be erased. Then, the pixel values ​​to be modified in the pixel matrix are modified to preset pixel values ​​to obtain a modified pixel matrix, and the erased sample image is obtained based on the modified pixel matrix. For example, in practical applications, the range of pixel values ​​to be modified in the pixel matrix is ​​[0, 255], and the preset pixel value can be set to 255.

[0087] By modifying the pixel matrix corresponding to the original sample image, the original sample image is erased, which facilitates the training of the image processing model for image recognition based on the unerased parts. At the same time, by restoring the erased parts, the noise of the sample image labels is avoided from affecting the training process of the image processing model.

[0088] The following describes some implementation methods for calculating the target loss value.

[0089] Please see Figure 4 , Figure 4 This is a flowchart illustrating an image processing method as shown in another exemplary embodiment of this application. Figure 4 As shown, the sample labeled image corresponds to a sample image label, which is obtained by erasing the original sample image; the process of calculating the target loss value based on the image recognition result and the image restoration result in step S230 may include the following steps S231 to S233:

[0090] Step S231: Calculate the image recognition loss value based on the image recognition results and sample image labels.

[0091] Image recognition results reflect the image recognition processing capability of the current image processing model; they represent the actual output of the current image processing model for the input sample labeled images. The sample image labels characterize the expected output for the input sample labeled images. Therefore, based on the difference between the image recognition results and the sample image labels, the image recognition loss value of the current image processing model can be calculated.

[0092] For example, the original sample image x i The corresponding sample image label is y i For the original sample image x i After erasing, the sample labeled image is obtained. Then, label the sample images. The image is input into the image processing model, and the image recognition result is obtained from the output of the image processing model. in, These are the weight parameters of the neural network used in the image recognition part of the image processing model. Then, the image recognition loss value is calculated based on the image recognition results corresponding to each sample labeled image and the sample image labels.

[0093] Image recognition loss can be calculated based on the cross-entropy loss function, and the calculation formula can be expressed as:

[0094]

[0095] in, denoted as the image recognition loss value, and i represents the number of original sample images.

[0096] Step S232: Calculate the image restoration loss value based on the image restoration result and the original sample image.

[0097] The image restoration result reflects the image restoration capability of the current image processing model; it is the actual output of the current image processing model for the input labeled sample image, while the original sample image represents the expected output for the input labeled sample image. Therefore, based on the difference between the image restoration result and the original sample image, the image restoration loss value of the current image processing model can be calculated.

[0098] For example, the original sample image x i After erasing, the sample labeled image is obtained. Then, label the sample images. The image is input into the image processing model, and the image restoration result is obtained from the output of the image processing model. in, These are the weight parameters of the neural network in the image recognition processing part of the image processing model. These are the weight parameters of the neural network used in the image restoration part of the image processing model. Then, the image restoration loss value is calculated based on the image restoration results corresponding to each sample labeled image and the original sample image.

[0099] The formula for calculating the image restoration loss value can be expressed as:

[0100]

[0101] in, denoted by , where i represents the number of original sample images.

[0102] It should be noted that there is no specific order of execution between steps S231 and S232. Step S231 can be executed first, followed by step S232; or step S232 can be executed first, followed by step S231; or steps S231 and S232 can be executed simultaneously. This application does not limit this.

[0103] It is understood that image recognition loss and image restoration loss can also be calculated using other loss functions, such as mean squared error loss (MSE Loss) and KL divergence loss (KL Div Loss), and this application does not limit these.

[0104] Step S233: Calculate the target loss value based on the image recognition loss value and the image restoration loss value.

[0105] Furthermore, the target loss value is calculated based on the image recognition loss value and the image restoration loss value. This is to effectively avoid the interference of noisy data from the sample image labels on the model training stage through the image restoration loss value, thereby increasing the robustness of the model.

[0106] In some implementations, the image processing model includes a first image processing model and a second image processing model, and the model structures corresponding to the first image processing model and the second image processing model are the same; the step S233, which calculates the target loss value based on the image recognition loss value and the image restoration loss value, includes: obtaining the first image recognition loss value and the first image restoration loss value of the first image processing model for the sample image set, and obtaining the second image recognition loss value and the second image restoration loss value of the second image processing model for the sample image set; calculating the consistency loss value based on the first image recognition loss value and the second image recognition loss value; and calculating the target loss value based on the consistency loss value, the first image recognition loss value, the first image restoration loss value, the second image recognition loss value, and the second image restoration loss value.

[0107] The labeled sample images are input into the first image processing model and the second image processing model respectively, so as to combine the image recognition results and image restoration results output by the first image processing model and the second image processing model respectively, and to perform collaborative training on the first image processing model and the second image processing model.

[0108] For example, please refer to Figure 5 , Figure 5 This is a schematic diagram illustrating the training of an image processing model, as shown in an exemplary embodiment of this application. Figure 5 As shown, the original sample image x i The corresponding sample image label is y i For the original sample image x i After the first erasure process, the first sample labeled image is obtained. For the original sample image x i After the second erasure process, the second sample labeled image is obtained. Then, label the sample images. and The images are input into the first image processing model and the second image processing model, respectively, to obtain the first image recognition result P1 and the first image restoration result output by the first image processing model.

[0109]

[0110]

[0111] The second image recognition result P2 and the second image restoration result output by the second image processing model are respectively

[0112]

[0113]

[0114] in, These are the weight parameters of the neural network in the image recognition processing part of the first image processing model. These are the weight parameters of the neural network in the image restoration part of the first image processing model. These are the weight parameters of the neural network in the image recognition processing part of the second image processing model. These are the weight parameters of the neural network in the image restoration part of the second image processing model. Then, the target loss value is calculated based on the first image recognition result, the first image restoration result, the second image recognition result, and the second image restoration result.

[0115] For example, the formulas for calculating the first image recognition loss and the first image restoration loss for the sample image set in the first image processing model can be:

[0116]

[0117]

[0118] in, This represents the first image recognition loss value. This represents the loss value for the first image restoration.

[0119] The formulas for calculating the second image recognition loss and the second image restoration loss for the sample image set in the second image processing model can be:

[0120]

[0121]

[0122] in, This represents the second image recognition loss value. This represents the second image recovery loss value.

[0123] Furthermore, based on the first image recognition loss value and the second image recognition loss value, the consistency loss value is calculated. The formula for calculating the consistency loss value can be:

[0124]

[0125] Since the sample labeled images input to the first image processing model and the second image processing model are generated by randomly erasing the same original sample image, a consistency loss value can be applied to the image recognition results output by the first image processing model and the second image processing model to enable collaborative training of multiple models, so that the individual model has more robust prediction results and improves the training efficiency of the model.

[0126] Then, the target loss value is calculated based on the consistency loss value, the first image recognition loss value, the first image restoration loss value, the second image recognition loss value, and the second image restoration loss value.

[0127] In some implementations, in addition to the first image processing model and the second image processing model, more image processing models may be included. If each image processing model has the same model structure, a consistency loss value is calculated based on the image recognition results of each image processing model, so that multiple models can be used for collaborative training.

[0128] In some implementations, if the first image processing model and the second image processing model have been trained, either image processing model can be selected as the target image processing model based on the model quality of the first image processing model and the second image processing model. For example, a set of test images can be obtained, and the test images in the set can be input into the first image processing model and the second image processing model respectively. Based on the image recognition results of the first image processing model and the second image processing model, the output accuracy of the first image processing model and the second image processing model can be calculated respectively, and the image processing model with the highest accuracy can be selected as the target image processing model.

[0129] In some implementations, calculating a target loss value based on a consistency loss value, a first image recognition loss value, a first image restoration loss value, a second image recognition loss value, and a second image restoration loss value includes: obtaining image recognition weight parameters and image restoration weight parameters; performing a weighted calculation on the first image recognition loss value and the second image recognition loss value based on the image recognition weight parameters to obtain a target image recognition loss value; performing a weighted calculation on the first image restoration loss value and the second image restoration loss value based on the image restoration weight parameters to obtain a target image restoration loss value; and calculating the target loss value based on the consistency loss value, the target image recognition loss value, and the target image restoration loss value.

[0130] Optionally, the first image recognition loss value and the second image recognition loss value can be summed to obtain the target image recognition loss value; alternatively, image recognition weight parameters can be obtained, including first model recognition weight parameters and second model recognition weight parameters. The first image recognition loss value is weighted according to the first model recognition weight parameters to obtain a first recognition loss weighted value, and the second image recognition loss value is weighted according to the second model recognition weight parameters to obtain a second recognition loss weighted value. Finally, the first recognition loss weighted value and the second recognition loss weighted value are summed to obtain the target image recognition loss value.

[0131] For example, summing the first image recognition loss value and the second image recognition loss value:

[0132]

[0133] in, The first image recognition loss value, This represents the second image recognition loss value.

[0134] The target image restoration loss value can be obtained by summing the first image restoration loss value and the second image restoration loss value. Alternatively, image restoration weight parameters can be obtained, including first model restoration weight parameters and second model restoration weight parameters. The first image restoration loss value is weighted according to the first model restoration weight parameters to obtain a first restoration loss weighted value. The second image restoration loss value is weighted according to the second model restoration weight parameters to obtain a second restoration loss weighted value. Finally, the first and second restoration loss weighted values ​​are summed to obtain the target image restoration loss value.

[0135] For example, summing the first image restoration loss value and the second image restoration loss value:

[0136]

[0137] in, The loss value for the first image recovery. To recover the loss value for the second image.

[0138] Then, based on the consistency loss value, the target image recognition loss value, and the target image recovery loss value, the formula for calculating the target loss value can be:

[0139] L=εL ce +αL reg +βL rec

[0140] Where L represents the target loss value, L reg L represents the consistency loss value.rec L represents the target image recovery loss value. ce The target image recognition loss value is represented by ε, α, and β, which are hyperparameters used to control the contribution of consistency loss, target image recognition loss, and target image restoration loss value.

[0141] The hyperparameters ε, α, and β can be preset manually or flexibly selected based on the current training status of the image processing model. For example, the specific values ​​of hyperparameters ε, α, and β can be calculated based on one or more of the following data: the current number of training iterations of the image processing model, the target loss value of the previous training iteration, and the image restoration loss value between the image restoration result of the previous training iteration and the original sample image.

[0142] The following describes some implementation methods for image recognition and image restoration.

[0143] In some implementations, the sample labeled images contained in the sample image set are input into an image processing model to perform image recognition processing and image restoration processing on the sample labeled images through the image processing model, so as to obtain the image recognition result and image restoration result of the sample labeled images. This includes: inputting the sample labeled images contained in the sample image set into an image processing model to extract features from the sample labeled images through the image processing model to obtain image features, and performing image recognition processing and image restoration processing on the sample labeled images based on the image features to obtain the image recognition result and image restoration result of the sample labeled images.

[0144] For example, the input layer of a Convolutional Neural Network (CNN) can be used to obtain the input matrix of the labeled sample image. The input layer is the input to the neural network, and the input matrix is ​​the pixel matrix of the labeled sample image. For example, the input layer of a CNN is an H*W*C matrix, representing the various dimensional attributes of the labeled sample image, where H and W represent the spatial dimensions of the input, with H being the height dimension and W being the width dimension. C is the depth dimension, which is the number of color channels. For example, an input layer of a 32*32*3 matrix indicates that the labeled sample image is a picture with a width of 32 and a height of 32, where 3 represents that in RGB (red, green, and blue) mode, an image is composed of three primary color images superimposed.

[0145] Then, feature extraction is performed on the input matrix to obtain image features. The processing of the input matrix by the convolutional neural network can include operations such as convolution calculation, ReLU nonlinear activation function calculation, and pooling calculation to extract image features from the sample labeled image. Based on the extracted image features, image recognition processing and image restoration processing are performed to obtain image recognition results and image restoration results. The image recognition loss value is determined by using a pre-set loss function combined with the difference between the image recognition result and the sample image label in the sample labeled image. The image restoration loss value is determined by using a pre-set loss function combined with the difference between the image restoration result and the original sample image corresponding to the sample labeled image. Based on the image recognition loss value and the image restoration loss value, the network parameters of the convolutional neural network are adjusted. The network training continues iteratively based on the adjusted network parameters until the pre-training termination condition is met, and the convolutional neural network corresponding to the network parameters at the end of pre-training is used as the target image processing model.

[0146] The convolutional neural network may include at least one of the following: Residual Neural Network (Res Net), Visual Geometry Group Network (VGG Net), Wide Residual Network (Wide Res Net), and Dense Neural Network (Dense Net), etc. This application does not limit the specific network structure of the image processing model.

[0147] In some implementations, the image processing model includes a feature extraction network, an image recognition network, and an image restoration network. The model inputs labeled sample images from a set of sample images into the image processing model to extract features from the labeled sample images, obtain image features, and then performs image recognition and image restoration processing on the feature images to obtain image recognition and image restoration results for the labeled sample images. This includes: inputting the labeled sample images into the feature extraction network for feature extraction to obtain image features corresponding to the labeled sample images output by the feature extraction network; inputting the image features into the image recognition network for image recognition processing to obtain image recognition results corresponding to the labeled sample images output by the image recognition network; and inputting the image features into the image restoration network for image restoration processing to obtain image restoration results corresponding to the labeled sample images output by the image restoration network.

[0148] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating an image processing model as shown in an exemplary embodiment of this application. Figure 6 As shown, the image processing model includes a feature extraction network, an image recognition network, and an image restoration network. The feature extraction network extracts features from the input labeled image and outputs the corresponding image features. The image recognition network performs image recognition processing on the image features output by the feature extraction network, obtaining the image recognition result corresponding to the labeled image output by the image recognition network. The image restoration network performs image restoration processing on the image features output by the feature extraction network, obtaining the image restoration result corresponding to the labeled image output by the image restoration network.

[0149] It is understood that feature extraction networks, image recognition networks, and image restoration networks can be implemented based on convolutional neural networks or other neural networks, and this application does not limit them.

[0150] The following describes some implementation methods for obtaining the target image processing model.

[0151] In some implementations, the image processing model includes a feature extraction network, an image recognition network, and an image restoration network; adjusting the parameters of the image processing model based on a target loss value includes: adjusting the parameters of the feature extraction network and the image recognition network based on the target loss value; and obtaining the target image processing model based on the adjusted feature extraction network and the image recognition network.

[0152] Since the image restoration network in the image processing model only restores the erased sample labeled images during the training process, so that the target loss value combined with the image restoration result can avoid the noise brought by the sample image labels, in practical applications, only the trained feature extraction network and image recognition network are needed.

[0153] For example, an image recognition loss value is determined based on a pre-defined loss function combined with the difference between the image recognition result and the sample image label in the sample annotation image. An image restoration loss value is determined based on a pre-defined loss function combined with the difference between the image restoration result and the original sample image corresponding to the sample annotation image. A target loss value is calculated based on the image recognition loss value and the image restoration loss value. The network parameters of the feature extraction network and the image recognition network are then adjusted according to the target loss value.

[0154] Optionally, the network parameters of the image restoration network can be pre-trained, in which case the network parameters of the image restoration network do not need to be adjusted when training the image processing model. Alternatively, the network parameters of the image restoration network can be untrained, in which case the network parameters of the image restoration network can be adjusted according to the target loss value when training the image processing model. This application does not impose any limitations on this comparison.

[0155] Then, based on the trained feature extraction network and image recognition network, a target image processing model is obtained, which has image recognition capabilities.

[0156] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating a target image processing model as an exemplary embodiment of this application. Figure 7 As shown, during the training phase, the image processing model includes a feature extraction network, an image recognition network, and an image restoration network. The parameters of the feature extraction network and the image recognition network are adjusted based on the target loss value. After multiple iterations of training, if the current training termination condition is met, the target image processing model is obtained by combining the current network parameters of the feature extraction network and the image recognition network.

[0157] The following describes some implementation methods for applying the target image processing model.

[0158] In some implementations, after adjusting the parameters of the image processing model based on the target loss value, the method further includes: obtaining a target image processing model based on the adjusted image processing model; acquiring an image to be recognized; inputting the image to be recognized into the target image processing model to obtain the image recognition result output by the target image processing model.

[0159] The image to be recognized can be an image processing object obtained by the server, or it can be a picture or video uploaded by the terminal to the server. Depending on the application's function, the image to be recognized sent by the terminal can be a picture or video stored locally on the terminal, or a picture or video captured by the terminal's image acquisition device during application execution. One application scenario is on an industrial product production line, where the image to be recognized is obtained by calling the images of industrial products captured by cameras positioned relative to the production line. Another application scenario is a mobile payment application using a camera to capture facial images during face authentication to obtain the image to be recognized. Another application scenario is a shopping application uploading locally stored product images during product search. Yet another application scenario is a video capture device uploading captured video data in real time and sensing facial images in the video to obtain the image to be recognized.

[0160] For example, taking the defect detection scenario of industrial products as an example, the application process of the target image processing model of this application will be introduced. Please refer to... Figure 8 , Figure 8 This is a schematic diagram illustrating image processing as an exemplary embodiment of this application. Figure 8As shown, images of industrial products captured by a camera are input into a trained target image processing model as images to be identified. The feature extraction network of the target image processing model extracts features from the industrial product images to obtain image features. The image recognition network of the target image processing model identifies defects based on these image features and outputs the defect confidence score for each image to be identified. A higher defect confidence score indicates a larger defect in the corresponding industrial product, while a lower defect confidence score indicates a smaller defect. Then, each image to be identified can be classified according to its defect confidence score and a confidence threshold. For example, industrial product images with a defect confidence score between 0 and 0.2 are classified as normal, those between 0.2 and 0.6 are classified as slightly defective, and those between 0.6 and 1 are classified as severely defective.

[0161] During model training, image recognition and image restoration are performed on the erased sample labeled images. The target loss value is calculated based on the image recognition and image restoration results. This allows the image recognition function to be learned through the image recognition results, and the interference of sample image labels on model training is avoided through the image restoration results. As a result, a more robust model is obtained and the accuracy of model image recognition is greatly improved.

[0162] Figure 9 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of this application. This image processing apparatus can be applied to... Figure 1 The implementation environment shown is different from that described above. This image processing apparatus can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the apparatus is applicable.

[0163] like Figure 9 As shown, the exemplary image processing apparatus includes:

[0164] The sample acquisition device 910 is configured to acquire a set of sample images, the set of sample images including erased sample labeled images;

[0165] The recognition and restoration device 920 is configured to input the sample labeled images contained in the sample image set into the image processing model, so as to perform image recognition processing and image restoration processing on the sample labeled images through the image processing model, and obtain the image recognition result and image restoration result of the sample labeled images;

[0166] The loss value calculation device 930 is configured to calculate the target loss value based on the image recognition result and the image restoration result;

[0167] The parameter adjustment device 940 is configured to adjust the parameters of the image processing model based on the target loss value.

[0168] In this exemplary image processing apparatus, erased sample labeled images included in a sample image set are acquired, and then the sample labeled images are input into an image processing model to perform image recognition and image restoration processing on the sample labeled images. The image recognition results and image restoration results of the sample labeled images are obtained, and a target loss value is calculated based on the image recognition results and image restoration results. This target loss value not only reflects the image recognition processing capability of the image processing model through the image recognition results, but also avoids noise from the sample image labels due to the combination of image restoration results. Finally, the parameters of the image processing model are adjusted according to the target loss value, so as to avoid the inaccuracy of model training due to noise from the sample image labels while achieving the image recognition processing capability of the image processing model, thereby improving the model training effect.

[0169] Based on the above exemplary embodiments, the sample labeled image corresponds to a sample image label, and the sample labeled image is obtained by erasing the original sample image; the loss value calculation device 930 includes: a recognition loss calculation module, configured to calculate an image recognition loss value based on the image recognition result and the sample image label; a recovery loss calculation module, configured to calculate an image recovery loss value based on the image recovery result and the original sample image; and a target loss calculation module, configured to calculate a target loss value based on the image recognition loss value and the image recovery loss value.

[0170] In this exemplary image processing apparatus, a target loss value is calculated based on the image recognition loss value and the image restoration loss value, so as to effectively avoid the interference of noisy data of sample image labels on the model training stage through the image restoration loss value, thereby increasing the robustness of the model.

[0171] Based on the above exemplary embodiments, the image processing model includes a first image processing model and a second image processing model, and the model structures corresponding to the first image processing model and the second image processing model are the same; the target loss calculation module includes: a first and a second recovery loss acquisition unit, configured to acquire a first image recognition loss value and a first image recovery loss value of the first image processing model for a sample image set, and to acquire a second image recognition loss value and a second image recovery loss value of the second image processing model for a sample image set; a consistency loss calculation unit, configured to calculate a consistency loss value based on the first image recognition loss value and the second image recognition loss value; and a target loss calculation unit, configured to calculate a target loss value based on the consistency loss value, the first image recognition loss value, the first image recovery loss value, the second image recognition loss value, and the second image recovery loss value.

[0172] In this exemplary image processing apparatus, since the sample labeled images input to the first image processing model and the second image processing model are generated by randomly erasing the same original sample image, a consistency loss value can be applied to the image recognition results output by the first image processing model and the second image processing model to enable collaborative training of multiple models, thereby making the individual model have more robust prediction results and improving the training efficiency of the model.

[0173] Based on the above exemplary embodiments, a target loss value is calculated based on the consistency loss value, the first image recognition loss value, the first image restoration loss value, the second image recognition loss value, and the second image restoration loss value. This includes: obtaining image recognition weight parameters and image restoration weight parameters; performing a weighted calculation on the first image recognition loss value and the second image recognition loss value based on the image recognition weight parameters to obtain the target image recognition loss value; performing a weighted calculation on the first image restoration loss value and the second image restoration loss value based on the image restoration weight parameters to obtain the target image restoration loss value; and calculating the target loss value based on the consistency loss value, the target image recognition loss value, and the target image restoration loss value.

[0174] Based on the above exemplary embodiments, the identification and recovery device 920 includes: a feature extraction module configured to input sample labeled images contained in the sample image set into an image processing model, so as to extract features from the sample labeled images through the image processing model to obtain image features; and an image processing module configured to perform image recognition processing and image recovery processing on the sample labeled images according to the image features to obtain image recognition results and image recovery results of the sample labeled images.

[0175] Based on the above exemplary embodiments, the image processing model includes a feature extraction network, an image recognition network, and an image restoration network. The model inputs labeled sample images from a set of sample images into the image processing model to extract features from the labeled sample images, obtain image features, and then performs image recognition and image restoration processing on the feature images to obtain image recognition and image restoration results for the labeled sample images. This includes: inputting the labeled sample images into the feature extraction network for feature extraction to obtain image features corresponding to the labeled sample images output by the feature extraction network; inputting the image features into the image recognition network for image recognition processing to obtain image recognition results corresponding to the labeled sample images output by the image recognition network; and inputting the image features into the image restoration network for image restoration processing to obtain image restoration results corresponding to the labeled sample images output by the image restoration network.

[0176] In this exemplary image processing apparatus, the networks corresponding to image restoration processing, feature extraction processing, and image recognition processing are separated to obtain a feature extraction network, an image recognition network, and an image restoration network, so that the target image processing network can be obtained subsequently based on the feature extraction processing and image recognition processing.

[0177] Based on the above exemplary embodiments, the image processing model includes a feature extraction network, an image recognition network, and an image restoration network; the parameter adjustment device 940 includes: an adjustment module configured to adjust the parameters of the feature extraction network and the image recognition network based on the target loss value; and a model acquisition module configured to obtain the target image processing model based on the adjusted feature extraction network and the image recognition network.

[0178] In this exemplary image processing apparatus, since the image restoration network in the image processing model only restores the erased sample labeled images during the training process, the target loss value combined with the image restoration result can avoid the noise brought by the sample image labels. In practical applications, only the trained feature extraction network and image recognition network are needed.

[0179] Based on the above exemplary embodiments, the sample acquisition device 910 includes: an original sample image acquisition module configured to acquire an original sample image; an erasure processing module configured to perform erasure processing on the area to be erased in the original sample image to obtain an erased sample image; an annotation processing module configured to perform annotation processing on the erased sample image to obtain a sample annotation image; and an image set acquisition module configured to obtain a sample image set based on the sample annotation image.

[0180] In this exemplary image processing apparatus, the original sample image is erased to facilitate the subsequent image processing model's training on the input sample labeled image for image restoration, and the erased sample image is labeled to facilitate the subsequent image processing model's training on the input sample labeled image for image recognition.

[0181] Based on the above exemplary embodiments, the erased area of ​​the original sample image is erased to obtain an erased sample image, including: obtaining the erase ratio corresponding to the original sample image; wherein, the erase ratio represents the ratio of the erased part to the unerased part of the original sample image; randomly selecting the erased area from the original sample image based on the erase ratio; and erasing the erased area to obtain an erased sample image.

[0182] In this exemplary image processing apparatus, the region to be erased is randomly selected from the original sample image by the erasure ratio, so as to avoid the fixed region to be erased from reducing the effect of model training. Randomly selecting the region to be erased can ensure the image processing capability of the image processing model and improve the robustness of the model.

[0183] Based on the above exemplary embodiments, the erasure process is performed on the area to be erased in the original sample image to obtain an erased sample image, including: obtaining the pixel matrix corresponding to the original sample image; determining the pixel values ​​to be modified corresponding to the area to be erased based on the pixel matrix; modifying the pixel values ​​to be modified in the pixel matrix to preset pixel values ​​to obtain a modified pixel matrix; and obtaining an erased sample image based on the modified pixel matrix.

[0184] In this exemplary image processing apparatus, the original sample image is erased by modifying the pixel matrix corresponding to the original sample image, which facilitates the training of the image processing model for image recognition based on the unerased parts. At the same time, the erased parts are restored to avoid the influence of noise from the sample image labels on the training process of the image processing model.

[0185] Based on the above exemplary embodiments, after adjusting the parameters of the image processing model based on the target loss value, the method further includes: obtaining a target image processing model based on the adjusted image processing model; acquiring an image to be recognized; inputting the image to be recognized into the target image processing model to obtain the image recognition result output by the target image processing model.

[0186] In this exemplary image processing apparatus, during model training, image recognition and image restoration are performed on the erased sample labeled images to calculate the target loss value. This allows the image recognition function to be learned through the image recognition results and the interference of sample image labels on model training to be avoided through the image restoration results, thereby obtaining a more robust model and significantly improving the accuracy of model image recognition.

[0187] It should be noted that the image processing apparatus and the image processing method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the image processing apparatus provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation.

[0188] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the image processing methods provided in the above embodiments.

[0189] Figure 10 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0190] like Figure 10 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0191] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0192] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0193] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0194] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image processing method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0195] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing methods provided in the various embodiments described above.

[0196] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtain a set of sample images, which includes erased sample labeled images. Each sample labeled image corresponds to a sample image label. The sample labeled images are obtained by erasing the original sample images. The sample labeled images contained in the sample image set are input into the image processing model to perform image recognition and image restoration processing on the sample labeled images through the image processing model, so as to obtain the image recognition result and image restoration result of the sample labeled images; the image processing model includes a first image processing model and a second image processing model, and the model structures corresponding to the first image processing model and the second image processing model are the same; Based on the image recognition result corresponding to the first image processing model and the sample image label, the first image recognition loss value is calculated, and based on the image restoration result corresponding to the first image processing model and the original sample image, the first image restoration loss value is calculated. Based on the image recognition result corresponding to the second image processing model and the sample image label, the second image recognition loss value is calculated, and based on the image restoration result corresponding to the second image processing model and the original sample image, the second image restoration loss value is calculated. Based on the first image recognition loss value and the second image recognition loss value, the consistency loss value is calculated; Based on the consistency loss value, the first image recognition loss value, the first image restoration loss value, the second image recognition loss value, and the second image restoration loss value, the target loss value is calculated. The parameters of the image processing model are adjusted based on the target loss value.

2. The method according to claim 1, characterized in that, The calculation of the target loss value based on the consistency loss value, the first image recognition loss value, the first image restoration loss value, the second image recognition loss value, and the second image restoration loss value includes: Obtain image recognition weight parameters and image restoration weight parameters; The first image recognition loss value and the second image recognition loss value are weighted and calculated based on the image recognition weight parameters to obtain the target image recognition loss value. The first image restoration loss value and the second image restoration loss value are weighted and calculated based on the image restoration weight parameters to obtain the target image restoration loss value. The target loss value is calculated based on the consistency loss value, the target image recognition loss value, and the target image recovery loss value.

3. The method according to claim 1, characterized in that, The step of inputting the sample labeled images contained in the sample image set into the image processing model, so as to perform image recognition processing and image restoration processing on the sample labeled images through the image processing model, and obtaining the image recognition result and image restoration result of the sample labeled images, includes: The sample labeled images contained in the sample image set are input into the image processing model so that the image processing model can extract features from the sample labeled images to obtain image features; Based on the image features, image recognition and image restoration processing are performed on the labeled sample image to obtain the image recognition result and image restoration result of the labeled sample image.

4. The method according to claim 3, characterized in that, The image processing model includes a feature extraction network, an image recognition network, and an image restoration network. The step of inputting the labeled sample images contained in the sample image set into the image processing model, extracting features from the labeled sample images using the image processing model to obtain image features, and performing image recognition and image restoration processing on the feature images to obtain the image recognition results and image restoration results of the labeled sample images includes: The labeled sample image is input into the feature extraction network for feature extraction, and the image features corresponding to the labeled sample image output by the feature extraction network are obtained. The image features are input into the image recognition network for image recognition processing to obtain the image recognition result corresponding to the sample labeled image output by the image recognition network; The image features are input into the image restoration network for image restoration processing, and the image restoration result corresponding to the sample labeled image output by the image restoration network is obtained.

5. The method according to claim 1, characterized in that, The image processing model includes a feature extraction network, an image recognition network, and an image restoration network; adjusting the parameters of the image processing model based on the target loss value includes: The parameters of the feature extraction network and the image recognition network are adjusted based on the target loss value; The target image processing model is obtained based on the adjusted feature extraction network and image recognition network.

6. The method according to claim 1, characterized in that, The acquisition of the sample image set includes: Obtain the original sample image; The area to be erased in the original sample image is erased to obtain an erased sample image; The erased sample image is annotated to obtain an annotated sample image; The sample image set is obtained based on the labeled sample images.

7. The method according to claim 6, characterized in that, The process of erasing the area to be erased in the original sample image to obtain an erased sample image includes: Obtain the erasure ratio corresponding to the original sample image; wherein, the erasure ratio represents the ratio of the erased portion to the unerased portion of the original sample image; Based on the erasure ratio, a region to be erased is randomly selected from the original sample image; The area to be erased is erased to obtain an erased sample image.

8. The method according to claim 6, characterized in that, The process of erasing the area to be erased in the original sample image to obtain an erased sample image includes: Obtain the pixel matrix corresponding to the original sample image; Based on the pixel matrix, determine the pixel values ​​to be modified corresponding to the area to be erased; The pixel values ​​to be modified in the pixel matrix are changed to preset pixel values ​​to obtain the modified pixel matrix; The erased sample image is obtained based on the modified pixel matrix.

9. The method according to any one of claims 1 to 8, characterized in that, After adjusting the parameters of the image processing model based on the target loss value, the method further includes: The target image processing model is obtained based on the adjusted image processing model; Acquire the image to be recognized; The image to be recognized is input into the target image processing model to obtain the image recognition result output by the target image processing model.

10. An image processing apparatus, characterized in that, The device includes: A sample acquisition device is configured to acquire a set of sample images, the set of sample images including erased sample annotation images; each sample annotation image corresponds to a sample image label, and the sample annotation images are obtained by erasing the original sample images; The recognition and restoration device is configured to input the sample labeled images contained in the sample image set into an image processing model, so as to perform image recognition processing and image restoration processing on the sample labeled images through the image processing model, and obtain the image recognition result and image restoration result of the sample labeled images; the image processing model includes a first image processing model and a second image processing model, and the model structures corresponding to the first image processing model and the second image processing model are the same; The loss value calculation device is configured to calculate a first image recognition loss value based on the image recognition result corresponding to the first image processing model and the sample image label; calculate a first image restoration loss value based on the image restoration result corresponding to the first image processing model and the original sample image; calculate a second image recognition loss value based on the image recognition result corresponding to the second image processing model and the sample image label; calculate a second image restoration loss value based on the image restoration result corresponding to the second image processing model and the original sample image; calculate a consistency loss value based on the first image recognition loss value and the second image recognition loss value; and calculate a target loss value based on the consistency loss value, the first image recognition loss value, the first image restoration loss value, the second image recognition loss value, and the second image restoration loss value. The parameter adjustment device is configured to adjust the parameters of the image processing model based on the target loss value.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image processing method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, include: processor; as well as A memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the image processing method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, are used to implement the image processing method as described in any one of claims 1 to 9.

Images