Image processing, classification method and device, electronic device and storage medium

Abstract

The present disclosure provides an image processing and classification method and apparatus, an electronic device, and a storage medium, wherein the method comprises: obtaining a first image selected from an image set, and a feature set stored in a storage buffer; performing a first enhancement process and a second enhancement process on the first image to obtain corresponding first enhanced images and second enhanced images; inputting the first enhanced image and the second enhanced image into a trained neural network for feature extraction to obtain first features and second features; performing association process on the first features and the second features with features in the feature set, and adjusting the neural network based on the similarity or difference between the first associated features and the second associated features obtained by the process. The loss function value in the present disclosure comprehensively considers the associated features of enhanced images with different enhancement strengths. This limitation can improve the training performance of the neural network, thereby improving the accuracy in subsequent classification applications.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more specifically, to an image processing and classification method and device, an electronic device, and a storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, end-to-end deep learning technology has become increasingly mature and widely used in various image processing fields, such as image classification, object tracking, scene analysis, etc. Among them, supervised learning has achieved great success in various image processing fields.

[0003] However, supervised learning often requires a large number of manually labeled image samples. Manual labels for some images are difficult to obtain, and the quality of the labels is also difficult to guarantee. Semi-supervised learning aims to guide learning from a large number of unlabeled samples based on a small number of labeled samples, and has broader practical significance in image processing. Summary of the Invention

[0004] The embodiments of the present disclosure at least provide an image processing and classification method and device, an electronic device, and a storage medium to improve network training performance, thereby further improving the accuracy of classification results.

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, comprising:

[0006] Obtaining a first image selected from an image set and a feature set stored in a storage buffer, wherein the feature set includes features of a second image selected from the image set; the second image includes a category label;

[0007] performing a first enhancement process on the first image to obtain a corresponding first enhanced image, and performing a second enhancement process on the first image to obtain a corresponding second enhanced image, wherein the degree of enhancement of the first enhancement process is weaker than the degree of enhancement of the second enhancement process;

[0008] inputting a first enhanced image and a second enhanced image corresponding to the first image into a trained neural network for feature extraction, respectively, to obtain a first feature of the first enhanced image and a second feature of the second enhanced image, wherein the neural network includes an image classification network trained based on the second image selected from the image set;

[0009] Associating the first feature and the second feature with features in the feature set respectively to obtain a first associated feature corresponding to the first feature and a second associated feature corresponding to the second feature;

[0010] The neural network is adjusted based on the similarity or difference between the first associated feature and the second associated feature.

[0011] By using the above-mentioned image processing method, when unlabeled samples (corresponding to the first image) and labeled samples (corresponding to the second image) in the semi-supervised learning are obtained, feature extraction can be performed on the first enhanced image and the second enhanced image obtained by enhancing the first image based on the trained neural network. The network adjustment is achieved by associating the two extracted features (i.e., the first feature and the second feature) with the features in the feature set respectively to obtain the similarity or difference between the first associated feature and the second associated feature, thereby obtaining a trained neural network.

[0012] In the process of adjusting the neural network, the present invention comprehensively considers the correlation features of enhanced images with different enhancement strengths. The correlation features here are an overall evaluation of the enhanced image based on the labeled samples in the image set. That is, the higher the correlation degree between the second image and the corresponding enhanced image, the more reasonable it is to assign the label of the corresponding second image to this enhanced image to a certain extent. Considering that two correlation features belonging to the same first image have a higher similarity, based on this limitation, the training of the neural network can be better guided, so that the training performance and training accuracy of the trained neural network are both higher, thereby further improving the accuracy in subsequent classification applications.

[0013] In a possible implementation, associating the first feature and the second feature with features in the feature set respectively includes:

[0014] Performing a dot product operation on the first feature and each feature in the feature set to obtain a first correlation feature; the first correlation feature is used to represent the degree of correlation between the category labels of the first enhanced image corresponding to the first feature and the second image corresponding to the feature in the feature set;

[0015] Performing a dot product operation on the second feature and each feature in the feature set to obtain the second associated feature; the second associated feature is used to characterize the degree of association between the second enhanced image corresponding to the second feature and the category label of the second image corresponding to the feature in the feature set.

[0016] In a possible implementation, adjusting the neural network based on the similarity or difference between the first correlation feature and the second correlation feature includes:

[0017] Determining a loss function value for the neural network based on a similarity or difference between the first associated feature and the second associated feature;

[0018] The neural network is adjusted at least once based on the loss function value to obtain a trained neural network.

[0019] In a possible implementation, after obtaining the first feature of the first enhanced image, the method further includes:

[0020] performing image classification on the first feature of the first enhanced image using an image classification network included in the neural network to obtain a probability distribution output by the image classification network; the probability distribution is used to indicate a predicted probability that the first enhanced image belongs to each of a plurality of preset categories;

[0021] The determining of a loss function value for the neural network based on the similarity or difference between the first associated feature and the second associated feature includes:

[0022] A loss function value for the neural network is determined based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature.

[0023] In one possible implementation, determining a loss function value for the neural network based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature includes:

[0024] Determine a first enhanced image whose predicted probability value indicated in the corresponding probability distribution is greater than a preset threshold as a first target enhanced image, and determine a category label of a second enhanced image paired with the first target enhanced image based on the probability distribution corresponding to the first target enhanced image;

[0025] When a second enhanced image corresponding to the first target enhanced image is input into a neural network, determining a first sub-loss function value for the neural network based on a probability distribution output by the image classification network and a degree of matching between the determined class labels;

[0026] A loss function value for the neural network is determined based on the first sub-loss function value and a second sub-loss function value determined based on the similarity or difference between the first association feature and the second association feature.

[0027] In one possible implementation, determining a loss function value for the neural network based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature includes:

[0028] updating the first correlation feature based on a probability distribution of an output corresponding to the first enhanced image to obtain an updated first correlation feature;

[0029] Based on the updated first correlation feature, updating the similarity or difference between the first correlation feature and the second correlation feature to obtain an updated similarity or difference;

[0030] Based on the updated similarity or difference, a loss function value for the neural network is determined.

[0031] In a possible implementation, updating the first correlation feature based on the probability distribution of the output corresponding to the first enhanced image to obtain the updated first correlation feature includes:

[0032] For each second image indicated by the feature set, selecting a predicted probability value corresponding to the category label of the second image from the probability distribution output corresponding to the first enhanced image; multiplying the predicted probability value by the feature of the second image to obtain an updated degree of association for the second image;

[0033] The updated correlation degrees for each of the second images are combined to obtain an updated first correlation feature.

[0034] In one possible implementation, determining a loss function value for the neural network based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature includes:

[0035] updating a probability distribution of an output corresponding to the first enhanced image based on the first correlation feature to obtain an updated probability distribution;

[0036] Based on the updated probability distribution and the similarity or difference between the first association feature and the second association feature, a loss function value for the neural network is determined.

[0037] In a possible implementation, updating the probability distribution of the output corresponding to the first enhanced image based on the first correlation feature to obtain an updated probability distribution includes:

[0038] For each preset category among the preset categories corresponding to the first enhanced image, selecting a correlation degree with respect to a second target image from the first correlation feature; wherein the second target image is a second image indicated by a category label as belonging to the preset category;

[0039] Performing a sum operation on each selected correlation degree to determine an updated correlation degree;

[0040] The probability distribution of the output corresponding to the first enhanced image is updated based on the updated association degree determined for each preset category to obtain an updated probability distribution.

[0041] In a possible implementation, determining a loss function value for the neural network based on the similarity or difference between the first correlation feature and the second correlation feature includes:

[0042] Inputting the second image into a neural network to be trained to obtain a probability distribution of an output of the neural network;

[0043] Determining a third sub-loss function value for the neural network based on a match between a probability distribution output by the neural network and a class label carried by the second image;

[0044] A loss function value for the neural network is determined based on the third sub-loss function value and a second sub-loss function value determined based on the similarity or difference between the first association feature and the second association feature.

[0045] In one possible implementation, the method further includes:

[0046] Inputting a second image selected from the image set into the neural network for feature extraction;

[0047] The feature set stored in the storage buffer is updated based on the extracted features of the second image to obtain an updated feature set.

[0048] In a second aspect, the present disclosure also provides a classification method, including:

[0049] Acquire a target image captured in a target scene;

[0050] The target image is processed using a neural network trained by the image processing method described in the first aspect and any of its various embodiments to obtain a processing result; wherein the processing result includes the category to which the target object in the target image belongs.

[0051] In a third aspect, an embodiment of the present disclosure further provides an image processing device, including:

[0052] an acquisition module, configured to acquire a first image selected from an image set and a feature set stored in a storage buffer, wherein the feature set includes features of a second image selected from the image set; the second image includes a category label;

[0053] an enhancement module, configured to perform a first enhancement process on the first image to obtain a corresponding first enhanced image, and perform a second enhancement process on the first image to obtain a corresponding second enhanced image, wherein the degree of enhancement of the first enhancement process is weaker than the degree of enhancement of the second enhancement process;

[0054] an extraction module, configured to input a first enhanced image and a second enhanced image corresponding to the first image into a trained neural network for feature extraction, respectively, to obtain a first feature of the first enhanced image and a second feature of the second enhanced image, wherein the neural network includes an image classification network trained based on the second image selected from the image set;

[0055] An associating module, configured to associate the first feature and the second feature with features in the feature set, respectively, to obtain a first associated feature corresponding to the first feature and a second associated feature corresponding to the second feature;

[0056] An adjustment module is used to adjust the neural network based on the similarity or difference between the first association feature and the second association feature.

[0057] In a fourth aspect, an embodiment of the present disclosure further provides a classification device, including:

[0058] An acquisition module is used to acquire a target image captured in a target scene;

[0059] A classification module is used to process the target image using a neural network trained by the image processing method described in the first aspect and any of its various embodiments to obtain a processing result; wherein the processing result includes the category to which the target object in the target image belongs.

[0060] In the fifth aspect, an embodiment of the present disclosure also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method described in any one of the first aspect and its various embodiments or the steps of the classification method described in the second aspect are performed.

[0061] In the sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method described in the first aspect and any of its various embodiments or the steps of the classification method described in the second aspect are executed.

[0062] For a description of the effects of the above-mentioned device, electronic device, and computer-readable storage medium, please refer to the description of the above-mentioned method, which will not be repeated here.

[0063] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0065] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure is shown;

[0066] Figure 2 A flow chart of a classification method provided by an embodiment of the present disclosure is shown;

[0067] Figure 3 A schematic diagram of an image processing device provided by an embodiment of the present disclosure is shown;

[0068] Figure 4 A schematic diagram of a classification device provided by an embodiment of the present disclosure is shown;

[0069] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0071] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0072] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0073] Research has found that most semi-supervised learning methods in related technologies are based on consistency constraints. That is, a model classifier can be trained using a small number of labeled samples. At the same time, for unsupervised samples, the model classifier should maintain "consistent" predictions for different image enhancements of the same picture, thereby constructing the loss function of unsupervised samples.

[0074] However, when the number of labeled samples is very small, the model classifier is inaccurate, and using consistency constraints on an inaccurate classifier will lead to poor model performance.

[0075] Based on the above research, the present disclosure provides an image processing method to improve network training performance, thereby further improving the accuracy in subsequent classification applications.

[0076] To facilitate understanding of this embodiment, an image processing method disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the image processing method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities, such as a terminal device, a server, or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, the image processing method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0077] See also Figure 1 FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, the method comprising steps S101 to S105, wherein:

[0078] S101: Acquire a first image selected from an image set and a feature set stored in a storage buffer, wherein the feature set includes features of a second image selected from the image set; the second image includes a category label;

[0079] S102: performing a first enhancement process on the first image to obtain a corresponding first enhanced image, and performing a second enhancement process on the first image to obtain a corresponding second enhanced image, wherein the enhancement degree of the first enhancement process is weaker than the enhancement degree of the second enhancement process;

[0080] S103: Inputting a first enhanced image and a second enhanced image corresponding to the first image into a trained neural network for feature extraction, respectively, to obtain a first feature of the first enhanced image and a second feature of the second enhanced image, wherein the neural network includes an image classification network trained based on the second image selected from the image set;

[0081] S104: Associating the first feature and the second feature with features in the feature set respectively to obtain a first associated feature corresponding to the first feature and a second associated feature corresponding to the second feature;

[0082] S105: Adjusting the neural network based on the similarity or difference between the first correlation feature and the second correlation feature.

[0083] To facilitate understanding of the image processing method provided by the embodiments of the present disclosure, the following briefly describes the application scenarios of the method. The image processing method of the embodiments of the present disclosure can be mainly applied to the field of vision. For example, based on a trained neural network, it can be applied to classification tasks such as object detection and image classification.

[0084] Considering that semi-supervised learning methods in related technologies can use a small number of labeled samples to train a model classifier, at the same time, for unsupervised samples, the model classifier should maintain "consistent" predictions for different image enhancements of the same image, thereby constructing a loss function for unsupervised samples. However, the number of labeled samples is usually relatively small, and in this case the resulting model classifier is inaccurate. Therefore, using consistency constraints will lead to poor model performance.

[0085] It is precisely to solve the above problems that the embodiments of the present disclosure provide a neural network training scheme based on consistency constraints at the feature level to improve the network training performance in semi-supervised learning.

[0086] Here, the neural network can be adjusted in combination with the associated feature information corresponding to the two enhanced images (the first enhanced image and the second enhanced image). Since the two enhanced images correspond to the same first image, the first and second features corresponding to the two enhanced images corresponding to the same first image, as well as the features in the feature set stored in the storage buffer, can be used to compare the similarity between the two associated features (i.e., the first associated feature and the second associated feature). This further enables the trained neural network to distinguish the classification characteristics of enhanced images belonging to different first images while integrating the classification characteristics of enhanced images belonging to the same first image, thereby increasing the accuracy and robustness of subsequent classification applications.

[0087] In the disclosed embodiments, the first and second features may be extracted based on a trained neural network. Before extracting the two features (the first and second features) using the neural network, the neural network may be pre-trained, that is, trained using a labeled second image.

[0088] During the neural network training process, the second image can be used as input data of the neural network, and the category label of the second image can be used as supervision data of the neural network output result. The neural network can be trained for one or more rounds until a pre-trained neural network is obtained.

[0089] In addition, the pre-trained neural network here can also be an initial neural network that has been pre-constructed and has not yet been input with training samples. In this way, during the network training process, the initial neural network can be trained based on the first image and the second image at the same time.

[0090] The first image and the second image in the embodiment of the present disclosure can both come from an image set, which can be a collection of multiple image samples collected in any field of computer vision. The image samples here can be labeled images (corresponding to the second image). For example, some image samples can be information about the position, size, etc. of the target object marked for the target detection task, and some image samples can be object semantic information marked for the target semantic segmentation task; in addition, the image samples here can also be original images without labels (corresponding to the first image).

[0091] In practical applications, the acquired first and second images may be partial images of the entire image set. For example, in an image set including 50,000 first images and 10,000 second images, the acquired first and second images may be 10 and 1, respectively.

[0092] Here, before training the neural network based on the associated features corresponding to the two enhanced images, image enhancement with different degrees of enhancement can be performed first, so that the two enhanced images obtained will have different degrees of enhancement features while retaining the basic features of the original first image.

[0093] In order to better perform unsupervised learning, the embodiment of the present disclosure can achieve consistency at the instance level based on the storage buffer, that is, the loss function value for the neural network can be determined based on the similarity or difference between the associated features corresponding to the two enhanced images. For example, the loss function value can be determined by the similarity between the two associated features. The higher the similarity, to a certain extent, it means that the characteristic attributes of the corresponding enhanced images are closer, and the corresponding loss function value is smaller. Conversely, the lower the similarity, to a certain extent, it means that the characteristic attributes of the corresponding enhanced images are farther apart, and the corresponding loss function value is larger. The loss function value can be adjusted by constraining the similarity, so that the entire neural network can be further adjusted, so that the performance of the neural network in image classification tasks can be further improved.

[0094] Among them, the associated features corresponding to the enhanced image can be determined based on the dot product operation results between the features extracted from the enhanced image and the various features included in the feature set, and are used to characterize the degree of association between the category labels of the second image corresponding to the enhanced image and the features in the feature set.

[0095] For an enhanced image, it is more likely to belong to the category label corresponding to the second image with a higher degree of correlation, and less likely to belong to the category label corresponding to the second image with a lower degree of correlation. In this way, when determining corresponding correlation features for two enhanced images, the adjustment of the loss function value can be guided by the similarity or difference between the correlation features.

[0096] When the loss function value changes, the neural network can be adjusted through back propagation, and then the second feature information of the currently acquired second image can be updated to obtain an updated feature set.

[0097] It can be seen that the feature set stored in the storage buffer can be updated as the network is adjusted. This is mainly because after the network parameter values ​​of the neural network are adjusted, the feature information output by the feature extraction layer based on the neural network will also change accordingly.

[0098] In this way, during the subsequent rounds of network adjustment, the similarity between the above-mentioned related features can be calculated based on the updated feature set, so that the obtained loss function value is closer and closer to the training target, and the performance indicators such as the neural network's classification accuracy of images are better.

[0099] The first feature / second feature extracted from the enhanced image can be image features such as texture, color, shape, spatial relationship, or other image-related features. In the actual feature extraction process, the feature dimensions extracted from different enhanced images can be the same, for example, all can be 128-dimensional image features.

[0100] It should be noted that in the embodiment of the present disclosure, the loss function value can be determined not only based on two associated features, but also in combination with other relationships. For example, the final loss function value can be determined by considering the probability distribution output after the second image with a label is input into the neural network and the matching degree between the category label carried by the second image. This takes into account the consistency at the semantic level to a certain extent. The consistency here mainly refers to the consistency between the classification result pointed to by the maximum predicted probability value in the probability distribution and the category label.

[0101] When the loss function value for the neural network is determined, the neural network can be adjusted in one or more rounds based on the loss function value. In practical applications, if the currently determined loss function value is not small enough, it can be said that the network performance of the current neural network does not meet the requirements. At this time, the loss function value can be used for back propagation to determine the network parameter values ​​of the adjusted neural network. After the neural network is adjusted, the enhanced image obtained by the enhancement process can be input into the neural network again for the next round of network adjustment, and the adjusted loss function value can be determined. The network is adjusted again according to the adjusted loss function value, and this cycle is repeated until the loss function value is small enough or other network convergence conditions are met, and the network training can be stopped.

[0102] The network convergence condition may be any of various network training termination conditions, such as the number of training times reaching a preset number, the number of training samples being traversed, etc., and no specific limitation is imposed here.

[0103] The loss function value in the disclosed embodiment is not only related to the similarity between the aforementioned associated features, but also combines the degree of influence of the probability distribution corresponding to the enhanced image with a weak enhancement effect (the first enhanced image) on the enhanced image with a strong enhancement effect (i.e., the second enhanced image). Specifically, the process of determining the above-mentioned loss function value can be illustrated using two paired enhanced images (i.e., the first enhanced image and the second enhanced image from the same first image), including the following steps:

[0104] Step 1: using an image classification network included in a neural network to perform an image classification on a first feature of the first enhanced image, thereby obtaining a probability distribution of an output of the image classification network; the probability distribution is used to indicate a predicted probability that the first enhanced image belongs to each of a plurality of preset categories;

[0105] Step 2: Determine a loss function value for the neural network based on the probability distribution of the image classification network output and the similarity or difference between the first associated feature and the second associated feature.

[0106] Here, for the two enhanced images corresponding to the first image, a probability distribution indicating the predicted probability that the first enhanced image belongs to each of multiple preset categories can be determined based on the first enhanced image with a smaller degree of enhancement, and then the loss function value for the neural network can be determined by combining the probability distribution and the similarity between the first associated feature and the second associated feature.

[0107] In a specific application, the first sub-loss function value related to semantic consistency can be determined in combination with the probability distribution corresponding to the first enhanced image, and the second sub-loss function value determined by the similarity or difference between the first associated feature and the second associated feature can be used to determine the loss function value for the neural network. This can be achieved by the following steps:

[0108] Step 1: Determine a first enhanced image whose predicted probability value in the corresponding probability distribution is greater than a preset threshold as a first target enhanced image, and determine a category label of a second enhanced image paired with the first target enhanced image based on the probability distribution corresponding to the first target enhanced image;

[0109] Step 2: When the second enhanced image corresponding to the first target enhanced image is input into the neural network, a first sub-loss function value for the neural network is determined based on the probability distribution output by the image classification network and the consistency between the determined class labels;

[0110] Step 3: Determine a loss function value for the neural network based on the first sub-loss function value and a second sub-loss function value determined based on the similarity or difference between the first associated feature and the second associated feature.

[0111] Here, we can first perform sample screening through the probability distribution corresponding to the first enhanced image to screen out a first target enhanced image with relatively high credibility. In this way, when assigning a category label to the corresponding second enhanced image (the second enhanced image and the first target enhanced image belong to the same first image) based on the probability distribution corresponding to the first target enhanced image, the corresponding category label is also accurate enough. That is, the label determined by the probability distribution of the screened weakly enhanced sample with relatively high credibility can be used to learn the corresponding strongly enhanced sample to improve the performance of the network.

[0112] In an embodiment of the present disclosure, when the second enhanced image is input into the neural network, the first sub-loss function value can be determined based on the probability distribution of the neural network output and the consistency between the determined category labels, and the overall loss function value can be determined in combination with the above-mentioned second sub-loss function value.

[0113] In practical applications, in order to make the loss function value more accurately represent the performance of the neural network and improve the training efficiency of the neural network, the corresponding associated features can be adjusted based on the probability distribution, and then the loss function value can be updated using the updated associated features. This can be achieved through the following steps:

[0114] Step 1: updating the first correlation feature based on the probability distribution of the output corresponding to the first enhanced image to obtain an updated first correlation feature;

[0115] Step 2: Based on the updated first correlation feature, the similarity or difference between the first correlation feature and the second correlation feature is updated to obtain an updated similarity or difference;

[0116] Step 3: Based on the updated similarity or difference, determine the loss function value for the neural network.

[0117] Here, the first correlation feature obtained from the first enhanced image can first be updated based on the probability distribution corresponding to the first enhanced image, thereby obtaining an updated first correlation feature. Specifically, for each second image included in the feature set, a predicted probability value corresponding to the category label of the second image can be selected from the probability distribution obtained from the first enhanced image; the predicted probability value is multiplied by the feature of the second image to obtain an updated correlation degree for the second image. In this way, the updated correlation degrees for each second image are combined to obtain the updated first correlation feature.

[0118] In order to facilitate the description of the updating process of the first correlation feature, a specific example will be provided below for illustration.

[0119] When performing classification prediction on the first enhanced image, the corresponding probability distribution p can be determined. Here, the probability distribution p includes a probability set of the predicted probability that the first enhanced image belongs to each preset category in multiple preset categories. At the same time, after performing a dot multiplication operation on the first feature extracted from the first enhanced image and the features of each second image in the memory buffer, a first correlation feature q can be obtained. Here, the first correlation feature q includes the degree of correlation between the first enhanced image and the category labels carried by each second image.

[0120] Here, for each second image included in the feature set, the predicted probability value of the category corresponding to the second image can be determined based on the category label of the second image, and then the predicted probability value is multiplied by the feature of the second image to obtain the updated degree of association for the second image.

[0121] For the i-th second image, its correlation degree can be updated using the following formula:

[0122]

[0123] in, Characterizes the degree of association after the i-th second image is updated, q i Characterize the features of the i-th second image, p j Characterizes the predicted probability value that the i-th second image belongs to the j-th class.

[0124] In the embodiment of the present disclosure, the updated first correlation feature can be obtained by combining the updated correlation degrees corresponding to the second images.

[0125] The updated first correlation feature may be normalized first, and then the similarity is updated based on the normalized first correlation feature.

[0126] In practical applications, in order to determine a more accurate loss function value, the probability distribution can be adjusted based on the first correlation feature, and then the loss function value can be updated using the updated probability distribution. This can be achieved by the following steps:

[0127] Step 1: updating the probability distribution of the output corresponding to the first enhanced image based on the first correlation feature to obtain an updated probability distribution;

[0128] Step 2: Based on the updated probability distribution and the similarity or difference between the first associated feature and the second associated feature, determine the loss function value for the neural network.

[0129] Here, the association degree determined for each preset category can be updated by summing the association degrees, and then the probability distribution can be updated based on the updated association degree. This allows the influence of association features to be comprehensively considered when determining the probability distribution, making the determined probability distribution more consistent with the actual task characteristics. Specifically, the probability distribution can be updated through the following steps:

[0130] Step 1: for each preset category corresponding to the first enhanced image, selecting a correlation degree with respect to a second target image from the first correlation feature; wherein the second target image is a second image indicated by a category label as belonging to the preset category;

[0131] Step 2: Sum the selected correlation levels to determine the updated correlation level;

[0132] Step three: updating the probability distribution of the output corresponding to the first enhanced image based on the updated correlation degree determined for each preset category to obtain an updated probability distribution.

[0133] Here, first, for each preset category, the degree of association with the second target image can be selected from the first association feature corresponding to the first enhanced image, that is, the degree of association of the second image whose category label indicates that it belongs to the preset category is selected, and the selected degrees of association are summed up to obtain the degree of association corresponding to each preset category (corresponding to the sum of the degree of association values), and then the probability distribution can be updated.

[0134] In the process of updating the probability distribution based on the correlation degrees and values ​​determined for each preset category, this can be achieved by performing a summation operation between the correlation degrees and values ​​and the original probability distribution. In the specific summation process, the correlation degrees and values ​​can be expanded to a data dimension consistent with the probability distribution to ensure the accuracy of the summation operation.

[0135] To facilitate explanation of the updating process of the above probability distribution, a specific example will be given below.

[0136] Here, the probability distribution p corresponding to the first enhanced image and the first correlation feature q obtained by multiplying the first feature extracted from the first enhanced image with the features of each second image in the memory buffer are still used as an example for explanation.

[0137] Here, for each preset category corresponding to the first enhanced image, the association degree of one or more second target images belonging to this preset category can be selected from the various association degrees included in the first association feature q, and the updated association degree for each preset category can be determined by summing up the various association degrees.

[0138] Among them, for the i-th preset category, a correlation degree and value can be determined, corresponding to In the process of updating the probability distribution of the first enhanced image, the following formula may be used for updating:

[0139]

[0140] in, represents the updated probability distribution, It represents the correlation degree and value corresponding to the i-th preset category, and α is an adjustable coefficient.

[0141] When the probability distribution is updated, the relevant loss function value is also adjusted accordingly.

[0142] In the embodiment of the present disclosure, in addition to determining the loss function value in the above manner, the loss function value may also be determined by comprehensively considering the consistency constraints of the second image, specifically including the following steps:

[0143] Step 1: input the second image into the neural network to be trained to obtain the probability distribution of the neural network output;

[0144] Step 2: Determine the value of the third sub-loss function for the neural network based on the probability distribution output by the neural network and the degree of match between the category label carried by the second image;

[0145] Step three: Determine a loss function value for the neural network based on the third sub-loss function value and the second sub-loss function value determined based on the similarity or difference between the first associated feature and the second associated feature.

[0146] Here, based on the degree of match between the probability distribution of the second image output and the category label carried by the second image, the third sub-loss function value for the neural network can be determined. In this way, the final loss function value can be determined in combination with the other sub-loss function values ​​determined by the above-mentioned first feature, thereby achieving better network training performance.

[0147] Based on the image processing method provided by the embodiment of the present disclosure, the embodiment of the present disclosure also provides a classification method, such as Figure 2 As shown, this can be achieved by following the steps below:

[0148] S201: Acquire a target image captured in a target scene;

[0149] S202: Processing the target image using the trained neural network to obtain a processing result; wherein the processing result includes the category to which the target object in the target image belongs.

[0150] The target image obtained here can be an image captured by an acquisition device set up in any target scene. After processing the target image using the above-mentioned trained neural network, a processing result including the category to which the target object in the image belongs can be obtained, for example, whether the target object belongs to a vehicle, a large truck, or a small car, etc.

[0151] For the training process of the neural network, please refer to the above description and will not be repeated here.

[0152] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0153] Based on the same inventive concept, an apparatus corresponding to the method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the apparatus in the embodiment of the present disclosure is similar to the above-mentioned method in the embodiment of the present disclosure, the implementation of the apparatus can refer to the implementation of the method, and the repeated parts will not be repeated.

[0154] Reference Figure 3 FIG. 1 is a schematic diagram of an image processing device provided by an embodiment of the present disclosure, the device comprising: an acquisition module 301, an enhancement module 302, an extraction module 303, an association module 304 and an adjustment module 305; wherein,

[0155] An acquisition module 301 is configured to acquire a first image selected from an image set and a feature set stored in a storage buffer, wherein the feature set includes features of a second image selected from the image set; the second image includes a category label;

[0156] an enhancement module 302 configured to perform a first enhancement process on the first image to obtain a corresponding first enhanced image, and perform a second enhancement process on the first image to obtain a corresponding second enhanced image, wherein the degree of enhancement of the first enhancement process is weaker than that of the second enhancement process;

[0157] an extraction module 303 for inputting a first enhanced image and a second enhanced image corresponding to the first image into a trained neural network for feature extraction, thereby obtaining a first feature of the first enhanced image and a second feature of the second enhanced image, wherein the neural network includes an image classification network trained based on the second image selected from the image set;

[0158] An association module 304 is configured to associate the first feature and the second feature with features in the feature set to obtain a first associated feature corresponding to the first feature and a second associated feature corresponding to the second feature;

[0159] The adjustment module 305 is configured to adjust the neural network based on the similarity or difference between the first correlation feature and the second correlation feature.

[0160] By using the above-mentioned image processing device, when unlabeled samples (corresponding to the first image) and labeled samples (corresponding to the second image) in semi-supervised learning are obtained, feature extraction can be performed on the first enhanced image and the second enhanced image obtained by enhancing the first image based on the trained neural network. The network adjustment is achieved by associating the two extracted features (i.e., the first feature and the second feature) with the features in the feature set to obtain the similarity or difference between the first associated features and the second associated features, thereby obtaining a trained neural network.

[0161] In the process of adjusting the neural network, the present invention comprehensively considers the correlation features of enhanced images with different enhancement strengths. The correlation features here are an overall evaluation of the enhanced image based on the labeled samples in the image set. That is, the higher the correlation degree between the second image and the corresponding enhanced image, the more reasonable it is to assign the label of the corresponding second image to this enhanced image to a certain extent. Considering that two correlation features belonging to the same first image have a higher similarity, based on this limitation, the training of the neural network can be better guided, so that the training performance and training accuracy of the trained neural network are both higher, thereby further improving the accuracy in subsequent classification applications.

[0162] In a possible implementation, the association module 304 is configured to associate the first feature and the second feature with features in the feature set according to the following steps:

[0163] Performing a dot product operation on the first feature and each feature in the feature set to obtain a first correlation feature; the first correlation feature is used to represent the degree of correlation between the category labels of the first enhanced image corresponding to the first feature and the second image corresponding to the feature in the feature set;

[0164] A dot product operation is performed on the second feature and each feature in the feature set to obtain a second correlation feature; the second correlation feature is used to characterize the degree of correlation between the second enhanced image corresponding to the second feature and the category label of the second image corresponding to the feature in the feature set.

[0165] In a possible implementation, the adjustment module 305 is configured to adjust the neural network based on the similarity or difference between the first correlation feature and the second correlation feature according to the following steps:

[0166] Determining a loss function value for the neural network based on a similarity or a difference between the first correlation feature and the second correlation feature;

[0167] The neural network is adjusted for at least one round based on the loss function value to obtain a trained neural network.

[0168] In one possible implementation, the adjustment module 305 is configured to determine a loss function value for the neural network based on the similarity or difference between the first correlation feature and the second correlation feature according to the following steps:

[0169] After obtaining the first feature of the first enhanced image, performing image classification on the first feature of the first enhanced image using an image classification network included in the neural network to obtain a probability distribution output by the image classification network; the probability distribution is used to indicate a predicted probability that the first enhanced image belongs to each of a plurality of preset categories;

[0170] A loss function value for the neural network is determined based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature.

[0171] In one possible implementation, the adjustment module 305 is configured to determine a loss function value for the neural network based on the probability distribution output by the image classification network and the similarity or difference between the first correlation feature and the second correlation feature according to the following steps:

[0172] Determine a first enhanced image whose predicted probability value indicated in the corresponding probability distribution is greater than a preset threshold as a first target enhanced image, and determine a category label of a second enhanced image paired with the first target enhanced image based on the probability distribution corresponding to the first target enhanced image;

[0173] When the second enhanced image corresponding to the first target enhanced image is input into the neural network, determining a first sub-loss function value for the neural network based on a probability distribution output by the image classification network and a degree of matching between the determined class labels;

[0174] A loss function value for the neural network is determined based on the first sub-loss function value and a second sub-loss function value determined based on the similarity or difference between the first associated feature and the second associated feature.

[0175] In one possible implementation, the adjustment module 305 is configured to determine a loss function value for the neural network based on the probability distribution output by the image classification network and the similarity or difference between the first correlation feature and the second correlation feature according to the following steps:

[0176] Updating the first correlation feature based on the probability distribution of the output corresponding to the first enhanced image to obtain an updated first correlation feature;

[0177] Based on the updated first correlation feature, updating the similarity or difference between the first correlation feature and the second correlation feature to obtain an updated similarity or difference;

[0178] Based on the updated similarity or difference, a loss function value for the neural network is determined.

[0179] In a possible implementation, the adjustment module 305 is configured to update the first correlation feature based on the probability distribution of the output corresponding to the first enhanced image according to the following steps to obtain an updated first correlation feature:

[0180] For each second image indicated by the feature set, selecting a predicted probability value corresponding to the category label of the second image from the probability distribution corresponding to the output of the first enhanced image; multiplying the predicted probability value by the feature of the second image to obtain an updated degree of association for the second image;

[0181] The updated correlation degrees for each second image are combined to obtain an updated first correlation feature.

[0182] In one possible implementation, the adjustment module 305 is configured to determine a loss function value for the neural network based on the probability distribution output by the image classification network and the similarity or difference between the first correlation feature and the second correlation feature according to the following steps:

[0183] updating a probability distribution of an output corresponding to the first enhanced image based on the first correlation feature to obtain an updated probability distribution;

[0184] A loss function value for the neural network is determined based on the updated probability distribution and the similarity or difference between the first associated feature and the second associated feature.

[0185] In a possible implementation, the adjustment module 305 is configured to update the probability distribution of the output corresponding to the first enhanced image based on the first correlation feature according to the following steps to obtain an updated probability distribution:

[0186] For each preset category among the preset categories corresponding to the first enhanced image, selecting a correlation degree for a second target image from the first correlation feature; wherein the second target image is a second image indicated by a category label as belonging to the preset category;

[0187] Performing a sum operation on each selected correlation degree to determine an updated correlation degree;

[0188] The probability distribution of the output corresponding to the first enhanced image is updated based on the updated association degree determined for each preset category to obtain an updated probability distribution.

[0189] In one possible implementation, the adjustment module 305 is configured to determine a loss function value for the neural network based on the similarity or difference between the first correlation feature and the second correlation feature according to the following steps:

[0190] Inputting the second image into the neural network to be trained to obtain a probability distribution of the neural network output;

[0191] Determining a third sub-loss function value for the neural network based on a match between a probability distribution output by the neural network and the category label carried by the second image;

[0192] A loss function value for the neural network is determined based on the third sub-loss function value and the second sub-loss function value determined based on the similarity or difference between the first association feature and the second association feature.

[0193] In a possible implementation, the above device further includes:

[0194] The updating module 306 is configured to input the second image selected from the image set into the neural network for feature extraction; and update the feature set stored in the storage buffer based on the features of the extracted second image to an updated feature set.

[0195] Reference Figure 4 FIG. 4 is a schematic diagram of a classification device provided by an embodiment of the present disclosure, the device comprising: an acquisition module 401 and a classification module 402; wherein,

[0196] An acquisition module 401 is used to acquire a target image captured in a target scene;

[0197] The classification module 402 is used to process the target image using the trained neural network to obtain a processing result; wherein the processing result includes the category to which the target object in the target image belongs.

[0198] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0199] The present disclosure also provides an electronic device, such as Figure 5 FIG. 5 is a schematic diagram of the electronic device structure provided by an embodiment of the present disclosure, including: a processor 501, a memory 502, and a bus 503. The memory 502 stores machine-readable instructions (e.g., Figure 3 The execution instructions corresponding to the acquisition module 301, the enhancement module 302, the extraction module 303, the association module 304 and the adjustment module 305 in the device; for example, Figure 4 When the electronic device is running, the processor 501 communicates with the memory 502 via the bus 503, and the machine-readable instructions are executed by the processor 501. Figure 1 The steps of the image processing method shown or Figure 2 The steps of the classification method are shown.

[0200] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to execute the steps of the method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0201] The present disclosure also provides a computer program product that carries program code. The program code includes instructions that can be used to execute the steps of the method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0202] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0203] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0204] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0205] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0206] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0207] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: include: Obtaining a first image selected from an image set and a feature set stored in a storage buffer, the feature set including features of a second image selected from the image set; The second image includes a category label; performing a first enhancement process on the first image to obtain a corresponding first enhanced image, and performing a second enhancement process on the first image to obtain a corresponding second enhanced image, wherein the degree of enhancement of the first enhancement process is weaker than the degree of enhancement of the second enhancement process; inputting a first enhanced image and a second enhanced image corresponding to the first image into a trained neural network for feature extraction, respectively, to obtain a first feature of the first enhanced image and a second feature of the second enhanced image, wherein the neural network includes an image classification network trained based on the second image selected from the image set; Associating the first feature and the second feature with features in the feature set respectively to obtain a first associated feature corresponding to the first feature and a second associated feature corresponding to the second feature; adjusting the neural network based on the similarity or difference between the first correlation feature and the second correlation feature; The associating the first feature and the second feature with features in the feature set respectively includes: Performing a dot product operation on the first feature and each feature in the feature set to obtain a first correlation feature; the first correlation feature is used to represent the degree of correlation between the category labels of the first enhanced image corresponding to the first feature and the second image corresponding to the feature in the feature set; Performing a dot product operation on the second feature and each feature in the feature set to obtain the second associated feature; the second associated feature is used to characterize the degree of association between the second enhanced image corresponding to the second feature and the category label of the second image corresponding to the feature in the feature set.

2. The method according to claim 1, characterized in that The adjusting the neural network based on the similarity or difference between the first correlation feature and the second correlation feature includes: Determining a loss function value for the neural network based on a similarity or difference between the first associated feature and the second associated feature; The neural network is adjusted at least once based on the loss function value to obtain a trained neural network.

3. The method according to claim 2, characterized in that After obtaining the first feature of the first enhanced image, the method further includes: performing image classification on the first feature of the first enhanced image using an image classification network included in the neural network to obtain a probability distribution output by the image classification network; the probability distribution is used to indicate a predicted probability that the first enhanced image belongs to each of a plurality of preset categories; The determining of a loss function value for the neural network based on the similarity or difference between the first associated feature and the second associated feature includes: A loss function value for the neural network is determined based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature.

4. The method according to claim 3, characterized in that The determining of a loss function value for the neural network based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature includes: Determine a first enhanced image whose predicted probability value indicated in the corresponding probability distribution is greater than a preset threshold as a first target enhanced image, and determine a category label of a second enhanced image paired with the first target enhanced image based on the probability distribution corresponding to the first target enhanced image; When a second enhanced image corresponding to the first target enhanced image is input into a neural network, determining a first sub-loss function value for the neural network based on a probability distribution output by the image classification network and a degree of matching between the determined class labels; A loss function value for the neural network is determined based on the first sub-loss function value and a second sub-loss function value determined based on the similarity or difference between the first association feature and the second association feature.

5. The method according to claim 3 or 4, characterized in that The determining of a loss function value for the neural network based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature includes: updating the first correlation feature based on a probability distribution of an output corresponding to the first enhanced image to obtain an updated first correlation feature; Based on the updated first correlation feature, updating the similarity or difference between the first correlation feature and the second correlation feature to obtain an updated similarity or difference; Based on the updated similarity or difference, a loss function value for the neural network is determined.

6. The method according to claim 5, characterized in that The updating of the first correlation feature based on the probability distribution of the output corresponding to the first enhanced image to obtain the updated first correlation feature includes: For each second image indicated by the feature set, selecting a predicted probability value corresponding to the category label of the second image from the probability distribution output corresponding to the first enhanced image; multiplying the predicted probability value by the feature of the second image to obtain an updated degree of association for the second image; The updated correlation degrees for each of the second images are combined to obtain an updated first correlation feature.

7. The method according to claim 3 or 4, characterized in that The determining of a loss function value for the neural network based on a probability distribution output by the image classification network and a similarity or difference between the first associated feature and the second associated feature includes: updating a probability distribution of an output corresponding to the first enhanced image based on the first correlation feature to obtain an updated probability distribution; Based on the updated probability distribution and the similarity or difference between the first association feature and the second association feature, a loss function value for the neural network is determined.

8. The method according to claim 7, characterized in that The updating of the probability distribution of the output corresponding to the first enhanced image based on the first correlation feature to obtain an updated probability distribution includes: For each preset category among the preset categories corresponding to the first enhanced image, selecting a correlation degree with respect to a second target image from the first correlation feature; wherein the second target image is a second image indicated by a category label as belonging to the preset category; Performing a sum operation on each selected correlation degree to determine an updated correlation degree; The probability distribution of the output corresponding to the first enhanced image is updated based on the updated association degree determined for each preset category to obtain an updated probability distribution.

9. The method according to any one of claims 2 to 4, characterized in that: The determining of a loss function value for the neural network based on the similarity or difference between the first associated feature and the second associated feature includes: Inputting the second image into a neural network to be trained to obtain a probability distribution of an output of the neural network; Determining a third sub-loss function value for the neural network based on a match between a probability distribution output by the neural network and a class label carried by the second image; A loss function value for the neural network is determined based on the third sub-loss function value and a second sub-loss function value determined based on the similarity or difference between the first association feature and the second association feature.

10. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Inputting a second image selected from the image set into the neural network for feature extraction; The feature set stored in the storage buffer is updated based on the extracted features of the second image to obtain an updated feature set.

11. A classification method, characterized in that include: Acquire a target image captured in a target scene; The target image is processed using a neural network trained by the image processing method according to any one of claims 1 to 10 to obtain a processing result; wherein the processing result includes the category to which the target object in the target image belongs.

12. An image processing device, characterized in that: include: an acquisition module, configured to acquire a first image selected from an image set, and a feature set stored in a storage buffer, wherein the feature set includes features of a second image selected from the image set; The second image includes a category label; an enhancement module, configured to perform a first enhancement process on the first image to obtain a corresponding first enhanced image, and perform a second enhancement process on the first image to obtain a corresponding second enhanced image, wherein the degree of enhancement of the first enhancement process is weaker than the degree of enhancement of the second enhancement process; an extraction module, configured to input a first enhanced image and a second enhanced image corresponding to the first image into a trained neural network for feature extraction, respectively, to obtain a first feature of the first enhanced image and a second feature of the second enhanced image, wherein the neural network includes an image classification network trained based on the second image selected from the image set; An associating module, configured to associate the first feature and the second feature with features in the feature set, respectively, to obtain a first associated feature corresponding to the first feature and a second associated feature corresponding to the second feature; an adjustment module, configured to adjust the neural network based on a similarity or a difference between the first correlation feature and the second correlation feature; The association module is specifically used for: Performing a dot product operation on the first feature and each feature in the feature set to obtain a first correlation feature; the first correlation feature is used to represent the degree of correlation between the category labels of the first enhanced image corresponding to the first feature and the second image corresponding to the feature in the feature set; Performing a dot product operation on the second feature and each feature in the feature set to obtain the second associated feature; the second associated feature is used to characterize the degree of association between the second enhanced image corresponding to the second feature and the category label of the second image corresponding to the feature in the feature set.

13. A classification device, characterized in that: include: An acquisition module is used to acquire a target image captured in a target scene; A classification module, configured to process the target image using a neural network trained by the image processing method according to any one of claims 1 to 10 to obtain a processing result; wherein the processing result includes the category to which the target object in the target image belongs.

14. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the image processing method according to any one of claims 1 to 10 or the steps of the classification method according to claim 11 are performed.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the image processing method according to any one of claims 1 to 10 or the steps of the classification method according to claim 11.

Images