Method and apparatus for image recognition

Abstract

A method and apparatus for image recognition are disclosed. An image recognition method includes receiving an input image of a first quality; extracting input features of a second quality of the input image from the input image by inputting the input image into an encoding model in an image recognition model; and generating a recognition result of the input image based on the input features.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0151482, filed on November 13, 2020, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to methods and devices for image recognition. Background Technology

[0003] The automation of recognition processing techniques can be achieved through neural network models, which are implemented as specialized computational structures, such as processors. After extensive training, these models provide an intuitive mapping between input and output patterns. The ability to generate such mappings is the learning capability of the neural network. Furthermore, specialized neural networks, trained through specific methods, can possess generalization capabilities, such as providing relatively accurate outputs for untrained input patterns. Summary of the Invention

[0004] This summary is provided to introduce, in a simplified form, the selection of concepts further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, an image recognition method includes: receiving an input image of first quality; extracting input features of second quality from the input image by inputting the input image into an encoding model in an image recognition model; and generating a recognition result of the input image based on the input features.

[0006] The encoding model can be configured to extract second-quality second-input features from the second-input image in response to input of a second-quality second-input image.

[0007] The encoding model can be a neural network-based model that is pre-trained to extract second-quality input features from the input image regardless of whether the input image has first-quality or second-quality features.

[0008] The method may include generating a second-quality output image by inputting input features into a decoding model in an image recognition model.

[0009] The decoding model can be a neural network-based model that is pre-trained to generate a second-quality output image based on second-quality input features.

[0010] The method may include both outputting recognition results and outputting images.

[0011] The method may include: determining decoding parameters by training a temporary model with a primary training image set of second quality; fixing the parameters of the decoding model as decoding parameters, wherein the image recognition model includes an encoding model and a decoding model; and training an image recognition model including fixed parameters with a secondary training image set of first quality and second quality.

[0012] A non-transitory computer-readable storage medium that stores instructions, when executed by a processor, configures the processor to perform the method.

[0013] In another general aspect, an image recognition method includes: determining decoding parameters by training a temporary model with a primary training image set of second quality; fixing the parameters of the decoding model in the image recognition model as decoding parameters, the image recognition model including an encoding model and a decoding model; and training the image recognition model including the fixed parameters with a secondary training image set of first quality and second quality.

[0014] The temporary model may include a temporary encoding model and a temporary decoding model, and the step of determining the decoding parameters may include: determining a recognition loss based on a first training feature extracted by the temporary encoding model from a first training image in the primary training image set; determining a reconstruction loss based on a first output image reconstructed by the temporary decoding model based on the first training feature; and updating the temporary model based on the recognition loss and the reconstruction loss.

[0015] The temporary model may include a temporary reversible coding model, and the step of determining the decoding parameters may include: determining a recognition loss based on a first training feature extracted by the temporary reversible coding model from a first training image in a primary training image set; and determining the decoding parameters by updating the temporary model based on the recognition loss.

[0016] The steps to fix the parameters of the decoding model may include determining the decoding model by inverting the temporary reversible coding model.

[0017] The steps of training an image recognition model may include: determining a recognition loss based on second training features extracted by an encoding model from a second training image set of secondary training images; determining a reconstruction loss based on a second output image reconstructed by a decoding model based on the second training features; and updating the image recognition model based on the recognition loss and the reconstruction loss.

[0018] In another general aspect, an electronic device includes a processor configured to: extract second-quality input features of an input image from an input image by inputting a first-quality input image into an encoding model in an image recognition model, and generate a recognition result of the input image based on the input features.

[0019] The encoding model can be configured to extract second-quality second-input features from the second-input image in response to input of a second-quality second-input image.

[0020] The encoding model can be a neural network-based model that is pre-trained to extract second-quality input features from the input image regardless of whether the input image has first-quality or second-quality features.

[0021] The processor can be configured to generate a second-quality output image by feeding input features into a decoding model within an image recognition model.

[0022] The decoding model can be a neural network-based model that is pre-trained to generate a second-quality output image based on second-quality input features.

[0023] The display module can be configured to output both recognition results and images.

[0024] The recognition result can be the category information of objects in the input image, and the output image can have improved quality compared to the input image.

[0025] The device may include: a camera configured to generate an input image; and a display module configured to output a recognition result.

[0026] In another general aspect, an image recognition method includes: training a first model comprising a first image recognition encoder and an image reconstruction decoder based on a training set comprising high-quality images but excluding low-quality images; and training a second model comprising a second image recognition encoder and a decoder based on a training set comprising both high-quality and low-quality images, wherein, during the training of the second model, the parameters of the decoder are fixed to the parameters of the decoder generated by the training of the first model.

[0027] The decoder of the trained second model can be configured to reconstruct a high-quality output image corresponding to the low-quality input image based on the feature vectors generated by the second encoder from the low-quality input image.

[0028] High-quality images can differ from low-quality images in any one or more of the following aspects: resolution, pixel size, amount of brightness, amount of noise, and amount of blur.

[0029] Other features and aspects will become clear from the following detailed description, the accompanying drawings, and the claims. Attached Figure Description

[0030] Figure 1 An example of the recognition operation of an image recognition device is shown.

[0031] Figure 2 An example configuration of an image recognition model is shown.

[0032] Figure 3 An example configuration of an image recognition model is shown.

[0033] Figure 4 An example of training operation of the training device is shown.

[0034] Figure 5 An example of training processing for an image recognition model is shown.

[0035] Figure 6 An example of the second stage in the training process is shown.

[0036] Figure 7 An example of training processing for an image recognition model is shown.

[0037] Figure 8 An example of the second stage in the training process is shown.

[0038] Figure 9 An example of an image recognition method is shown.

[0039] Figure 10 An example of a training method is shown.

[0040] Figure 11 An example configuration of an image recognition device is shown.

[0041] Figure 12 An example of the configuration of an electronic device is shown.

[0042] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0043] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and brevity, descriptions of features known in the art upon understanding this disclosure may be omitted.

[0044] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts are not limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0045] Throughout the specification, when a component is described as "connected to" or "combined to" another component, that component may be directly "connected to" or "combined to" that other component, or there may be one or more other components in between. Conversely, when an element is described as "directly connected to" or "directly combined to" another element, there may be no other elements in between.

[0046] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more. The terms "comprising," "including," and "having" specify the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof. The use of the term "may" herein with respect to examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the presence of at least one example or embodiment that includes or implements such a feature, and that all examples are not limited thereto.

[0047] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and based on the understanding of the disclosure of this application. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and shall not be interpreted in an idealized or overly formalized sense.

[0048] In the following description, examples will be described in detail with reference to the accompanying drawings. When describing examples with reference to the accompanying drawings, the same reference numerals denote the same components, and repeated descriptions related to the same components will be omitted.

[0049] Figure 1 This illustrates an example of the recognition operation of an image recognition device. (See reference...) Figure 1The image recognition device 100 can receive an input image 101. The input image 101 may be generated by a camera and sent to the image recognition device 100 for image recognition. The image recognition device 100 can perform recognition processing on the input image 101 and output a recognition result 102. The input image 101 may include objects, and the recognition processing may include processing for recognizing objects in the input image 101 (e.g., object recognition).

[0050] As a non-limiting example, object recognition may include object detection, object tracking, object classification, object identification, and / or object authentication. Object detection detects objects in the input image 101, object tracking tracks objects, object classification specifies the category of the object, object identification specifies the identity of the object, and object authentication confirms the permissions of the object. The recognition result 102 may include the result of such recognition processing. For example, as a non-limiting example, the recognition result 102 may include the location information of the object obtained through object detection or object tracking (e.g., coordinates of the object bounding box), the category information obtained through object classification (e.g., person, animal, dog, cat, etc.), the identity information obtained through object identification (e.g., user, celebrity, etc.), and / or the authentication result obtained through object authentication (e.g., authentication value for unlocking a smartphone).

[0051] Image recognition device 100 can perform recognition processing on input image 101 using image recognition model 110. Image recognition model 110 can be a deep learning-based neural network model. The neural network can perform operations (e.g., object recognition operations) based on the learning objective by mapping input and output data in a non-linear relationship using deep learning. Deep learning is a machine learning technique used to solve problems (such as image recognition or speech recognition) on large datasets. Deep learning can be interpreted as an optimization problem-solving process that finds the point where energy is minimized while training a neural network using prepared training data.

[0052] Through supervised or unsupervised learning in deep learning, the structure of a neural network, or the weights corresponding to the model, can be obtained, and input and output data can be mapped to each other through the weights. If the width and depth of the neural network are large enough, it can have the capability to achieve a predetermined function. When trained on a sufficiently large amount of training data through appropriate training processes, the neural network can achieve optimized performance.

[0053] In the following text, a neural network or its parameters (e.g., weights) may be referred to as being “pre-trained.” Here, “pre-trained” means before the neural network is “started.” Starting a neural network means that the neural network is ready to perform inference. For example, starting a neural network may include loading the neural network into memory, or, after the neural network is loaded into memory, input data for inference being fed into the neural network.

[0054] A neural network may include multiple layers. In this example, the neural network may also be referred to as a deep neural network (DNN). Multiple layers may include an input layer, one or more hidden layers, and an output layer. A neural network can be or includes various types of networks (such as fully connected networks, convolutional neural networks (CNNs), and / or recurrent neural networks (RNNs)).

[0055] The performance of a neural network can depend on its training characteristics. A typical neural network trained with high-quality training images can identify high-quality images well, but exhibits relatively poor performance when identifying low-quality images. High and low quality can be determined based on reference values ​​for various factors, such as resolution, size (e.g., pixel size), brightness, noise, and / or blur. For example, high-quality and low-quality images can be determined based on a predetermined resolution. In addition to resolution, noise or blur can also be used as a criterion. More different levels can be defined to determine quality. For example, in addition to high and low quality, intermediate quality can be defined between high and low quality.

[0056] While typical neural networks can be trained using both high-quality and low-quality training images, they may perform relatively poorly in recognizing high-quality images compared to a typical neural network trained solely on high-quality training images. For a typical neural network trained on high-quality training images to recognize low-quality images, a typical approach can be used: improve the quality of the low-quality image and then feed the improved image into the neural network. However, using such a typical approach may require additional processing for quality improvement, and there is no guarantee that such quality improvement will necessarily contribute to image recognition. This is because the quality of the quality improvement may not be consistent with the quality of recognition within the neural network. Furthermore, the typical approach of using separate high-quality image-specific neural networks trained on high-quality training images and low-quality image-specific neural networks trained on low-quality training images may be inefficient.

[0057] In contrast, a two-stage training method using one or more embodiments of the decoder can be used to obtain an image recognition model 110 robust to images of various qualities. Therefore, as a single model, the image recognition model 110 of one or more embodiments can maintain high recognition performance for input images 101 of various qualities (e.g., both high-quality and low-quality input images 101). More specifically, the image recognition model 110 can be trained in two stages using an encoder that extracts features and a decoder that reconstructs the image from the features.

[0058] In Stage 1 of the two-stage training method in one or more embodiments, a primary recognition model (also referred to as a temporary model) including a temporary encoder and a temporary decoder is trained using high-quality training images. In this case, the loss function reflects the recognition loss associated with image recognition via the temporary encoder and the reconstruction loss associated with image reconstruction via the temporary decoder. Thus, the temporary encoder can be trained to improve the performance of recognizing high-quality images, and the temporary decoder can be trained to reconstruct high-quality images from the features of the high-quality images. In Stage 1, the primary recognition model may optionally include a temporary reversible encoder instead of a temporary encoder and a temporary decoder; non-limiting examples of this will be described in more detail later.

[0059] In stage 2 of the two-stage training method of one or more embodiments, a novel secondary recognition model including an encoder and a decoder is trained using both high-quality and low-quality training images. In this example, the decoder parameters can be fixed to the training parameters of a temporary decoder trained in stage 1 (e.g., the decoder parameters can be set to the training parameters of a temporary decoder trained in stage 1 and fixed at that value so that the parameters do not change during training in stage 2). The loss function in stage 2 reflects the recognition loss associated with image recognition via the encoder and the reconstruction loss associated with image reconstruction via the decoder. Since the parameters of the temporary decoder are trained to reconstruct high-quality images from features of high-quality images, the reconstruction loss can increase when features of low-quality training images are input into the decoder. Therefore, the two-stage training method of one or more embodiments can train the encoder to extract features from low-quality images that correspond to high-quality images.

[0060] Image recognition model 110 can be configured to include both an encoder and a decoder of this secondary recognition model, or it can be configured to include an encoder but not a decoder. In the former case, image recognition model 110 can extract features from input image 101 and generate a high-quality output image based on the features. Therefore, image recognition device 100 can output the output image together with recognition result 102, or recognition result 102 can also include the output image. Furthermore, in both cases, image recognition model 110 includes an encoder, and according to the above description of the training process, the encoder can extract high-quality features regardless of whether the input image 101 is of high or low quality. Therefore, regardless of the quality of the input image 101, image recognition model 110 of one or more embodiments can exhibit a consistent level of recognition performance. In the following, non-limiting examples of the configuration of image recognition model 110 will be described in more detail.

[0061] Figure 2 This shows an example of the configuration of an image recognition model. (See reference...) Figure 2 The image recognition model 200 may include an encoding model 210 (e.g., an encoder) and a decoding model 220 (e.g., a decoder). The encoding model 210 and decoding model 220 may be pre-trained neural network-based models. The image recognition model 200 may be configured to include the above-mentioned references. Figure 1 The image recognition model is described in terms of both the encoder and decoder of the secondary recognition model during the training process.

[0062] Encoding model 210 can extract input features from input image 201, and image recognition model 200 can generate recognition result 202 of input image 201 based on the input features. For example, image recognition device and / or image recognition model 200 can generate input feature vector by performing convolution and / or global average pooling (GAP) on input image 201, and generate recognition result 202 by performing softmax operation on input feature vector. In this case, recognition result 202 may include category information obtained through object classification. However, recognition result 202 may include various other information, and image recognition model 200 may process input features in various other ways to generate recognition result 202.

[0063] By using a two-stage training method for the decoder, the encoding model 210 can have network parameters robust to images of various qualities. Therefore, the encoding model 210 of one or more embodiments can extract input features of a consistent quality level, regardless of the quality of the input image 201. For example, the encoding model 210 can consistently extract second-quality input features from the input image 201, regardless of whether the input image 201 has a first quality (e.g., low quality) or a second quality (e.g., high quality). Therefore, the image recognition model 200 of one or more embodiments can exhibit a consistent level of recognition performance regardless of the quality of the input image 201.

[0064] Decoding model 220 can generate output image 203 based on input features. For example, decoding model 220 can generate second-quality output image 203 based on second-quality (e.g., high-quality) input features. Image recognition device and / or image recognition model 200 can directly input input features into decoding model 220, or apply necessary processing (e.g., depth-to-space transformation) to the input features and input the processed input features into decoding model 220.

[0065] The image recognition device can output both a recognition result 202 and an output image 203. For example, the image recognition device can output both the recognition result 202 and the output image 203 through a display module. In this case, the recognition result 202 can be the category information of the object in the input image 201, and the output image 203 can have improved quality compared to the input image 201. As an example, the input image 201 can be a dark, very noisy, low-quality image obtained by photographing the object at low brightness; the recognition result 202 can include the category information of the object, and the output image 203 can be a high-quality image with improved brightness and noise. As another example, the input image 201 can be a low-quality image obtained by photographing a person at a small size; the recognition result 202 can include the location information of the person's detection / tracking box and / or the person's identity information, and the output image 203 can be a high-quality image with improved resolution by magnifying the person.

[0066] Figure 3 This shows an example of the configuration of an image recognition model. (See reference...) Figure 3 The image recognition model 300 may include an encoding model 310 (e.g., an encoder). The encoding model 310 may be a pre-trained neural network-based model. When compared with... Figure 2 Compared to image recognition model 200, image recognition model 300 may not include decoding model 220. In other words, image recognition model 300 may be an encoder configured to include the secondary recognition model in the training process described above, but not a decoder.

[0067] Except that image recognition model 300 may neither include decoding model 220 nor output image 203, the description of image recognition model 200 can be applied to image recognition model 300. For example, encoding model 310 can extract input features from input image 301, and image recognition model 300 can generate recognition result 302 of input image 301 based on the input features. Since encoding model 310 can correspond to encoding model 210, similar to encoding model 210, encoding model 310 can have network parameters robust to images of various qualities by using a two-stage training method for the decoder.

[0068] Figure 4 An example of training operation of the training device is shown. (See reference...) Figure 4 The training device 400 can use training input 401 and training output 402 to train the image recognition model 410. For example, the training device 400 can input training input 401 into the image recognition model 410 and update the image recognition model 410 to reduce the difference between the corresponding training output 402 and the ground truth (GT). More specifically, the training device can update the network parameters of the image recognition model 410 to reduce the loss according to the loss function.

[0069] The training device 400 can train the image recognition model 410 using a two-stage training method with a decoder. More specifically, in stage 1, the training device 400 can train a temporary model with a primary training image set including training images of second quality (e.g., high quality) and fix the parameters of the decoding model of the image recognition model 410 to the decoding parameters of the temporary model, and then in stage 2, train the image recognition model 410 with a secondary training image set including training images of first quality (e.g., low quality) and second quality (high quality). A non-limiting example of the training process of the image recognition model 410 will be described in more detail below.

[0070] Figure 5 An example of training processing for an image recognition model is shown. (See reference...) Figure 5In Phase 1, a temporary model 510 may be trained based on training images 501. Training images 501 may be extracted from a primary training image set including training images of a second quality (e.g., high quality). The temporary model 510 may include a temporary encoding model 511 (e.g., an encoder) and a temporary decoding model 512 (e.g., a decoder) based on a neural network model. The temporary encoding model 511 may extract training features from the training images 501 and obtain a recognition result 502 based on the training features. Furthermore, the temporary decoding model 512 may reconstruct an output image 503 by decoding the training features. The training device may update (e.g., train) the temporary model 510 based on a loss calculated based on the recognition result 502 and the output image 503. The temporary model 510 may be iteratively updated (e.g., trained) using other training images from the primary training image set. In Phase 1, the temporary model 510 may be iteratively updated until the loss is less than or equal to a predetermined threshold, and Phase 1 may terminate when the loss is less than or equal to the predetermined threshold. When Phase 1 is terminated after multiple iterations, the training network parameters of the temporary model 510 can be determined.

[0071] For example, in stage 1, the temporary model 510 can be trained to satisfy the following equation 1.

[0072] Equation 1:

[0073] f H -1 (f H (x))=g H (f H (x))

[0074] x = g H (f H (x))

[0075] In Equation 1, x represents the training image 501, f H This represents the temporary coding model 511, g H This represents the temporary decoding model 512. Therefore, when training according to Equation 1 is complete, the same output image 503 as the training image 501 can be obtained through the temporary model 510 in response to the input of the training image 501. When the training image 501 has a second quality (e.g., high quality), through stage 1, the temporary encoding model 511 can be trained to extract second-quality features from the second-quality training image 501, and the temporary decoding model 512 can be trained to reconstruct the second-quality output image 503 based on the second-quality features.

[0076] In phase 2, image recognition model 520 may be trained based on training images 505 and 506. Image recognition model 520 may include a neural network-based encoding model 521 and a decoding model 522. The parameters of temporary decoding model 512 and / or temporary decoding model 512 may be passed to decoding model 522, and the parameters of decoding model 522 may be fixed to the parameters of temporary decoding model 512, such that the parameters of decoding model 522 are the same as those of temporary decoding model 512 trained in phase 1. Training images 505 and 506 may be extracted from a secondary training image set including training images of a first quality (e.g., low quality) and a second quality (e.g., high quality). For example, training image 505 may have the first quality, and training image 506 may have the second quality.

[0077] Encoding model 521 can extract training features from training images 505 and 506, and obtain a recognition result 507 based on the training features. Furthermore, decoding model 522 can reconstruct an output image 508 by decoding the training features. Training images 505 and 506 can be input sequentially, rather than simultaneously, to train image recognition model 520. As a non-limiting example of sequential input, encoding model 521 can extract one or more first training features from training image 505 and obtain a first recognition result based on one or more first training features; encoding model 521 can extract one or more second training features from training image 506 and obtain a second recognition result based on one or more second training features. Furthermore, as a non-limiting example of sequential input, decoding model 522 can reconstruct a first output image by decoding one or more first training features; decoding model 522 can reconstruct a second output image by decoding one or more second training features. In this example, training images 505 and 506 can be images obtained based on the same object or different objects; the first recognition result and the second recognition result can be the same or different; and the first output image and the second output image can be the same or different. The training device can update the image recognition model 520 based on the loss calculated based on the recognition result 507 and the output image 508. Since the parameters of the decoding model 522 are fixed, the parameters of the encoding model 521 can be updated in stage 2, and the parameters of the decoding model 522 may not be updated. The image recognition model 520 can be iteratively updated using other training images in the secondary training image set, and the network parameters of the image recognition model 520 can be determined when stage 2 is terminated. In stage 2, the image recognition model 520 can be iteratively updated until the loss is less than or equal to a predetermined threshold, and stage 2 can be terminated when the loss is less than or equal to the predetermined threshold.

[0078] For example, in stage 2, the image recognition model 520 can be trained to satisfy the following equation 2.

[0079] Equation 2:

[0080] f H (x)=f s (x)

[0081]

[0082] In equation 2, Let f represent training image 505, x represent training image 506, and f H This represents the temporary coding model 511, f s Let 521 be the encoding model. Therefore, when training according to Equation 2 is complete, the extracted features can be the same as those obtained by inputting the second-quality training image 506 into the temporary encoding model 511, regardless of whether the first-quality training image 505 or the second-quality training image 506 is input into the encoding model 521. This indicates that if the first quality is low and the second quality is high, then when the low-quality image is input into the encoding model 521, the features corresponding to the high-quality image are extracted. For example, Equations 3 and 4 can be derived from Equation 2, as shown below.

[0083] Equation 3:

[0084] g H (f H (x))=g H (f s (x))

[0085] x = g H (f s (x))

[0086] Equation 4:

[0087]

[0088]

[0089] Equations 3 and 4 indicate that a training image 506 of the second quality can be obtained as an output image 508, regardless of whether a training image 505 of the first quality or a training image 506 of the second quality is input into the image recognition model 520. In other words, if the first quality is low quality and the second quality is high quality, then when a low-quality image is input into the image recognition model 520, a high-quality image can be obtained. Therefore, through two-stage training, the encoding model 521 can extract features corresponding to the high-quality image from the low-quality image, and the decoding model 522 can reconstruct the high-quality image based on the features corresponding to the high-quality image.

[0090] To achieve the relationship in Equation 1 of Phase 1, the loss function shown in Equation 5 below can be used.

[0091] Equation 5:

[0092]

[0093] In equation 5, L stage1 Let θ represent the loss function for stage 1. H φ represents the parameters of the temporary coding model 511. H This represents the parameters of the temporary decoding model 512. This represents a dataset that includes training images of second quality (e.g., high quality). Flexibility maximization can be used in the processing of obtaining recognition results 502 based on features extracted by the temporary coding model 511. In this case, ω represents the flexibility maximization weight. L softmax L represents the recognition loss. quality Let λ represent the reconstruction loss, and λ represent the adjusted weights. For example, the identification loss and reconstruction loss can be represented by Equations 6 and 7 below.

[0094] Equation 6:

[0095]

[0096] Equation 6 represents the recognition loss calculated based on maximum flexibility and cross-entropy. In Equation 6, This indicates temporary coding model 511. This represents Global Average Pooling (GAP). GAP is an example, and other operations used to transform the output features of the temporary coding model 511 into the most flexible input can be used alternatively. x represents the input image, y represents the class of x, and C represents the total number of classes.

[0097] Equation 7:

[0098]

[0099] In Equation 7, d(a,b) represents the distance or divergence between a and b, and, for example, the L1 norm or L2 norm can be used to calculate this distance. This indicates temporary decoding model 512.

[0100] For example, to achieve the relationship in Equation 2 in Stage 2, a loss function as shown in Equation 8 below can be used. The difference between Stage 2 and Stage 1 is that the parameters of the decoding model 522 can be transferred from the temporary decoding model 512 and fixed in the decoding model 522, and different training image sets are used. Equation 8 reflects this.

[0101] Equation 8:

[0102]

[0103] In equation 8, L stage2 Let θ represent the loss function for stage 2. s This represents the parameters of the coding model 521, where ω represents the maximum flexible weight. This represents a dataset that includes training images of second quality (e.g., high quality). This represents a dataset that includes training images of the first quality (e.g., low quality). L softmax L represents the recognition loss. quality Let λ represent the reconstruction loss, and λ represent the adjusted weights. For example, the identification loss and reconstruction loss can be represented by Equations 9 and 10 below.

[0104] Equation 9:

[0105]

[0106] In equation 9, This indicates the encoding model 521. GAP is indicated by Equation 9, which specifies only datasets of the first quality (e.g., low quality). Used for training. However, optionally, a second-quality (e.g., high-quality) dataset. And datasets of first quality (e.g., low quality) Both can be used.

[0107] Equation 10:

[0108]

[0109] In equation 10, This represents the temporary decoding model 512. Therefore, the decoding model 522 can correspond to the state in which the parameters of the temporary decoding model 512 are transmitted and fixed (e.g., the parameters of the decoding model 522 can be fixed to the training parameters of the temporary decoding model 512 trained in stage 1).

[0110] Figure 6 This illustrates the second phase of the training process (e.g., Figure 5 An example of stage 2 in the training process. See [reference]. Figure 6 The encoding model 610 can be trained using training images 601 and 602. For example, training image 601 may have a first quality (e.g., low quality), and training image 602 may have a second quality (e.g., high quality). Training images 601 and 602 may be input sequentially, rather than simultaneously, to train the encoding model 610.

[0111] Encoding model 610 generates feature map 603 by extracting features from training images 601 and 602. Feature map 603 can be converted into feature vectors through operations such as GAP, and recognition result 605 can be generated through a flexibility maximization operation on the feature vectors. Decoding model 620 can reconstruct output image 604 based on feature map 603. In this case, processing (such as depth-to-space transformation) can be applied to feature map 603 used as input to decoding model 620. For example, the size of feature map 603 can be 7×7×512, and feature map 603 can be converted into 112×112×2 data through depth-to-space transformation and input into decoding model 620. The size of feature vector can be 1×1×512. These values ​​are only examples, and various other processing can be applied to feature map 603.

[0112] Figure 7 An example of training processing for an image recognition model is shown. (See reference...) Figure 7 ,and Figure 5 Unlike the temporary model 510, the temporary model 710 may include a temporary reversible coding model 711. The temporary reversible coding model 711 may be a neural network-based model and may be simply referred to as the reversible coding model 711, and can be converted into a reversible decoding model 722 by inverting. For example, the relationship between the reversible coding model 711 and the reversible decoding model 722 can be represented by the following equation 11.

[0113] Equation 11:

[0114] g H =f H -1

[0115] In equation 11, f H Represents the reversible coding model 711, g H Reversible decoding model 722 is represented. Reversible encoding model 711 can be an encoder for extracting features from an image, and reversible decoding model 722 can be a decoder for reconstructing an image from the features. If features are extracted from the original image via reversible encoding model 711 and the image is reconstructed from the features via reversible decoding model 722, the original image and the reconstructed image are matchable. This can be compensation for the difference between the original image and the reconstructed image, which can occur when features are extracted from the original image and the image is reconstructed from the features via temporary encoding model 511 and temporary decoding model 512 of temporary model 510. Furthermore, unlike temporary model 510, which is trained with both recognition loss and reconstruction loss, temporary model 710 can be trained with recognition loss instead of reconstruction loss.

[0116] More specifically, in Phase 1, a temporary model 710 may be trained based on training images 701. Training images 701 may be extracted from a primary training image set that includes training images of a second quality (e.g., high quality). A temporary reversible encoding model 711 may extract training features from training images 701 and obtain a recognition result 702 based on the training features. The training device may update the temporary model 710 based on the loss according to the recognition result 702. The temporary model 710 may be iteratively updated using other training images from the primary training image set. When Phase 1 is terminated, the network parameters of the temporary model 710 may be determined.

[0117] When the training image 701 has a second quality (e.g., high quality), through stage 1, the temporary reversible coding model 711 can be trained to extract second-quality features from the second-quality training image 701. When the temporary reversible coding model 711 is inverted into a reversible decoding model 722, the reversible decoding model 722 can reconstruct the second-quality second image 708 based on the second-quality features.

[0118] For example, in stage 1, a loss function as shown in Equation 12 below can be used.

[0119] Equation 12:

[0120]

[0121] In equation 12, L stage1 Let θ represent the loss function for stage 1. H The parameters of the temporary reversible coding model 711 are represented by w, where w represents the maximum flexible weight. This represents a dataset that includes training images of second-highest quality (e.g., high quality). L softmax This represents the recognition loss. Since the temporary model 710 does not include a decoding model, the reconstruction loss may not be used. Equation 6 can be used to calculate the recognition loss in Equation 12.

[0122] In stage 2, the image recognition model 720 may be trained based on training images 705 and 706. The parameters of the temporary reversible encoding model 711 and / or the temporary reversible encoding model 711 may be inverted and passed to the reversible decoding model 722, whose parameters may be fixed. Training images 705 and 706 may be extracted from a secondary training image set including training images of a first quality (e.g., low quality) and a second quality (e.g., high quality). For example, training image 705 may have the first quality, and training image 706 may have the second quality.

[0123] Encoding model 721 extracts training features from training images 705 and 706, and obtains recognition result 707 based on the training features. Furthermore, reversible decoding model 722 reconstructs output image 708 by decoding the training features. Reversible decoding model 722 can be a decoder for reconstructing second-quality output image 708 based on second-quality features. The training device can update image recognition model 720 based on a loss calculated based on recognition result 707 and output image 708. Since the parameters of decoding model 722 are fixed, the parameters of encoding model 721 can be updated in stage 2, while the parameters of decoding model 722 may not be updated. Image recognition model 720 can be iteratively updated using other training images in the secondary training image set, and the network parameters of image recognition model 720 can be determined when stage 2 is terminated. In stage 2, the loss functions according to Equations 8 to 10 can be used.

[0124] Figure 8 This illustrates the second phase of the training process (e.g., Figure 7 An example of stage 2 in the training process. See [reference]. Figure 8 The encoding model 810 can be trained using training images 801 and 802. For example, training image 801 may have a first quality (e.g., low quality), and training image 802 may have a second quality (e.g., high quality). Training images 801 and 802 may be input sequentially, rather than simultaneously, to train the encoding model 810.

[0125] Encoding model 810 generates feature map 803 by extracting features from training images 801 and 802. Feature map 803 can be converted into feature vectors through operations such as GAP, and recognition result 805 can be generated through a flexibility maximization operation on the feature vectors. Reversible decoding model 820 can reconstruct output image 804 based on feature map 803. In this case, processing (such as convolution) can be applied to feature map 803 used as input to reversible decoding model 820. For example, the size of feature map 803 can be 7×7×2048, and feature map 803 can be converted into 7×7×3072 data through convolution and input to reversible decoding model 820. The size of feature vector can be 1×1×2048. These values ​​are only examples, and various other processing can be applied to feature map 803.

[0126] Figure 9 An example of an image recognition method is shown. (See reference...) Figure 9In operation 910, the image recognition device can receive an input image of first quality. In operation 920, the image recognition device can extract second-quality input features from the input image by inputting the input image into an encoding model in the image recognition model. In operation 930, the image recognition device can generate a recognition result of the input image based on the input features. The encoding model can extract second-quality second input features from the second input image in response to the input of a second-quality second input image. The encoding model can be a neural network-based model pre-trained to output second-quality input features from the input image regardless of whether the input image has first quality or second quality. The image recognition model may also include a decoding model, and the image recognition device can generate a second-quality output image by inputting the input features into the decoding model. The decoding model can be a neural network-based model pre-trained to generate a second-quality output image based on the second-quality input features. The image recognition device can output both the recognition result and the output image. Additionally, refer to... Figures 1 to 8 and Figures 10 to 12 The provided description can be applied to image recognition methods.

[0127] Figure 10 An example of a training method is shown. (See reference...) Figure 10 In operation 1010, the training device can determine decoding parameters by training a temporary model using a primary training image set of second quality. The temporary model may include a temporary encoding model and a temporary decoding model. In this case, the training device can determine a recognition loss based on a first training feature extracted by the temporary encoding model from a first training image in the primary training image set, determine a reconstruction loss based on a first output image reconstructed by the temporary decoding model based on the first training feature, and update the temporary model based on the recognition loss and the reconstruction loss. In another example, the temporary model may include a temporary reversible encoding model. In this case, the training device can determine a recognition loss based on a first training feature extracted by the temporary reversible encoding model from a first training image in the primary training image set, and determine the decoding parameters by updating the temporary model based on the recognition loss.

[0128] In operation 1020, the training device can fix the parameters of the decoding model in the image recognition model to the decoding parameters. The image recognition model includes an encoding model and a decoding model. In operation 1030, the training device can train the image recognition model using secondary training image sets of first and second quality. The training device can determine the recognition loss based on the second training features extracted by the encoding model from the second training image set, determine the reconstruction loss based on the second output image reconstructed by the decoding model based on the second training features, and update the image recognition model based on the recognition loss and the reconstruction loss. Additionally, refer to... Figures 1 to 9 , Figure 11 and Figure 12The provided description can be applied to training methods.

[0129] Figure 11 An example configuration of an image recognition device is shown. (See reference...) Figure 11 The image recognition device 1100 may include a processor 1110 (e.g., one or more processors) and a memory 1120 (e.g., one or more memories). The memory 1120 may be connected to the processor 1110 and may store instructions executable by the processor 1110, data to be computed by the processor 1110, and / or data processed by the processor 1110. The memory 1120 may include a non-transitory computer-readable medium (e.g., high-speed random access memory) and / or a non-volatile computer-readable storage medium (e.g., one or more magnetic storage devices, flash memory devices, or other non-volatile solid-state memory devices).

[0130] Processor 1110 can execute instructions to perform reference Figures 1 to 10 and Figure 12 The described operation. For example, processor 1110 may receive a first-quality input image, extract second-quality input features from the input image by inputting the input image into an encoding model in an image recognition model, and generate a recognition result of the input image based on the input features. Additionally, refer to... Figures 1 to 10 and Figure 12 The provided description can be applied to image recognition device 1100.

[0131] Figure 12 An example configuration of an electronic device is shown. (See reference) Figure 12 Electronic device 1200 may include a processor 1210 (e.g., one or more processors), a memory 1220 (e.g., one or more memories), a camera 1230, a storage device 1240, an input device 1250, an output device 1260, and a network interface 1270, all of which can communicate with each other via a communication bus 1280. For example, electronic device 1200 may be implemented as at least a part of a mobile device (such as a mobile phone, smartphone, PDA, netbook, tablet computer, or laptop computer), a wearable device (such as a smartwatch, smart bracelet, or smart glasses), a computing device (such as a desktop computer or server), a home appliance (such as a television, smart TV, or refrigerator), a security device (such as a door lock or closed-circuit television (CCTV)), or a vehicle (such as an autonomous vehicle or intelligent vehicle).

[0132] Electronic device 1200 can acquire an input image and perform recognition processing on the acquired input image. Furthermore, electronic device 1200 can perform operations associated with the recognition result. Electronic device 1200 may structurally and / or functionally include... Figure 1 Image recognition device 100 and / or Figure 11 Image recognition device 1100.

[0133] Processor 1210 can execute instructions or functions that will be executed in electronic device 1200. For example, processor 1210 can process instructions stored in memory 1220 or storage device 1240. Processor 1210 can execute instructions via... Figures 1 to 11 One or more operations described.

[0134] The memory 1220 may store data used for image recognition. The memory 1220 may include a computer-readable storage medium or a computer-readable storage device. The memory 1220 may store instructions to be executed by the processor 1210, and may store relevant information when the electronic device 1200 executes software and / or applications.

[0135] Camera 1230 can capture photos and / or videos. For example, camera 1230 can capture facial images including the user's face. Camera 1230 can be a three-dimensional (3D) camera that includes depth information of objects, or a CCTV camera for security purposes.

[0136] Storage device 1240 may include a computer-readable storage medium or a computer-readable storage device. Compared to memory 1220, storage device 1240 can store a greater amount of information for a longer period of time. For example, storage device 1240 may include a magnetic hard disk, optical disk, flash memory, floppy disk, or other non-volatile memory known in the art.

[0137] Input device 1250 can receive input from a user via a keyboard and mouse in both traditional and new input methods, such as touch input, voice input, and image input. For example, input device 1250 may include a keyboard, mouse, touchscreen, microphone, or any other device that detects input from the user and transmits the detected input to electronic device 1200.

[0138] Output device 1260 can provide the output of electronic device 1200 to a user through a visual channel, an auditory channel, or a tactile channel. Output device 1260 may include, for example, a display module, a touch screen, a speaker, a vibration generator, or any other device that provides output to the user. The display module can simultaneously output the recognition result obtained through image recognition and the output image. For example, the recognition result may be category information of objects in the input image (e.g., human, animal, dog, cat, etc.), and the output image may have improved quality (e.g., improved resolution, brightness, etc.) compared to the input image.

[0139] The network interface 1270 can communicate with external devices via wired or wireless networks.

[0140] Regarding Figures 1 to 12The described image recognition devices, processors, memories, electronic devices, cameras, storage devices, input devices, output devices, network interfaces, communication buses, image recognition devices 100 and 1100, processor 1110, memory 1120, electronic devices 1200 and 1210, memory 1220, camera 1230, storage device 1240, input device 1250, output device 1260, network interface 1270, communication bus 1280, and other devices, equipment, units, modules, and components are implemented or represent hardware components by hardware components. Examples of hardware components that can be used to perform the operations described in this application suitably include: controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer may be implemented using one or more processing elements, such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by the processor or computer may execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described herein. The hardware components may also access, manipulate, process, create, and store data in response to the execution of instructions or software. For simplicity, the singular terms “processor” or “computer” may be used in the description of the examples described herein, but in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components may be implemented using a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or processors and controllers, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or processors and controllers, may implement a single hardware component or two or more hardware components.The hardware components can have any one or any of different processing configurations, examples of which include: a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0141] Figures 1 to 12 The methods for performing the operations described in this application, as shown, are executed by computing hardware (e.g., by one or more processors or a computer), which is implemented as described above to execute instructions or software to perform the operations performed by the methods described in this application. For example, a single operation or two or more operations may be executed by a single processor, or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation or two or more operations.

[0142] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers, such as machines or special-purpose computers, to perform operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written using any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification, which disclose algorithms for performing operations performed by the hardware components and methods described above.

[0143] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, may be recorded, stored, or fixed on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-RLTH, BD-RE Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide the instructions or software and any associated data, data files, and data structures to one or more processors or computers, such that one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0144] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is to be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

Claims

1. An image recognition method, comprising: Receive an input image of the highest quality. By feeding the input image into the encoding model in the image recognition model, second-quality input features of the input image are extracted from the input image; and Generate recognition results for the input image based on input features. The image recognition method further includes: determining decoding parameters by training a temporary model using a primary training image set of second quality; fixing the parameters of the decoding model as decoding parameters, wherein the image recognition model includes an encoding model and a decoding model; and training an image recognition model including the fixed parameters of the decoding model using secondary training image sets of first and second quality. The encoding model is trained to extract second-quality input features from the input image regardless of whether the input image has first-quality or second-quality features.

2. The image recognition method according to claim 1 further includes: By inputting a second-quality second input image into the encoding model, second-quality second input features are extracted from the second input image.

3. The image recognition method according to claim 1, wherein, The encoding model is a neural network-based model.

4. The image recognition method according to claim 1 further includes: A second-quality output image is generated by feeding the input features into the decoding model of the image recognition model.

5. The image recognition method according to claim 4, wherein, The decoding model is a neural network-based model that is pre-trained to generate a second-quality output image based on second-quality input features.

6. The image recognition method according to claim 4 further includes: Both the recognition result and the output image are output.

7. A method for training an image recognition model, comprising: Decoding parameters are determined by training a temporary model using a primary training image set of second quality; The parameters of the decoding model in the image recognition model are fixed as decoding parameters. The image recognition model includes an encoding model and a decoding model. and The image recognition model, with parameters fixed, is trained using secondary training image sets of first and second quality. The encoding model is trained to extract second-quality input features from the input image regardless of whether the input image has first-quality or second-quality features.

8. The method according to claim 7, wherein, Temporary models include temporary encoding models and temporary decoding models, and The steps to determine the decoding parameters include: The recognition loss is determined based on the first training features extracted from the first training images in the primary training image set by the temporary coding model. The reconstruction loss is determined based on the first output image reconstructed by the temporary decoding model based on the first training features; and The temporary model is updated based on the identification loss and reconstruction loss.

9. The method according to claim 7, wherein, Temporary models include temporary reversible coding models, and The steps to determine the decoding parameters include: The recognition loss is determined based on the first training features extracted from the first training images in the primary training image set by the temporary invertible coding model; and Decoding parameters are determined by updating a temporary model based on recognition loss.

10. The method according to claim 9, wherein, The steps to fix the parameters of the decoding model include determining the decoding model by inverting the temporary reversible coding model.

11. The method according to claim 7, wherein, The steps for training an image recognition model include: The recognition loss is determined based on the second training features extracted by the encoding model from the second training images in the secondary training image set; The reconstruction loss is determined based on the second output image reconstructed by the decoding model based on the second training features; and The image recognition model is updated based on recognition loss and reconstruction loss.

12. An electronic device comprising: The processor is configured as follows: The second-quality input features of the input image are extracted by feeding the first-quality input image into the encoding model in the image recognition model, and Generate recognition results for the input image based on input features. The processor is further configured to: determine decoding parameters by training a temporary model using a primary training image set of second quality; fix the parameters of the decoding model as decoding parameters, wherein the image recognition model includes an encoding model and a decoding model; and train an image recognition model including the fixed parameters of the decoding model using secondary training image sets of first and second quality. The encoding model is trained to extract second-quality input features from the input image regardless of whether the input image has first-quality or second-quality features.

13. The electronic device according to claim 12, wherein, The encoding model is configured to extract second-quality second-input features from the second-input image in response to input of the second-quality second-input image.

14. The electronic device according to claim 12, wherein, The encoding model is a neural network-based model.

15. The electronic device according to claim 12, wherein, The processor is also configured to generate a second-quality output image by feeding input features into a decoding model within an image recognition model.

16. The electronic device according to claim 15, wherein, The decoding model is a neural network-based model that is pre-trained to generate a second-quality output image based on second-quality input features.

17. The electronic device according to claim 15, wherein, The display module is also configured to output both the recognition results and the image.

18. The electronic device according to claim 15, wherein, The recognition result is the category information of the objects in the input image, and The output image has improved quality compared to the input image.

19. The electronic device according to any one of claims 12 to 18, further comprising: The camera is configured to generate the input image; and The display module is configured to output the recognition results.

20. A training method, comprising: A first model, comprising a first image recognition encoder and an image reconstruction decoder, is trained based on a training set that includes high-quality images but excludes low-quality images. and A second model, comprising a second image recognition encoder and a decoder, is trained using a training set that includes both high-quality and low-quality images. During the training of the second model, the parameters of the decoder are fixed to the parameters of the image reconstruction decoder generated from the training of the first model. The second image recognition encoder is trained to extract high-quality input features from the input image, regardless of whether the input image is of high or low quality.

21. The training method according to claim 20, wherein, The decoder of the trained second model is configured to reconstruct a high-quality output image corresponding to the low-quality input image based on the feature vectors generated by the second image recognition encoder from the low-quality input image.

22. The training method according to claim 20 or 21, wherein, High-quality images differ from low-quality images in any one or more of the following aspects: resolution, pixel size, brightness, noise level, and blur level.

23. A non-transitory computer-readable storage medium storing instructions, said instructions, when executed by a processor, configuring the processor to perform the method according to any one of claims 1 to 11 and 20 to 22.

Images