Image processing method and device

Abstract

The invention discloses an image processing method and device, and belongs to the technical field of image processing. The method comprises the following steps: acquiring an original image and a cut image, wherein the cut image is an image obtained by cutting the original image according to a zoom ratio; respectively carrying out scene perception on the original image and the cut image to obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cut image; fusing the first scene description text information and the second scene description text information to obtain fused scene description text information; and performing image quality enhancement on the cut image according to the fused scene description text information to obtain a target image.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to an image processing method and apparatus. Background Technology

[0002] As electronic devices become increasingly powerful in their camera functions, taking pictures with these devices has become a more and more common need. In particular, using telephoto lenses for high-magnification shooting greatly enhances users' ability to capture images from a distance. However, when shooting from a distance, magnifying the image results in a lower actual effective pixel count, leading to a blurry image where the subject is difficult to discern.

[0003] Currently, Artificial Intelligence Generated Content (AIGC) technology is commonly used to generate image details based on the magnified captured image, in order to obtain a clearer image with more details.

[0004] The above scheme relies on the quality of the input image. However, images taken with telephoto lenses and high magnification often have low pixel counts and poor image quality, making it difficult to identify the image content. Therefore, the above scheme has the problem of incorrect image content judgment when generating images based on magnified images, which in turn leads to erroneous generated images. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method and apparatus that can improve the accuracy of image content judgment and obtain images with more accurate, clearer, and more detailed content.

[0006] In a first aspect, embodiments of this application provide an image processing method, the method comprising: Acquire an original image and a cropped image, wherein the cropped image is an image obtained by cropping from the original image according to the zoom ratio; Scene perception is performed on the original image and the cropped image respectively to obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image; By fusing the first scene description text information and the second scene description text information, a fused scene description text information is obtained; The cropped image is enhanced based on the fusion scene description text information to obtain the target image.

[0007] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising: An acquisition module is used to acquire an original image and a cropped image, wherein the cropped image is an image obtained by cropping from the original image according to the zoom ratio; The first determining module is used to perform scene perception on the original image and the cropped image respectively, and obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image; The fusion module is used to fuse the first scene description text information and the second scene description text information to obtain fused scene description text information; The second determining module is used to enhance the image quality of the cropped image based on the fusion scene description text information to obtain the target image.

[0008] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can run on the processor, and the programs or instructions, when executed by the processor, implement the method as described in the first aspect.

[0009] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0010] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0011] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0012] In this embodiment, scene perception is performed on the acquired original image and the cropped image obtained by cropping from the original image according to the zoom ratio. First scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image are obtained respectively. Then, the first scene description text information and the second scene description text information are fused to obtain fused scene description text information. The cropped image is enhanced by image content generation using the fused scene description text information to obtain the target image. That is, in this embodiment, in the process of generating image content, in addition to referring to the cropped image obtained by cropping from the original image according to the zoom ratio, not only the magnified image is referenced, but also the original image. Since the original image has high image quality and clear and complete details, scene perception on the original image improves the accuracy of image content judgment and obtains more accurate first scene description text information. In this way, an image with more accurate, clear, and detailed content can be obtained. Attached Figure Description

[0013] Figure 1 This is a schematic flowchart of an image processing method provided in some embodiments of this application; Figure 2 This is a schematic diagram illustrating the image processing method performed by the diffusion model provided in some embodiments of this application; Figure 3 These are schematic diagrams illustrating the structure of an image processing apparatus according to some embodiments of this application; Figure 4 These are schematic diagrams illustrating the structure of an electronic device according to some embodiments of this application; Figure 5 These are schematic diagrams illustrating the hardware structure of an electronic device according to some embodiments of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0015] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or N objects. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0016] The terminology used in the embodiments of this invention will be explained below.

[0017] AIGC technology: It uses artificial intelligence to learn from large amounts of data to automatically generate various types of content, such as text, images, audio, and video.

[0018] Diffusion models are advanced machine learning algorithms that uniquely generate high-quality data by progressively adding noise to a dataset and then learning how to reverse the process. This approach enables diffusion models to create highly accurate and detailed outputs, ranging from realistic images to coherent sequences of text. Essentially, diffusion models generate new data by reversing the diffusion process; that is, they create new data by adding random noise to the data and then reversing this process to recover the original data distribution from the noisy data. The core function of diffusion models is to gradually reduce the quality of data and then reconstruct it to its original form or transform it into something new. It's like painting a picture with messy colors, then slowly erasing them until you get a beautiful painting again; this method makes the generated images and text look more realistic.

[0019] An autoencoder (AE) is a type of artificial neural network structure that includes an encoder and a decoder. An autoencoder is a neural network with the same input and learning objective, and its structure consists of two parts: an encoder and a decoder. Given an input space and a feature space, the autoencoder solves for the mapping between the two to minimize the reconstruction error of the input features. After solving, the hidden layer features output by the encoder, i.e., the "encoded features," can be regarded as a representation of the input data. The encoded features vary depending on the type of autoencoder. For a shrinking autoencoder, the encoded features can be the compression of the input data; for a sparse autoencoder, the encoded features can be the sparsification features of the input data; and for a variational autoencoder, the encoded features can be the latent variables of the input data.

[0020] Variational autoencoder: A type of autoencoder that accelerates the learning process of information generation tasks through feature reuse mechanisms, effectively reducing model training time and computational resource consumption.

[0021] A text encoder is a model or component that converts human-readable natural language text, such as words, sentences, and paragraphs, into a numerical representation. This numerical representation is typically a fixed-dimensional vector. Simply put, its core task is to transform unstructured text data into a mathematical form that computers can understand and process—a string of numbers, also known as a vector or embedding.

[0022] A decoder is a model or component that transforms a compressed, machine-readable vector representation back into a more natural, readable form such as text, images, or speech. Simply put, if the encoder is about "understanding," then the decoder is about "generating." A decoder receives an abstract, semantically meaningful encoded vector, also known as a latent representation or context vector, and based on the information contained in this vector, generates a completely new, human-readable sequence or data structure, such as a sentence, image, or code.

[0023] Interface: Refers to the graphical interactive layer seen by users through the screen of an electronic device. Also known as the "user interface (UI)," it is the medium through which applications or operating systems interact and exchange information with users, converting the internal form of information into a form acceptable to the user. The user interface is source code written in specific computer languages ​​such as Java and XML. This source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. The most common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements displayed on the screen of an electronic device, such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and web widgets.

[0024] Application: A computer program developed to perform one or more specific tasks and running on an operating system. Applications run in user mode, can interact with the user, and have a visual user interface.

[0025] Photo preview interface: This is the real-time view that the user sees on the device screen before taking a photo. It is a visual interactive area that is rendered in real time after the image data captured by the camera sensor is processed. It can be a GUI, which can display visible interface elements such as buttons, navigation bars, and widgets.

[0026] The technical solution of this application embodiment can be applied to scenarios involving the generation of image content details from magnified images. For example, a user wants to photograph the ear of their cat. The user uses the camera app on their phone to take a picture of the cat. Initially, the user captures image 1, which contains all information about the cat, based on the camera magnification information of 1X. That is, the object in image 1 is a complete cat. Then, the user wants to capture an image that only contains the cat's left ear. Therefore, the user increases the camera magnification information, for example, to 3X, and then captures the cat's left ear based on the camera magnification information of 3X, resulting in image 2, which only contains the cat's left ear. However, the user feels that the cat's left ear in image 2 is not clear enough, and the user wants to process the left ear area in image 2 to obtain a clear image that includes the cat's left ear.

[0027] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0028] Figure 1This is a flowchart illustrating an image processing method provided in an embodiment of this application. The subject executing the image processing method can be an electronic device, which can be, but is not limited to, a personal computer (PC), a smartphone, a tablet computer, or a personal digital assistant (PDA).

[0029] like Figure 1 As shown, the image processing method provided in this application embodiment may include steps S110-140.

[0030] S110, Obtain the original image and crop the image.

[0031] The original image can be an image taken using the original magnification information of the camera application, such as image 1 in the example above.

[0032] The aforementioned raw magnification information can be the default magnification information of the camera application. The default magnification information here is the magnification information used by the camera application after the user clicks the application icon of the camera application and enters the photo preview interface. For example, the raw magnification information here is 1X.

[0033] The cropped image can be an image obtained by cropping from the original image according to a zoom ratio, where the zoom ratio can be the magnification information used to obtain the cropped image. For example, if the cropped image is image 2 in the example above, then the zoom ratio is the magnification information 3X in the above embodiment.

[0034] S120. Perform scene perception on the original image and the cropped image respectively to obtain the first scene description text information corresponding to the original image and the second scene description text information corresponding to the cropped image.

[0035] The first scene description text information can be the scene description text information of the original image obtained after scene perception is performed on the original image. This first scene description text information can be text information used to describe the original image. For example, in the example above, the first scene description text information can be "This is a cat".

[0036] The second scene description text information can be the scene description text information of the cropped image obtained after scene perception. This second scene description text information can be text information used to describe the cropped image. For example, in the example above, the second scene description text information could be "This is a cat's ear".

[0037] In some embodiments of this application, the original image and the cropped image can be input into a Contrastive Language-Image Pre-training (CLIP) model for scene perception, resulting in a first scene description text corresponding to the original image and a second scene description text corresponding to the cropped image. The CLIP model is a cross-modal artificial intelligence model capable of understanding the semantic relationships between images and text. Its core objective is to map information from different modalities to the same semantic space, thereby enabling the calculation of their similarity.

[0038] In some embodiments of this application, in order to improve the accuracy of the generated image, S120 may specifically include: Generate the first image feature vector corresponding to the original image and the second image feature vector corresponding to the cropped image, respectively; From the scene description text library, determine the first scene description text information that matches the feature vector of the first image, and the second scene description text information that corresponds to the feature vector of the second image.

[0039] The first image feature vector can be the image feature vector of the original image. The second image feature vector can be the image feature vector of the cropped image. The specific methods for determining the first and second image feature vectors will be described in detail in later embodiments.

[0040] The aforementioned scene description text library can be a pre-set database containing a massive amount of scene description text. For example, in the example above, the scene description text library may include the scene description text "This is a cat" and the scene description text "These are a cat's ears".

[0041] In some embodiments of this application, a first image feature vector corresponding to the original image and a second image feature vector corresponding to the cropped image can be generated firstly, and then a first scene description text information matching the first image feature vector can be determined from the scene description text library, and a second scene description text information matching the second image feature vector can be determined from the scene description text library.

[0042] In the embodiments of this application, an indirect path of "image → feature vector → text" is adopted instead of direct "end-to-end" text generation. This is because directly obtaining descriptive text from an image is a generative task that requires generating a complete sentence word by word. When the input image quality is low, the uncertainty of generating a complete sentence based on blurry features is extremely high, which can easily lead to factual errors or illusions. However, the embodiments of this application adopt the indirect path of "image → feature vector → text," which does not "generate" a sentence. Instead, it compares the feature vector of the input image with the feature vectors of all texts in a predefined text library for similarity, and then selects the best match. Thus, even if the input image is blurry and its feature vector may not match the most accurate description, it will still have a very high probability of matching a semantically closest, safe, and pre-stored description. This avoids absurd errors that may occur during image generation, and the output results are always within the range of the predefined text library, significantly improving reliability. That is, the solution of this application transforms the complex content generation problem into a more controllable semantic matching problem. For scenarios that process low-quality input, it ensures that even when the information is incomplete, the electronic device's understanding of the scene can remain within a reasonable semantic range, laying a reliable foundation for subsequent high-quality image generation.

[0043] In some embodiments of this application, the step of generating the first image feature vector corresponding to the original image and the second image feature vector corresponding to the cropped image may specifically include: Divide the first image into multiple image blocks; Convert each image patch into a digital vector; Based on the position of each image patch in the first image, add image patch position encoding to the digital vector; An attention mechanism is used to encode each numeric vector with image patch location encoding to obtain the image feature vector corresponding to the image.

[0044] The first image can be either the original image or the cropped image.

[0045] For the first image, multiple image blocks can be multiple image blocks obtained by segmenting the first image.

[0046] For any image patch, its corresponding digital vector can be the vector obtained by flattening it.

[0047] For any image patch, the image patch position encoding can be an encoding added to the image patch based on its position in its corresponding first image. For example, in the example above, image 1 is segmented, for instance, into four image patches: image patch 1, image patch 2, image patch 3, and image patch 4. Then, for any image patch 1 within these four patches, it can be converted into a digital vector. For example, for image patch 1, it can be converted into digital vector 1. Since image patch 1 is the first image patch in these four patches, an image patch position encoding can be added to its digital vector 1, such as adding image patch position encoding 1. Correspondingly, image patches 2, 3, and 4 are encoded in the same way as image patch 1, with image patch position encoding added to their digital vectors.

[0048] In some embodiments of this application, the first image can be divided into multiple image blocks, as shown in the following formula (1), the first image can be divided into multiple image blocks.

[0049]

[0050] In the above formula (1), The first image has a size of [size missing]. Since both the original and cropped images are RGB images, the 3 mentioned above refers to 3 color channels. For the length and width of the image, The number of image blocks into which the first image is divided. For an image patch, the length and width are both .

[0051] In the above formula (1), it is to... The first image It was divided into 196 16×16 image blocks.

[0052] For any image in the original image or cropped image, after obtaining multiple image blocks of the image, each image block can be converted into a digital vector according to the following formula (2). Specifically, each image block can be input into the linear layer of the dual-tower model, and the image block can be mapped to a preset dimension D through the linear layer to obtain the digital vector of the image block. The preset dimension D mentioned above is the same as the size of the image block.

[0053]

[0054] In the above formula (2), for the first image, This is a digital vector converted from an image patch in the first image. , , here For preset dimensions, .

[0055] As shown in formula (1) above, the size of an image patch is 16×16×3=768, then the preset dimension in formula (2) is... It is 768.

[0056] Then, based on the position of each image block in the first image, an image block position code is added to the digital vector, as shown in formula (3) below.

[0057]

[0058] In the above formula (3), It is a learnable image feature vector, which It is the initial token of a model, derived from the model's training process. It is a vector obtained through data-driven learning, used to represent the global features of an image. ; For the first Numerical vectors of image patches; For the location encoding matrix, positional information is injected into the digital vector of each image patch; ,Should To unfold the image into image feature vector form, this It includes image feature vectors from 196 image patches and a learnable initial token.

[0059] Next, attention mechanism encoding is performed on each digital vector with added image block position encoding, as shown in the following formula (4), to obtain the image feature vector corresponding to the image.

[0060]

[0061] In the above formula (4), For the first in the Transformer encoder Layer coding layer.

[0062] After passing through the last encoding layer in the Transformer encoder, the output of the learnable initial token is taken as the image feature vector of the image. , here This represents the total number of layers in the Transformer encoder.

[0063] Thus, for any image in the original image or the cropped image, according to the above formulas (1)-(4), the image feature vector corresponding to the original image and the image feature vector corresponding to the cropped image can be obtained respectively. Since formula (2) maps each image patch to a preset dimension... In this way, both the image feature vector and the image feature vector obtained have a preset dimension. .

[0064] It should be noted that the image feature vectors corresponding to the original image and the cropped image obtained according to the above formulas (1)-(4) can be used as the first image feature vector and the second image feature vector in the embodiments of this application.

[0065] However, since the image feature vectors corresponding to the original image and the cropped image need to be matched with the feature vectors corresponding to the scene description text in the scene description text library, but the image feature dimensions and text dimensions are different, they need to be converted into vectors with the same dimensions. Therefore, after obtaining the image feature vectors corresponding to the original image and the cropped image according to the above formulas (1)-(4), in order to correspond with the subsequent text feature vectors, the image feature vectors obtained by the above formula (4) can be projected into the shared space according to the following formula (5), that is, mapped to the same dimensional space as the text feature vectors.

[0066]

[0067] In the above formula (5), , , The value “512” in the above formula (5) is the dimension of the shared space.

[0068] Then, the image feature vector projected into the shared space is normalized to obtain the normalized image feature vector, as shown in the following formula (6).

[0069]

[0070] For any image in the original image or the cropped image, after obtaining the image feature vector of the image according to the above formula (6), it can be used as the image feature vector for subsequent processing, that is, as the first image feature vector or the second image feature vector. That is, for the original image, after processing it according to the above formula (1)-(6), the first image feature vector is obtained. For the cropped image, after processing it according to the above formula (1)-(6), the second image feature vector is obtained.

[0071] In the embodiments of this application, for any image in the original image or cropped image, the image is divided into multiple image blocks, and each image block is converted into a digital vector. Based on the position of each image block in the image, an image block position code is added to the digital vector. An attention mechanism is used to encode each digital vector with the added image block position code. In this way, the scene information of any image in the original image or cropped image can be deeply perceived, and the corresponding image feature vector can be obtained.

[0072] In some embodiments of this application, determining the first scene description text information that matches the first image feature vector and the second scene description text information that corresponds to the second image feature vector from the scene description text library may specifically include: Each text information in the scene description text library is segmented into words to obtain a segmented text sequence. Add a sequence identifier to each segmented text sequence; Each segmented text sequence with a sequence identifier is encoded into a text vector to obtain the corresponding text feature vector; For any image feature vector in the first image feature vector or the second image feature vector, calculate the similarity between the image feature vector and each text feature vector respectively; The text information corresponding to the text feature vector with the best similarity is determined as the scene description text information corresponding to the image feature vector.

[0073] In this context, for each text information in the scene description text library, its corresponding segmented text sequence can be the text sequence obtained after segmenting the text information into words. For example, in the example above, the text information "This is a cat" can be segmented into 5 words: "this", "is", "one", "only", and "cat". Therefore, the resulting segmented text sequence is [this, is, one, only, cat].

[0074] For each segmented text sequence, its sequence identifier can be an identifier used to identify its integrity. The sequence identifier can include a start identifier and an end identifier of the segmented text sequence. For example, the start identifier can be [SOS] and the end identifier can be [EOS].

[0075] For each segmented text sequence, its corresponding text feature vector can be the vector obtained by text vector encoding of the segmented text sequence with sequence identifiers.

[0076] In some embodiments of this application, each text information in the scene description text library can be segmented into words to obtain a segmented text sequence corresponding to each text information. Then, sequence identifiers are added to the beginning and end of each segmented text sequence to indicate the start and end of the segmented text sequence. Then, each segmented text sequence with added sequence identifiers is encoded into a text vector to obtain the corresponding text feature vector.

[0077] Specifically, for each segmented text sequence with added sequence identifiers, the text vector encoding can be performed as follows: each segmented text sequence with added sequence identifiers is converted into an embedding vector through an embedding layer, as shown in the following formula (7).

[0078]

[0079] In the above formula (7), The parameters for each embedding layer are of size . Here, 49408 is the size of the preset embedding table used, that is, the preset embedding table contains information on 49408 lexical units. The preset embedding table is a vocabulary that determines the embedding vector of each lexical unit in a segmented text sequence with added sequence identifiers. =512, which is the hidden layer dimension of the Transformer model, i.e., the preset dimension; For a word in a segmented text sequence with added sequence identifiers, This is the embedding vector of a word.

[0080] Then, add positional encoding to each word in the segmented text sequence with sequence identifiers, as shown in formula (8) below.

[0081]

[0082] In the above formula (8), Here, 77 is the length of the segmented text sequence with sequence markers. A learnable positional encoding matrix is ​​used to add positional encoding to each word in a segmented text sequence with added sequence identifiers; , This is the feature vector of a word.

[0083] For each segmented text sequence with added sequence identifiers, the feature vector of each word in the segmented text sequence with added sequence identifiers is obtained according to the above formulas (6)-(8). The feature vectors of each word are merged to obtain the feature vector of the segmented text sequence with added sequence identifiers. Then, the feature vector of the segmented text sequence with added sequence identifiers is input into the Transformer encoding for Transformer encoding.

[0084] The Transformer encoders mentioned above have a total of The layer performs Transformer encoding on the segmented text sequence with added sequence identifiers, as shown in the following formula (9).

[0085]

[0086] In the above formula (9), For the Transformer encoder's first The output of the layer, For the Transformer encoder's first The output of a layer, that is, in a Transformer encoder, the output of the previous layer is the input of the next layer.

[0087] Then, the representation of the last word of the segmented text sequence with added sequence identifiers obtained from the last layer of the Transformer encoder is taken as the representation of the segmented text sequence with added sequence identifiers, as shown in the following formula (10).

[0088]

[0089] In formula (10), This indicates the position of the end marker [EOS] within the segmented text sequence with added sequence markers.

[0090] Then the representation obtained by formula (10) is projected into the same shared space as the image feature vector, as shown in formula (11) below.

[0091]

[0092] In the above formula (11), , , The value “512” in the above formula (11) is the dimension of the shared space.

[0093] Then, the text feature vector projected into the shared space is normalized to obtain the normalized text feature vector, as shown in the following formula (12).

[0094]

[0095] For each text information in the scenario description text library, after being processed according to the above formulas (7)-(12), the text feature vector corresponding to each text information can be obtained.

[0096] It should be noted that for each token in the tokenized text sequence with sequence identifiers added, the specific process of the above formulas (7)-(8) is as follows: Search for the embedding vector corresponding to the token in the preset embedding table to obtain the embedding vector corresponding to each token; then, according to the position information of each token in the tokenized text sequence, perform positional encoding on each token respectively to obtain the position vector of each token; then, according to the embedding vector and position vector corresponding to each token, obtain the feature vector corresponding to each token, and based on the feature vectors corresponding to each token, the text feature vector corresponding to the tokenized text sequence can be obtained.

[0097] The above-mentioned preset embedding table can be a pre-trained semantic embedding table, which contains semantic feature vectors of each token in different dimensions. For example, for the token "this", the preset embedding table contains semantic feature vectors of this token "this" in different dimensions. For example, the preset embedding table contains semantic feature vectors of this token "this" in different dimensions such as the sentiment dimension, domain dimension, and词性 dimension. The above-mentioned sentiment dimension can be the sentiment represented by the token, such as whether the token represents a positive sentiment or a negative sentiment. The domain dimension can be the domain represented by the token, such as whether the token is in the literary field or the financial field. The词性 dimension can be the词性 represented by the token, such as whether the token is a noun or a verb, etc.

[0098] For a token, its corresponding embedding vector can be obtained by concatenating the semantic feature vectors of different dimensions of the token found in the preset embedding table. For example, in the above example, the dimension of the token in the preset embedding table is 512, that is, for a certain token, the preset embedding table contains semantic feature vectors of this token in 512 dimensions. For example, for the token "this" in "This is a cat", the semantic feature vectors of this token in 512 dimensions can be obtained respectively, and by concatenating the semantic feature vectors of this token in 512 dimensions, a vector with 1 row and 512 columns can be obtained, and this vector with 1 row and 512 columns is the embedding vector of the token "this".

[0099] For a token, its corresponding position vector can be a vector generated according to the ordinal position of the token in its corresponding word segmentation text sequence. For example, in the above example, for the token "这" in "这是一只猫", it is in the first position in "这是一只猫", and position encoding is performed according to the position of the token "这" in "这是一只猫" to obtain the position vector of the token "这".

[0100] Then, the embedding vector and the position vector of each token are combined to obtain the feature vector of the token. Specifically, the embedding vector and the position vector corresponding to each token can be added to obtain the feature vector corresponding to each token.

[0101] It should be noted that the dimension of the position vector of a token is the same as the dimension of the embedding vector, so that the position vector and the embedding vector of the token can be added to obtain the feature vector corresponding to the token. Specifically, the vector elements at the same positions of the position vector and the embedding vector can be added. For example, the element in the first row and first column of the position vector is added to the element in the first row and first column of the embedding vector to obtain the element in the first row and first column of the feature vector of the token. For example, in the above example, the element in the first row and first column of the embedding vector corresponding to the token "这" is added to the element in the first row and first column of the position vector corresponding to the token "这" to obtain the element in the first row and first column of the feature vector of the token "这". The element in the first row and second column of the embedding vector corresponding to the token "这" is added to the element in the first row and second column of the position vector corresponding to the token "这" to obtain the element in the first row and second column of the feature vector of the token "这", and so on, until the element in the first row and 512th column of the embedding vector corresponding to the token "这" is added to the element in the first row and 512th column of the position vector corresponding to the token "这" to obtain the element in the first row and 512th column of the feature vector of the token "这". That is, the obtained feature vector of the token "这" is also a vector with 1 row and 512 columns.

[0102] Then, the feature vectors of each token are combined to obtain the text feature vector corresponding to the word segmentation text sequence. For example, in the above example, the feature vectors of the tokens "这", "是", "一", "只", and "猫" are combined to obtain the text feature vector of the word segmentation text sequence "这是一只猫". The text feature vector of the word segmentation text sequence "这是一只猫" is a vector with 5 rows and 512 columns.

[0103] After obtaining the text feature vector corresponding to each segmented text sequence with sequence identifiers, for any image feature vector from the first or second image feature vector, the similarity between the image feature vector and each text feature vector is calculated. This means calculating the similarity between the first image feature vector and each text feature vector, and the similarity between the second image feature vector and each text feature vector. Then, the text information corresponding to the text feature vector with the best similarity between the first image feature vector and each text feature vector is determined as the first scene description text information corresponding to the first image feature vector. Similarly, the text information corresponding to the text feature vector with the best similarity between the second image feature vector and each text feature vector is determined as the second scene description text information corresponding to the second image feature vector.

[0104] In the embodiments of this application, each text information in the scene description text library is converted into a text feature vector. Then, the first image feature vector and the second image feature vector are matched with the text feature vector corresponding to each text information, respectively. This is because directly matching text and images can only perform surface keyword-based retrieval, which fails to understand semantic relationships and leads to matching errors. By converting both text and images into high-dimensional vectors, the model learns the abstract relationships between concepts. This method transcends rigid keyword matching, possesses strong generalization ability, and ensures both computational efficiency and the reliability of matching results.

[0105] In some embodiments of this application, calculating the similarity between the image feature vector and each text feature vector may specifically include: The image feature vector is multiplied by each text feature vector to obtain the dot product.

[0106] The dot product can be used to characterize the similarity between the image feature vector and each text feature vector.

[0107]

[0108] In some embodiments of this application, the image feature vector and each text feature vector can be multiplied by the dot product operation shown in the following formula (13).

[0109] In the above formula (13), For the first scene description text in the library A text message, For image feature vectors and the first The similarity of text feature vectors corresponding to each piece of text information; For the first scene description text in the library The sum of the text feature vectors of each text information. This represents the total number of text entries in the scene description text library. This is the image feature vector.

[0110] Since one image feature vector can match multiple text feature vectors, the above formula (13) can be expanded into the form shown in the following formula (14).

[0111]

[0112] The higher the dot product result, the higher the similarity between the two pieces of information. Therefore, the dot product can be used to characterize the similarity between image feature vectors and each text feature vector.

[0113] In this embodiment, the dot product is obtained by performing a dot product operation between the image feature vector and each text feature vector. The dot product is then used to represent the similarity between the image feature vector and each text feature vector. Since the dot product can unify the magnitude of all vectors to 1, this means that the dot product calculation completely eliminates the influence of the original magnitude of the vectors and only focuses on the direction of the vectors. The direction of the vectors encodes semantic information. After normalization, the dot product ensures that the similarity comparison is based on pure semantic content, and will not be biased due to the generally large or small values ​​of some vectors. This improves the accuracy of the similarity calculation between the image feature vector and each text feature vector.

[0114] S130. Merge the first scene description text information and the second scene description text information to obtain the merged scene description text information.

[0115] The fused scene description information can be the text information obtained by fusing the first scene description text information and the second scene description text information. For example, in the example above, the first scene description text information is "This is a cat" and the second scene description text information is "This is a cat's ear". By fusing the two pieces of information, we can obtain the fused scene description text information "This is a cat's ear fur".

[0116] Step 140: Enhance the image quality of the cropped image based on the fusion scene description text information to obtain the target image.

[0117] The target image can be an image obtained by enhancing the image quality of a cropped image based on the fused scene description text information. Specifically, it can be an image content-generating image quality enhancement method that enhances the image content in the cropped image based on the fused scene description text information. For example, in the example above, the image quality of the cat ears can be enhanced, resulting in a clear image of the cat ears as the target image.

[0118] In some embodiments of this application, in order to improve the efficiency of acquiring the target image, step 140 may specifically include: The fused scene description text and the cropped image are input into the diffusion model. The diffusion model uses the fused scene description text as prompts to enhance the image quality of the cropped image, thus obtaining the target image.

[0119] In some embodiments of this application, the fused scene description text information and the cropped image can be input into a diffusion model. The diffusion model uses the fused scene description text information as prompts to enhance the image quality of the cropped image, thereby obtaining the target image.

[0120] In the embodiments of this application, the diffusion model uses the fused scene description text information as prompt words to process the fused scene description text information and the cropped image to obtain a target image with clear details. The prompt words ensure that the generation process of the diffusion model does not "go astray". This strong semantic constraint makes the generated result highly accurate in content and improves the accuracy of the generated content.

[0121] In some embodiments of this application, the diffusion model may include a variational autoencoder, a text encoder, a noise removal network, and a decoder.

[0122] The process of inputting the fused scene description text information and the cropped image into a diffusion model, and using the fused scene description text information as prompts, enhances the image quality of the cropped image to obtain the target image, may specifically include: The cropped image is encoded in the latent space using a variational autoencoder to obtain the initial latent space vector; The text information describing the fused scene is encoded using a text encoder to obtain a text embedding vector; Using the text embedding vector as a condition, the initial latent space vector is iteratively denoised through a noise removal network to obtain the denoised latent space vector. The target image is obtained by decoding the denoised latent space vector using a decoder.

[0123] The initial latent space vector can be a vector obtained by performing latent space encoding on the cropped image using a variational autoencoder.

[0124] A text embedding vector can be a vector obtained by encoding the fusion scene description text information through a text encoder.

[0125] The denoised latent space vector can be obtained by iteratively denoising the initial latent space vector using a noise removal network, conditioned on the text embedding vector.

[0126] In some embodiments of this application, reference may be made to Figure 2 The fused scene description text information and the cropped image are input into the diffusion model 21. Then, the cropped image is encoded in the latent space by the variational autoencoder 22 to obtain the initial latent space vector. Then, the fused scene description text information is encoded by the text encoder 23 to obtain the text embedding vector. Then, the initial latent space vector is iteratively denoised by the noise removal network 24 based on the text embedding vector to obtain the denoised latent space vector. Finally, the denoised latent space vector is decoded by the decoder 25 to obtain the target image.

[0127] In the embodiments of this application, the fused scene description text information and cropped image are processed by the variational autoencoder, text encoder, noise removal network, and decoder in the diffusion model to obtain a target image with clear details. Since the variational autoencoder compresses the image into a low-dimensional latent space, the dimensionality of all subsequent operations is reduced, saving computational resources and improving the efficiency of target image acquisition. The text encoder converts the fused scene description text information into text embedding vectors, which forces the model to use the fused scene description text information to firmly "lock" the correct content that the telephoto cropped image should have, greatly suppressing the "illusion" that the AIGC model may produce due to the blurriness of the input image. The noise removal network performs denoising in the latent space, which is equivalent to repairing and enhancing in the "essential" space of the image, improving the accuracy of content generation and avoiding the influence of noise on content generation.

[0128] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0129] Figure 3 This is a schematic diagram illustrating the structure of an image processing apparatus according to an exemplary embodiment. Figure 3 As shown, the image processing apparatus 300 may include: The acquisition module 310 is used to acquire an original image and a cropped image, wherein the cropped image is an image obtained by cropping from the original image according to the zoom ratio; The first determining module 320 is used to perform scene perception on the original image and the cropped image respectively, and obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image; The fusion module 330 is used to fuse the first scene description text information and the second scene description text information to obtain fused scene description text information; The second determining module 340 is used to enhance the image quality of the cropped image based on the fusion scene description text information to obtain the target image.

[0130] In this embodiment, scene perception is performed on the acquired original image and the cropped image obtained by cropping from the original image according to the zoom ratio. First scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image are obtained respectively. Then, the first scene description text information and the second scene description text information are fused to obtain fused scene description text information. The cropped image is enhanced by image content generation using the fused scene description text information to obtain the target image. That is, in this embodiment, in the process of generating image content, in addition to referring to the cropped image obtained by cropping from the original image according to the zoom ratio, not only the magnified image is referenced, but also the original image. Since the original image has high image quality and clear and complete details, scene perception on the original image improves the accuracy of the judgment of the image content, and more accurate first scene description text information can be obtained. In this way, an image with more accurate, clear, and detailed image content can be obtained.

[0131] In some embodiments of this application, the first determining module is specifically used for: Generate a first image feature vector corresponding to the original image and a second image feature vector corresponding to the cropped image, respectively. From the scene description text library, determine the first scene description text information that matches the first image feature vector, and the second scene description text information that corresponds to the second image feature vector.

[0132] In some embodiments of this application, the first determining module is specifically used for: The first image is divided into multiple image blocks, where the first image is any one of the original image and the cropped image; Each of the image blocks is converted into a digital vector; Based on the position of each image patch in the first image, add an image patch position code to the digital vector; An attention mechanism is used to encode each digital vector with added image patch location codes to obtain the image feature vector corresponding to the image, and the image feature vector has a preset dimension.

[0133] In some embodiments of this application, the first determining module is specifically used for: Each text information in the scene description text library is segmented into words to obtain the segmented text sequence corresponding to each text information. Add a sequence identifier to each of the segmented text sequences; Each segmented text sequence with added sequence identifier is encoded into a text vector to obtain a corresponding text feature vector; the text feature vector has a preset dimension. For any image feature vector in the first image feature vector or the second image feature vector, calculate the similarity between the image feature vector and each of the text feature vectors respectively; The text information corresponding to the text feature vector with the best similarity is determined as the scene description text information corresponding to the image feature vector.

[0134] In some embodiments of this application, the second determining module is specifically used for: The fused scene description text and the cropped image are input into a diffusion model. The diffusion model uses the fused scene description text as prompts to enhance the cropped image, thereby obtaining the target image.

[0135] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0136] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0137] The image processing apparatus provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0138] Optionally, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401 and a memory 402. The memory 402 stores a program or instructions that can run on the processor 401. When the program or instructions are executed by the processor 401, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0139] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0140] Figure 5 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0141] The electronic device 500 includes, but is not limited to, components such as: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, and processor 510.

[0142] Those skilled in the art will understand that the electronic device 500 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0143] The processor 510 is configured to acquire an original image and a cropped image, wherein the cropped image is an image cropped from the original image according to a zoom ratio; perform scene perception on the original image and the cropped image respectively to obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image; fuse the first scene description text information and the second scene description text information to obtain fused scene description text information; and perform image content generation-based image quality enhancement on the cropped image based on the fused scene description text information to obtain a target image.

[0144] Thus, by performing scene perception on the acquired original image and the cropped image obtained by cropping from the original image according to the zoom ratio, the first scene description text information corresponding to the original image and the second scene description text information corresponding to the cropped image are obtained respectively. Then, the first scene description text information and the second scene description text information are fused to obtain fused scene description text information. The cropped image is then enhanced with image content generation using the fused scene description text information to obtain the target image. That is, in this embodiment of the application, in addition to referring to the cropped image obtained by cropping from the original image according to the zoom ratio, the image content generation also refers to the original image. Since the original image has high image quality and clear and complete details, scene perception on the original image improves the accuracy of image content judgment, and more accurate first scene description text information can be obtained, thereby obtaining an image with more accurate, clearer, and more detailed image content.

[0145] Optionally, the processor 510 is configured to generate a first image feature vector corresponding to the original image and a second image feature vector corresponding to the cropped image, respectively; and determine, from a scene description text library, first scene description text information that matches the first image feature vector and second scene description text information that matches the second image feature vector.

[0146] Optionally, the processor 510 is configured to divide the first image into multiple image blocks, wherein the first image is any one of the original image and the cropped image; convert each image block into a digital vector; add an image block position code to the digital vector according to the position of each image block in the image; and perform attention mechanism encoding on each digital vector with added image block position code to obtain an image feature vector corresponding to the image, wherein the image feature vector has a preset dimension.

[0147] Optionally, the processor 510 is configured to perform word segmentation on each text information in the scene description text library to obtain a word segmentation text sequence corresponding to each text information; add a sequence identifier to each word segmentation text sequence; encode each word segmentation text sequence with added sequence identifier into a text vector to obtain a corresponding text feature vector; the text feature vector has a preset dimension; for any image feature vector in the first image feature vector or the second image feature vector, calculate the similarity between the image feature vector and each text feature vector; and determine the text information corresponding to the text feature vector with the best similarity as the scene description text information corresponding to the image feature vector.

[0148] Optionally, the processor 510 is configured to input the fused scene description text information and the cropped image into a diffusion model, and use the fused scene description text information as prompt words to enhance the image quality of the cropped image to obtain a target image.

[0149] It should be understood that, in this embodiment, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The GPU 5041 processes image data of still images or videos obtained by an image capture device (such as a color camera) in video capture mode or image capture mode. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0150] The memory 509 can be used to store software programs and various data. The memory 509 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 509 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 509 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0151] Processor 510 may include one or more processing units; optionally, processor 510 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 510.

[0152] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0153] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0154] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0155] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0156] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the information image method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0157] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0158] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0159] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Acquire an original image and a cropped image, wherein the cropped image is an image obtained by cropping from the original image according to the zoom ratio; Scene perception is performed on the original image and the cropped image respectively to obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image; By fusing the first scene description text information and the second scene description text information, a fused scene description text information is obtained; The cropped image is enhanced based on the fusion scene description text information to obtain the target image.

2. The method according to claim 1, characterized in that, The step of performing scene perception on the original image and the cropped image respectively to obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image includes: Generate a first image feature vector corresponding to the original image and a second image feature vector corresponding to the cropped image, respectively. From the scene description text library, determine the first scene description text information that matches the first image feature vector, and the second scene description text information that corresponds to the second image feature vector.

3. The method according to claim 2, characterized in that, The step of generating the first image feature vector corresponding to the original image and the second image feature vector corresponding to the cropped image includes: The first image is divided into multiple image blocks, where the first image is any one of the original image and the cropped image; Each of the image blocks is converted into a digital vector; Based on the position of each image patch in the first image, add an image patch position code to the digital vector; An attention mechanism is used to encode each digital vector with added image patch location codes to obtain the image feature vector corresponding to the image, and the image feature vector has a preset dimension.

4. The method according to claim 2 or 3, characterized in that, The step of determining, from the scene description text library, the first scene description text information that matches the first image feature vector, and the second scene description text information that corresponds to the second image feature vector, includes: Each text information in the scene description text library is segmented into words to obtain the segmented text sequence corresponding to each text information. Add a sequence identifier to each of the segmented text sequences; Each segmented text sequence with added sequence identifier is encoded into a text vector to obtain a corresponding text feature vector; the text feature vector has a preset dimension. For any image feature vector in the first image feature vector or the second image feature vector, calculate the similarity between the image feature vector and each of the text feature vectors respectively; The text information corresponding to the text feature vector with the best similarity is determined as the scene description text information corresponding to the image feature vector.

5. The method according to claim 1, characterized in that, The step of enhancing the image quality of the cropped image based on the fused scene description text information to obtain the target image includes: The fused scene description text information and the cropped image are input into the diffusion model. The diffusion model uses the fused scene description text information as prompt words to enhance the image quality of the cropped image, thereby obtaining the target image.

6. An image processing apparatus, characterized in that, The device includes: An acquisition module is used to acquire an original image and a cropped image, wherein the cropped image is an image obtained by cropping from the original image according to the zoom ratio; The first determining module is used to perform scene perception on the original image and the cropped image respectively, and obtain first scene description text information corresponding to the original image and second scene description text information corresponding to the cropped image; The fusion module is used to fuse the first scene description text information and the second scene description text information to obtain fused scene description text information; The second determining module is used to enhance the image quality of the cropped image based on the fusion scene description text information to obtain the target image.

7. The apparatus according to claim 6, characterized in that, The first determining module is specifically used for: Generate a first image feature vector corresponding to the original image and a second image feature vector corresponding to the cropped image, respectively. From the scene description text library, determine the first scene description text information that matches the first image feature vector, and the second scene description text information that corresponds to the second image feature vector.

8. The apparatus according to claim 7, characterized in that, The first determining module is specifically used for: The first image is divided into multiple image blocks, where the first image is any one of the original image and the cropped image; Each of the image blocks is converted into a digital vector; Based on the position of each image patch in the first image, add an image patch position code to the digital vector; An attention mechanism is used to encode each digital vector with added image patch location codes to obtain the image feature vector corresponding to the image, and the image feature vector has a preset dimension.

9. The apparatus according to claim 8, characterized in that, The first determining module is specifically used for: Each text information in the scene description text library is segmented into words to obtain the segmented text sequence corresponding to each text information. Add a sequence identifier to each of the segmented text sequences; Each segmented text sequence with added sequence identifier is encoded into a text vector to obtain a corresponding text feature vector; the text feature vector has a preset dimension. For any image feature vector in the first image feature vector or the second image feature vector, calculate the similarity between the image feature vector and each of the text feature vectors respectively; The text information corresponding to the text feature vector with the best similarity is determined as the scene description text information corresponding to the image feature vector.

10. The apparatus according to claim 6, characterized in that, The second determining module is specifically used for: The fused scene description text information and the cropped image are input into the diffusion model. The diffusion model uses the fused scene description text information as prompt words to enhance the image quality of the cropped image, thereby obtaining the target image.

Images