An image processing method, apparatus, and electronic device
By combining image convolutional models and graph network models, and utilizing multi-scale attention networks to process images, the problem of wasted human resources in deep learning models is solved, and automated image enhancement is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2026-02-06
- Publication Date
- 2026-06-05
AI Technical Summary
Existing deep learning models require extensive manual debugging and data annotation in image enhancement algorithms, resulting in a waste of human resources.
By employing image convolution models, feature algorithms, and graph network models, and through multi-scale attention networks and graph network processing, images are automatically processed to obtain matching matrices, reducing manual intervention.
It achieves automated image enhancement processing, avoids wasting human resources, and improves processing efficiency.
Smart Images
Figure CN122155972A_ABST
Abstract
Description
Technical Field
[0001] This application relates to artificial intelligence, and includes, but is not limited to, an image processing method, apparatus, and electronic device. Background Technology
[0002] With social development and technological advancements, image enhancement algorithms have become an important image processing and detection technology, aiming to improve image resolution through automated means. Image enhancement algorithms are divided into traditional image processing methods and deep learning-based image enhancement algorithms.
[0003] Existing deep learning models require a large number of parameters for manual debugging and a significant amount of time for data filtering and labeling, resulting in a waste of human resources. Summary of the Invention
[0004] In view of the above, embodiments of this application provide an image processing method, apparatus, and electronic device.
[0005] The technical solution of this application embodiment is implemented as follows: This application provides an image processing method, the method comprising: acquiring an original image; processing the original image using a first image convolution model to obtain a first semantic and a first feature image; calculating the first semantic using a first feature algorithm to obtain a second feature image; obtaining a first resolution image based on the first feature image and the second feature image; obtaining a tensor set based on a first extraction algorithm and the first resolution image; processing the tensor set using a first graph network model to obtain a first description matrix and a second description matrix; obtaining a matching matrix based on the first description matrix and the second description matrix; and processing the image based on the matching matrix.
[0006] Optionally, obtaining the tensor set based on the first extraction algorithm and the first resolution image includes: obtaining keypoint vectors and descriptor vectors based on the first extraction algorithm and the first resolution image; and obtaining the tensor set based on the keypoint vectors and the descriptor vectors.
[0007] Optionally, obtaining the tensor set based on the keypoint vector and the descriptor vector includes: normalizing the descriptor vector and adding it to the keypoint vector to obtain the tensor set.
[0008] Optionally, based on the first extraction algorithm and the first resolution image, keypoint vectors and descriptor vectors are obtained, including: The formula is: The steps are: ;in, It is the updated keypoint vector. The original keypoint vector, express The i-th element of the vector, where Here, n represents the keypoint index of the original image, and n is the total number of keypoints. For the parameters of a one-dimensional convolution, subscript Representing vectors The first in Each element.
[0009] Optionally, the step of processing the tensor set through the first graph network model to obtain the first description matrix and the second description matrix includes: processing the tensor set through the first graph network model to obtain distinguishable feature points; and obtaining the first description matrix and the second description matrix based on the distinguishable feature points.
[0010] Optionally, obtaining a matching matrix based on the first description matrix and the second description matrix includes: performing one-dimensional convolutional encoding on the first description matrix to obtain a first score matrix; performing one-dimensional convolutional encoding on the second description matrix to obtain a second score matrix; and performing an inner product of the first score matrix and the second score matrix to obtain the matching matrix.
[0011] Optionally, after obtaining the matching matrix, the process includes: obtaining a matching score based on the matching matrix.
[0012] Optionally, a matching score is obtained based on the matching matrix, including the following calculation formula: .
[0013] A processing apparatus, comprising: an acquisition unit, an analysis unit, and a processing unit; the acquisition unit being configured to acquire an original image; the analysis unit being configured to process the original image using a first image convolution model to obtain a first semantic and a first feature image; to calculate the first semantic using a first feature algorithm to obtain a second feature image; to obtain a first resolution image based on the first feature image and the second feature image; to obtain a tensor set based on a first extraction algorithm and the first resolution image; and to process the tensor set using a first graph network model to obtain a first description matrix and a second description matrix; the processing unit being configured to obtain a matching matrix based on the first description matrix and the second description matrix, and to process the image based on the matching matrix.
[0014] An electronic device includes: a memory for storing at least one set of instructions; and a processor for acquiring raw images. The original image is processed using a first image convolution model to obtain a first semantic image and a first feature image; the first semantic image is calculated using a first feature algorithm to obtain a second feature image; a first resolution image is obtained based on the first feature image and the second feature image; a tensor set is obtained based on a first extraction algorithm and the first resolution image; the tensor set is processed using a first graph network model to obtain a first description matrix and a second description matrix; a matching matrix is obtained based on the first description matrix and the second description matrix; and the image is processed based on the matching matrix.
[0015] This application provides an image processing method, apparatus, and electronic device. First, an original image is acquired. Then, the original image is processed using a first image convolution model to obtain a first semantic image and a first feature image. Next, the first semantic image is calculated using a first feature algorithm to obtain a second feature image. Based on the first and second feature images, a first resolution image is obtained. Then, based on a first extraction algorithm and the first resolution image, a tensor set is obtained. The tensor set is processed using a first graph network model to obtain a first description matrix and a second description matrix. Finally, based on the first and second description matrices, a matching matrix is obtained. The image is then automatically processed based on the matching matrix, thereby avoiding the waste of human resources. Attached Figure Description
[0016] Figure 1 A flowchart of the image processing method provided in the embodiments of this application; Figure 2 Another flowchart of the image processing method provided in the embodiments of this application; Figure 3 Another flowchart of the image processing method provided in the embodiments of this application; Figure 4 Another flowchart of the image processing method provided in the embodiments of this application; Figure 5 A schematic diagram of the processing apparatus provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structural composition of the electronic device provided in the embodiments of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0018] Please refer to Figure 1 , Figure 2 ,in, Figure 1 A flowchart illustrating an implementation of the image processing method provided in this application embodiment may include: Step S101: Obtain the original image; Step S102: Process the original image using the first image convolution model to obtain the first semantic and first feature images; Step S103: Calculate the first semantic meaning using the first feature algorithm to obtain the second feature image; Step S104: Obtain a first resolution image based on the first feature image and the second feature image; Step S105: Obtain the tensor set based on the first extraction algorithm and the first resolution image; Step S106: Process the tensor set through the first graph network model to obtain the first description matrix and the second description matrix; Step S107: Obtain a matching matrix based on the first description matrix and the second description matrix, and process the image based on the matching matrix.
[0019] The original image to be processed consists of a first image convolutional model composed of 4 convolutional layers and 3 max-pooling layers. The first semantics is the high-dimensional semantics of the image. The first feature image is a 1024-layer feature map, and the second feature image is a single-channel feature map, i.e., a 1025-channel feature map. The tensor set includes at least: a descriptor tensor. and descriptor tensors The first description matrix is the matching description matrix. The matching matrix is the matching score matrix.
[0020] Specifically, this scheme utilizes a multi-scale attention network as the generator and a graph network as the discriminator in an adversarial generative network. Its main structure includes an image convolution module, a feature enhancement module, an image generation module, a keypoint extraction module, a graph network processing module, and a score calculation module.
[0021] Step 1: Image Convolution Module; Specifically, as shown in Table 1, this module consists of four convolutional layers and three max-pooling layers. The larger convolutional kernels have a larger receptive field, effectively capturing complete semantic information. Each pooling operation transforms the input image to its original size. ,in This represents the kernel size. Adding pooling layers can significantly reduce the size of the feature maps, thus reducing the amount of training required in later stages of the model.
[0022]
[0023] Table 1 Step 2, Feature Enhancement Module; After obtaining the high-dimensional semantics of the image in step 1, the following operations are performed:
[0024]
[0025] Actually one The purpose of convolution is to transform the output channel map to be the same as that of the input, and to ensure the "plug-and-play" functionality of the entire attention mechanism module. It is a residual connection to prevent the feature enhancement module from disturbing the entire model. It is the output of the feature enhancement module.
[0026] The feature enhancement module can establish effective connections between the current pixel and other pixels, and enhance the model's attention to these regions by weighting the values, thereby improving the feature representation of image details.
[0027] Step 3, Image Generation Module; As shown in Table 2, the input to the image generation module mainly consists of the 1024-layer feature map output from step 1 and the single-channel feature map output from step 2, i.e., a 1025-channel feature map. Meanwhile, to improve the model's ability to reconstruct features at different scales, the corresponding feature maps extracted from the first and third pooling operations in step 1 are directly mapped and connected to the feature maps from the first and third upsampling operations, respectively.
[0028]
[0029] Table 2 Step 4: Key Point Extraction Module; First, the original image is used as the standard image, and the generated image is used as the image to be detected. The same keypoint extraction algorithm is used to obtain keypoint vectors and object descriptor vectors. The keypoint vectors are then re-encoded using multi-layer one-dimensional convolution to obtain higher-dimensional features. The implementation steps are as follows: ; in, It is the updated keypoint vector. The original keypoint vector, express The i-th element of the vector, where is the key point number of the original image, and n is the total number of key points. For the parameters of a one-dimensional convolution, subscript Representing vectors The first in Each element.
[0030] Finally, the descriptor vector is normalized to reduce computation and added to the original keypoint vector to obtain a tensor.
[0031] Step 5: Graph network processing module; Step 4 will yield a new descriptor tensor. and As input, it is fed into a graph network, and the output is a matching description matrix. and matching description matrix By utilizing the attention mechanism of graph networks, key point features that differ significantly between the images to be detected and the standard images are selected, resulting in a more robust matching description matrix.
[0032] The matching description matrix output by the self-attention map network and matching description matrix The input is passed through the attention graph network, and the output of the attention graph network is the matching description matrix. and matching description matrix Cross-attention map networks compare key points in the image to be detected and the standard image, selecting key points that show significant differences between the two images.
[0033] Step 6, Fraction Calculation Module; The output after processing by the self-attention and cross-attention methods described in step 5 is the matching description matrix. and Each keypoint corresponds to a matching descriptor vector in the matrix. The two matching descriptor matrices are then... and One-dimensional convolutional encoding is performed separately (the formula for one-dimensional convolutional encoding is the same as that in step three). Through encoding, a score matrix with higher confidence can be obtained. and The two matrices are then multiplied to obtain a matching score matrix. Elements in the matching score matrix represent the matching scores between keypoints in the standard image and keypoints in the image to be detected. A higher matching score indicates smaller differences between the keypoints.
[0034] The formula for calculating the matching score matrix is:
[0035] If the standard image contains One key point is that the image to be detected contains... There are 1 key points, and the descriptor vector corresponding to each key point is 1. Wei, at this time Dimensions , Dimensions The matching score matrix has a dimension of .
[0036] Please refer to Figure 2 The method in this embodiment may include: obtaining a tensor set based on a first extraction algorithm and a first resolution image, including: Step S201: Based on the first extraction algorithm and the first resolution image, obtain the key point vector and descriptor vector; Step S202: Obtain the tensor set based on the key point vector and descriptor vector.
[0037] The method in this embodiment may include: obtaining a tensor set based on keypoint vectors and descriptor vectors, including: normalizing the descriptor vectors and adding them to the keypoint vectors to obtain the tensor set.
[0038] The method in this embodiment may include: obtaining key point vectors and descriptor vectors based on a first extraction algorithm and a first resolution image, including: the formula is: the steps are: ; in, It is the updated keypoint vector. The original keypoint vector, express The i-th element of the vector, where Here, n represents the keypoint index of the original image, and n is the total number of keypoints. For the parameters of a one-dimensional convolution, subscript Representing vectors The first in Each element.
[0039] Please refer to Figure 3 The method in this embodiment may include: processing the tensor set through a first graph network model to obtain a first description matrix and a second description matrix, including: Step S301: Process the tensor set using the first graph network model to obtain distinguishable feature points; Step S302: Obtain the first description matrix and the second description matrix based on the distinguishing feature points.
[0040] Please refer to Figure 4 The method in this embodiment may include: obtaining a matching matrix based on a first description matrix and a second description matrix, including: Step S401: Perform one-dimensional convolutional encoding on the first description matrix to obtain the first score matrix; Step S402: Perform one-dimensional convolutional encoding on the second description matrix to obtain the second score matrix; Step S403: Perform an inner product of the first score matrix and the second score matrix to obtain the matching matrix.
[0041] The method in this embodiment may include: after obtaining the matching matrix, obtaining the matching score based on the matching matrix.
[0042] The method in this embodiment may include: obtaining a matching score based on a matching matrix, including: the calculation formula is: .
[0043] Please refer to Figure 5 The apparatus of this embodiment may include the following structure: Acquisition unit 501 is used to acquire the original image; Analysis unit 502 is used to process the original image through a first image convolution model to obtain a first semantic and a first feature image; to calculate the first semantic through a first feature algorithm to obtain a second feature image; to obtain a first resolution image based on the first feature image and the second feature image; to obtain a tensor set based on a first extraction algorithm and the first resolution image; and to process the tensor set through a first graph network model to obtain a first description matrix and a second description matrix. The processing unit 503 is used to obtain a matching matrix based on the first description matrix and the second description matrix, and to process the image based on the matching matrix.
[0044] Please refer to Figure 6 This embodiment of the present application also discloses an electronic device, which includes at least one processor 601, and at least one memory 602 and a bus 603 connected to the processor 601; wherein the processor 601 and the memory 602 communicate with each other through the bus 603; the processor 601 is used to call program instructions in the memory 602 to execute the above-described image processing method.
[0045] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An image processing method, characterized in that, The method includes: Obtain the original image; The original image is processed by a first image convolution model to obtain a first semantic and a first feature image; The first semantic meaning is calculated using the first feature algorithm to obtain the second feature image; Based on the first feature image and the second feature image, a first resolution image is obtained; Based on the first extraction algorithm and the image at the first resolution, a tensor set is obtained; The tensor set is processed by the first graph network model to obtain the first description matrix and the second description matrix; A matching matrix is obtained based on the first description matrix and the second description matrix, and the image is processed based on the matching matrix.
2. The method according to claim 1, characterized in that, The process of obtaining a tensor set based on the first extraction algorithm and the first resolution image includes: Based on the first extraction algorithm and the first resolution image, key point vectors and descriptor vectors are obtained; The tensor set is obtained based on the keypoint vector and the descriptor vector.
3. The method according to claim 2, characterized in that, Based on the keypoint vector and the descriptor vector, the tensor set is obtained, including: The descriptor vector is normalized and added to the keypoint vector to obtain the tensor set.
4. The method according to claim 2, characterized in that, Based on the first extraction algorithm and the first resolution image, key point vectors and descriptor vectors are obtained, including: The formula is as follows: The steps are as follows: ； in, It is the updated keypoint vector. The original keypoint vector, express The i-th element of the vector, where Here, n represents the keypoint index of the original image, and n is the total number of keypoints. For the parameters of a one-dimensional convolution, subscript Representing vectors The first in Each element.
5. The method according to claim 3, characterized in that, The step of processing the tensor set through the first graph network model to obtain the first description matrix and the second description matrix includes: The tensor set is processed using a first graph network model to obtain distinguishable feature points; Based on the distinguishing feature points, the first description matrix and the second description matrix are obtained.
6. The method according to claim 5, characterized in that, Based on the first description matrix and the second description matrix, a matching matrix is obtained, including: Perform one-dimensional convolutional encoding on the first description matrix to obtain the first score matrix; The second description matrix is subjected to one-dimensional convolutional encoding to obtain the second score matrix; The matching matrix is obtained by performing an inner product of the first score matrix and the second score matrix.
7. The method according to claim 6, characterized in that, After obtaining the matching matrix, the process includes: Based on the matching matrix, the matching score is obtained.
8. The method according to claim 6, characterized in that, Based on the matching matrix, a matching score is obtained, including: The calculation formula is: .
9. A processing apparatus, characterized in that, The device includes: an acquisition unit, an analysis unit, and a processing unit. The acquisition unit is used to acquire the original image; The analysis unit is configured to process the original image using a first image convolution model to obtain a first semantic and a first feature image; calculate the first semantic using a first feature algorithm to obtain a second feature image; obtain a first resolution image based on the first feature image and the second feature image; obtain a tensor set based on a first extraction algorithm and the first resolution image; and process the tensor set using a first graph network model to obtain a first description matrix and a second description matrix. The processing unit is configured to obtain a matching matrix based on the first description matrix and the second description matrix, and process the image based on the matching matrix.
10. An electronic device, characterized in that, include: Memory, used to store at least one set of instructions; Processor, used to acquire the raw image; The original image is processed by a first image convolution model to obtain a first semantic and a first feature image; The first semantic meaning is calculated using the first feature algorithm to obtain the second feature image; Based on the first feature image and the second feature image, a first resolution image is obtained; Based on the first extraction algorithm and the image at the first resolution, a tensor set is obtained; The tensor set is processed by the first graph network model to obtain the first description matrix and the second description matrix; A matching matrix is obtained based on the first description matrix and the second description matrix, and the image is processed based on the matching matrix.