Image feature extraction method and device based on quantum computing operation improvement, and image classification, object detection and image segmentation methods

Abstract

The invention discloses an improved image feature extraction method and device based on quantum computing operation, which can be used for downstream tasks such as image classification, object detection, image segmentation and the like, and comprises the following steps: constructing a ViT model infrastructure adaptive to the quantum computing operation; the basic framework of the ViT model is improved; conventional characteristics are converted into quantum state representation based on quantum random characteristics; integrating quantum state calculation into a self-attention mechanism of the ViT model by utilizing the influence of quantum conjugate calculation and quantum state superposition on information interaction; a quantized linear conditional random field is introduced to improve the ViT model; quantum state feature enhancement, quantization self-attention mechanism optimization and quantization linear conditional random field improvement jointly act on the image feature extraction process of the ViT model. Innovative fusion of the quantum calculation and the computer vision field is realized, and unique advantages of the quantum calculation are introduced into an image feature extraction task, so that the image feature extraction task can be better coped with a complex image feature extraction task.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to an image feature extraction method, device, image classification, object detection and image segmentation method improved based on quantum computing operations. Background Art

[0002] Image feature extraction is a key task in computer vision, and plays a decisive role in the performance of various downstream tasks such as image classification, object detection, image segmentation, etc. Traditional image feature extraction methods are mainly based on manual features, such as edge detection, texture analysis, and color histograms. Although these methods can describe the characteristics of images to a certain extent, they rely on manually designed feature descriptors, have great limitations, and are difficult to handle complex image content and scene changes.

[0003] With the development of deep learning, convolutional neural networks (CNNs) have gradually become the mainstream method for image feature extraction. However, CNNs have limitations in processing long-distance dependencies and processing sequences of variable lengths. To solve these problems, the Transformer architecture was introduced into the field of computer vision. Its core is the self-attention mechanism, which enables the model to focus on the dependencies between different positions in the input sequence, and has powerful expression and parallel computing capabilities. The Visual Transformer (ViT) model is a successful attempt to apply the Transformer architecture to image tasks. It divides images into small blocks and converts them into sequences, and processes these sequences through the self-attention mechanism to extract features.

[0004] Although the ViT model has achieved remarkable results, it still has some shortcomings. First, in terms of position information processing, it mainly relies on a simple position encoding strategy, and the capture of the local and global spatial relationship of the image is superficial, making it difficult to deeply explore the complex relationships, resulting in certain limitations in the model's understanding of the spatial structure of the image. Secondly, in the feature extraction process, there is a lot of information loss, especially when facing complex texture and structure images, the extracted features are still insufficient in accuracy and representativeness, which affects the performance of downstream tasks. In addition, when processing images of different resolutions, scenes and types, the performance stability of the traditional ViT model is low, and the adaptability and generalization ability need to be further enhanced to meet the diverse visual task requirements.

[0005] Quantum computing technology has developed rapidly in recent years, and its unique advantages in processing complex computing tasks have brought new inspiration to deep learning algorithms in the field of computer vision. The parallelism and superposition of quantum computing have great potential, and are expected to reshape the image feature extraction model architecture, optimize the feature processing process, and make up for the shortcomings of the existing ViT model. However, quantum computing technology is still in its early stages of development. How to apply quantum computing to the field of computer vision and improve its computing performance is an important issue that needs to be solved in the interdisciplinary integration research of quantum computing and computer vision. Summary of the invention

[0006] The purpose of the present invention is to propose an image feature extraction method and device based on quantum computing operation improvement to address the problems existing in the prior art, so as to overcome the limitations of the ViT model in image feature extraction, improve the accuracy and effectiveness of feature extraction, and further improve the performance of downstream image analysis tasks such as image classification, object detection, and image segmentation.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: An improved image feature extraction method based on quantum computing operation, the method comprising: Build a ViT model infrastructure that adapts to quantum computing operations and improve the ViT model infrastructure in the following ways: Quantum state feature enhancement: converting conventional features into quantum state representation based on quantum random properties; Optimization of the quantized self-attention mechanism: Incorporate quantum state computing into the self-attention mechanism of the ViT model by utilizing the influence of quantum conjugate computing and quantum state superposition on information interaction; Quantized linear conditional random field improvement: Quantized linear conditional random field is introduced to improve the ViT model; The quantum state feature enhancement, quantized self-attention mechanism optimization and quantized linear conditional random field improvement work together in the feature extraction process of the ViT model for the image.

[0008] In the above-mentioned image feature extraction method improved based on quantum computing operation, the construction of the ViT model infrastructure adapted to quantum computing operation includes: Build a visual Transformer (ViT) model consisting of multiple stacked Transformer encoder layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network, and has a reserved quantum operation interface; The calculation formula of the multi-head self-attention mechanism is as follows: in, represents the query matrix, represents the key matrix, represents the value matrix, represents the dimension of the key matrix; Introducing quantum parallelism factors for attention weights , the adjusted attention weight The calculation is as follows: in , which is initially determined by the parallel resources and the complexity of the image processing task, and will be dynamically adjusted according to the model training situation. If the computing device has abundant parallel resources and the image processing task is complex (such as high-resolution image classification, multi-category object detection, etc.), The initial value of is set close to 1; otherwise, Set it low, close to 0. During the training process, the loss value of the validation set and the change of GPU idle rate will be adjusted. Dynamic updates.

[0009] In the above-mentioned image feature extraction method based on quantum computing operation improvement, quantum state feature enhancement specifically includes: Generate phase factors based on quantum random properties: Generate random phases based on the principle of quantum random number generation , , Represents the feature dimension of the feature vector; Calculate the phase factor ; The general features Multiplying with the phase factor, we get the quantum state characteristics in It represents element-by-element multiplication, and the real part is taken as the initial quantum state enhancement feature.

[0010] In the above-mentioned image feature extraction method based on quantum computing operation improvement, the optimization of the quantized self-attention mechanism includes: Calculate the query vector based on the quantum state conjugation principle Conjugation ; use With key vector Perform matrix multiplication and then add the scaling factor Multiply them together to get the attention score matrix : in is the dimension of each head; The attention weight matrix is ​​obtained through the Softmax function , the formula is: in Represents the row index, Represents the column index, yes The number of columns in the matrix. This process simulates the effect of quantum state superposition on information interaction, aiming to improve the accuracy and effectiveness of the self-attention mechanism, so that the model can focus on key information more accurately when processing image features.

[0011] In the above-mentioned image feature extraction method improved based on quantum computing operation, the quantized linear conditional random field training process includes: Construct a conditional probability model: in, represents the input image, represents the extracted features, is the normalization factor, is a characteristic function, used to describe the local feature relationship; The conditional probability model is optimized using a loss function. A custom loss function is used to optimize the conditional probability model so that the extracted features are more consistent with the actual feature distribution of the image. Additional regularization terms, such as L1 or L2 regularization, can be added to the loss function according to actual conditions. The adjusted loss function can be expressed as: in, is the regularization parameter, A weight vector representing the conditional random field.

[0012] During training, the linear layer weights Perform the following quantum rotation operation in, It is a learnable quantum rotation angle that is continuously adjusted through training. This operation is analogous to quantum state rotation and can change the distribution of weights, thereby introducing the unique characteristics of quantum computing and bringing a new optimization dimension to the model.

[0013] Use the rotated weights With input features Perform matrix multiplication and add the bias vector , and get the quantized output: .

[0014] Through the above-mentioned loss function optimization and quantum rotation operation, the weights of the feature function and parameters such as the quantum rotation angle are adjusted according to the local and global context information of the input image, so that the model can more accurately optimize the feature extraction process, make full use of the relevant characteristics of quantum computing to improve the linear conditional random field, reduce the feature extraction loss in the feature space, and thus improve the performance of the model in image feature extraction tasks.

[0015] An image classification method based on an image feature extraction method improved by quantum computing operations, the method comprising: constructing a ViT model improved by quantum computing operations by the method, and adding a classification module to obtain an image classification model; The image classification model is trained using an image classification dataset so that it has corresponding image classification capabilities.

[0016] Specifically, a classification head is added after the "Quantized Linear Conditional Random Field Improvement" module. The classification head can adopt a fully connected layer structure, whose input is the quantized output features after the quantized linear conditional random field processing. The fully connected layer learns and distinguishes the image feature patterns of different categories, and finally outputs the category prediction of the corresponding image.

[0017] An object detection method based on an image feature extraction method improved by quantum computing operations, the method comprising: constructing a ViT model improved by quantum computing operations by the method, and adding a detection module to obtain an object detection model; The object detection model is trained using an object detection dataset so that it has corresponding object detection capabilities.

[0018] Specifically, add an object detection head after the "Quantized Linear Conditional Random Field Improvement" module. The detection head can adopt a convolutional neural network structure, which contains convolution layers, pooling layers and fully connected layers. The quantized output features are first extracted by the convolution layer with different convolution kernels to extract multi-scale semantic features, and then downsampled by the pooling layer to enhance robustness. Finally, the mapping relationship is learned by the fully connected layer to output the object location and category information.

[0019] An image segmentation method based on an image feature extraction method improved by quantum computing operation, the method comprising: constructing a ViT model improved by quantum computing operation by the method, and adding a segmentation module to obtain an image segmentation model; The image segmentation model is trained using an image segmentation dataset so that it has corresponding image segmentation capabilities.

[0020] Specifically, a segmentation head is added after the "Quantized Linear Conditional Random Field Improvement" module. The segmentation head can use a combination of convolutional layers and upsampling layers. The convolutional layer extracts local features, and the upsampling layer restores the feature map size to a size close to the original image size. Its input is the quantized output features after quantized linear conditional random field processing, and the output is the segmentation mask of the image, which assigns category labels to the original image pixels and distinguishes different target areas.

[0021] An image feature extraction device improved based on quantum computing operations, the device comprising a model architecture building module, a self-attention optimization module, a feature enhancement module and a quantized linear conditional random field module; Model architecture building module, used to build the ViT model infrastructure adapted to quantum computing operations; Self-attention optimization module, which is used to integrate quantum conjugate calculation and the influence of quantum state superposition on information interaction into the self-attention mechanism of the ViT model; A feature enhancement module for converting conventional features into quantum state representations based on quantum random properties; The quantized linear conditional random field module is used to introduce quantized linear conditional random fields into the ViT model.

[0022] In the above-mentioned image feature extraction device improved based on quantum computing operation, the quantized linear conditional random field training process includes: Construct a conditional probability model: in, represents the input image, represents the extracted features, is the normalization factor, is a characteristic function, used to describe the local feature relationship; Use loss function to optimize the conditional probability model; During training, the linear layer weights Perform the following quantum rotation operation in, It is a learnable quantum rotation angle that is continuously adjusted through training; Use the rotated weights With input features Perform matrix multiplication and add the bias vector , and get the quantized output: .

[0023] The advantages of the present invention are: 1. This solution provides a solid framework for the integration of quantum computing and the ViT model by building a ViT model infrastructure that is adapted to quantum computing. It gives full play to the advantages of quantum computing and fundamentally improves the performance potential of the ViT model, enabling it to better cope with complex image feature extraction tasks. 2. This solution greatly enriches the expression dimension of features through quantum state feature enhancement operations, enabling the model to capture subtle feature information that is difficult to detect with traditional methods, effectively enhancing the model's ability to understand image content, and providing a richer and more valuable data foundation for subsequent analysis and processing; 3. The quantized self-attention mechanism optimization proposed in this scheme significantly improves the information focusing ability and interaction accuracy of the model when processing image features, reduces the interference of irrelevant information, improves the quality and efficiency of feature extraction, and makes the extracted features more discriminative and representative; 4. The quantized linear conditional random field improvement proposed in this scheme effectively uses operations such as quantum rotation to optimize the feature extraction process, reduce information loss, enhance the model's modeling ability of feature space, and improve the generalization performance of the model, making it more stable and outstanding in different data sets and task scenarios; 5. This solution achieves an innovative integration of quantum computing and computer vision, introduces the unique advantages of quantum computing into image feature extraction tasks, and helps promote the progress of related cross-disciplinary research. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A method flow chart of an improved image feature extraction method based on quantum computing operation of the present invention; Figure 2 A flowchart of the application of the improved image feature extraction method based on quantum computing operation of the present invention to downstream tasks of image classification tasks; Figure 3 The training process of the image feature extraction method improved by the quantum computing operation of the present invention on the CIFAR100 data set; Figure 4 This is a comparison chart of the accuracy of the image feature extraction method improved by the present invention based on quantum computing operation and other feature extraction methods in the control group on the CIFAR100 data set; Figure 5 This is a comparison chart of the computational consumption of the image feature extraction method improved based on quantum computing operations in the present invention and other feature extraction methods in the control group on the CIFAR100 dataset. DETAILED DESCRIPTION

[0025] This solution provides an improved image feature extraction method based on quantum computing operations, which can be used for downstream tasks such as image classification, object detection, and image segmentation. This embodiment takes the image classification task as an example to introduce this solution. Figure 1 As shown, firstly, an image feature extraction module is built based on the proposed method, and then an image classification model is established by introducing a classification module. Finally, it is compared and analyzed with several traditional feature extraction methods. Among them, the image feature extraction module of this embodiment is common to image classification, object detection, and image segmentation tasks. For different downstream tasks, only the corresponding data sets and post-modules need to be replaced, which will not be described here.

[0026] The specific steps of this embodiment are as follows: S1.CIFAR100 is selected as the standard dataset, which consists of 60,000 color images divided into 100 categories, each category contains 600 images.

[0027] S2. Select some feature extraction methods for comparison with the proposed method: In addition to the traditional feature extraction methods SIFT and HOG, a classic feature extraction method based on neural network VGG16 is also included. The above three methods are added to the control group. At the same time, the difference between the ViT improved by the proposed method and the unimproved ViT is also examined.

[0028] Among them, the SIFT algorithm can detect local features in the image and describe the texture information near the feature points; HOG constructs features by calculating and counting the gradient direction histogram of the local area of ​​the image, and both use SVM for subsequent classification.

[0029] VGG16 is a deep convolutional neural network structure composed of multiple convolutional layers and pooling layers stacked alternately. The network depth is 16 and a fully connected layer is used for classification.

[0030] S3. Construct a ViT model infrastructure adapted to quantum computing operations. The basic ViT model constructed in this embodiment is composed of 6 stacked Transformer encoder layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network.

[0031] In the multi-head self-attention mechanism, the bond matrix dimension .

[0032] When calculating the attention weight, for the CIFAR100 dataset, considering that the complexity of the dataset is at a moderate level and the parallel computing resources used in this embodiment are relatively abundant, the quantum parallel factor is initially set to 0.5. After every 10 training batches, the average loss value on the validation set is calculated. If the validation set loss value does not decrease for 3 consecutive times and the current GPU idle rate is greater than 30%, the quantum parallel factor is increased by a fixed value of 0.05 to try to use more parallel resources to improve model performance; otherwise, it is reduced by 0.05 to reduce the computational burden and stabilize model training.

[0033] S4. Figure 2 As shown, the ViT model built in S3 is optimized based on the proposed method.

[0034] Quantum state feature enhancement: For the feature vector of the input image after preliminary processing, its feature dimension is .

[0035] According to the principle of quantum random number generation, a random phase is generated for each dimension , . Calculate the phase factor , and then the regular features Multiply the phase factor element by element, and take the real part as the initial quantum state enhancement feature. This quantum state feature enhancement method can significantly enrich the diversity and expressiveness of features, enabling the model to capture more complex and subtle image feature information, providing reliable support for subsequent feature processing and analysis.

[0036] Optimization of quantized self-attention mechanism: For the query vector , according to the quantum state conjugation principle to calculate its conjugation ; The dimensions of each head ,use With key vector Perform matrix multiplication and then multiply by the scaling factor , get the attention score matrix , and then through Function gets the attention weight matrix .

[0037] Quantized linear conditional random field improvements: Building a conditional probability model , in the CIFAR100 dataset, the image size is , for the characteristic function ,Taking into account the fact that adjacent pixels in CIFAR100 images usually have a high correlation, and combining the rule that the correlation of diagonally adjacent pixels is slightly weaker and the correlation of non-adjacent pixels is even weaker, a weight relationship that can reflect the pixel position is set: the feature relationship weight for horizontally or vertically adjacent pixels is initially set to 0.8, the feature relationship weight for diagonally adjacent pixels is set to 0.6, and the feature relationship weight for non-adjacent pixels is set to 0.3.

[0038] Then use a custom loss function Optimize the conditional probability model and regularize the parameters During training, the linear layer weights for the quantized linear conditional random field , the initial quantum rotation angle Set to 0.1 and will be adjusted as the training progresses. Use the rotated weights With input features Perform matrix multiplication and add the bias vector , and get the quantized output: .

[0039] Introduction of classification module: After establishing the feature extraction method and obtaining the quantized output, the ViT overall structure is connected to the classification head module. In this embodiment, since the CIFAR100 dataset has 100 categories, when setting the fully connected layer of the classification head, the feature dimension of the quantized output is mapped to 100 dimensions, and the above loss function is used to train and optimize the entire model including feature extraction and classification heads to complete the classification task.

[0040] S5. During the training process, the learning rate is set to 0.001, the batch size is 64, the number of training rounds is 50, and the performance is evaluated on the CIFAR100 dataset. In the specific experiment, after each round of training, the performance of the model on the test set is evaluated, and performance indicators such as accuracy and recall are statistically analyzed. Figure 3 The results show that after 20 rounds of training, the accuracy rate increased by about 0.3.

[0041] S6. The proposed optimized ViT model is compared with VGG16, SIFT, HOG, and traditional ViT feature extraction methods. The settings in S5 are kept unchanged and retrained to obtain the accuracy, time consumption, and feature representation distribution of each model in the feature space.

[0042] from Figure 3 From the optimized ViT model training performance graph shown, it can be seen that as the number of training rounds increases, the training loss gradually decreases and the verification accuracy gradually increases, indicating that the model is constantly being optimized.

[0043] Depend on Figure 4 It can be seen that the ViT model optimized by the proposed method finally achieved an accuracy of 59.32% after 50 epochs of training, which is about 2% to 3% higher than the accuracy of 56.45% of the traditional ViT model, which is quite impressive on the CIFAR100 standard dataset. This proves the feasibility of the ViT model improved by quantum computing in image classification tasks.

[0044] Depend on Figure 5 It can be seen that by comparing the computational time of the method of the present invention with that of each control group on the CIFAR100 dataset, the proposed methods are superior to traditional methods such as VGG16, HOG, and SIFT; and no excessive additional resource consumption is generated on the basis of traditional ViT, indicating that the method of the present invention has achieved better performance within a reasonable range of computational resource consumption.

[0045] It can be seen that this scheme effectively enhances the diversity and expressiveness of image features through the coordinated optimization of quantum computing and linear conditional random fields, optimizes the information interaction efficiency in the self-attention mechanism, reduces information loss in the feature extraction process, improves the accuracy of feature extraction, and ultimately improves the performance of the model in image analysis tasks.

[0046] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. An image feature extraction method based on quantum computing operation improvement, characterized in that: The method includes: Build a ViT model infrastructure that adapts to quantum computing operations and improve the ViT model infrastructure in the following ways: Quantum state feature enhancement: converting conventional features into quantum state representation based on quantum random properties; Optimization of the quantized self-attention mechanism: Incorporate quantum state computing into the self-attention mechanism of the ViT model by utilizing the influence of quantum conjugate computing and quantum state superposition on information interaction; Quantized linear conditional random field improvement: Quantized linear conditional random field is introduced to improve the ViT model; The quantum state feature enhancement, quantized self-attention mechanism optimization and quantized linear conditional random field improvement work together in the feature extraction process of the ViT model for the image.

2. The image feature extraction method based on quantum computing operation improvement according to claim 1 is characterized in that: The ViT model infrastructure for quantum computing operations includes: Build a ViT model consisting of multiple stacked Transformer encoder layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network, and has a reserved quantum operation interface; The calculation formula of the multi-head self-attention mechanism is as follows: in, represents the query matrix, represents the key matrix, represents the value matrix, represents the dimension of the key matrix; Introducing quantum parallelism factors for attention weights , the adjusted attention weight The calculation is as follows: in , updated with training.

3. The image feature extraction method based on quantum computing operation improvement according to claim 1 is characterized in that: The quantum state feature enhancement specifically includes: Generate phase factors based on quantum random properties: Generate random phases based on the principle of quantum random number generation , , Represents the feature dimension of the feature vector; Calculate the phase factor ; The general features Multiplying with the phase factor gives the quantum state characteristics , in It represents element-by-element multiplication, and the real part is taken as the initial quantum state enhancement feature.

4. The image feature extraction method based on quantum computing operation improvement according to claim 1 is characterized in that: Optimization of the quantized self-attention mechanism includes: Calculate the query vector based on the quantum state conjugation principle Conjugation ; use With key vector Perform matrix multiplication and then add the scaling factor Multiply them together to get the attention score matrix : in is the dimension of each head; The attention weight matrix is ​​obtained through the Softmax function , the formula is: in Represents the row index, Represents the column index, yes The number of columns in the matrix.

5. The image feature extraction method based on quantum computing operation improvement according to claim 1 is characterized in that: The quantized linear conditional random field training process includes: Construct a conditional probability model: in, represents the input image, represents the extracted features, is the normalization factor, is a characteristic function, used to describe the local feature relationship; Use loss function to optimize the conditional probability model; During training, the linear layer weights Perform the following quantum rotation operation in, It is a learnable quantum rotation angle that is continuously adjusted through training; Use the rotated weights With input features Perform matrix multiplication and add the bias vector , and get the quantized output: .

6. An image classification method based on an improved image feature extraction method using quantum computing operations, characterized in that: The method includes: Constructing a ViT model improved based on quantum computing operations by the method described in any one of claims 1 to 5, and adding a classification module to obtain an image classification model; The image classification model is trained using an image classification dataset so that it has corresponding image classification capabilities.

7. An object detection method based on an improved image feature extraction method using quantum computing operations, characterized in that: The method includes: Constructing a ViT model improved based on quantum computing operations by the method described in any one of claims 1 to 5, and adding a detection module to obtain an object detection model; The object detection model is trained using an object detection dataset so that it has corresponding object detection capabilities.

8. An image segmentation method based on an improved image feature extraction method using quantum computing operations, characterized in that: The method includes: Constructing a ViT model improved based on quantum computing operations by the method described in any one of claims 1 to 5, and adding a segmentation module to obtain an image segmentation model; The image segmentation model is trained using an image segmentation dataset so that it has corresponding image segmentation capabilities.

9. An image feature extraction device improved based on quantum computing operation, characterized in that: The device includes a model architecture building module, a self-attention optimization module, a feature enhancement module, and a quantized linear conditional random field module; Model architecture building module, used to build the ViT model infrastructure adapted to quantum computing operations; Self-attention optimization module, which is used to integrate quantum conjugate calculation and the influence of quantum state superposition on information interaction into the self-attention mechanism of the ViT model; A feature enhancement module for converting conventional features into quantum state representations based on quantum random properties; The quantized linear conditional random field module is used to introduce quantized linear conditional random fields into the ViT model.

10. The image feature extraction device improved based on quantum computing operation according to claim 9, characterized in that: The quantized linear conditional random field training process includes: Construct a conditional probability model: in, represents the input image, represents the extracted features, is the normalization factor, is a characteristic function, used to describe the local feature relationship; Use loss function to optimize the conditional probability model; During training, the linear layer weights Perform the following quantum rotation operation in, It is a learnable quantum rotation angle that is continuously adjusted through training; Use the rotated weights and input features Perform matrix multiplication and add the bias vector , and get the quantized output: 。

Images