Visual language corresponding AI generated panoramic image quality evaluation method and system

Abstract

The invention provides a visual language corresponding AI generation panoramic image quality evaluation method and system, and the method comprises the steps: obtaining an AI generation panoramic image, and carrying out the sampling of the AI generation panoramic image; based on the image block set, utilizing a visual encoder to carry out feature representation on the image blocks; performing feature representation on text description attached to the panoramic image generated by the AI by using a language encoder; l2 normalization processing and cosine similarity calculation are sequentially carried out on the visual features of the image blocks and the text features of the text description; and processing the fused feature vector by using a full-connection network and a normalization function. According to the method, visual language corresponding analysis, the AI generated panoramic image and the corresponding text description are used, linkage analysis of the visual language corresponding analysis and the AI generated panoramic image and the corresponding text description is carried out, and the learned visual language corresponding relation is utilized, so that the quality score of the AI generated panoramic image is efficiently and accurately predicted.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and multimedia digital image processing, and in particular to a method and system for evaluating the quality of panoramic images generated by AI based on visual language correspondence. Background Art

[0002] The popularity of artificial intelligence (AI) technology, especially generative AI and large language models (LLMs), has had a significant impact on all aspects of human life. Among these developments, AI-generated images (AGIs) are created by adopting text-to-image (T2I) generative models, including generative adversarial networks, variational autoencoders, and diffusion-based models. AI-generated omnidirectional images (AGOIs) are a specific category of AGI, which are promising in virtual reality (VR) applications. Compared with two-dimensional images, AGOIs have the properties of generated content and omnidirectional images. However, AGOIs currently also have some problems, such as unrealistic structures and inappropriate combinations. Therefore, objective methods are urgently needed to evaluate the quality of AGOIs.

[0003] Recent studies have leveraged contrastive language-image pre-trained models to enhance AGOI quality assessment by emulating human preferences and evaluating visual-text correspondence. However, these methods have significant limitations. First, most methods for predicting human preferences for AGOI lack a comprehensive sense of quality and fail to consider both perception and alignment. Second, although measuring text-to-image alignment can indicate consistency, these methods heavily rely on task-specific prompting strategies (e.g., antonym prompt pairings, text words, and text templates), which limits their generalization ability and hinders unprompted image generation tasks. Current methods typically ignore the correlations and interactions between features from different modalities, resulting in unsatisfactory performance. Summary of the invention

[0004] In view of the above situation, the main purpose of the present invention is to propose an AI-generated panoramic image quality evaluation method and system based on visual language correspondence to solve the above technical problems.

[0005] The present invention proposes a method for evaluating the quality of panoramic images generated by AI based on visual language correspondence, the method comprising the following steps: Step 1: Build a visual encoder and a language encoder based on the Transformer architecture, build a multimodal feature fusion module based on the feature fusion mechanism, and the language encoder, visual encoder and multimodal feature fusion module constitute a quality evaluation model; Step 2: Obtain an AI-generated panoramic image, and sample the AI-generated panoramic image to obtain an image block set; Step 3: Based on the image block set, a visual encoder is used to represent the image block to obtain the visual features of the image block; Step 4: Use a language encoder to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description; Step 5: Perform L2 normalization and cosine similarity calculation on the visual features of the image block and the text features of the text description in turn to obtain a fused feature vector; Step 6: Use the fully connected network and normalization function to process the fused feature vector to obtain the prediction quality score; Based on the predicted quality score, mean absolute error loss and Pearson correlation induced loss are constructed, and the quality evaluation model is optimized using mean absolute error loss and Pearson correlation induced loss to obtain the optimized quality evaluation model; The evaluation results are obtained using the optimized quality evaluation model.

[0006] The present invention also proposes an AI-generated panoramic image quality evaluation system based on visual language correspondence, the system comprising: Random block sampling module for: Based on the Transformer architecture, a visual encoder and a language encoder are constructed. Based on the feature fusion mechanism, a multimodal feature fusion module is constructed. The language encoder, the visual encoder and the multimodal feature fusion module constitute a quality evaluation model. Acquire an AI-generated panoramic image, and sample the AI-generated panoramic image to obtain an image block set; Visual feature extraction module for: Based on the image block set, a visual encoder is used to represent the image block to obtain the visual features of the image block; Text prompt embed module: Using a language encoder to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description; Multimodal feature fusion module, used for: The visual features of the image block and the text features of the text description are subjected to L2 normalization and cosine similarity calculation in turn to obtain a fused feature vector; Quality regression module for: The fused feature vector is processed using a fully connected network and a normalization function to obtain a prediction quality score; Based on the predicted quality score, mean absolute error loss and Pearson correlation induced loss are constructed, and the quality evaluation model is optimized using mean absolute error loss and Pearson correlation induced loss to obtain the optimized quality evaluation model; The evaluation results are obtained using the optimized quality evaluation model.

[0007] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention is based on a random block sampling method that is independent of the viewport. While efficiently extracting visual features, it retains local texture details and takes into account the overall spatial structure, thereby reflecting the quality of the original image to the greatest extent and greatly improving the efficiency of visual feature extraction and calculation; 2. By introducing causal mask matrix and position encoding into the language encoder, the present invention not only ensures the autoregressive generation characteristics of the text sequence but also accurately models the word order dependency, thereby significantly improving the alignment accuracy of text features and visual features, and further enhancing the semantic relevance of quality evaluation; 3. The present invention uses visual language correspondence analysis to analyze the AI-generated panoramic images and their corresponding text descriptions, and uses the learned visual language correspondence to efficiently and accurately predict the quality score of the AI-generated panoramic images.

[0008] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a flow chart of the AI-generated panoramic image quality evaluation method based on visual language correspondence proposed by the present invention; Figure 2 This is an algorithm framework diagram of the AI-generated panoramic image quality evaluation method based on visual language correspondence proposed by the present invention; Figure 3 This is a system framework diagram of the AI-generated panoramic image quality evaluation system based on visual language correspondence proposed in the present invention. DETAILED DESCRIPTION

[0010] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0011] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0012] See also Figure 1 This embodiment provides a method for evaluating the quality of panoramic images generated by AI based on visual language, and the method comprises the following steps: Step 1: Construct a visual encoder and a language encoder based on the Transformer architecture, and construct a multimodal feature fusion module based on the feature fusion mechanism. The language encoder, visual encoder and multimodal feature fusion module constitute a quality evaluation model.

[0013] Step 2: Obtain an AI-generated panoramic image, and sample the AI-generated panoramic image to obtain a set of image blocks.

[0014] See also Figure 2 In step 2, the panoramic image generated by AI is obtained, and the panoramic image generated by AI is sampled to obtain a set of image blocks, which specifically includes the following sub-steps: Randomly sample blocks in the AI-generated panoramic image in ERP format to obtain a set of image blocks , the corresponding process has the following relationship: ; in, Indicates image blocks and , represents a random sampling operation, Indicates that the sampled AI generates a panoramic image and , Indicates The spatial coordinates of the upper left corner of the image block, represents the height of the image block, Indicates the width of the image block, represents the index of the image block, Indicates the image height, Indicates the image width, Indicates the number of image channels, Indicates the total number of image blocks.

[0015] It should be noted that the attached Figure 2 middle Represents matrix multiplication.

[0016] Furthermore, the present invention samples the AI-generated panoramic image based on a random block sampling method that is independent of the viewport. In order to ensure that the sampling range remains within the image boundary, the spatial coordinates of the upper left corner of each image block are The following requirements must be met: ; in, Indicated in and Uniform distribution within the range.

[0017] This embodiment is carried out on the AIGCOIQA2024 database, which is the first subjective database of AGOI (AI generated panoramic images) and consists of 300 AGOIs generated by five AIGC models based on 25 text prompts. Human visual preferences for each AGOI are collected from three perspectives: quality, comfort, and correspondence, and all preferences range from 0 to 100. The database is divided into 60% for training and 40% for testing according to the text prompts. In order to reduce the deviation caused by random segmentation, it was repeated ten times and the average effect was taken. The random sampling strategy adopted in the present invention can effectively capture the spatial and texture quality of the image, while significantly improving the model efficiency and minimizing the computational cost and model complexity.

[0018] Step 3: Based on the image block set, a visual encoder is used to represent the image blocks to obtain visual features of the image blocks.

[0019] In step 3, based on the image block set, the visual encoder is used to represent the image block to obtain the visual features of the image block. The present invention uses the VisionTransformer feature encoder as the visual encoder. The Image blocks When performing feature extraction, the following sub-steps are included: The image block is flattened into a vector and mapped to a high-dimensional embedding space to obtain a vector of the image block mapped to the high-dimensional embedding space. The following relationship exists in the corresponding process: ; in, Indicates The image patches are mapped to vectors in a high-dimensional embedding space. represents the linear projection matrix, Represents the bias term.

[0020] The vectors of the image blocks mapped to the high-dimensional embedding space are configured with learnable position encoding to construct a position encoding matrix. The position encoding matrix is ​​used to embed the vectors of the image blocks mapped to the high-dimensional embedding space to obtain an embedding vector that integrates the position information. The following relationship exists in the corresponding process: ; in, The first The embedding vector corresponding to the image block is Indicates The position code corresponding to each image block.

[0021] It should be noted that the learnable position encoding is added to preserve the position information of the image blocks.

[0022] The embedded vector of the fused position information is sent to the Trnsformaer encoder for feature extraction to obtain the visual features of the image block. The following relationship exists in the corresponding process: ; in, Indicates The visual features of the image blocks, Indicates that the parameter is Transformer feature encoder.

[0023] Step 4: Use a language encoder to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description.

[0024] In step 4, a language encoder is used to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description. The present invention uses GPT-2 as a language encoder, and its generative architecture is more suitable for open text descriptions. The process of extracting features by the language encoder specifically includes the following sub-steps: Decompose the text descriptions attached to the AI-generated panoramic images (e.g., “The panorama shows a dining area with a classic chandelier and a round table, next to a cozy living room with a fireplace and comfortable chairs.”) to get token, the following relationship exists in the corresponding process: ; in, It represents the text description attached to the AI-generated panoramic image. Token represents the text processing unit. Indicates the total number of tokens. Both represent tokens.

[0025] The embedding matrix is ​​used to map the token to a high-dimensional space to obtain the word vector. The following relationship exists in the corresponding process: ; in, represents the embedding matrix and , represents the embedding matrix operation, Indicates Tokens, Indicates The word vector corresponding to the token, Indicates the index of the token. represents the vocabulary size, Represents the embedding dimension.

[0026] The word vector is configured to be positionally encoded to construct a position encoding matrix, and the word vector is embedded using the position encoding matrix to obtain a word vector fused with position encoding. The following relationship exists in the corresponding process: ; in, Indicates the fusion position encoding The word vector corresponding to the token, Indicates The position code corresponding to the token and .

[0027] The word vectors of the fused positional encoding are integrated to obtain the input sequence of the fused positional encoding. The following relationship exists in the corresponding process: ; in, represents the input sequence fused with positional encoding, Both represent word vectors fused with positional encoding.

[0028] The input sequence of the fused positional encoding is fed into the Transformer encoder for Layer encoding is performed to obtain a feature sequence. The following relationship exists in the corresponding process: ; in, Indicates passing The feature sequence after the layer Transformer encoder is processed.

[0029] Each layer of Transformer encoder contains masked multi-head self-attention, feedforward neural network and layer normalization; When performing masked multi-head self-attention calculations, the following relationship exists in the corresponding process: ; in, Represents the causal mask matrix, ensuring that the model can only access the current and previous tokens; represents the Query matrix, represents the Key matrix, Represents the Value matrix.

[0030] The feature sequence is processed using a trainable projection matrix to obtain text features. The following relationship exists in the corresponding process: ; in, Represents text features and , Represents a trainable projection matrix.

[0031] It should be noted that the trainable projection matrix is ​​to ensure the text features With visual features Have the same feature dimensions .

[0032] Step 5: Perform L2 normalization and cosine similarity calculation on the visual features of the image block and the text features of the text description in turn to obtain a fused feature vector.

[0033] In step 5, the visual features of the image block and the text features of the text description are subjected to L2 normalization processing and cosine similarity calculation in sequence to obtain a fused feature vector, which specifically includes the following sub-steps: The cosine similarity of the visual features and the text features is calculated to obtain the cosine similarity of the visual features and the text features. The following relationship exists in the corresponding process: ; in, Indicates The cosine similarity between the visual features and text features of the image blocks, Indicates The visual features of the image blocks, Represents text features, represents the vector dot product, represents the 2-norm, Indicates the total number of image blocks.

[0034] The visual features are subjected to L2 normalization to obtain normalized visual features. The following relationship exists in the corresponding process: ; in, Represents the normalized Visual features of image patches.

[0035] The text features are subjected to L2 normalization to obtain normalized text features. The following relationship exists in the corresponding process: ; in, Represents the normalized text features.

[0036] It should be noted that after L2 normalization of the visual features and the text features in turn, cosine similarity calculation is performed on the normalized visual features and the normalized text features, which can eliminate the influence of feature amplitude differences on the cosine similarity calculation.

[0037] The cosine similarity of the normalized visual features and the normalized text features is calculated to obtain the cosine similarity of the enhanced visual features and the text features. The following relationship exists in the corresponding process: ; in, represents the cosine similarity between the enhanced visual features and text features, Represents the transposed normalized text features.

[0038] In the step of calculating the average value of the cosine similarity between the enhanced visual features and the text features to obtain the fused feature vector, the following relationship exists in the corresponding process: ; in, Represents the fused feature vector.

[0039] Step 6: Use the fully connected network and normalization function to process the fused feature vector to obtain the prediction quality score; Based on the predicted quality score, mean absolute error loss and Pearson correlation induced loss are constructed, and the quality evaluation model is optimized using mean absolute error loss and Pearson correlation induced loss to obtain the optimized quality evaluation model; The evaluation results are obtained using the optimized quality evaluation model.

[0040] In step 6, the fused feature vector is processed using a fully connected network and a normalization function to obtain a prediction quality score, which specifically includes the following sub-steps: The fused feature vector is processed using a fully connected network to obtain the output score. The following relationship exists in the corresponding process: ; in, Represents the output score of the fused feature vector after passing through the fully connected layer, Represents the weight matrix of the fully connected layer.

[0041] The normalization function is used to normalize the output score of the fused feature vector after passing through the fully connected layer to obtain the prediction quality score. The following relationship exists in the corresponding process: ; in, represents the prediction quality score, It means that it has been processed by general normalization method.

[0042] Furthermore, in order to optimize the parameters in the quality evaluation model, the present invention adopts mean absolute error loss and Pearson correlation induced loss as loss functions to balance coverage speed and prediction accuracy. The total loss is the weighted sum of mean absolute error loss and Pearson correlation induced loss. The following relationship exists in the corresponding process: ; in, represents the total loss, represents the mean absolute error loss, represents the Pearson correlation induced loss, Represents a hyperparameter.

[0043] It should be noted that Used to measure the difference between the predicted score and the true human preference score, Used to ensure linearity and monotonicity of the prediction scores, Used for balance and contribution.

[0044] The mean absolute error formula is as follows: ; in, represents the number of samples, Indicates The true value of samples (from labeled data), Indicates The predicted value of samples (from the model output); The Pearson correlation induced loss formula is as follows: ; in, represents the Pearson correlation coefficient, represents the average subjective score, represents the average objective prediction score; According to the above description, the total loss can be expressed as the following relationship: ; in, represents the mean of the true values, represents the average of the predicted values; By comparing the average prediction results with the subjective evaluation scores, we can get the various indicators of the model. The subjective evaluation scores are given by professional scorers of the AIGCOIQA2024 database builders. The test indicators include the following three types: Forecast monotonicity indicators, including the Spearman correlation coefficient (SRCC), are specifically expressed as: ; in, represents the Spearman correlation coefficient, represents the number of distorted images in the data, Indicates The difference between the subjective score of an image and the objective predicted score; Forecast accuracy indicators, including the Pearson correlation coefficient (PLCC), are specifically expressed as: ; in, Indicates The subjective score of the image, Indicates The objective prediction score of an image; Prediction ranking correlation indicators, including Kendall rank correlation coefficient (KRCC), are specifically expressed as: ; in, represents the number of rank-inconsistent pairs in paired data, represents the number of rank-consistent pairs of data, Indicates the total number of data.

[0045] Please refer to Figure 3 This embodiment further provides a panoramic image quality assessment system generated by AI based on visual language correspondence, the system applies the panoramic image quality assessment method generated by AI based on visual language correspondence as described above, and the system includes: Random block sampling module for: Based on the Transformer architecture, a visual encoder and a language encoder are constructed. Based on the feature fusion mechanism, a multimodal feature fusion module is constructed. The language encoder, the visual encoder and the multimodal feature fusion module constitute a quality evaluation model. Acquire an AI-generated panoramic image, and sample the AI-generated panoramic image to obtain an image block set; Visual feature extraction module for: Based on the image block set, a visual encoder is used to represent the image block to obtain the visual features of the image block; Text prompt embed module: Using a language encoder to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description; Multimodal feature fusion module, used for: The visual features of the image block and the text features of the text description are subjected to L2 normalization and cosine similarity calculation in turn to obtain a fused feature vector; Quality regression module for: The fused feature vector is processed using a fully connected network and a normalization function to obtain a prediction quality score; Based on the predicted quality score, mean absolute error loss and Pearson correlation induced loss are constructed, and the quality evaluation model is optimized using mean absolute error loss and Pearson correlation induced loss to obtain the optimized quality evaluation model; The evaluation results are obtained using the optimized quality evaluation model.

[0046] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0047] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for evaluating the quality of panoramic images generated by AI based on visual language correspondence, characterized in that: The method comprises the following steps: Step 1: Build a visual encoder and a language encoder based on the Transformer architecture, build a multimodal feature fusion module based on the feature fusion mechanism, and the language encoder, visual encoder and multimodal feature fusion module constitute a quality evaluation model; Step 2: Obtain an AI-generated panoramic image, and sample the AI-generated panoramic image to obtain an image block set; Step 3: Based on the image block set, a visual encoder is used to represent the image block to obtain the visual features of the image block; Step 4: Use a language encoder to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description; Step 5: Perform L2 normalization and cosine similarity calculation on the visual features of the image block and the text features of the text description in turn to obtain a fused feature vector; Step 6: Use the fully connected network and normalization function to process the fused feature vector to obtain the prediction quality score; Based on the predicted quality score, mean absolute error loss and Pearson correlation induced loss are constructed, and the quality evaluation model is optimized using mean absolute error loss and Pearson correlation induced loss to obtain the optimized quality evaluation model; The evaluation results are obtained using the optimized quality evaluation model.

2. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 1, characterized in that: In step 2, the panoramic image generated by AI is obtained, and the panoramic image generated by AI is sampled to obtain a set of image blocks. The following relationship exists in the corresponding process: ； in, Indicates image blocks, represents a random sampling operation, Indicates the sampled AI generated panoramic image, Indicates The spatial coordinates of the upper left corner of the image block, represents the height of the image block, Indicates the width of the image block, Represents the index of the image block.

3. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 2, characterized in that: In step 3, based on the image block set, a visual encoder is used to perform feature representation on the image block to obtain the visual features of the image block, which specifically includes the following sub-steps: Flatten the image block into a vector and map it to a high-dimensional embedding space to obtain a vector of the image block mapped to the high-dimensional embedding space; Perform learnable position encoding configuration on the vectors mapping the image blocks to the high-dimensional embedding space to construct a position encoding matrix, and use the position encoding matrix to perform embedding operation on the vectors mapping the image blocks to the high-dimensional embedding space to obtain an embedding vector that integrates position information; The embedded vector fused with position information is sent to the Trnsformaer encoder for feature extraction to obtain the visual features of the image block.

4. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 3, characterized in that: The image block is flattened into a vector and mapped to a high-dimensional embedding space to obtain a vector of the image block mapped to the high-dimensional embedding space. The following relationship exists in the corresponding process: ； in, Indicates The image patches are mapped to vectors in a high-dimensional embedding space. represents the linear projection matrix, represents the bias term; In the steps of configuring a learnable position encoding for the vector mapped from the image block to the high-dimensional embedding space to construct a position encoding matrix, and using the position encoding matrix to embed the vector mapped from the image block to the high-dimensional embedding space to obtain an embedding vector that fuses the position information, the following relationship exists in the corresponding process: ； in, The first The embedding vector corresponding to the image block is Indicates The position encoding corresponding to each image block; In the step of sending the embedding vector of the fused position information to the Trnsformaer encoder for feature extraction to obtain the visual features of the image block, the following relationship exists in the corresponding process: ； in, Indicates The visual features of the image blocks, Indicates that the parameter is Transformer feature encoder.

5. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 4, characterized in that: In step 4, a language encoder is used to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description, which specifically includes the following sub-steps: Decompose the text description attached to the AI-generated panoramic image to obtain Tokens; Use the embedding matrix to map the token to a high-dimensional space to obtain the word vector; Perform position encoding on the word vector to construct a position encoding matrix, and use the position encoding matrix to embed the word vector to obtain a word vector fused with the position encoding; Integrate the word vectors of the fused positional encoding to obtain the input sequence of the fused positional encoding; The input sequence of the fused positional encoding is fed into the Transformer encoder for Layer encoding processing to obtain feature sequence; The feature sequence is processed using a trainable projection matrix to obtain text features.

6. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 5, characterized in that: Decompose the text description attached to the AI-generated panoramic image to obtain token, the following relationship exists in the corresponding process: ； in, It represents the text description attached to the AI-generated panoramic image. Token represents the text processing unit. Indicates the total number of tokens. All represent tokens; In the step of using the embedding matrix to map the token to a high-dimensional space to obtain the word vector, the following relationship exists in the corresponding process: ； in, represents the embedding matrix, represents the embedding matrix operation, Indicates Tokens, Indicates The word vector corresponding to the token, Indicates the index of the token; In the steps of configuring the position encoding of the word vector to construct the position encoding matrix, and using the position encoding matrix to embed the word vector to obtain the word vector fused with the position encoding, the following relationship exists in the corresponding process: ； in, Indicates the first The word vector corresponding to the token, Indicates The position code corresponding to each token; In the step of integrating the word vectors of the fused position encoding to obtain the input sequence of the fused position encoding, the following relationship exists in the corresponding process; ； in, represents the input sequence fused with positional encoding, Both represent word vectors fused with positional encoding; When the input sequence of the fused positional encoding is input into the Transformer encoder Layer encoding is performed to obtain a feature sequence. The following relationship exists in the corresponding process: ； in, Indicates passing The feature sequence after the layer Transformer encoder is processed; In the step of processing the feature sequence using the trainable projection matrix to obtain text features, the following relationship exists in the corresponding process: ； in, Represents text features, represents the causal mask matrix, Represents a trainable projection matrix.

7. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 6, characterized in that: In step 5, the visual features of the image block and the text features of the text description are subjected to L2 normalization processing and cosine similarity calculation in sequence to obtain a fused feature vector, which specifically includes the following sub-steps: Calculate the cosine similarity between the visual feature and the text feature to obtain the cosine similarity between the visual feature and the text feature; Perform L2 normalization on the visual features to obtain normalized visual features; Perform L2 normalization on the text features to obtain normalized text features; Calculate the cosine similarity between the normalized visual features and the normalized text features to obtain the cosine similarity between the enhanced visual features and the text features; The cosine similarities of the enhanced visual features and text features are averaged to obtain the fused feature vector.

8. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 7, characterized in that: The cosine similarity of the visual features and the text features is calculated to obtain the cosine similarity of the visual features and the text features. The following relationship exists in the corresponding process: ； in, Indicates The cosine similarity between the visual features and text features of the image blocks, Indicates The visual features of the image blocks, Represents text features, represents the vector dot product, represents the 2-norm, Indicates the total number of image blocks; In the step of performing L2 normalization on the visual features to obtain normalized visual features, the following relationship exists in the corresponding process: ； in, Represents the normalized Visual features of image patches; In the step of performing L2 normalization on the text features to obtain normalized text features, the following relationship exists in the corresponding process: ； in, Represents the normalized text features; In the step of calculating the cosine similarity of the normalized visual features and the normalized text features to obtain the cosine similarity of the enhanced visual features and text features, the following relationship exists in the corresponding process: ； in, represents the cosine similarity between the enhanced visual features and text features, Represents the transpose of normalized text features; In the step of calculating the average value of the cosine similarity of the enhanced visual features and the text features to obtain the fused feature vector, the following relationship exists in the corresponding process: ； in, Represents the fused feature vector.

9. The method for evaluating the quality of panoramic images generated by AI based on visual language correspondence according to claim 8, characterized in that: In step 6, the fused feature vector is processed using a fully connected network and a normalization function to obtain a prediction quality score, which specifically includes the following sub-steps: The fused feature vector is processed using a fully connected network to obtain the output score. The following relationship exists in the corresponding process: ； in, Represents the output score of the fused feature vector after passing through the fully connected layer, Represents the weight matrix of the fully connected layer; The normalization function is used to normalize the output score of the fused feature vector after passing through the fully connected layer to obtain the prediction quality score. The following relationship exists in the corresponding process: ； in, represents the prediction quality score, It means that it has been processed by general normalization method.

10. An AI-generated panoramic image quality evaluation system based on visual language correspondence, characterized in that: The system applies the AI-generated panoramic image quality assessment method based on visual language correspondence according to any one of claims 1 to 9, and the system comprises: Random block sampling module for: Based on the Transformer architecture, a visual encoder and a language encoder are constructed. Based on the feature fusion mechanism, a multimodal feature fusion module is constructed. The language encoder, the visual encoder and the multimodal feature fusion module constitute a quality evaluation model. Acquire an AI-generated panoramic image, and sample the AI-generated panoramic image to obtain an image block set; Visual feature extraction module for: Based on the image block set, a visual encoder is used to represent the image block to obtain the visual features of the image block; Text prompt embed module: Using a language encoder to perform feature representation on the text description attached to the AI-generated panoramic image to obtain text features of the text description; Multimodal feature fusion module, used for: The visual features of the image block and the text features of the text description are subjected to L2 normalization and cosine similarity calculation in turn to obtain a fused feature vector; Quality regression module for: The fused feature vector is processed using a fully connected network and a normalization function to obtain a prediction quality score; Based on the predicted quality score, mean absolute error loss and Pearson correlation induced loss are constructed, and the quality evaluation model is optimized using mean absolute error loss and Pearson correlation induced loss to obtain the optimized quality evaluation model; The evaluation results are obtained using the optimized quality evaluation model.

Images