Image meaning analysis scene consistency evaluation system based on visual model

Abstract

The invention relates to the technical field of image processing, discloses an image meaning analysis scene consistency evaluation system based on a visual model, and aims to solve the problem of insufficient recognition stability in complex environments such as illumination variation, angle deviation and local shielding in the prior art. The system comprises an image input module used for receiving and preprocessing a plurality of images; the visual large model analysis module is used for carrying out multi-dimensional semantic feature extraction on the preprocessed image; the structured description generation module is used for converting the semantic features into structured text description in a unified format; the scene consistency evaluation module is used for performing logic consistency analysis on the structured text descriptions of the plurality of images; and the result output module is used for generating and outputting a final consistency evaluation report. According to the technical scheme, the method can effectively improve the recognition stability of the system in a complex environment, achieves the multi-dimensional semantic understanding of the image content, and remarkably reduces the misjudgment rate of multi-view image evaluation.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a scene consistency evaluation system for image meaning analysis based on visual models. Background Technology

[0002] With the in-depth development of artificial intelligence and computer vision technologies, image meaning analysis is increasingly widely used in fields such as business auditing, scene verification, and content compliance. Its core objective is to accurately understand and consistently judge the semantic information carried by images through algorithms. In practical applications, images are often used to prove the real existence of a specific scene, such as verifying the operational status of a store, its spatial layout, or environmental features. Therefore, higher demands are placed on the deep semantic analysis capabilities of image content. Traditional image processing methods mainly rely on low-level visual features for analysis, which is insufficient to meet the needs of accurately assessing the consistency of image meaning in complex real-world scenarios.

[0003] Among these, image meaning analysis and scene consistency assessment based on visual models has become a key technological direction for improving the accuracy and automation of image review. This direction aims to generate structured scene descriptions by comprehensively understanding high-order semantic elements such as text, texture, environmental atmosphere, and spatial layout in images, and then use these descriptions to determine whether multiple images originate from the same logical scene. Compared to traditional methods that rely solely on pixel-level similarity or geometric matching, semantic-level consistency assessment is closer to human cognitive logic and can effectively cope with the diversity and complexity of real-world shooting conditions.

[0004] Existing technologies still face significant limitations in image scene consistency assessment: First, traditional algorithms are extremely sensitive to interference factors such as changes in lighting, shooting angle shifts, and partial occlusion, resulting in poor recognition stability in complex environments and difficulty in ensuring the stability of assessment results. Second, existing methods are usually limited to low-dimensional features such as color, edges, or textures, lacking the ability to extract semantic information such as material type, text content, and decoration style, leading to a narrow recognition dimension and an inability to support refined scene comparison. Third, when multiple images come from different perspectives or positions in the same location, traditional systems often misjudge reasonable differences as inconsistencies because they cannot understand the inherent logical connections within the scene, resulting in a high misjudgment rate. These problems are particularly prominent in practical application scenarios such as store authenticity verification and remote inspection, severely restricting the reliability and applicability of automated review systems. There is an urgent need for an image analysis and assessment mechanism that can achieve high robustness, multi-dimensional semantic understanding, and logical consistency judgment. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a scene consistency evaluation system for image meaning analysis based on visual models, which can effectively solve the problems in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A scene consistency evaluation system for image meaning analysis based on visual models, comprising the following components:

[0008] The image input module is used to receive multiple images to be evaluated and perform standardized preprocessing.

[0009] The quality assessment module, connected to the image input module, is used to perform quality screening on images based on sharpness, brightness uniformity, and color saturation; a comprehensive scoring mechanism is employed, and the calculation formula is as follows:

[0010] Q = 0.4 × C + 0.3 × B + 0.3 × S, where C represents the sharpness score, B represents the brightness uniformity score, and S represents the color saturation score. When the overall score Q is lower than 0.75, the system automatically rejects the image.

[0011] The visual large model parsing module, connected to the quality assessment module, is used to extract multi-dimensional semantic features from images that have passed the quality assessment.

[0012] The structured description generation module, connected to the visual large model parsing module, is used to convert the extracted semantic features into structured text descriptions in a unified format.

[0013] The scene consistency assessment module, connected to the structured description generation module, is used to perform logical consistency analysis on the structured text descriptions of multiple images. This module adopts a dual assessment mechanism based on semantic similarity calculation and logical contradiction detection. Semantic similarity calculation obtains the basic consistency score by calculating the cosine similarity of the structured text descriptions of multiple images, while logical contradiction detection identifies potential inconsistencies by analyzing whether there are contradictory semantic elements in the descriptions of multiple images.

[0014] The results output module connects to the scenario consistency assessment module and is used to generate and output the final consistency assessment report.

[0015] Preferably, the large visual model adopts a pre-trained visual model based on the Transformer architecture, which includes a feature extraction network and a semantic understanding network. The feature extraction network uses a multi-scale convolutional neural network to extract the low-level visual features of the image, and the semantic understanding network uses a self-attention mechanism to weight and fuse the extracted features to generate a high-dimensional semantic feature vector containing text content, environmental atmosphere, material type and spatial layout.

[0016] Furthermore, the multi-dimensional semantic feature extraction of the visual large model parsing module includes text feature extraction, environment feature extraction, material feature extraction, and layout feature extraction;

[0017] Text feature extraction combines optical character recognition technology with visual semantic understanding to identify the text content and its spatial location in an image.

[0018] Environmental feature extraction analyzes the overall environmental attributes and atmospheric features of an image through the collaborative work of a scene classification network and an atmosphere perception network.

[0019] Material feature extraction employs a combination of texture analysis algorithms and material classification models to identify the material types in different regions of an image.

[0020] Layout feature extraction constructs a spatial distribution map of the main objects in the image through object detection and spatial relationship reasoning.

[0021] Preferably, the structured description generation module includes a feature encoder and a text decoder. The feature encoder projects high-dimensional semantic feature vectors onto the text semantic space, and the text decoder generates structured text containing text content description, environmental feature description, material type description and spatial layout description according to a preset template structure.

[0022] Preferably, the structured description generation module adopts a hierarchical description architecture. First, it generates basic object-level descriptions, then integrates them into scene-level descriptions, and finally forms a complete semantic description text. Each level of description follows a fixed grammatical structure and vocabulary to ensure the consistency and comparability of the descriptions.

[0023] The structured text description includes required and optional fields. Required fields include the main text content, dominant material type, and core spatial relationships, while optional fields include auxiliary environmental features and detailed decorative elements.

[0024] Preferably, the semantic similarity calculation in the scene consistency assessment module adopts a text embedding method based on the BERT model, which converts the structured text description into a 768-dimensional semantic vector, and then calculates the cosine similarity between the vectors.

[0025] The semantic similarity calculation uses a weighted similarity algorithm, assigning different weights to the similarity of different semantic dimensions.

[0026] Preferably, the logical contradiction detection in the scenario consistency assessment module adopts a combination of rule engine and machine learning model. The rule engine performs matching detection based on a preset contradiction rule library to identify obvious semantic conflicts, and the machine learning model identifies potential semantic conflicts through a trained contradiction classifier and analyzes whether there are mutually exclusive semantic statements in the description.

[0027] Preferably, the scene consistency assessment module further includes a time series analysis unit, which is used to analyze the impact of the time interval between multiple image captures on scene consistency, and adjust the strictness of the consistency assessment by comparing the relationship between image timestamp differences and scene change sensitivity.

[0028] Preferably, the result output module integrates the basic consistency score obtained from semantic similarity calculation with the problem points identified by logical contradiction detection to generate a final consistency score from 0 to 100; at the same time, it lists in detail the specific inconsistencies detected, including differences in text content, contradictions in material description, inconsistent areas of environmental features, and conflicting locations in spatial layout.

[0029] In summary, this application includes at least one of the following beneficial technical effects:

[0030] 1. By performing deep semantic analysis on images using a large visual model, the sensitivity of traditional methods to changes in illumination, angle shifts, and local occlusion is effectively overcome, significantly improving the system's recognition stability in complex environments;

[0031] 2. It achieves multi-dimensional semantic understanding of image content, and can accurately identify high-order semantic features such as text content, environmental atmosphere, material type and spatial layout, which greatly expands the recognition dimensions and accuracy;

[0032] 3. The semantic-level consistency evaluation mechanism based on structured text description can understand the internal logical connections of the scene, effectively distinguish reasonable differences from real contradictions, and significantly reduce the misjudgment rate of multi-view image evaluation;

[0033] 4. By adopting a hierarchical quality assessment and feature extraction process, the quality of the system input data and the reliability of the processing results are ensured, thereby improving the overall practicality and applicability of the system. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall technical architecture of the image meaning parsing scene consistency evaluation system based on visual models proposed in this invention;

[0035] Figure 2 This is a schematic diagram of the core principle framework of the scenario consistency assessment module in this invention. Detailed Implementation

[0036] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the specific embodiments according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0037] In the product display image review scenario, the image meaning analysis scenario consistency assessment system based on visual models verifies the consistency of multi-angle display images of the same product. The image input module receives five product display images uploaded by the merchant, which respectively show the product's front view, side view, close-up detail, usage scenario, and packaging. The image input module performs standardized preprocessing operations on the received raw images, including scaling images of different resolutions to a standard size of 1024×768 pixels, performing sRGB color space standardization conversion to eliminate color shifts caused by device differences, using histogram equalization technology to adjust image contrast to ensure clear and distinguishable visual features, and finally performing Gaussian filtering to suppress high-frequency noise introduced during the acquisition process.

[0038] The quality assessment module performs quality screening on the five pre-processed product images. This module evaluates image quality by calculating three key indicators for each image: sharpness score, brightness uniformity score, and color saturation score. The sharpness score is obtained by calculating the sum of squared gradients of the image, with a value ranging from zero to one; a higher value indicates sharper edge details. The brightness uniformity score is calculated by analyzing the brightness variance of different regions of the image, with a value ranging from zero to one; a higher value indicates more uniform illumination distribution.

[0039] Color saturation score is obtained by statistically analyzing the average saturation channel of the image in the HSV color space. The value ranges from zero to one, and the higher the value, the richer the color representation.

[0040] The quality assessment module employs a comprehensive scoring mechanism. The calculation formula is: the comprehensive quality score equals 0.4 times the sharpness score plus 0.3 times the brightness uniformity score plus 0.3 times the color saturation score. When the comprehensive quality score of an image falls below the preset threshold of 0.75, the system automatically rejects the image and sends a prompt to the merchant to re-collect the image, ensuring the reliability of the input data.

[0041] See Figure 1 The large-scale visual model parsing module is connected to the image input module and uses a pre-trained visual model based on the Transformer architecture to extract multi-dimensional semantic features from images that have passed quality assessment. This large-scale visual model gains deep semantic understanding of image content through contrastive learning training on a large-scale image-text pair dataset.

[0042] The visual large-scale model parsing module comprises two core components: a feature extraction network and a semantic understanding network. The feature extraction network employs a multi-scale convolutional neural network to extract low-level visual features from images. It processes image features at different scales through three parallel convolutional branches: the first branch uses a 7x7 convolutional kernel to capture global features, the second branch uses a 3x3 convolutional kernel to extract local features, and the third branch uses a 1x1 convolutional kernel for feature dimensionality reduction.

[0043] The semantic understanding network uses a self-attention mechanism to weight and fuse the extracted features, generating a high-dimensional semantic feature vector containing text content, environmental atmosphere, material type, and spatial layout. The vector dimension is 2,048.

[0044] The multi-dimensional semantic feature extraction process of the visual large-scale model parsing module includes four specialized feature extraction tasks. Text feature extraction combines optical character recognition technology with visual semantic understanding to identify the text content and its spatial location in an image, specifically including three sub-steps: detecting text regions, recognizing text content, and analyzing text semantics.

[0045] Environmental feature extraction analyzes the overall environmental attributes and atmospheric features of images through the collaborative work of scene classification network and atmosphere perception network. The scene classification network classifies images into indoor scenes, outdoor scenes or specific environmental types, while the atmosphere perception network analyzes the atmospheric features conveyed by the images, such as warmth, coldness, luxury or simplicity.

[0046] Material feature extraction employs a combination of texture analysis algorithms and material classification models to identify the material types in different regions of an image. The texture analysis algorithm captures the surface texture of the material by calculating local binary pattern features and gray-level co-occurrence matrix features, while the material classification model maps texture features to specific material categories based on a deep convolutional network.

[0047] The layout feature extraction constructs a spatial distribution map of the main objects in the image through object detection and spatial relationship reasoning. The object detection component identifies salient objects in the image and marks their bounding boxes, while the spatial relationship reasoning component analyzes the relative positional relationships and size ratios between objects.

[0048] The structured description generation module connects to the visual large-scale model parsing module and uses a template-based text generation method to convert extracted semantic features into structured text descriptions in a unified format. This module comprises two core components: a feature encoder and a text decoder. The feature encoder projects a 2048-dimensional high-dimensional semantic feature vector onto a 512-dimensional text semantic space, achieving feature dimensionality compression and semantic alignment through fully connected layers and layer normalization operations. The text decoder generates structured text containing text content descriptions, environmental feature descriptions, material type descriptions, and spatial layout descriptions based on a pre-defined template structure.

[0049] The structured text description employs a hierarchical architecture. First, it generates basic object-level descriptions, recording the detailed attributes of each important object in the image. Then, these are integrated into scene-level descriptions, elucidating the relationships between objects and the environmental context. Finally, a complete semantic description is formed, ensuring consistency and comparability. Each level of description follows a fixed grammatical structure and vocabulary. The structured text description includes required and optional fields. Required fields include the main text content, dominant material type, and core spatial relationships, while optional fields include auxiliary environmental features and detailed decorative elements.

[0050] See Figure 2 The scene consistency assessment module connects to the structured description generation module and employs a dual assessment mechanism based on semantic similarity calculation and logical contradiction detection to perform logical consistency analysis on the structured text descriptions of five product images. Semantic similarity calculation uses a BERT-based text embedding method to convert each structured text description into a 768-dimensional semantic vector, then calculates the cosine similarity between vectors to obtain a basic consistency score. The semantic similarity calculation uses a weighted similarity algorithm, assigning different weights to the similarity of different semantic dimensions: 0.4 for text content similarity, 0.3 for material type similarity, 0.2 for environmental atmosphere similarity, and 0.1 for spatial layout similarity. Logical contradiction detection combines a rule engine and a machine learning model. The rule engine performs matching detection based on a pre-set contradiction rule library, identifying obvious semantic conflicts such as inconsistent material descriptions and contradictory functional descriptions. The machine learning model uses a trained contradiction classifier to identify potential semantic conflicts and analyze whether there are mutually exclusive semantic statements in the descriptions. The scene consistency assessment module also includes a time series analysis unit, which is used to analyze the impact of the time interval between multiple image captures on scene consistency. By comparing the relationship between image timestamp differences and scene change sensitivity, the strictness of the consistency assessment can be adjusted.

[0051] The results output module connects to the scene consistency assessment module, generating and outputting the final consistency assessment report. This module integrates the basic consistency score calculated from semantic similarity with the identified logical contradictions to generate a final consistency score ranging from zero to one hundred. Simultaneously, the results output module details the specific inconsistencies detected, including differences in text content, contradictions in material descriptions, areas of inconsistency in environmental features, and locations of spatial layout conflicts, providing reviewers with clear decision-making support. For image groups with scores below 60, the system automatically marks them as high-risk inconsistencies, recommending manual review; for image groups with scores between 60 and 80, the system marks them as medium-risk, prompting attention to specific inconsistencies; and for image groups with scores above 80, the system marks them as low-risk, allowing for rapid approval.

[0052] In the actual implementation process, the system processes five product display images throughout the entire process.

[0053] The first front view image received a comprehensive score of 0.85 after quality assessment. The visual large model analysis module extracted the product name text features, plastic material features, solid color background environmental features, and central symmetry layout features.

[0054] The second side view image received a comprehensive score of 0.82, and features such as the same product name text, plastic material, solid color background environment, and side view layout were extracted.

[0055] The third close-up image received a comprehensive score of 0.79, with features extracted including local label text, plastic material, macro shooting environment, and local close-up layout.

[0056] The fourth usage scenario image received a comprehensive score of 0.88, and its environmental background text features, plastic material features, life scene environmental features, and contextual layout features were extracted.

[0057] The fifth packaging image received a comprehensive score of 0.80, and its features included packaging description text, paper material, display stand environment, and planar layout.

[0058] The structured description generation module converts these features into text descriptions in a unified format. The scene consistency assessment module calculates semantic similarity and finds that the text content similarity is 0.92, material type similarity is 0.65, environmental atmosphere similarity is 0.76, and spatial layout similarity is 0.58. The weighted basic consistency score is 78. Logical contradiction detection identifies the main inconsistency as a discrepancy between the material description in the fifth image and other images. The front view, side view, and close-up detail all describe the product as plastic, while the packaging image describes it as paper. The system determines this to be a reasonable difference rather than a logical contradiction. The final results output module generates a consistency assessment report with an overall score of 78, marked as medium risk, prompting reviewers to pay attention to the material description discrepancies, but recommending approval.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A scene consistency evaluation system for image meaning analysis based on visual models, characterized in that, The system includes the following components: The image input module is used to receive multiple images to be evaluated and perform standardized preprocessing. The quality assessment module, connected to the image input module, is used to perform quality screening on the image based on sharpness, brightness uniformity, and color saturation. A comprehensive scoring mechanism is adopted, and the calculation formula is as follows: Q = 0.4 × C + 0.3 × B + 0.3 × S, where C represents the sharpness score, B represents the brightness uniformity score, and S represents the color saturation score. When the overall score Q is lower than 0.75, the system automatically rejects the image. The visual large model parsing module, connected to the quality assessment module, is used to extract multi-dimensional semantic features from images that have passed the quality assessment. The structured description generation module, connected to the visual large model parsing module, is used to convert the extracted semantic features into structured text descriptions in a unified format. The scene consistency assessment module, connected to the structured description generation module, is used to perform logical consistency analysis on the structured text descriptions of multiple images. This module adopts a dual assessment mechanism based on semantic similarity calculation and logical contradiction detection. Semantic similarity calculation obtains the basic consistency score by calculating the cosine similarity of the structured text descriptions of multiple images, while logical contradiction detection identifies potential inconsistencies by analyzing whether there are contradictory semantic elements in the descriptions of multiple images. The results output module connects to the scenario consistency assessment module and is used to generate and output the final consistency assessment report.

2. The image meaning parsing scene consistency evaluation system based on visual models according to claim 1, characterized in that, The large visual model adopts a pre-trained visual model based on the Transformer architecture, which includes a feature extraction network and a semantic understanding network. The feature extraction network uses a multi-scale convolutional neural network to extract the low-level visual features of the image, and the semantic understanding network uses a self-attention mechanism to weight and fuse the extracted features to generate a high-dimensional semantic feature vector containing text content, environmental atmosphere, material type and spatial layout.

3. The image meaning parsing scene consistency evaluation system based on visual models according to claim 2, characterized in that, The multi-dimensional semantic feature extraction of the visual large model parsing module includes text feature extraction, environment feature extraction, material feature extraction and layout feature extraction; Text feature extraction combines optical character recognition technology with visual semantic understanding to identify the text content and its spatial location in an image. Environmental feature extraction analyzes the overall environmental attributes and atmospheric features of an image through the collaborative work of a scene classification network and an atmosphere perception network. Material feature extraction employs a combination of texture analysis algorithms and material classification models to identify the material types in different regions of an image. Layout feature extraction constructs a spatial distribution map of the main objects in the image through object detection and spatial relationship reasoning.

4. The image meaning parsing scene consistency evaluation system based on visual models according to claim 1, characterized in that, The structured description generation module includes a feature encoder and a text decoder. The feature encoder projects high-dimensional semantic feature vectors onto the text semantic space, and the text decoder generates structured text containing text content description, environmental feature description, material type description and spatial layout description according to a preset template structure.

5. The image meaning parsing scene consistency evaluation system based on visual models according to claim 4, characterized in that, The structured description generation module adopts a hierarchical description architecture. First, it generates basic object-level descriptions, then integrates them into scene-level descriptions, and finally forms a complete semantic description text. Each level of description follows a fixed grammatical structure and vocabulary to ensure the consistency and comparability of the descriptions. The structured text description includes required and optional fields. Required fields include the main text content, dominant material type, and core spatial relationships, while optional fields include auxiliary environmental features and detailed decorative elements.

6. The image meaning parsing scene consistency evaluation system based on visual models according to claim 1, characterized in that, The semantic similarity calculation in the scenario consistency assessment module adopts a text embedding method based on the BERT model, which converts the structured text description into a 768-dimensional semantic vector, and then calculates the cosine similarity between the vectors. The semantic similarity calculation uses a weighted similarity algorithm, which assigns different weights to the similarity of different semantic dimensions.

7. The image meaning parsing scene consistency evaluation system based on visual models according to claim 1, characterized in that, The logical contradiction detection in the scenario consistency assessment module adopts a combination of rule engine and machine learning model. The rule engine performs matching detection based on a preset contradiction rule library to identify obvious semantic conflicts. The machine learning model identifies potential semantic conflicts through a trained contradiction classifier and analyzes whether there are mutually exclusive semantic statements in the description.

8. The image meaning parsing scene consistency evaluation system based on visual models according to claim 1, characterized in that, The scene consistency assessment module also includes a time series analysis unit, which is used to analyze the impact of the time interval between multiple image captures on scene consistency. By comparing the relationship between image timestamp differences and scene change sensitivity, the strictness of the consistency assessment can be adjusted.

9. The image meaning analysis scene consistency evaluation system based on visual models according to claim 1, characterized in that, The result output module integrates the basic consistency score obtained from semantic similarity calculation with the problem points identified by logical contradiction detection to generate a final consistency score from 0 to 100; at the same time, it lists in detail the specific inconsistencies detected, including differences in text content, contradictions in material description, inconsistent areas of environmental features, and conflicting locations in spatial layout.

Images