Scene consistency evaluation system for image meaning resolution based on visual model

Abstract

The application relates to the technical field of image processing, and discloses an image meaning analysis scene consistency evaluation system based on a visual model, which aims to solve the problem of insufficient recognition stability of the prior art under complex environments such as illumination change, angle offset and local occlusion. The system comprises: an image input module for receiving and preprocessing multiple images; a visual large model analysis module for performing multi-dimensional semantic feature extraction on the preprocessed images; a structured description generation module for converting the semantic features into structured text descriptions in a unified format; a scene consistency evaluation module for performing logical consistency analysis on the structured text descriptions of the multiple images; and a result output module for generating and outputting a final consistency evaluation report. Through the above technical scheme, the system can effectively improve the recognition stability under complex environments, realize multi-dimensional semantic understanding of image content, and significantly reduce the misjudgment rate of multi-view image evaluation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a scene consistency evaluation system for image meaning analysis based on a visual model. BACKGROUND

[0002] With the in-depth development of artificial intelligence and computer vision technology, image meaning analysis is increasingly widely used in commercial auditing, scene verification and content compliance, etc. The core goal is to accurately understand and consistently judge the semantic information carried by the image through algorithms. In practical applications, images are often used to prove the real existence of a specific scene, such as the verification of store operating status, spatial layout or environmental characteristics, so higher requirements are put forward for the deep semantic analysis capability of image content. Traditional image processing methods mainly rely on bottom-level visual features for analysis, which is difficult to meet the needs of accurate evaluation of image meaning consistency in complex real scenes.

[0003] Among them, the scene consistency evaluation of image meaning analysis based on visual model has become a key technology direction to improve the accuracy and automation level of auditing. This direction aims to comprehensively understand high-level semantic elements such as text, material, environmental atmosphere and spatial layout in images, generate structured scene descriptions, and judge whether multiple images come from the same logical scene based on this. Compared with traditional methods that only rely on pixel-level similarity or geometric matching, semantic-level consistency evaluation is closer to human cognitive logic and can effectively cope with the diversity and complexity of real shooting conditions.

[0004] The existing technology still has significant limitations in implementing image scene consistency evaluation: first, traditional algorithms are extremely sensitive to interference factors such as light changes, shooting angle shifts and local occlusions, and have poor recognition stability in complex environments, making it difficult to ensure the stability of the evaluation results; second, existing methods are usually limited to low-dimensional features such as color, edge or texture, lacking the ability to extract semantic information such as material type, text content and decoration style, resulting in narrow recognition dimension and inability to support fine scene comparison; third, when multiple images come from different angles or positions of the same place, traditional systems often misjudge reasonable differences as inconsistencies due to their inability to understand the internal logical association of the scene, resulting in high misjudgment rates. The above problems are particularly prominent in real store verification, remote inspection and other practical application scenarios, severely restricting the reliability and scope of application of automated auditing systems, and an image analysis and evaluation mechanism that can realize high robustness, multi-dimensional semantic understanding and logical consistency judgment is urgently needed. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art and provide a scene consistency evaluation system for image meaning analysis based on a visual model, which can effectively solve the problems in the background art.

[0006] To achieve the above object, the present application provides the following technical solutions:

[0007] An image meaning analysis scene consistency evaluation system based on a visual model, the system comprising the following components:

[0008] An image input module for receiving a plurality of images to be evaluated and performing standardization preprocessing;

[0009] A quality evaluation module connected to the image input module for quality screening of images based on sharpness, brightness uniformity and color saturation; a comprehensive scoring mechanism is adopted, and the calculation formula is:

[0010] Q=0.4xC+0.3xB+0.3xS, wherein C represents the sharpness score, B represents the brightness uniformity score, and S represents the color saturation score, and when the comprehensive score Q is lower than 0.75, the system automatically rejects the image;

[0011] A visual large model analysis module connected to the quality evaluation module for multi-dimensional semantic feature extraction of images passing the quality evaluation;

[0012] A structured description generation module connected to the visual large model analysis module for converting the extracted semantic features into structured text descriptions in a unified format;

[0013] A scene consistency evaluation module connected to the structured description generation module for logical consistency analysis of the structured text descriptions of the plurality of images, the module adopting a dual evaluation mechanism based on semantic similarity calculation and logical contradiction detection, the semantic similarity calculation obtaining a basic consistency score by calculating the cosine similarity of the structured text descriptions of the plurality of images, and the logical contradiction detection identifying potential inconsistent areas by analyzing whether there are mutually contradictory semantic elements in the descriptions of the plurality of images;

[0014] A result output module connected to the scene consistency evaluation module for generating and outputting a final consistency evaluation report.

[0015] Preferably, the visual large model adopts a pre-trained visual model based on a Transformer architecture, comprising a feature extraction network and a semantic understanding network, the feature extraction network adopts a multi-scale convolutional neural network to extract the bottom layer visual features of the image, and the semantic understanding network generates a high-dimensional semantic feature vector containing text content, environmental atmosphere, material type and spatial layout by weighted fusion of the extracted features through a self-attention mechanism.

[0016] Further, the multi-dimensional semantic feature extraction of the visual large model analysis module includes text feature extraction, environmental feature extraction, material feature extraction and layout feature extraction.

[0017] The text feature extraction combines optical character recognition technology with visual semantic understanding to identify the text content and its spatial position in the image.

[0018] The environmental feature extraction analyzes the overall environmental attributes and atmosphere features of the image through the collaborative work of the scene classification network and the atmosphere perception network.

[0019] The material feature extraction identifies the material types of different regions in the image by combining texture analysis algorithms with material classification models.

[0020] The layout feature extraction constructs the spatial distribution relationship graph of the main objects in the image through target detection and spatial relationship reasoning.

[0021] Preferably, the structured description generation module includes a feature encoder and a text decoder. The feature encoder projects the high-dimensional semantic feature vector into a text semantic space, and the text decoder generates a structured text containing a text content description, an environmental feature description, a material type description, and a spatial layout description according to a preset template structure.

[0022] Preferably, the structured description generation module adopts a hierarchical description architecture. First, a basic object-level description is generated, then integrated into a scene-level description, and finally a complete semantic description text is formed. Each level of description follows a fixed syntax structure and vocabulary table to ensure consistency and comparability of the description.

[0023] The structured text description includes mandatory fields and optional fields. The mandatory fields include the main text content, the dominant material type, and the core spatial relationship. The optional fields include auxiliary environmental features and detailed decorative elements.

[0024] Preferably, the semantic similarity calculation in the scene consistency evaluation module adopts a text embedding method based on the BERT model to convert the structured text description into a 768-dimensional semantic vector, and then calculate the cosine similarity between the vectors.

[0025] The semantic similarity calculation adopts a weighted similarity algorithm, which assigns different weights to the similarity of different semantic dimensions.

[0026] Preferably, the logic contradiction detection in the scene consistency evaluation module adopts a combination of rule engines and machine learning models. The rule engine matches and detects based on a preset contradiction rule library to identify obvious semantic conflicts. The machine learning model identifies potential semantic conflicts through a trained contradiction classifier to analyze whether there are mutually exclusive semantic statements in the description.

[0027] Preferably, the scene consistency evaluation module further comprises a time series analysis unit for analyzing the influence of the time interval of multiple image shooting on scene consistency, adjusting the strictness of consistency evaluation by comparing the time stamp difference of the images with the scene change sensitivity.

[0028] Preferably, the result output module integrates the basic consistency score calculated by semantic similarity calculation and the problem points detected by logical contradiction detection to generate a final consistency score of 0 to 100; at the same time, detailed listing of detected specific inconsistency items, including text content difference points, material description contradiction points, environment feature inconsistency areas and space layout conflict positions.

[0029] In summary, the present application includes at least one of the following beneficial technical effects:

[0030] 1. The deep semantic analysis of images by a visual large model effectively overcomes the sensitivity of traditional methods to light changes, angle shifts and local occlusions, significantly improving the recognition stability of the system in complex environments;

[0031] 2. Multi-dimensional semantic understanding of image content is achieved, which can accurately identify high-level semantic features such as text content, environment atmosphere, material type and space layout, greatly expanding the recognition dimension and accuracy;

[0032] 3. The semantic-level consistency evaluation mechanism based on structured text description can understand the internal logical association of the scene, effectively distinguish between reasonable differences and real contradictions, and significantly reduce the misjudgment rate of multi-view image evaluation;

[0033] 4. The hierarchical quality evaluation and feature extraction process ensures the quality of system input data and the reliability of processing results, improving the practicality and scope of application of the overall system. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is the overall technical scheme architecture diagram of the image meaning analysis scene consistency evaluation system based on a visual model proposed by the present application;

[0035] Figure 2 is the core principle framework diagram of the scene consistency evaluation module in the present application. DETAILED DESCRIPTION

[0036] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined invention purpose, the specific embodiments according to the present application are described in detail as follows in combination with the drawings and preferred embodiments.

[0037] In the commodity display image review scene, the image meaning analysis scene consistency evaluation system based on the visual model verifies the consistency of multi-angle display images of the same commodity. The image input module receives five commodity display image data uploaded by the merchant, which respectively show the front view, side view, detail close-up view, use scene view and packaging view of the commodity. The image input module performs standardization preprocessing operations on the received original images, including uniformly scaling images of different resolutions to a standard size of 1024x768 pixels, performing sRGB standardization conversion on the image color space to eliminate color deviation caused by device differences, adjusting the contrast of the image using histogram equalization technology to ensure that the visual features are clear and identifiable, and finally performing Gaussian filtering on the image to suppress high-frequency noise introduced during acquisition.

[0038] The quality evaluation module performs quality screening on the five preprocessed commodity images. This module evaluates image quality by calculating three key indicators: sharpness score, brightness uniformity score and color saturation score. The sharpness score is obtained by calculating the sum of squares of image gradients, with a value range of zero to one, and a higher value representing clearer image edge details. The brightness uniformity score is calculated by analyzing the brightness variance of different regions of the image, with a value range of zero to one, and a higher value representing more uniform light distribution.

[0039] The color saturation score is obtained by calculating the mean value of the saturation channel in the HSV color space, with a value range of zero to one, and a higher value representing richer color performance.

[0040] The quality evaluation module uses a comprehensive scoring mechanism, with the formula being: quality comprehensive score equals zero point four times sharpness score plus zero point three times brightness uniformity score plus zero point three times color saturation score. When the quality comprehensive score of a certain image is lower than the preset threshold of zero point seven five, the system automatically rejects the image and sends a re-collection prompt to the merchant, ensuring the reliability of the input data.

[0041] Referring to Figure 1 , the visual large model analysis module is connected to the image input module and uses a pre-trained visual model based on the Transformer architecture to extract multi-dimensional semantic features from images that have passed quality evaluation. This visual large model has deep semantic understanding of image content by training through contrastive learning on large-scale image-text pair datasets.

[0042] The visual large model analysis module includes two core components: a feature extraction network and a semantic understanding network. The feature extraction network uses a multi-scale convolutional neural network to extract low-level visual features from images. Three parallel convolutional branches process image features at different scales. The first branch uses a seven-by-seven convolutional kernel to capture global features, the second branch uses a three-by-three convolutional kernel to extract local features, and the third branch uses a one-by-one convolutional kernel for feature dimension reduction.

[0043] The semantic understanding network uses a self-attention mechanism to weight and fuse the extracted features, generating a high-dimensional semantic feature vector containing text content, environmental atmosphere, material type, and spatial layout. The vector has a dimension of 2,048.

[0044] The multi-dimensional semantic feature extraction process of the visual large model analysis module includes four specialized feature extraction tasks. Text feature extraction combines optical character recognition technology with visual semantic understanding to identify text content and its spatial position in the image. This includes three sub-steps: detecting text regions, recognizing text content, and analyzing text semantics.

[0045] Environmental feature extraction involves the collaborative work of a scene classification network and an atmosphere perception network to analyze the overall environmental attributes and atmosphere features of an image. The scene classification network classifies the image as an indoor scene, an outdoor scene, or a specific environmental type, while the atmosphere perception network analyzes the warm, cold, luxurious, or minimalist atmosphere features conveyed by the image.

[0046] Material feature extraction uses a combination of texture analysis algorithms and material classification models to identify the material types in different regions of the image. Texture analysis algorithms capture material surface textures by calculating local binary pattern features and gray level co-occurrence matrix features, while material classification models map texture features to specific material categories based on deep convolutional networks.

[0047] Layout feature extraction involves target detection and spatial relationship reasoning to construct a spatial distribution relationship graph of the main objects in the image. The target detection component identifies significant objects in the image and labels their bounding boxes, while the spatial relationship reasoning component analyzes the relative position relationships and size proportions between objects.

[0048] The structured description generation module connects the visual large model analysis module and uses a template-based text generation method to convert extracted semantic features into structured text descriptions in a unified format. This module includes two core components: a feature encoder and a text decoder. The feature encoder projects the high-dimensional semantic feature vector of 2,048 dimensions into a text semantic space of 512 dimensions, achieving feature dimension compression and semantic alignment through fully connected layers and layer normalization operations. The text decoder generates structured text containing text content descriptions, environmental feature descriptions, material type descriptions, and spatial layout descriptions based on a pre-set template structure.

[0049] The structured text description adopts a hierarchical description architecture, first generates a basic object-level description, records the attribute characteristics of each important object in the image in detail, then integrates it into a scene-level description, describes the mutual relationship between objects and the environment background, and finally forms a complete semantic description text to ensure the consistency and comparability of the description. Each level of description follows a fixed syntax structure and vocabulary table. The structured text description contains mandatory fields and optional fields. The mandatory fields include main text content, dominant material type and core spatial relationship. The optional fields include auxiliary environmental features and detailed decorative elements.

[0050] Referring to Figure 2 The scene consistency evaluation module connects the structured description generation module and adopts a dual evaluation mechanism based on semantic similarity calculation and logical contradiction detection to analyze the logical consistency of the structured text descriptions of the five product images. Semantic similarity calculation converts each structured text description into a 768-dimensional semantic vector through a text embedding method based on the BERT model, then calculates the cosine similarity between the vectors to obtain a basic consistency score. Semantic similarity calculation uses a weighted similarity algorithm to assign different weights to the similarity of different semantic dimensions. The weight of the text content similarity is set to 0.4, the weight of the material type similarity is set to 0.3, the weight of the environmental atmosphere similarity is set to 0.2, and the weight of the spatial layout similarity is set to 0.1. Logical contradiction detection uses a combination of rule engines and machine learning models. The rule engine matches and detects based on a pre-set contradiction rule library to identify obvious semantic conflicts such as inconsistent material descriptions and contradictory function descriptions. The machine learning model uses a trained contradiction classifier to identify potential semantic conflicts and analyze whether there are mutually exclusive semantic statements in the description. The scene consistency evaluation module also includes a time series analysis unit for analyzing the impact of the time interval between multiple images on scene consistency. By comparing the image timestamp difference with the scene change sensitivity, the strictness of the consistency evaluation is adjusted.

[0051] The result output module connects the scene consistency evaluation module to generate and output the final consistency evaluation report. This module integrates the basic consistency score obtained by semantic similarity calculation with the problem points identified by logical contradiction detection to generate a final consistency score of zero to one hundred. At the same time, the result output module lists the specific inconsistency items detected in detail, including text content differences, material description contradictions, environmental feature inconsistencies and spatial layout conflict positions, providing clear decision-making basis for the review personnel. For image groups with a score below sixty, the system automatically marks them as high-risk inconsistencies and suggests manual review. For image groups with a score between sixty and eighty, the system marks them as medium-risk and prompts attention to specific inconsistency items. For image groups with a score above eighty, the system marks them as low-risk and allows them to pass through the review quickly.

[0052] In the implementation process, the system processes five product display images in the whole process.

[0053] The first front view image obtains a comprehensive score of 0.85 after quality assessment, and the visual large model analysis module extracts product name text features, plastic material features, pure color background environment features, and central symmetry layout features.

[0054] The second side view image obtains a comprehensive score of 0.82, and extracts the same product name text features, plastic material features, pure color background environment features, and side view angle layout features.

[0055] The third detail close-up image obtains a comprehensive score of 0.79, and extracts local identification text features, plastic material features, micro-distance shooting environment features, and local close-up layout features.

[0056] The fourth use scene image obtains a comprehensive score of 0.88, and extracts environment background text features, plastic material features, life scene environment features, and scenario layout features.

[0057] The fifth packaging image obtains a comprehensive score of 0.80, and extracts packaging instruction text features, paper material features, display table environment features, and planar expansion layout features.

[0058] The structured description generation module converts these features into text descriptions in a unified format, and the scene consistency evaluation module finds that the text content similarity is 0.92, the material type similarity is 0.65, the environment atmosphere similarity is 0.76, and the spatial layout similarity is 0.58 by semantic similarity calculation. The weighted basis consistency score is 78 points. The logic contradiction detection identifies that the main inconsistency item is the material description of the fifth image, which is inconsistent with other images. The front view, side view and detail close-up image all describe the product as plastic material, while the packaging image describes the material as paper. The system determines that this is a reasonable difference rather than a logical contradiction. The final result output module generates a consistency evaluation report with a total score of 78 points, marked as medium risk, and prompts the reviewer to pay attention to the material description difference, but suggests passing the review.

[0059] The above is only a preferred embodiment of the present application, and is not intended to limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content without departing from the scope of the technical solution of the present application, and any equivalent embodiments with equivalent changes are equivalent. Any modification, change, modification and modification of the above embodiments according to the technical essence of the present application are still within the scope of the technical solution of the present application.

Claims

1. A visual model-based image meaning parsing scene consistency evaluation system, characterized in that, The system comprises the following components: An image input module for receiving multiple images to be evaluated and performing standardization preprocessing; A quality evaluation module connected to the image input module for quality screening of the images based on sharpness, brightness uniformity, and color saturation; An integrated scoring mechanism is used, with the formula being: Q = 0.4 × C + 0.3 × B + 0.3 × S, where C represents the sharpness score, B represents the brightness uniformity score, and S represents the color saturation score. When the integrated score Q is lower than 0.75, the system automatically rejects the image; A visual large model analysis module connected to the quality evaluation module for multi-dimensional semantic feature extraction of the images that pass the quality evaluation; A structured description generation module connected to the visual large model analysis module for converting the extracted semantic features into structured text descriptions in a unified format; A scene consistency evaluation module connected to the structured description generation module for logical consistency analysis of the structured text descriptions of multiple images. This module uses a dual evaluation mechanism based on semantic similarity calculation and logical contradiction detection. Semantic similarity calculation obtains a basic consistency score by calculating the cosine similarity of the structured text descriptions of multiple images. Logical contradiction detection identifies potential inconsistent areas by analyzing whether there are mutually contradictory semantic elements in the descriptions of multiple images. A result output module connected to the scene consistency evaluation module for generating and outputting a final consistency evaluation report. 2.The visual model based image meaning parsing scene consistency evaluation system according to claim 1, characterized in that, The visual large model uses a pre-trained visual model based on the Transformer architecture, which includes a feature extraction network and a semantic understanding network. The feature extraction network uses a multi-scale convolutional neural network to extract the low-level visual features of the image, and the semantic understanding network uses a self-attention mechanism to weight and fuse the extracted features to generate a high-dimensional semantic feature vector containing text content, environmental atmosphere, material type, and spatial layout. 3.The visual model based image meaning parsing scene consistency evaluation system of claim 2, wherein, The multi-dimensional semantic feature extraction of the visual large model analysis module includes text feature extraction, environmental feature extraction, material feature extraction, and layout feature extraction; Text feature extraction combines optical character recognition technology with visual semantic understanding to identify the text content and its spatial position in the image; Environmental feature extraction analyzes the overall environmental attributes and atmosphere features of the image through the collaborative work of a scene classification network and an atmosphere perception network; Material feature extraction uses a combination of texture analysis algorithms and material classification models to identify the material types in different regions of the image; Layout feature extraction constructs a spatial distribution relationship graph of the main objects in the image through target detection and spatial relationship reasoning.

4. The visual model based image meaning parsing scene consistency evaluation system according to claim 1, characterized in that, The structured description generation module includes a feature encoder and a text decoder. The feature encoder projects the high-dimensional semantic feature vector into a text semantic space, and the text decoder generates a structured text containing text content description, environmental feature description, material type description, and spatial layout description according to a pre-set template structure.

5. The visual model based image meaning parsing scene consistency evaluation system according to claim 4, characterized in that, The structured description generation module adopts a hierarchical description architecture, first generating a basic object-level description, then integrating it into a scene-level description, and finally forming a complete semantic description text. Each level of description follows a fixed syntax structure and vocabulary table to ensure consistency and comparability of the description. The structured text description contains mandatory fields and optional fields. The mandatory fields include the main text content, the dominant material type, and the core spatial relationship. The optional fields include auxiliary environmental features and detailed decorative elements. 6.The visual model based image meaning parsing scene consistency evaluation system of claim 1, wherein, The semantic similarity calculation in the scene consistency evaluation module uses a text embedding method based on the BERT model to convert structured text descriptions into 768-dimensional semantic vectors, and then calculates the cosine similarity between vectors. The semantic similarity calculation uses a weighted similarity algorithm, which assigns different weights to the similarity of different semantic dimensions.

7. The visual model based image meaning parsing scene consistency evaluation system according to claim 1, characterized in that, The logic contradiction detection in the scene consistency evaluation module uses a combination of rule engines and machine learning models. The rule engine matches based on a pre-set contradiction rule library to identify obvious semantic conflicts. The machine learning model uses a trained contradiction classifier to identify potential semantic conflicts and analyze whether there are mutually exclusive semantic statements in the description. 8.The visual model based image meaning parsing scene consistency evaluation system of claim 1, wherein, The scene consistency evaluation module also includes a time series analysis unit for analyzing the impact of the time interval between multiple image captures on scene consistency. By comparing the image timestamp difference with the scene change sensitivity, the strictness of the consistency evaluation is adjusted. 9.The visual model based image meaning parsing scene consistency evaluation system of claim 1, wherein, The result output module integrates the basic consistency score obtained by semantic similarity calculation and the problem points identified by logic contradiction detection to generate a final consistency score of 0 to 100. It also lists the specific inconsistency items detected, including text content differences, material description contradictions, environmental feature inconsistencies, and spatial layout conflict positions.

Images