Image quality analysis method, electronic device, and storage medium

Abstract

Embodiments of the present application provide an image quality analysis method, an electronic device and a storage medium, and relate to the technical field of image processing and artificial intelligence. The method comprises: acquiring an input image; performing pixel-level operation on the input image through a computer vision algorithm, extracting a pixel-level quantization index, and the pixel-level quantization index being used to represent a pixel-level physical imaging quality feature of the input image; converting the pixel-level quantization index into a structured text, adding the structured text to a prompt word of a multi-modal large model, and generating a target prompt word; inputting the input image and the target prompt word into the multi-modal large model, and performing multi-modal analysis on the input image by the multi-modal large model in combination with the structured text, and generating a quality analysis report described in natural language. Through the fusion of the computer vision algorithm and the multi-modal large model, the technical problem that it is difficult to balance the pixel-level precision and the semantic-level interpretability in image quality analysis is solved.

Description

Technical Field

[0001] This application relates to the fields of image processing and artificial intelligence technology, and in particular to an image quality analysis method, electronic device, and storage medium. Background Technology

[0002] In the fields of mobile devices, smart terminals, and industrial imaging, image quality assessment is a crucial step in ensuring that the images output by imaging equipment meet quality standards.

[0003] Currently, image quality assessment mainly adopts the following methods: visual inspection of images by professional test engineers to provide subjective evaluation from multiple image quality dimensions; or pixel-level quantitative analysis of images using traditional computer vision algorithms to output numerical results of various image quality indicators; or semantic-level analysis of images using multimodal large models to generate quality evaluations described in natural language.

[0004] However, all of the above solutions suffer from the problem of failing to simultaneously consider pixel-level detail perception and semantic-level comprehensive judgment in image quality assessment. Summary of the Invention

[0005] This application provides an image quality analysis method, electronic device, and storage medium to solve the technical problem that multimodal large models are difficult to balance pixel-level accuracy and semantic-level interpretability in image quality analysis.

[0006] In a first aspect, embodiments of this application provide an image quality analysis method, the method comprising:

[0007] Obtain the input image;

[0008] The input image is processed using computer vision algorithms at the pixel level to extract pixel-level quantization metrics. These metrics are used to characterize the pixel-level physical imaging quality features of the input image. The pixel-level processing includes fine-grained processing of the pixel matrix of the input image using computer vision algorithms to obtain at least one pixel-level quantization metric. The pixel-level quantization metric includes at least one of sharpness quantization value, noise quantization value, color quantization value, and exposure quantization value.

[0009] The pixel-level quantization metrics are assembled into a predefined structured text format, converted into structured text, and then added to the prompt words of the multimodal large model to generate target prompt words.

[0010] Input the input image and target prompts into the multimodal large model. The multimodal large model combines structured text to perform multimodal analysis on the input image to analyze the image quality of the corresponding dimension of pixel-level quantification indicators and generate a quality analysis report described in natural language.

[0011] In this embodiment, computer vision algorithms are used to perform pixel-level calculations on the input image to extract quantitative indicators that characterize pixel-level physical imaging quality features. These quantitative indicators are then converted into structured text and added to the prompts of the multimodal large model. This allows the multimodal large model to simultaneously obtain accurate quantitative data and image semantic information during analysis, achieving a fusion analysis of pixel-level precision and semantic understanding. Furthermore, by injecting the quantitative indicators into the prompts in a structured text format, the numerical output of the computer vision algorithm can participate in reasoning in a form understandable to the multimodal large model, avoiding the problem of direct fusion due to different data modalities.

[0012] Compared to existing technologies that rely on the visual encoder of the multimodal large model for image feature extraction and suffer from high-frequency detail loss due to downsampling, the embodiments of this application extract quantization indicators and inject prompt words on the original resolution image through computer vision algorithms. This enables the multimodal large model to have pixel-level detail perception capabilities without retraining or fine-tuning, solving the technical problem in existing technologies where multimodal large models cannot accurately detect minor image signal processor (ISP) defects (such as brightness noise, color noise, purple fringing, and moiré patterns).

[0013] In one possible embodiment, the method further includes:

[0014] Cross-validate the quality analysis report with pixel-level quantitative indicators;

[0015] When the defect judgment conclusion in the quality analysis report is consistent with the defect judgment conclusion reflected by the pixel-level quantitative indicators, the analysis result with the first confidence level is output.

[0016] When the defect judgment conclusion in the quality analysis report is inconsistent with the defect judgment conclusion reflected by the pixel-level quantitative indicators, the analysis result of the second confidence level is output, and the input image is marked as to be reviewed.

[0017] The first confidence level is higher than the second confidence level.

[0018] In this implementation, the quality analysis report generated by the multimodal large model is cross-validated with the pixel-level quantification indicators extracted by the computer vision algorithm. This ensures that each quality analysis conclusion undergoes a second-channel verification independent of the Vision Language Model (VLM) reasoning process, thus constructing a reliability guarantee mechanism that mutually verifies semantic and numerical conclusions. At the same time, by setting a hierarchical processing strategy that outputs high-confidence results when conclusions are consistent and low-confidence results and marks them for review when conclusions are inconsistent, the system can automatically identify and mark low-confidence cases when VLM reasoning deviates, avoiding the direct adoption of erroneous conclusions caused by misjudgment from a single information source.

[0019] Compared to existing technologies where VLM output results lack independent verification methods and users find it difficult to judge the reliability of analysis conclusions, this application embodiment uses computer vision (CV) quantitative indicators to perform real-time cross-validation of VLM outputs, assigning a quantifiable confidence assessment to each analysis result, thus solving the technical problems of lacking reliability quantitative indicators for fused analysis results and users being unable to distinguish between credible conclusions and conclusions awaiting verification.

[0020] In one possible embodiment, the defect determination conclusion includes determining whether a defect exists and determining the defect level.

[0021] In this implementation, by specifying the defect determination conclusion as the determination of whether a defect exists and the determination of the defect level, cross-validation can not only determine whether an image has a defect, but also perform multi-level quantitative evaluation of the severity of the defect. By introducing multi-level defect level determination, the analysis results can accurately reflect the severity of the defect, and users can determine the priority of defect processing accordingly. At the same time, it provides a more granular comparison dimension for cross-validation, that is, it can compare not only whether the qualitative conclusions of whether there is a defect are consistent, but also whether the quantitative classification of the defect level matches, thereby improving the accuracy of cross-validation and the operability of the evaluation results.

[0022] In one possible embodiment, structured text is added to the cue words of the multimodal large model, including:

[0023] The pixel-level quantization metrics are assembled into a JSON (JavaScript Object Notation) object according to a preset key-value pair format;

[0024] Add the JSON object to the prompt word template according to the preset position to form the target prompt word, which includes system role definition, analysis dimension definition, defect level standard, structured text and output format requirements.

[0025] In this implementation, by assembling pixel-level quantization metrics into JSON objects according to a preset key-value pair format, the quantization data is organized in a structured key-value pair format. VLM can quickly parse and accurately locate the quantization value corresponding to each dimension, avoiding parsing deviations caused by chaotic data formats. By adding the JSON object to the prompt word template according to preset positions, the quantization data is precisely embedded in the prompt words at the positions corresponding to each analysis dimension. This allows VLM to accurately obtain the quantization basis corresponding to a certain dimension (such as sharpness) when performing analysis, achieving a precise correspondence between quantization data and analysis dimensions.

[0026] Meanwhile, by generating target prompts that include system role definitions, analysis dimension definitions, defect level standards, structured text, and output format requirements, VLM can obtain four layers of information—task role positioning, analysis rule constraints, quantitative data support, and output format specifications—within the same prompt, thus constructing a complete reasoning context and avoiding inference biases or non-compliant output formats caused by incomplete prompt information.

[0027] In one possible embodiment, the pixel-level quantization metrics include at least sharpness quantization, noise quantization, color quantization, and exposure quantization.

[0028] In this implementation, by limiting the pixel-level quantization metrics to include at least sharpness quantization, noise quantization, color quantization, and exposure quantization, computer vision algorithms can comprehensively evaluate the input image at the pixel level from four dimensions: sharpness, noise level, color accuracy, and exposure accuracy. These four dimensions correspond to the core evaluation metrics of image quality: sharpness quantization reflects the sharpness of image details and the ability to preserve edge details; noise level reflects the signal-to-noise ratio of the image sensor and the effect of noise reduction algorithms; color accuracy reflects the ability of white balance correction and color restoration; and exposure accuracy reflects the dynamic range control capability of automatic exposure algorithms.

[0029] The above four-dimensional quantitative indicators cover the key elements of image quality evaluation, avoiding evaluation bias caused by single-dimensional assessment, and providing a complete multi-dimensional quantitative basis for multimodal large models, enabling subsequent quality analysis to make accurate judgments from the four dimensions of sharpness, noise, color, and exposure.

[0030] In one possible embodiment, before assembling the pixel-level quantization metrics into a predefined structured text format and converting it into structured text, the process further includes:

[0031] The clarity quantization value, noise quantization value, color quantization value, and exposure quantization value are mapped to the evaluation dimensions of the multimodal large model, and the correlation between each quantization value and the corresponding evaluation dimension is established.

[0032] In this implementation, by mapping the clarity quantization value, noise quantization value, color quantization value, and exposure quantization value to the evaluation dimensions of the multimodal large model, the correlation between each quantization value and the corresponding evaluation dimension is established, so that the multimodal large model can accurately call the quantization data corresponding to each evaluation dimension when analyzing each evaluation dimension.

[0033] Specifically, clarity quantization is used for analysis of the autofocus evaluation dimension, noise quantization for analysis of the image signal processing evaluation dimension, color quantization for analysis of the automatic white balance evaluation dimension, and exposure quantization for analysis of the automatic exposure evaluation dimension, thus ensuring a precise correspondence between quantized data and evaluation dimensions. This approach solves the problem that when quantized data is broadly incorporated into prompts, large models cannot accurately correlate each quantized value with its corresponding analysis dimension, improving the accuracy and effectiveness of quantized data in guiding VLM inference.

[0034] In one possible embodiment, the sharpness quantization value is calculated using the Laplacian variance method; the noise quantization value is calculated using the median filter difference method; the color quantization value is calculated using the HSV (Hue, Saturation, Value) color space analysis method; and the exposure quantization value is calculated using the luminance histogram analysis method.

[0035] In this implementation, the sharpness quantification value is calculated using the Laplacian variance method. This method effectively quantifies the concentration of edge energy and the richness of detail by performing the second derivative operation of Laplacian on the grayscale image and calculating the variance. The larger the variance value, the sharper the edges and the richer the details. The noise quantification value is calculated using the median filtering difference method. This method performs median filtering smoothing on the grayscale image, calculates the absolute difference between each pixel in the original image and the smoothed image, and takes the average of the entire image. This effectively estimates the noise level of the image. The smaller the average difference, the smoother the image and the less noise. The color quantification value is calculated using the HSV color space analysis method. This method converts the image from the BGR (Blue, Green, Red) color space to the HSV color space and calculates the average saturation and average brightness of the entire image. This effectively evaluates the vividness of colors and the uniformity of brightness distribution. The exposure quantification value is calculated using the brightness histogram analysis method. This method effectively evaluates the exposure accuracy by statistically analyzing the proportion of overexposed and underexposed pixels in multi-level brightness histograms.

[0036] By combining the above four algorithms, precise quantification of four dimensions of input image sharpness, noise, color, and exposure is achieved, providing reliable pixel-level data support for semantic analysis of multimodal large models and improving the accuracy and reliability of subsequent quality analysis.

[0037] In one possible embodiment, the clarity quantization value and the noise quantization value are respectively normalized to a preset scoring range through a preset empirical threshold, wherein the clarity quantization value adopts a positive normalization method, and the higher the clarity quantization value, the higher the normalized score.

[0038] The noise quantization value is normalized in reverse. The lower the noise quantization value, the higher the score after normalization.

[0039] In this implementation, both the sharpness quantization value and the noise quantization value are normalized to a preset scoring range (e.g., 0-100 points) using preset empirical thresholds. This allows physical measurement values ​​of different dimensions and scales to be mapped to a unified scoring scale for comparison and evaluation. Specifically, the sharpness quantization value uses forward normalization, meaning a higher sharpness quantization value results in a higher normalized score. This allows the sharpness score to directly reflect the detail clarity of the image; a higher score indicates a sharper image. Conversely, the noise quantization value uses reverse normalization, meaning a lower noise quantization value results in a higher normalized score. This allows the noise score to directly reflect the purity of the image; a higher score indicates less noise and a cleaner image quality.

[0040] By using the above method, the original measured values ​​(such as Laplacian variance and median filtered difference mean) that originally had no intuitive physical meaning and uncertain numerical range are transformed into score values ​​with clear physical meaning within the range of 0-100. This allows the analysis results to be presented in an intuitive score form, making it easier for users to quickly understand the quality performance of the image in various dimensions. At the same time, it provides a unified quantitative basis for the semantic analysis of multimodal large models, improving the interpretability and usability of the quantitative data.

[0041] In one possible embodiment, pixel-level quantization metrics are assembled into a predefined structured text format, converted into structured text, and then added to the cue words of the multimodal large model to generate target cue words, including:

[0042] When the quantization value of any dimension in the pixel-level quantization metric is lower than the preset threshold, add the corresponding region analysis instruction to the prompt word and generate a target prompt word containing the region analysis instruction.

[0043] The region analysis command is used to instruct a multimodal large model to perform defect analysis on the image regions corresponding to the dimensions when analyzing the input image.

[0044] In this implementation, when the quantization value of any dimension in the pixel-level quantization index is lower than a preset threshold, it is determined that there is an anomaly in that dimension. A region analysis instruction corresponding to that dimension is added to the prompt word, and a target prompt word containing the region analysis instruction is generated to instruct the multimodal large model to perform defect analysis on the image region corresponding to the abnormal dimension when analyzing the input image.

[0045] The above method guides the VLM analysis process using CV quantification metrics. The CV layer first identifies dimensions with quality issues by comparing quantification metrics with preset thresholds. Based on the preset correspondence between these dimensions and image regions (e.g., sharpness corresponds to edge regions, noise corresponds to dark regions), it determines the image regions requiring focused analysis. Then, prompts in the form of region analysis instructions inform the VLM which region needs defect analysis. This allows the VLM to concentrate its analysis resources on potentially defective image regions during inference, avoiding the problem of missing local anomalies during global uniform image analysis. This improves the detection accuracy and analysis efficiency of multimodal large models for local quality defects.

[0046] Secondly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0047] The memory stores the instructions that the computer executes;

[0048] The processor executes computer execution instructions stored in memory to implement the methods described above.

[0049] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided above. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0051] Figure 1 This is a schematic diagram of the overall architecture of the image quality multimodal intelligent analysis system provided in the embodiments of this application;

[0052] Figure 2 A flowchart illustrating an image quality analysis method provided in an embodiment of this application;

[0053] Figure 3 A schematic diagram of the dual-channel feature fusion mechanism provided in the embodiments of this application;

[0054] Figure 4A schematic diagram illustrating the resolution calculation process provided in this application embodiment;

[0055] Figure 5 A schematic diagram illustrating the noise level calculation process provided in an embodiment of this application;

[0056] Figure 6 A schematic diagram illustrating the color accuracy calculation process provided in this application embodiment;

[0057] Figure 7 This is a schematic diagram of the exposure accuracy calculation process provided in an embodiment of this application.

[0058] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0059] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0060] It should be noted that the image quality analysis method and electronic device provided in this application embodiment can be used in the fields of image processing and artificial intelligence technology, or in any field other than image processing and artificial intelligence technology. This application embodiment does not limit the application field of the image quality analysis method and electronic device.

[0061] This application can be applied to image quality assessment of mobile phone cameras, vehicle cameras, drone cameras, and other smart terminals. In such scenarios, imaging devices acquire images under different lighting conditions, shooting scenes, and motion states. The acquired images exhibit varying degrees of deviation in detail, noise levels, color reproduction, and brightness distribution. Image quality assessment methods are needed to quantitatively analyze and subjectively evaluate the image quality in order to pinpoint the optimization directions for each signal processing stage in the imaging system (such as autofocus, auto exposure, and auto white balance).

[0062] For example, in mobile phone camera R&D testing scenarios, some images (such as backlit portraits) have problems with overexposure in highlight areas and loss of detail in shadow areas, so the exposure accuracy needs to be evaluated first to optimize the automatic exposure algorithm; some images (such as night scenes) have luminance noise or color noise, so the noise level needs to be evaluated first to optimize the noise reduction algorithm; some images (such as indoor warm light environments) have problems with inaccurate skin tone reproduction, so the color accuracy needs to be evaluated first to optimize the automatic white balance algorithm.

[0063] For the above application scenarios, existing technologies typically employ the following image quality assessment schemes.

[0064] One method is manual visual inspection, in which professional test engineers visually observe the images output by the imaging equipment under standard viewing conditions, and subjectively evaluate the images from multiple image quality dimensions such as exposure, color, noise, and sharpness based on their personal experience, and give corresponding evaluation conclusions.

[0065] Another approach is to use traditional computer vision algorithms for evaluation. This involves preprocessing the image (e.g., grayscale conversion, size normalization) and then performing pixel-level operations using algorithms such as edge detection, frequency domain analysis, or statistical modeling to extract the image's low-level visual features. For example, edge detection or high-frequency component analysis can yield quantified values ​​related to sharpness, local variance analysis or difference analysis with the smoothed image can provide estimates of noise levels, and brightness histogram statistics can provide reference values ​​for exposure shift. Finally, the system outputs numerical results for various image quality metrics.

[0066] With the development of multimodal large models, image quality analysis methods based on visual language models have also emerged. These methods utilize pre-trained multimodal large models to perform semantic-level analysis of images. Specifically, the image to be evaluated is input into the multimodal large model. The model's visual encoder divides the image into multiple image patches and extracts high-level semantic features. The text encoder encodes the input question text. The cross-modal attention module jointly models the image features and text features. Finally, the model's text decoder generates a natural language description of the image word by word, including scene recognition results and a semantic evaluation of image quality.

[0067] In addition, there is a method that combines computer vision algorithms with multimodal large models independently. In this method, the computer vision algorithm and the multimodal large model run independently in the same image evaluation process, without any data interaction or information transfer between them. The system displays the numerical indicators output by the computer vision algorithm and the natural language description output by the multimodal large model separately on the interface, allowing users to comprehensively judge the overall quality of the image by reading both sets of information.

[0068] In current image quality assessment schemes, manual visual inspection relies on the professional experience and subjective judgment of testers. The difficulty in standardizing evaluation criteria among different personnel leads to poor consistency and repeatability of assessment results, making it hard to meet the reliability requirements of batch testing scenarios. While traditional computer vision algorithms can output precise numerical indicators, their assessment results lack the ability to understand the semantic content of images, cannot identify the type of shooting scene, and cannot describe and evaluate subjective quality dimensions such as "blurred image" or "unnatural skin texture," making it difficult to generate readable, structured analysis reports.

[0069] In the multimodal large model approach, the model's visual encoder typically divides the image into fixed-size image blocks for feature extraction when processing the image. This downsampling operation leads to the loss of high-frequency pixel-level detail information, making it difficult to detect subtle defects generated in the imaging signal processing stage (such as brightness noise, color noise, purple fringing, moiré patterns, etc.). The detection rate of defects at the ISP level is low, and the evaluation accuracy is insufficient.

[0070] Furthermore, in the independent combination of computer vision algorithms and multimodal large models, the two methods operate independently, the quantitative data cannot effectively guide the direction of semantic reasoning, and the semantic analysis results cannot supplement and verify the detection conclusions. The lack of an effective fusion mechanism between the two results in a fragmented evaluation outcome and insufficient overall reliability.

[0071] In view of this, embodiments of this application provide an image quality analysis method, the core concept of which is to construct a multimodal fusion analysis architecture that integrates computer vision algorithms and a multimodal large model in a hierarchical and collaborative manner. The computer vision algorithm performs pixel-level operations on the input image to extract quantitative indicators characterizing pixel-level physical imaging quality features, thus preserving high-frequency detail information of the image. Furthermore, the extracted quantitative indicators are converted into structured text and added to the prompts of the multimodal large model, enabling the model to simultaneously obtain accurate underlying quantitative data during semantic analysis. Specifically, the computer vision algorithm is responsible for pixel-level detail perception, while the multimodal large model is responsible for semantic-level comprehensive judgment. The two achieve cross-modal information transmission and fusion through structured text prompts.

[0072] Based on this, the quality analysis report generated by the multimodal large model is cross-validated with the pixel-level quantization indicators extracted by the computer vision algorithm. When the defect judgment conclusions of the two are consistent, a high-confidence analysis result is output. When the defect judgment conclusions of the two are inconsistent, the input image is marked as pending review, thus forming a dual-channel quality analysis mechanism that combines evaluation and verification.

[0073] Compared to traditional computer vision algorithms that lack semantic interpretability, multimodal large models that lose details due to downsampling, and solutions that lack a fusion mechanism when used independently, this application's embodiments solve the technical problem in the prior art that image quality assessment is difficult to balance pixel-level accuracy and semantic interpretability by synergistically fusing computer vision algorithms and multimodal large models, thereby improving the accuracy, interpretability, and reliability of the assessment results.

[0074] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0075] First, combine Figure 1 This application introduces the image quality multimodal intelligent analysis system provided in its embodiments. Figure 1 This is a schematic diagram of the overall architecture of the image quality multimodal intelligent analysis system provided in the embodiments of this application.

[0076] See Figure 1 The system consists of four core layers: feature extraction layer, prompt word assembly layer, multimodal large model inference layer, and result fusion output layer. These four layers are connected in sequence to form a complete processing link from image input to quality analysis report output.

[0077] The feature extraction layer, located at the system's input, receives the input image to be analyzed. This layer contains sharpness quantization, noise quantization, color quantization, and exposure quantization modules, which perform pixel-level calculations on the input image to extract sharpness, noise, color, and exposure quantization values, respectively. The results from each module are aggregated to form pixel-level quantized indicators (structured text), which are then passed to the prompt word assembly layer.

[0078] The prompt assembly layer receives the output pixel-level quantized metrics (structured text) and assembles them uniformly with system role definitions, analysis dimension definitions, and defect level standards to generate target prompts. These target prompts contain quantified numerical constraints and analysis rules, guiding subsequent multimodal large-scale models to reference precise underlying quantized data during semantic analysis.

[0079] The multimodal large-scale model inference layer receives the input image and target prompts. A visual encoder performs semantic understanding on the input image, extracting high-level semantic features. A text encoder parses the target prompts, extracting analysis rules and quantified data. The two sets of information are then fused across modalities and input into the multimodal large-scale model for multimodal analysis, generating a quality analysis report.

[0080] The result fusion output layer receives the quality analysis report output by the multimodal large model, performs structured parsing, and extracts information such as problem summaries, scores, and optimization suggestions. Simultaneously, this layer cross-validates the parsing results with the output pixel-level quantization metrics. When the defect determination conclusions of the two are consistent, a high-confidence analysis result is output; when the defect determination conclusions are inconsistent, the input image is marked as requiring further review, ensuring the reliability and accuracy of the final analysis report.

[0081] The following is combined Figure 2 The image quality analysis method provided in the embodiments of this application is described. Among them, Figure 2 A flowchart illustrating an image quality analysis method provided in this application embodiment is shown below. Figure 2 As shown, the method includes:

[0082] S201. Obtain the input image.

[0083] The input image in this step is the object to be analyzed, providing the basic image content for subsequent pixel-level operations and multimodal analysis. After acquiring the input image, the executing entity uses it as the processing object of the computer vision algorithm to extract pixel-level quantization indicators, and also as the visual input of the multimodal large model to participate in joint inference.

[0084] In this embodiment, the input image can be obtained through local reading, interface reception, or import from an external device. For example, the input image can be a static image; when the input image is in compressed format, the analysis program can first perform a decoding operation to restore the image to a pixel matrix in memory before sending it to subsequent modules.

[0085] In one possible embodiment, to ensure the consistency of subsequent pixel-level operations, preprocessing can be performed after the input image is acquired. This preprocessing may include at least one of image integrity verification, size normalization, or color channel unification.

[0086] It should be understood that the above compression optimization coding is only an example. In other implementations, multi-resolution coding, region clipping coding, or adaptive compression and other coding strategies may also be used. This application does not limit the implementation of such strategies.

[0087] S202. Perform pixel-level operations on the input image using computer vision algorithms to extract pixel-level quantization indicators. The pixel-level quantization indicators are used to characterize the pixel-level physical imaging quality features of the input image. The pixel-level operations include fine-grained processing of the pixel matrix of the input image using computer vision algorithms to obtain at least one pixel-level quantization indicator. The pixel-level quantization indicator includes at least one of sharpness quantization value, noise quantization value, color quantization value, and exposure quantization value.

[0088] In this step, the execution entity sends the input image to the index extraction module. Computer vision algorithms are used to perform fine-grained processing on the pixel matrix of the input image, extracting pixel-level quantization indicators to characterize the physical imaging quality of the image. The results are then summarized into a structured index set. Specifically, any computer vision algorithm suitable for characterizing the physical imaging quality of an image can be used to perform pixel-level analysis to obtain the corresponding quantization results.

[0089] Based on the above analysis, it can be seen that by performing pixel-level operations on the input image using computer vision algorithms, the physical imaging differences in the original image can be converted into calculable quantitative results, providing a foundation for injecting fine-grained imaging quality information into the semantic analysis link.

[0090] S203. Assemble the pixel-level quantization indicators according to the predefined structured text format, convert them into structured text, and add the structured text to the prompt words of the multimodal large model to generate target prompt words.

[0091] In this step, after obtaining pixel-level quantization metrics, the executing entity calls the text construction module to organize the quantization metrics into structured text that can be stably parsed by a multimodal large model according to a preset syntax. This structured text is then concatenated with a preset prompt word template to generate target prompt words for subsequent joint inference. Specifically, the structured text can be formatted text that the model can parse, such as key-value pairs or JSON.

[0092] For example, structured text can be organized in a key-value pair format, and may include fields such as sharpness_score (float, example value 85.3), noise_level (float, example value 12.1), color_accuracy (float, example value 72.5), exposure_score (float, example value 68.9), highlight_clipping_ratio (percentage string, example value 15%), and shadow_clipping_ratio (percentage string, example value 8%).

[0093] The field names and data types mentioned above are for illustrative purposes only. In practical applications, fields can be added, deleted, or adjusted according to specific needs. Optionally, to improve the model's consistent understanding of the meaning of each indicator, explanatory notes can be added to the prompts.

[0094] Based on the above analysis, it can be seen that converting pixel-level quantization indicators into structured text and embedding prompt words injects the numerical basis obtained by computer vision algorithms into the language context, enabling the multimodal large model to refer to fine-grained imaging quality information when performing semantic judgment, so that the subsequent generated results have both indicator basis and semantic integrity.

[0095] In one possible embodiment, the prompt word template may include task descriptions, analysis requirements, and output format requirements. The text construction module inserts structured text into a preset position in the prompt word template, thereby enabling the model to obtain a quantitative context corresponding to the input image before generating an analysis report. For example, the target prompt word may include the following continuous semantic content: requiring a comprehensive judgment on image quality based on the input image content and given quality indicators, and outputting a natural language report in a preset format.

[0096] In one possible embodiment, the structured text can also be grouped into cue words according to the analysis dimension, enabling the multimodal large model to be analyzed item by item according to the corresponding dimension when generating the report.

[0097] S204. Input the input image and target prompt words into the multimodal large model. The multimodal large model combines structured text to perform multimodal analysis on the input image to analyze the image quality of the corresponding dimension of the pixel-level quantification index and generate a quality analysis report described in natural language.

[0098] In this step, after the execution entity generates the target prompt word, it sends the input image as visual input and the target prompt word as text input to the multimodal large model. The multimodal large model extracts visual features from the input image, encodes the target prompt word into text, and generates a quality analysis report corresponding to the input image through cross-modal joint reasoning.

[0099] In practice, the multimodal large model first extracts scene-level visual features from the input image, and simultaneously text-encodes the target prompts to identify the task requirements and pixel-level quantification indicators. Then, through cross-modal joint processing, the visual and text features are aligned, enabling the model to make comprehensive judgments by referring to the corresponding numerical information in the structured text when generating semantic conclusions. The resulting quality analysis report is output in natural language, including at least a description of defects, and may further include a quality score and optimization suggestions. For example, the report could state that the image has problems such as underexposure, insufficient edge sharpness, or prominent noise, and provide a corresponding quality score.

[0100] In one possible embodiment, multimodal analysis employs a combined processing approach of image content and structured text, enabling the model to simultaneously reference visual content and quantitative metrics when generating a quality analysis report.

[0101] Based on the above analysis, this step inputs the input image and the target prompt words carrying pixel-level information into the multimodal large model. This allows the model output to no longer rely solely on visual semantic information, but also combine quantifiable physical imaging features to complete the reasoning. The resulting quality analysis report can meet the implementation requirements of image quality analysis scenarios in terms of defect localization, cause description, and consistency of conclusions.

[0102] It should be understood that the embodiments of this application are not limited to the specific implementation of the multimodal large model, and any multimodal large model that supports image and text input can be used.

[0103] In this embodiment, computer vision algorithms are used to perform pixel-level operations on the input image to extract pixel-level quantization indicators that characterize physical imaging quality. This allows high-frequency detail information and physical imaging differences in the original image to be quantified into calculable numerical data, compensating for the loss of high-frequency details caused by downsampling of the visual encoder in multimodal large models. Simultaneously, by converting the pixel-level quantization indicators into structured text and embedding them into the prompts of the multimodal large model, the multimodal large model can simultaneously obtain accurate quantization data and image semantic information during semantic analysis. This achieves the fusion of pixel-level accuracy and semantic understanding, solving the technical problem that multimodal large models struggle to balance pixel-level accuracy and semantic interpretability in image quality analysis.

[0104] In this embodiment, the above-described image quality analysis method also involves a dual-channel feature fusion mechanism. Figure 3 This is a schematic diagram of the dual-channel feature fusion mechanism provided in an embodiment of this application. The following is in conjunction with... Figure 3 This fusion mechanism will be explained in detail.

[0105] See Figure 3 The dual-channel feature fusion mechanism provided in this application includes a fine-grained pixel channel and a coarse-grained semantic channel.

[0106] Fine-grained pixel channels are used to acquire pixel-level quantization metrics of the input image. These metrics include sharpness quantization, noise quantization, color quantization, and exposure quantization. These four pixel-level quantization metrics are aggregated to form a quantization context structured text. This quantization context structured text is then input to both the attention redirection module and the multi-granularity alignment module.

[0107] The coarse-grained semantic channel is used for scene understanding of the input image, obtaining scene understanding results, defect semantic descriptions, and optimization suggestions. Specifically, the scene understanding results are input to the multi-granularity alignment module, where they are aligned with the quantized contextual structured text. The defect semantic descriptions and optimization suggestions are input to the confidence-weighted module.

[0108] The outputs of the attention redirection module, the multi-granularity alignment module, and the confidence weighting module are all input into the structured analysis report module, which contains a cross-validation mechanism to verify the fusion results and finally generate a quality analysis report.

[0109] Through the aforementioned dual-channel feature fusion mechanism, the precise quantization data provided by the fine-grained pixel channel and the scene semantic information provided by the coarse-grained semantic channel are deeply integrated in the three modules of attention redirection, multi-granularity alignment, and confidence weighting. This enables the multimodal large model to make full use of pixel-level detailed information during semantic analysis, thereby improving the accuracy and reliability of the evaluation results.

[0110] Building upon the aforementioned embodiments, in this embodiment, the executing entity inputs the quality analysis report generated in step S204 and the pixel-level quantization indicators extracted in step S202 into the cross-validation module. The cross-validation module first parses the defect determination conclusion from the quality analysis report, i.e., whether a defect exists and its corresponding level, and simultaneously obtains the defect determination conclusion reflected by the pixel-level quantization indicators. Subsequently, the cross-validation module performs a consistency comparison between the two based on a preset mapping relationship between indicators and defect types.

[0111] In one possible embodiment, the mapping relationship between pixel-level quantization metrics and defect determination conclusions can be pre-established as follows: sharpness quantization values ​​correspond to sharpness defects, noise quantization values ​​correspond to noise defects, color quantization values ​​correspond to color defects, and exposure quantization values ​​correspond to exposure defects. When any quantization value is lower than or higher than a preset threshold, it is determined that a corresponding defect exists in that dimension, and the defect level can be classified according to the degree of deviation from the threshold.

[0112] Based on the above mapping relationship, if the defect judgment conclusion in the quality analysis report is consistent with the defect judgment conclusion reflected by the pixel-level quantization index (e.g., both point to the same defect type and the defect level matches, or both point to no defect), then the analysis result with the first confidence level is output, and this result is written to the quality analysis record. If the two are inconsistent (e.g., the quality analysis report determines that a defect exists, but the pixel-level quantization index reflects normality, or the defect types do not match, or the defect levels are inconsistent), then the analysis result with the second confidence level is output, and a "to be reviewed" mark is added to the input image, and the conflict information is associated and saved. The first confidence level is higher than the second confidence level.

[0113] For example, when the quality analysis report determines that "the image has noise defects, with a defect level of medium," and the noise quantization value in the pixel-level quantization index is lower than a preset threshold and the deviation corresponds to the medium defect range, the two conclusions are consistent, and the cross-validation module outputs the analysis result with a first confidence level. When the quality analysis report determines that "the image has sharpness defects," but the sharpness quantization value in the pixel-level quantization index is within the normal range, the two conclusions are inconsistent, and the cross-validation module outputs the analysis result with a second confidence level and marks the input image as pending review. It should be understood that the specific expression of confidence level can be in the form of numerical range, percentage, level label, or score, etc., and this application embodiment does not limit this.

[0114] By cross-validating the quality analysis report generated by the multimodal large model with pixel-level quantification indicators extracted by computer vision algorithms, high-confidence analysis results can be output when semantic conclusions and underlying quantification indicators are consistent. When they are inconsistent, the confidence level is automatically reduced and the result is marked for further verification, thus assigning a quantifiable reliability assessment to each analysis result. This method, through mutual verification of semantic and numerical conclusions, effectively identifies and marks cases where multimodal large model inference may produce biases, avoiding the direct adoption of erroneous conclusions caused by misjudgments from a single information source. This ensures that the final analysis results possess both interpretability and traceability.

[0115] Based on the aforementioned embodiments, the defect determination conclusion includes determining whether a defect exists and determining the defect level.

[0116] Specifically, when the cross-validation module parses the defect judgment conclusion from the quality analysis report, it simultaneously obtains information from two dimensions: first, the determination of whether a defect exists, and second, the determination of the defect level. The determination of whether a defect exists characterizes whether the image has quality problems in the current evaluation dimension; the determination of the defect level is used to grade and evaluate the severity of the defect if it exists. For example, the defect level can be divided into four levels according to severity from high to low: Blocker, Critical, Major, and Minor. Other classification methods, such as three or five levels, can also be used depending on the actual application scenario; this embodiment does not limit this.

[0117] During cross-validation, the cross-validation module compares the defect determination conclusions in the quality analysis report with those reflected in the pixel-level quantitative indicators. Specifically, it compares whether the determination of defect existence is consistent and whether the defect levels match. If both are consistent in determining the existence of defects and the defect levels match, the conclusions are considered consistent, and an analysis result with a first confidence level is output. If the two are inconsistent in determining the existence of defects or the defect levels do not match, the conclusions are considered inconsistent, and an analysis result with a second confidence level is output.

[0118] By simultaneously incorporating the determination of whether a defect exists and the determination of its level into the cross-validation comparison scope, cross-validation is upgraded from a single qualitative comparison of "whether a defect exists" to a multi-level quantitative comparison of "whether a defect exists and its level," thereby improving the accuracy of cross-validation and the operability of the evaluation results.

[0119] Based on the aforementioned embodiments, this embodiment describes the specific method for generating target prompt words.

[0120] After obtaining the pixel-level quantization metrics, the executing entity first assembles each quantization metric into a JSON object according to a preset key-value pair format. Specifically, the key is used to identify the metric category (e.g., sharpness, noise, color, exposure, etc.), and the value is used to record the corresponding numerical result. For example, the assembled JSON object can be represented as {"sharpness_score":85.3,"noise_level":12.1,"color_accuracy":72.5,"exposure_score":68.9}. It should be understood that the above field names and values ​​are only examples, and fields can be added, deleted, or adjusted according to specific needs in actual applications.

[0121] Subsequently, the executing entity adds the JSON object to the prompt word template according to the preset positions, forming the target prompt word. This prompt word template contains the following: system role definition, used to limit the multimodal large model to perform inference tasks as an image quality analyzer; analysis dimension definition, used to limit the evaluation direction of the input image, such as dimensions like autofocus, auto exposure, auto white balance, and image signal processing; defect level standards, used to constrain the model's description boundaries of the existence and severity of defects; structured text, i.e., the pixel-level quantification indicators carried by the aforementioned JSON object; and output format requirements, used to limit the organization method when the model generates a natural language report.

[0122] For example, the target prompt can be expressed semantically as follows: Acting as an image quality assessment engineer, combining the input image content with given quantitative indicators such as sharpness score, noise level, color accuracy, and exposure score, comprehensively judge the image quality according to the defined analysis dimensions and defect level standards, and output a natural language analysis report according to preset format requirements. After the target prompt is generated, it is input along with the input image into a multimodal large model, allowing the model to perform joint reasoning under the constraints of the system role definition, combining the defined analysis dimensions, defect level standards, and structured text to generate a quality analysis report.

[0123] It should be understood that the JSON format described above is only an example. In other implementations, structured text can also be described using natural language, structured tags, or dimensional injection, as long as the pixel-level quantization metrics can be organized in a form that can be parsed by a multimodal large model.

[0124] By assembling pixel-level quantization indicators into JSON objects according to a preset key-value pair format, the quantization data is organized in a structured key-value pair form. This allows the multimodal large model to quickly parse and accurately locate the quantization value corresponding to each dimension, avoiding parsing bias caused by data format chaos. By adding JSON objects to the prompt word template according to preset positions, the quantization data is accurately embedded in the prompt words at the positions corresponding to each analysis dimension. This enables the multimodal large model to accurately obtain the corresponding quantization basis for a certain dimension when performing analysis, achieving a precise correspondence between quantization data and analysis dimensions. Furthermore, by forming target prompt words that include system role definitions, analysis dimension definitions, defect level standards, structured text, and output format requirements, the multimodal large model obtains four layers of information—task role positioning, analysis rule constraints, quantization data support, and output format specifications—within the same prompt word, constructing a complete inference context and avoiding inference bias or output format non-compliance issues caused by incomplete prompt word information.

[0125] Based on the aforementioned embodiments, the pixel-level quantization metrics include at least the sharpness quantization value, noise quantization value, color quantization value, and exposure quantization value.

[0126] Specifically, the executing entity performs pixel-level calculations on the input image using computer vision algorithms, outputting sharpness quantization values, noise quantization values, color quantization values, and exposure quantization values. The sharpness quantization value characterizes the edge sharpness and detail retention of the input image; the noise quantization value characterizes the noise level and smoothness of the input image; the color quantization value characterizes the color reproduction accuracy and saturation performance of the input image; and the exposure quantization value characterizes the brightness distribution and dynamic range performance of the input image. These four quantization values ​​numerically characterize the physical imaging quality of the input image from four dimensions: edge detail, random noise, color distribution, and brightness distribution, collectively forming a complete pixel-level quantization index system.

[0127] After the four quantified values ​​are output, they are aggregated into a structured set of indicators for subsequent structured text conversion and multimodal analysis. This allows physical imaging defects in the input image, such as blurred edges, grainy noise, color distortion, and brightness imbalance, to be characterized in a unified data format. This avoids evaluation bias caused by single-dimensional assessments and provides a complete multi-dimensional quantitative basis for large multimodal models.

[0128] Based on the aforementioned embodiments, before assembling the pixel-level quantization indicators into a predefined structured text format and converting them into structured text, the method further includes mapping the clarity quantization value, noise quantization value, color quantization value, and exposure quantization value to the evaluation dimensions of the multimodal large model, and establishing the correlation between each quantization value and the corresponding evaluation dimension.

[0129] Specifically, the executing entity configures a dimension mapping relationship for each type of quantized value. This mapping relationship records the correspondence between the quantized value type and the evaluation dimension name. Specifically, the sharpness quantized value is mapped to the sharpness evaluation dimension, the noise quantized value to the noise evaluation dimension, the color quantized value to the color evaluation dimension, and the exposure quantized value to the exposure evaluation dimension. After establishing this association, when converting the quantized values ​​into structured text, the text fields can be named and organized according to the established evaluation dimensions, allowing the structured text to carry both numerical information and semantic meaning.

[0130] For example, if the clarity quantification value is 85.3, after mapping, this quantification value is labeled as a value associated with the clarity evaluation dimension; when the multimodal large model reads this structured text, it can directly understand this value as "clarity score of 85.3 points", rather than an isolated number.

[0131] By establishing the above-mentioned relationships, quantitative indicators no longer exist only in the form of isolated numerical values, but participate in subsequent reasoning as structured information bound to the evaluation dimensions. This enables the multimodal large model to directly establish a connection between numerical indicators and corresponding evaluation dimensions when reading prompt words, thereby improving the model's consistency in identifying sharpness, noise, color, and exposure defects and the interpretability of the results.

[0132] In one possible implementation, the sharpness quantization is calculated using the Laplacian variance method; the noise quantization is calculated using the median filter difference method; the color quantization is calculated using the HSV color space analysis method; and the exposure quantization is calculated using the luminance histogram analysis method.

[0133] Figures 4 to 7 The following are schematic flowcharts illustrating the sharpness calculation, noise level calculation, color accuracy calculation, and exposure accuracy calculation provided in embodiments of this application. Figures 4 to 7 Each will be explained separately.

[0134] Figure 4 This is a schematic diagram illustrating the resolution calculation process provided in an embodiment of this application. Figure 4As shown, the execution unit first converts the input image to grayscale to obtain a grayscale image; then, it performs a Laplacian second derivative operation on the grayscale image to obtain the Laplacian response value; finally, it calculates the variance of this Laplacian response value to obtain a sharpness quantification index. Specifically, the sharper the image and the more angular the edges, the larger the variance of the Laplacian response value; conversely, the blurrier the image and the smoother the edges, the smaller the variance of the Laplacian response value. By using the Laplacian variance method, the edge detail information of an image can be quantified into a calculable numerical value, providing an objective basis for sharpness evaluation.

[0135] Figure 5 This is a schematic diagram illustrating the noise level calculation process provided in an embodiment of this application. Figure 5 As shown, the execution unit first converts the input image to grayscale to obtain a grayscale image; then, it performs median filtering smoothing on the grayscale image to obtain a denoised reference image; next, it calculates the absolute difference between each pixel in the grayscale image and the denoised reference image to obtain a difference map; finally, it calculates the pixel mean of the difference map as an indicator of noise level. The larger the mean of the difference map, the stronger the high-frequency random noise in the image; the smaller the mean of the difference map, the smoother the image. The median filtering difference method can effectively separate noise components in an image, providing a quantitative basis for evaluating noise levels.

[0136] Figure 6 This is a schematic diagram illustrating the color accuracy calculation process provided in an embodiment of this application. Figure 6 As shown, the execution unit first converts the input image from the BGR color space to the HSV color space; then, it calculates the average saturation and average brightness of the entire image; finally, it calculates the color quantization value using a weighted formula. For example, the weighted formula could be: Color Score = Average Saturation / Maximum Saturation Value × 40 + Average Brightness / Maximum Brightness Value × 60. Through HSV color space analysis, the color performance of an image can be comprehensively evaluated from both saturation and brightness dimensions, providing a quantitative basis for color accuracy evaluation.

[0137] Figure 7 This is a schematic diagram illustrating the exposure accuracy calculation process provided in an embodiment of this application. Figure 7 As shown, the execution unit first converts the input image to grayscale to obtain a grayscale image; then, it calculates a multi-level brightness histogram of the grayscale image; it statistically analyzes the percentage of overexposed pixels (brightness values ​​above the upper threshold) and the percentage of underexposed pixels (brightness values ​​below the lower threshold); finally, it calculates the exposure quantization value using the formula: Exposure Score = 100 - Percentage of Overexposed Pixels - Percentage of Underexposed Pixels. Through brightness histogram analysis, the exposure level of an image can be evaluated from the perspective of pixel brightness distribution, providing a quantitative basis for evaluating exposure accuracy.

[0138] For example, the upper limit threshold of brightness value can be understood as the overexposure highlight threshold, and the lower limit threshold of brightness value can be understood as the underexposure dark area threshold. Various methods can be used to determine the upper and lower limits.

[0139] In one possible implementation, a fixed upper and lower threshold value can be set using a general standard threshold method.

[0140] For example, based on the actual needs of the application scenario, the overexposure highlight threshold and underexposure shadow area threshold corresponding to the application scenario can be set as the upper and lower limits, respectively. For instance, in a face recognition scenario, where the fixed overexposure highlight threshold is 40 and the underexposure shadow area threshold is 220, the upper limit threshold can be set to 40 and the lower limit threshold to 220. As another example, in a daily mobile phone photo quality inspection scenario, where the fixed overexposure highlight threshold is 30 and the underexposure shadow area threshold is 225, the upper limit threshold can be set to 30 and the lower limit threshold to 225.

[0141] In one possible implementation, an adaptive dynamic threshold method can be used to set the upper and lower thresholds of the dynamic value.

[0142] For example, automatically calculating the underexposed dark area threshold based on the image histogram involves taking the histogram. Figure 5 The 5% quantile brightness is used to determine the exposure level; values ​​below this value indicate underexposure. The overexposure highlight threshold is automatically calculated based on the image histogram, using the 95th quantile brightness; values ​​above this value indicate overexposure. Of course, the 5% and 95th quantile brightness values ​​can be dynamically determined according to the scene; for example, 2% and 98th quantile values ​​can also be used, which is suitable for high-precision image detection.

[0143] Through the above four calculation methods, sharpness, noise, color, and exposure can be output as independent quantized values, and each quantized value corresponds to a clear image statistical feature, which makes the image quality analysis results highly consistent and interpretable, and provides stable quantized input for subsequent structured text generation and multimodal judgment.

[0144] Based on the aforementioned embodiments, the sharpness quantization value and the noise quantization value are respectively normalized to a preset scoring range using preset empirical thresholds. Specifically, the sharpness quantization value adopts a forward normalization method, that is, the higher the sharpness quantization value, the higher the normalized score; the noise quantization value adopts a reverse normalization method, that is, the lower the noise quantization value, the higher the normalized score.

[0145] Specifically, the sharpness quantification value is represented by the variance of the Laplacian response value. The larger the variance value, the sharper the image edges and the richer the details. Therefore, when mapping it to the preset scoring interval, a forward normalization method is used to ensure that the normalized score maintains a positive correlation with image sharpness; that is, the sharper the image, the higher the score. The noise quantification value is represented by the mean of the median filter difference map. The larger the mean difference value, the stronger the random noise in the image. Therefore, when mapping it to the preset scoring interval, a reverse normalization method is used to ensure that the normalized score maintains a positive correlation with image purity; that is, the purer the image (i.e., the lower the noise level), the higher the score.

[0146] For example, the sharpness quantification value can be normalized to a preset scoring range using the formula "Sharpness Score = min(Laplacian variance / Empirical threshold × 100, 100)"; the noise quantification value can be normalized to a preset scoring range using the formula "Noise Score = max(0, 100 - Noise level / Empirical threshold × 100)".

[0147] Through the above normalization method, quantitative indicators of different dimensions are mapped to a unified preset scoring range, so that the scoring results and the actual imaging quality maintain a corresponding relationship, providing a unified quantitative basis for the subsequent structured analysis and cross-validation of multimodal large models.

[0148] Based on the aforementioned embodiments, the pixel-level quantization indicators are converted into structured text, and the structured text is added to the prompt words of the multimodal large model. The process of generating target prompt words also includes: when the quantization value of any dimension in the pixel-level quantization indicators is lower than a preset threshold, a region analysis instruction corresponding to that dimension is added to the prompt words, and a target prompt word containing the region analysis instruction is generated; the region analysis instruction is used to instruct the multimodal large model to perform defect analysis on the image region corresponding to that dimension when analyzing the input image.

[0149] For example, a preset threshold can be understood as a pre-set threshold used to determine anomalies in one or more dimensions of a pixel-level quantization metric. For instance, if the quantization value of any dimension in the pixel-level quantization metric is lower than the preset threshold, it can be determined that there is an anomaly in that dimension, and a region analysis command corresponding to that dimension can be added to the prompt.

[0150] When setting preset thresholds, you can make presets based on the actual needs of the application scenario. For example, you can set preset thresholds based on the statistical distribution of the measured data. For instance, if the pixel-level quantization index includes a sharpness quantization value, you can perform a statistical analysis on the sharpness quantization values ​​of multiple sample images in advance, and set the lower limit of the quantization value interval in which the sharpness quantization value is most likely (e.g., accounting for 60%, 80%, or 90%) as the judgment threshold for the sharpness quantization value.

[0151] For example, by performing Laplacian variance calculations on a large number of sample images, the sharpness quantification value of each sample image can be obtained. Statistical analysis can be performed on these sharpness quantification values. Assuming that 80% of the sharpness quantification values ​​fall within the quantization value range of 100 to 300, the lower limit of the range of 100 can be set as the preset threshold for the sharpness quantification value.

[0152] Similarly, the noise quantization value, color quantization value, exposure quantization value, or other indicators in pixel-level quantization can also be set with preset thresholds for each dimension using a similar method.

[0153] Specifically, in the process of generating target prompts, the executing entity first compares each pixel-level quantization indicator with its corresponding preset threshold. When the quantization value of any dimension is lower than the preset threshold, the executing entity generates a corresponding region analysis instruction based on the preset correspondence between that dimension and the image region. This region analysis instruction is then added to the prompt in the form of a natural language description, forming a target prompt containing the region analysis instruction.

[0154] For example, the preset correspondence between each dimension and the image region includes: the clarity quantization value corresponds to the edge region and the region with rich detail in the image; the noise quantization value corresponds to the dark region and the flat region in the image; the color quantization value corresponds to the colored region and the skin color region in the image; and the exposure quantization value corresponds to the highlight region and the shadow region in the image.

[0155] For example, when the sharpness quantization value is lower than a preset threshold, a region analysis instruction is generated: "Please perform defect analysis on edge regions and areas with rich detail in the image." When the noise quantization value is lower than a preset threshold, a region analysis instruction is generated: "Please perform defect analysis on dark regions and flat regions in the image." When the color quantization value is lower than a preset threshold, a region analysis instruction is generated: "Please perform defect analysis on colored regions and skin-toned regions in the image." When the exposure quantization value is lower than a preset threshold, a region analysis instruction is generated: "Please perform defect analysis on highlight regions or shadow regions in the image." These region analysis instructions are added to the prompt words in the form of natural language descriptions, and together with the structured text, they constitute the target prompt words.

[0156] In this way, when there is an anomaly in the imaging quality of a certain dimension of the image, the multimodal large model can perform defect analysis on the region corresponding to the abnormal dimension in the image based on the region analysis instructions in the target prompt. This allows the analysis resources to focus on the region where there may be defects, avoiding the problem of missing local anomalies when analyzing the whole image uniformly. This improves the detection accuracy and analysis efficiency of the multimodal large model for local quality defects.

[0157] The electronic device provided in this application embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0158] This application provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, these instructions are used to implement the method described above. The implementation principle and technical effects are similar, and will not be elaborated upon here.

[0159] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods in any of the above method embodiments.

[0160] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0161] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0163] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0164] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the equivalent technology of this application, this application also intends to include these modifications and variations.

[0165] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0166] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0167] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0168] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0169] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0170] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0171] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. An image quality analysis method, characterized in that, The method includes: Obtain the input image; The input image is subjected to pixel-level operations using computer vision algorithms to extract pixel-level quantization metrics, which are used to characterize the pixel-level physical imaging quality features of the input image. The pixel-level operations include fine-grained processing of the pixel matrix of the input image using computer vision algorithms to obtain at least one pixel-level quantization metric. The pixel-level quantization metric includes at least one of sharpness quantization, noise quantization, color quantization, and exposure quantization. The pixel-level quantization indicators are assembled into a predefined structured text format, converted into structured text, and then added to the prompt words of the multimodal large model to generate target prompt words. The input image and the target prompt are input into the multimodal large model. The multimodal large model combines the structured text to perform multimodal analysis on the input image to analyze the image quality of the dimension corresponding to the pixel-level quantization index and generate a quality analysis report described in natural language.

2. The method according to claim 1, characterized in that, The method further includes: Cross-validate the quality analysis report with the pixel-level quantification indicators; When the defect determination conclusion in the quality analysis report is consistent with the defect determination conclusion reflected by the pixel-level quantitative index, the analysis result with the first confidence level is output. When the defect determination conclusion in the quality analysis report is inconsistent with the defect determination conclusion reflected by the pixel-level quantitative index, the analysis result of the second confidence level is output, and the input image is marked as to be reviewed. The first confidence level is higher than the second confidence level.

3. The method according to claim 2, characterized in that, The defect determination conclusion includes determining whether a defect exists and determining the defect level.

4. The method according to claim 1, characterized in that, Adding the structured text to the prompts in the multimodal large model includes: The pixel-level quantization metrics are assembled into a JSON object according to a preset key-value pair format; The JSON object is added to the prompt word template according to the preset position to form a target prompt word that includes system role definition, analysis dimension definition, defect level standard, the structured text, and output format requirements.

5. The method according to claim 1, characterized in that, Before assembling the pixel-level quantization indicators into structured text according to a predefined structured text format, the process further includes: The clarity quantization value, noise quantization value, color quantization value, and exposure quantization value are respectively mapped to the evaluation dimensions of the multimodal large model to establish the correlation between each quantization value and the corresponding evaluation dimension.

6. The method according to claim 1, characterized in that, The clarity quantization value is calculated using the Laplacian variance method; the noise quantization value is calculated using the median filter difference method; the color quantization value is calculated using the HSV color space analysis method; and the exposure quantization value is calculated using the luminance histogram analysis method.

7. The method according to claim 6, characterized in that, The clarity quantization value and the noise quantization value are respectively normalized to a preset scoring range through a preset empirical threshold. The clarity quantization value adopts a positive normalization method, which means that the higher the clarity quantization value, the higher the normalized score. The noise quantization value is obtained by reverse normalization, which means that the lower the noise quantization value, the higher the score after normalization.

8. The method according to claim 1, characterized in that, The step of assembling the pixel-level quantization indicators according to a predefined structured text format, converting them into structured text, and adding the structured text to the prompt words of the multimodal large model to generate target prompt words includes: When the quantization value of any dimension in the pixel-level quantization index is lower than a preset threshold, a region analysis instruction corresponding to the dimension is added to the prompt word, and a target prompt word containing the region analysis instruction is generated. The region analysis instruction is used to instruct the multimodal large model to perform defect analysis on the image region corresponding to the dimension when analyzing the input image.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

Images