Device testing method, apparatus, and device

Abstract

Embodiments of the present application provide a device testing method, device and equipment to improve testing efficiency, wherein the method comprises: grouping a plurality of first devices according to reference images corresponding to the first devices, determining at least one first device combination, comparing a reference image corresponding to a target first device in each first device combination with a test image corresponding to a second device to obtain a comparison result between the target first device and the second device, and determining a test result of the second device according to the comparison result corresponding to each target first device, wherein the first device combination includes at least two first devices with similar performance, the comparison result includes at least a scene consistency comparison result, a picture quality comparison result and a dimension consistency comparison result under different model temperatures, and the test result includes the comparison result corresponding to the second device and each first device. The method is used to improve testing efficiency.

Description

Technical Field

[0001] This application relates to the fields of image processing and artificial intelligence, and in particular to a device testing method, apparatus, and device. Background Technology

[0002] In the research and development and quality evaluation of smart terminal devices (such as mobile phones, action cameras, drone cameras, and vehicle cameras), multi-device comparative testing is a core step. This step requires comparing and analyzing the shooting effects of the test device with multiple reference devices (such as industry benchmark devices, historical iteration devices, or competing devices) to identify the strengths and weaknesses of the test device in image shooting.

[0003] In related technologies, the test images of the test equipment and the reference images of N reference equipment (N≥2) are usually input into a multimodal image comparison model one by one. The semantic understanding capability of the multimodal image comparison model is used to compare the differences between the test images and each reference image to determine the performance of the test equipment.

[0004] However, the testing methods in related technologies suffer from poor testing efficiency. Summary of the Invention

[0005] This application provides a device testing method, apparatus, and device to improve testing efficiency.

[0006] In a first aspect, embodiments of this application provide a device testing method, the method comprising:

[0007] Based on the reference images corresponding to multiple first devices, each first device is grouped to determine at least one combination of first devices, and the combination of first devices includes at least two first devices with similar performance.

[0008] The reference image corresponding to the target first device in each first device combination is compared with the test image corresponding to the second device to obtain the comparison results between each target first device and the second device. The comparison results include at least the image scene consistency comparison results, the image quality comparison results, and the dimensionality consistency comparison results under different model temperatures.

[0009] The test results of the second device are determined based on the comparison results of the first device for each target. The test results include the comparison results between the second device and each first device.

[0010] In one possible embodiment, based on reference images corresponding to multiple first devices, the first devices are grouped to determine at least one combination of first devices, including:

[0011] Feature extraction is performed on the reference image to obtain the image features corresponding to the reference image. The image features include device visual features, scene features, and color distribution features.

[0012] Based on a preset clustering algorithm, each first device is clustered according to image features to determine at least one combination of first devices.

[0013] In one possible embodiment, the method further includes:

[0014] Determine the cluster center corresponding to the first equipment combination;

[0015] At least one target first device corresponding to the first device combination is determined based on the cluster center of the first device combination.

[0016] In one possible embodiment, the reference image corresponding to the target first device in each first device combination is compared with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device, including:

[0017] Based on the first prompt word, a multimodal image comparison model is used to compare the reference image corresponding to the first target device with the test image corresponding to the second device in batches to determine the first comparison result corresponding to each target first device. In the image quality comparison process, the first prompt word only compares the differences between the test image and each reference image in multiple image quality dimensions, including sharpness, noise, color, and exposure.

[0018] Based on the first comparison results, at least one weak dimension device is identified from the target first device. The weak dimension device is the target first device with the best imaging performance in the image quality dimension where the second device has a weakness, and is used for the second comparison.

[0019] Based on the second prompt word, a multimodal image comparison model is used to compare the reference image corresponding to the weak dimension device with the test image corresponding to the second device to determine the second comparison result. In the image quality comparison process, in addition to comparing the differences between the test image and each reference image in multiple image quality dimensions, the second prompt word also compares the performance differences between the test image and each reference image in the same image area, which includes faces, sky and text.

[0020] The second comparison result corresponding to the weak dimension device and the first comparison result corresponding to the target first device other than the weak dimension device are used as the comparison result corresponding to each target first device.

[0021] In one possible embodiment, based on the second cue word, a second comparison is performed between the reference image corresponding to the weaker dimension device and the test image corresponding to the second device using a multimodal image comparison model, including:

[0022] If the number of devices with weaker dimensions exceeds the maximum number of comparisons in a single instance, the devices with weaker dimensions will be divided into multiple batches based on the image quality dimension corresponding to the devices with weaker dimensions.

[0023] Based on the priority of the image quality dimension corresponding to each batch of weak dimension devices, and using the second prompt word, a second comparison is conducted on the reference image corresponding to the weak dimension device and the test image corresponding to the second device in batches through a multimodal image comparison model.

[0024] In one possible embodiment, the test result of the second device is determined based on the comparison results corresponding to each target first device, including:

[0025] Based on the comparison results corresponding to each target first device, determine the comparison results corresponding to each first device in the first device combination to which the target first device belongs;

[0026] Based on the comparison results of each first device, the test results of the second device are determined.

[0027] In one possible embodiment, the method further includes:

[0028] The confidence level of the test results is determined based on the comparison results of the first device for each target. The confidence level is used to characterize the credibility of the test results.

[0029] The test results for the second device are updated based on the confidence level of the results.

[0030] In one possible embodiment, the result confidence level corresponding to the test result is determined based on the comparison results of each target first device. The result confidence level is used to characterize the credibility of the test result, including:

[0031] Based on the degree of matching of scene features between the reference image and the test image in the image scene consistency comparison results of each comparison result, the first confidence factor is determined.

[0032] Based on the cosine similarity between the device visual features of the reference image and the test image in the image quality comparison results of each comparison result, the second confidence factor is determined.

[0033] The third confidence factor is determined based on the dimensionality consistency comparison results under different model temperatures in each comparison result.

[0034] The weighted sum of the first confidence factor, the second confidence factor, and the third confidence factor is used to determine the result confidence level.

[0035] Secondly, embodiments of this application provide a device testing apparatus, the apparatus comprising:

[0036] The clustering and grouping module is used to group each first device according to the reference images corresponding to multiple first devices, and determine at least one combination of first devices, wherein the combination of first devices includes at least two first devices with similar performance.

[0037] The comparison module is used to compare the reference image corresponding to the target first device in each first device combination with the test image corresponding to the second device to obtain the comparison results between each target first device and the second device. The comparison results include at least the image scene consistency comparison results, the image quality comparison results, and the dimensionality consistency comparison results under different model temperatures.

[0038] The result generation module is used to determine the test results of the second device based on the comparison results corresponding to each target first device. The test results include the comparison results between the second device and each first device.

[0039] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0040] The memory stores instructions that the computer executes;

[0041] The processor executes computer execution instructions stored in memory, causing the processor to perform the methods described above.

[0042] This application provides a device testing method, apparatus, and device to improve testing efficiency. The method includes: grouping each first device according to reference images corresponding to multiple first devices to determine at least one combination of first devices; comparing the reference image corresponding to a target first device in each combination of first devices with the test image corresponding to a second device to obtain a comparison result between each target first device and the second device; and determining the test result of the second device based on the comparison result of each target first device. The first device combination includes at least two first devices with similar performance. The comparison result includes at least image scene consistency comparison result, image quality comparison result, and dimensional consistency comparison result under different model temperatures. The test result includes the comparison result between the second device and each first device. In this way, based on the reference images corresponding to multiple first devices, each first device is grouped to determine at least one combination of first devices containing at least two first devices with similar performance. Based on the comparison results obtained by comparing the reference image corresponding to the target first device in each combination of first devices with the test image corresponding to the second device, the test results including the comparison results between the second device and all first devices can be obtained. Compared with the process of comparing each first device with the second device in related technologies, the technical solution provided by this application greatly reduces the number of comparisons and improves the testing efficiency. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] Figure 1A flowchart illustrating the device testing method provided in the embodiments of this application. Figure 1 ;

[0045] Figure 2 A flowchart illustrating the device testing method provided in the embodiments of this application. Figure 2 ;

[0046] Figure 3 A flowchart illustrating the device testing method provided in the embodiments of this application. Figure 3 ;

[0047] Figure 4 Flowchart of the device testing method provided in the embodiments of this application Figure 4 ;

[0048] Figure 5 This is a schematic diagram of the structure of the equipment testing apparatus provided in the embodiments of this application;

[0049] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0050] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] This technology relates to the field of intelligent terminal device testing and image semantic comparison analysis, and is particularly applicable to image research and development testing scenarios for devices such as mobile phones, action cameras, and vehicle cameras. In such scenarios, test images are typically acquired using a second device, and corresponding reference images are acquired using multiple first devices. The imaging performance of the second device is then evaluated based on the image comparison results.

[0053] In the research and development and quality evaluation of smart terminal devices (such as mobile phones, action cameras, drone cameras, and vehicle cameras), multi-device comparative testing is a core component. Test engineers need to compare and analyze the shooting effects of the device under test (test machine) with multiple reference devices (such as industry benchmark devices, historical iteration devices, or competing devices) to identify the strengths and weaknesses of the test machine in terms of image quality, color reproduction, noise control, dynamic range, and other dimensions.

[0054] For example, in mobile image optimization, test engineers may need to compare the test device with eight reference devices of different brands or models at the same time to evaluate the differences in performance in different scenarios (such as night scene, backlight, macro).

[0055] In related technologies, this testing process mainly relies on the following schemes:

[0056] (1) Manual pairwise visual comparison: The test engineer pairs the image of the test equipment with the image of each reference equipment one by one, and subjectively judges the advantages and disadvantages of each dimension after visual inspection. This method is highly dependent on the engineer's experience, has low efficiency (about 3-5 minutes for each comparison, about 24-40 minutes for 8 comparisons), and as the number of reference equipment increases, the engineer's memory load and judgment consistency decrease sharply.

[0057] (2) Algorithm comparison based on pixel-level residuals: Pixel-level differences (such as mean square error, structural similarity, histogram differences, etc.) are calculated between the test image and the reference image. This type of method requires that the test image and the reference image have highly consistent viewpoints, times, and scene conditions when they are captured, and is extremely sensitive to small viewpoint deviations or time differences that are common in actual tests.

[0058] (3) Comparison based on single-dimensional quantization: Traditional computer vision (CV) algorithms are used to calculate single-dimensional quantization indicators such as sharpness, noise, color, and exposure for each image, and then numerical comparisons are performed. This method cannot output semantic-level differences, and there is a lack of a unified evaluation benchmark between the quantization scores of different dimensions.

[0059] (4) Pairwise semantic comparison based on multimodal large model: The semantic understanding capability of multimodal large model is used to input the test image and a single reference image simultaneously for semantic level comparison. However, in the multi-machine comparison scenario of 1 test device and N reference devices, the existing technology usually adopts the "looping pairwise call" method - the multimodal large model is called once for each reference device, which requires a total of N model calls.

[0060] Understandably, when the second device (test device) needs to compare with multiple first devices (reference devices) at the same time, the manual method relies heavily on experience, which not only has a long processing cycle, but also makes it difficult to unify the judgment standards among different people. Traditional image quality indicators also rely heavily on the consistency of shooting time, angle and lighting conditions. Once there is a scene deviation between the reference image and the test image, the results are difficult to stably reflect the real semantic differences.

[0061] Using a large multimodal image comparison model requires calling the model for each of the first devices and comparing it with the second device. The number of comparisons increases significantly with the number of first devices, resulting in high computational and interface call overhead and low testing efficiency.

[0062] Meanwhile, in practice, it has been found that when N≥6, the multimodal image comparison model tends to focus only on the first 2-3 images and the last 1-2 images, while the images in the middle are easily ignored or confused with other images. Therefore, when the number of first devices is too large, it will also affect the accuracy of the final test results.

[0063] Therefore, the testing methods in related technologies all suffer from poor testing efficiency. In view of this, the embodiments of this application provide a device testing method, apparatus, and device to improve testing efficiency. The method includes: grouping each first device according to reference images corresponding to multiple first devices to determine at least one combination of first devices; comparing the reference image corresponding to the target first device in each combination of first devices with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device; and determining the test result of the second device according to the comparison result corresponding to each target first device. The first device combination includes at least two first devices with similar performance. The comparison result includes at least the image scene consistency comparison result, the image quality comparison result, and the dimensionality consistency comparison result under different model temperatures. The test result includes the comparison result between the second device and each first device. In this way, based on the reference images corresponding to multiple first devices, each first device is grouped to determine at least one combination of first devices containing at least two first devices with similar performance. Based on the comparison results obtained by comparing the reference image corresponding to the target first device in each combination of first devices with the test image corresponding to the second device, the test results including the comparison results between the second device and all first devices can be obtained. Compared with the process of comparing each first device with the second device in related technologies, the technical solution provided by this application greatly reduces the number of comparisons and improves the testing efficiency.

[0064] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0065] Figure 1 Flowchart of the device testing method provided in the embodiments of this application Figure 1 In the embodiments of this application, reference is made to... Figure 1 The method may include:

[0066] S101, based on the reference images corresponding to the multiple first devices, group each first device into groups to determine at least one combination of first devices.

[0067] The first equipment combination includes at least two first equipment with similar performance.

[0068] Here, the first device may refer to a device that serves as a reference object, and the reference image is image data used to characterize the imaging effect of each first device.

[0069] Correspondingly, the first equipment combination is a set of first equipment with similar performance formed after grouping.

[0070] In this application embodiment, it is necessary to obtain a set of reference images collected and stored by multiple first devices under the same test theme. In some embodiments, each first device corresponds to at least one reference image, and in some embodiments, the reference image corresponding to each first device can cover different scenarios.

[0071] Regarding the process of acquiring a reference image, for example, a reference image actively provided by the user can be acquired through an external input device; for example, a reference image pre-stored at a preset storage location can be read from a specified reading path.

[0072] After obtaining the reference image corresponding to each first device, the first devices can be grouped according to the reference image to form multiple combinations of first devices, and the first devices in each combination of first devices have similar performance.

[0073] Regarding the grouping process, in one possible implementation, each reference image can be directly input into a preset classification model to obtain the classification result output by the model.

[0074] In another possible implementation, image features of each reference image can be extracted based on different feature dimensions, and each first device can be classified and combined based on the image features.

[0075] It is understandable that the reference image can reflect the imaging performance of its corresponding first device. Therefore, the extracted image features are performance-related features, such as device visual features, scene features, and color distribution features.

[0076] Regarding the process of classifying and combining each first device based on image features, in one possible implementation, the image features of each reference image can be processed by a preset clustering algorithm, such as the k-means clustering algorithm, to obtain multiple clusters, and the first devices in each cluster form a first device combination.

[0077] S102, compare the reference image corresponding to the target first device in each first device combination with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device.

[0078] The comparison results include at least the image scene consistency comparison results, the image quality comparison results, and the dimensionality consistency comparison results under different model temperatures.

[0079] Here, the target first device is the representative device selected from each first device combination for comparison.

[0080] Here, the second device is the device to be tested, and the test image is the image output by the second device and used as the comparison object.

[0081] Here, the comparison results are the comparison outputs formed between the first target device and the second target device. Among them, the image scene consistency comparison results are used to measure the degree of scene matching between the reference image and the test image; the image quality comparison results are used to measure the differences between the two in terms of sharpness, noise, color and exposure; and the dimensional consistency comparison results at different model temperatures are used to measure the degree of consistency of the image comparison model's judgments on the same pair of images under different temperature parameter conditions.

[0082] Regarding the process of determining the target first device from each first device combination, if there is only one first device in each first device combination, then that first device can be directly determined as the target first device.

[0083] If the first device combination includes multiple first devices, in one possible implementation, one of the first devices can be randomly selected as the target first device; in another possible implementation, if the first device combination is determined based on a clustering algorithm, the first device closest to the cluster center can be selected as the target first device of the first device combination.

[0084] In this embodiment of the application, for each target first device, a relevant annotation of the first device group to which the target first device belongs can be added to its corresponding features or comparison prompts to indicate that the target first device is a representative device of the first device group and its performance is superior within the first device group. For example,

[0085] Once the target first device corresponding to each combination of first devices is determined, a comparison process can be performed based on the reference image corresponding to each target first device and the test image corresponding to the second device.

[0086] In one possible implementation, for a single comparison process, in some implementations, a reference image corresponding to a single target first device and a test image of the second device may be compared; in some implementations, the test images corresponding to each target first device may be divided into multiple batches, and the differences between multiple reference images and test images in each batch may be compared simultaneously to improve comparison efficiency.

[0087] Regarding the comparison process, in one possible implementation, the reference and test images to be compared can be preprocessed, such as alignment processing. Alignment processing includes at least scene label matching, image size unification, cropping region standardization, and necessary geometric correction, so that subsequent comparisons are based on comparable inputs.

[0088] After obtaining the preprocessed test image and reference image, image scene consistency comparison, image quality comparison, and comparison of the same reference image and test image at different model temperatures can be performed to obtain the corresponding comparison results.

[0089] For each round of comparison, in one possible implementation, the test image and reference image to be compared can be directly input into a preset multimodal image comparison model for processing, thereby obtaining the comparison results output by the model. The comparison results include image scene consistency comparison results, image quality comparison results, and dimensional consistency comparison results at different model temperatures.

[0090] In another possible implementation, three comparisons can be performed according to preset comparison rules. For example, for the image scene consistency comparison process, the image scenes of the reference image and the test image can be identified and consistency matching can be performed to obtain a scene consistency score. When the scene consistency score reaches a preset threshold, it is determined that the two images are scene comparable and the corresponding image scene consistency comparison result is output. When the threshold is not reached, a low consistency result is also output and the state is written into the comparison record. For the image quality comparison process, the image quality dimension of the reference image and the test image is compared and analyzed to obtain the image quality comparison result. The image quality comparison result can be a multi-dimensional value or a relative relationship label. For example, the second device is higher than the target first device in the clarity dimension, lower than the target first device in the noise control dimension, and close to the target first device in the color reproduction dimension.

[0091] Understandably, the comparison process can also be completed by the model. Therefore, to improve the dimensions of the comparison, it is also necessary to compare the differences between the comparison results output by the test image and the reference image under the same comparison rules at different model temperatures. Here, the model temperature is used to adjust the dispersion of the model output distribution. For each temperature value, the model will output the judgment results for several evaluation dimensions. The evaluation dimensions include at least scene understanding consistency, subject restoration degree, color tendency, brightness and darkness level, and detail performance. Then, the consistency of the judgment results of the same dimension at each temperature is statistically analyzed. When the judgment label of the same dimension is the same at multiple temperatures, or the corresponding score deviation does not exceed the preset tolerance, it is determined that the dimension has consistent judgment. The consistency of each dimension is further summarized to obtain the dimension consistency comparison results at different model temperatures.

[0092] S103, determine the test results of the second device based on the comparison results of the first device for each target.

[0093] The test results include comparisons between the second device and each of the first devices.

[0094] Here, the test results represent the final test conclusions for the second device.

[0095] In this embodiment of the application, the comparison result corresponding to each first device can be determined firstly based on the comparison result corresponding to each target first device, and then the test result can be determined based on the comparison result corresponding to each first device.

[0096] Regarding the process of determining the comparison results of each first device based on the comparison results corresponding to each target first device, in one possible implementation, the process may include: determining the comparison results corresponding to each first device in the first device combination to which the target first device belongs, based on the comparison results corresponding to each target first device; and determining the test results of the second device based on the comparison results corresponding to each first device.

[0097] In the embodiments of this application, each first device has its corresponding first device combination, and each first device combination has its corresponding target first device.

[0098] Therefore, in this embodiment of the application, the comparison result of the first device in the first device combination to which the target first device belongs can be determined by the target first device.

[0099] In one possible implementation, the comparison result corresponding to each target first device can be directly used as the comparison result of each corresponding first device.

[0100] In another possible implementation, although the first devices in the same first device combination have similar performance, they also have differences. Different compensation coefficients can be set according to the differences between the target first device and other first devices, and the comparison results of each first device can be determined according to the comparison results of each compensation coefficient and the target first device.

[0101] In this way, by deducing the comparison results of each target first device from the comparison results of each first device, the number of comparisons is greatly reduced, and the efficiency of the testing process is further improved.

[0102] Based on the above analysis, this application provides a device testing method, including grouping each first device according to reference images corresponding to multiple first devices to determine at least one combination of first devices, wherein the first device combination includes at least two first devices with similar performance; comparing the reference image corresponding to the target first device in each first device combination with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device, wherein the comparison result includes at least the image scene consistency comparison result, the image quality comparison result, and the dimensionality consistency comparison result under different model temperatures; and determining the test result of the second device according to the comparison result corresponding to each target first device, wherein the test result includes the comparison result between the second device and each first device. In the above embodiments, each first device is grouped according to the reference images corresponding to multiple first devices, and at least one group of first device combinations containing at least two first devices with similar performance is determined. The test results, including the comparison results between the second device and all first devices, can be obtained by comparing the reference image corresponding to the target first device in each first device combination with the test image corresponding to the second device. Compared with the process of comparing each first device with the second device in related technologies, the technical solution provided by this application greatly reduces the number of comparisons and improves the test efficiency.

[0103] Figure 2 Flowchart of the device testing method provided in the embodiments of this application Figure 2 In the embodiments of this application, reference is made to... Figure 2 The process of step S101 above, "grouping each first device according to the reference image corresponding to the multiple first devices, and determining at least one combination of first devices", may include:

[0104] S1011, Perform feature extraction on the reference image to obtain the image features corresponding to the reference image.

[0105] Image features include device visual features, scene features, and color distribution features.

[0106] Here, device visual features can be quantified feature vectors that characterize the basic image quality capabilities of device hardware imaging. These vectors can include multiple dimensions, which can be composed of traditional computer vision quantification metrics, such as sharpness score, noise level, color accuracy, and exposure score. In this embodiment, a preset feature extraction network can be used to process the reference image to obtain the device visual features corresponding to the reference image.

[0107] Here, scene features can refer to structured feature vectors that characterize the content attributes of an image. The extraction method can be to use a multimodal large model lightweight inference to identify image scene category labels, and then convert the category labels into one-hot encoded vectors to distinguish different scene scenes such as faces, sky, text, and night scenes.

[0108] Here, color distribution features can refer to numerical feature vectors that characterize the overall tone style and pixel color distribution patterns of an image. The extraction method can be to convert the test image to a preset color space, such as the hue, saturation, and brightness (HSV) color space, and to statistically analyze a 16-segment tone histogram. The histogram values ​​are used to construct a fixed-dimensional feature vector, thereby quantifying the differences in color distribution in the image.

[0109] S1012, based on a preset clustering algorithm, cluster each first device according to image features to determine at least one combination of first devices.

[0110] In this embodiment of the application, the preset clustering algorithm can adopt the K-means (K-means clustering) algorithm to iteratively calculate the performance feature vector, complete the classification by comparing the distance between the image features corresponding to each reference image and the cluster center, and classify the first devices corresponding to each reference image into the same first device group according to the clustering results.

[0111] In some embodiments, when there are a large number of reference images, the number of clusters can be set to a value based on the number of images estimated, so that the images in the same group have a high degree of similarity in image quality attributes, shooting environment and color distribution.

[0112] In the above embodiments, this scheme jointly extracts multi-dimensional image features from the reference image and performs unsupervised clustering, grouping first devices with similar imaging performance into the same group, thereby forming a consistent set of first devices. By adopting this method, when multiple first devices are compared, devices with similar features can be grouped together, reducing invalid comparison objects in subsequent comparisons. This makes the division results of the first device groups closer to actual imaging differences, thereby improving the accuracy of device grouping and the stability of subsequent test conclusions.

[0113] Based on the foregoing embodiments, the process of determining a target first device from a first device combination may include: determining a cluster center corresponding to the first device combination; and determining at least one target first device corresponding to the first device combination based on the cluster center of the first device combination.

[0114] In this embodiment of the application, for each cluster corresponding to the first device combination, a cluster center can be calculated. Based on the cluster center, the distance between each first device and the cluster center can be calculated, and the one with the smallest distance is taken as the target first device.

[0115] In some implementations, if there are multiple devices in the same first device group, the correspondence between the cluster center and each device can stably reflect the performance distribution within the group, and the selection result of the target first device can also be directly used to compare its reference image with the test image of the second device.

[0116] In this way, the representative equipment in the first equipment group can be determined by the cluster center constraint. The selection of the target first equipment does not depend on manual judgment and can take into account both the overall characteristics of the group and the performance of a single dimension, thereby making the subsequent comparison objects more representative and ensuring the consistency and repeatability of the test conclusions of the second equipment.

[0117] Figure 3 Flowchart of the device testing method provided in the embodiments of this application Figure 3 In the embodiments of this application, reference is made to... Figure 3 The process of step S102 above, "comparing the reference image corresponding to the target first device in each first device combination with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device," may include:

[0118] S1021, based on the first prompt word, through a multimodal image comparison model, the reference image corresponding to the first target device is compared with the test image corresponding to the second device in batches to determine the first comparison result corresponding to each first target device.

[0119] In the image quality comparison process, the first prompt word only compares the differences between the test image and each reference image in multiple image quality dimensions, including sharpness, noise, color, and exposure.

[0120] In this embodiment of the application, the comparison process is completed based on a large multimodal image comparison model and preset prompts. It is understood that the total number of first devices is large, and directly using detailed and complete prompts to conduct in-depth comparisons of each first device would result in huge computing power consumption and extremely long time consumption. Therefore, coarse prompts can be used for rough screening and comparison first. Coarse prompts have short instructions, less output content, and faster reasoning speed, making them more suitable for rapid assessment of large batches.

[0121] In the embodiments of this application, the first comparison is a rough comparison process, and the corresponding first prompt word is a lightweight coarse screening prompt word. In the process of image quality comparison, the model is only required to determine the macroscopic advantages and disadvantages of the four dimensions of sharpness, noise, color and exposure, without analyzing the details of local areas.

[0122] It is understandable that the number of images that a multimodal image comparison model can process in a single run is limited. This application can group all reference images of the target first device, with the number of reference images in each group not exceeding the upper limit of the multimodal image comparison model's single image input. Each group is paired with the same test image, input into the multimodal image comparison model, and pass in the first prompt word to constrain the model to compare only the four major image quality dimensions. After each group's inference is completed, the first comparison result corresponding to each first device in the group can be generated.

[0123] In the embodiments of this application, the image quality comparison results in the first comparison results only include the judgment of the superiority or inferiority of each image quality dimension between the test image and the reference image, without any local detailed description.

[0124] In this way, the initial screening of all devices can be completed quickly in batches, which greatly reduces the overall call cost and time consumption. The image quality comparison results in the first comparison result only retain simple judgments at the dimensional level, reducing redundant output of the model and improving processing efficiency. In addition, the number of images input to the model is limited during each batch comparison, and batch execution avoids errors due to exceeding the limit.

[0125] S1022, Based on the first comparison result, at least one weak dimension device is identified from the target first device.

[0126] Among them, the weak dimension device is the first target device that has the best imaging performance in the image quality dimension where the second device has a weakness and is used for the second comparison.

[0127] Here, the weak dimension refers to the set of image quality dimensions in which the second device's imaging is significantly inferior to that of most first devices, and the weak dimension device is the target first device that performs best in the weak dimension.

[0128] Understandably, the difference between most ordinary first devices and the second devices under test is too small to warrant a detailed comparison process, and their comparative reference value is low. Therefore, only the benchmark device with the best performance in the weakest dimension (the weak dimension device) is retained for the subsequent detailed second comparison process. This can accurately expose the real image quality difference of the device under test, further reduce the number of comparisons, and improve testing efficiency.

[0129] Regarding the process of identifying weak dimension devices, in one possible implementation, the first comparison results corresponding to each target first device can be summarized to statistically determine the overall weaker image quality dimension of the second device; then, the reference images corresponding to all target first devices are traversed, and their image features are identified and compared to select a batch of first devices with the best imaging effect in the above-mentioned weak dimension and marked as weak dimension devices.

[0130] S1023, Based on the second prompt word, a second comparison is performed between the reference image corresponding to the weak dimension device and the test image corresponding to the second device through a multimodal image comparison model, and the second comparison result is determined.

[0131] In the image quality comparison process, the second prompt word not only compares the differences between the test image and each reference image in multiple image quality dimensions, but also compares the performance differences between the test image and each reference image in the same image area, which includes faces, sky and text.

[0132] In this embodiment, the second comparison refers to a deep and detailed comparison task for multi-dimensional devices. This process is based on a large multi-dimensional image comparison model and more detailed and complete second prompt words. In this embodiment, the second comparison result obtained after processing based on the second prompt words not only includes the quality dimension of the image, but also includes a complete structured conclusion on the differences in details and local defects between the reference image and the test image in the same area.

[0133] The process of step S1023, "Based on the second prompt word, using a multimodal image comparison model, perform a second comparison between the reference image corresponding to the weak dimension device and the test image corresponding to the second device to determine the second comparison result," may include: if the number of weak dimension devices is greater than the maximum number of comparisons in a single instance, the weak dimension devices are divided into multiple batches according to the image quality dimension corresponding to the weak dimension devices; based on the priority of the image quality dimension corresponding to each batch of weak dimension devices, using a multimodal image comparison model, the reference image corresponding to the weak dimension device and the test image corresponding to the second device are compared in batches according to the second prompt word.

[0134] Here, the maximum number of comparisons in a single instance can refer to the maximum number of reference images that the multi-dimensional image comparison large model interface supports being passed in at one time, which is limited by the model input window.

[0135] It is understandable that large multimodal image comparison models have a fixed upper limit on the number of images that can be input. If too many images are fed into a batch, it will trigger interface errors, distract the model's attention, and reduce the accuracy of the judgment. Directly randomizing the batch will result in messy optimization dimensions for the same batch of devices, and subsequent analysis will be fragmented.

[0136] Therefore, in this embodiment, devices with each weak dimension can be grouped according to their corresponding weak image quality dimension so that the weak dimension devices in the same batch undergoing the second comparison are aligned in the same direction.

[0137] Regarding the batching process, in one possible implementation, the total number of all selected weak dimension devices can be counted and compared with the maximum number of comparisons in a single batch. If the total number of dimension devices exceeds the maximum number of comparisons in a single batch, they can be classified and grouped according to the weak dimensions that each weak dimension device is good at optimizing.

[0138] For example, if the weak dimensions of the second device are noise and color, then devices with excellent noise control are grouped into one group and devices with excellent color reproduction are grouped into another group, and the number of devices in each dimension does not exceed the maximum number of comparisons in a single instance.

[0139] Understandably, during the iterative processing of a large multimodal image comparison model, the model's attention becomes increasingly scattered as the number of processing iterations increases. Furthermore, during testing, priority is given to weaker dimensions, such as prioritizing the resolution of severe, critical image quality issues. Therefore, it's advisable to perform batch comparisons of devices with high-priority weaker dimensions first, generating detailed regional difference data for high-priority defects, thus making the comparison results for these devices more accurate.

[0140] In this embodiment, batches can be sorted from high to low priority, with high-priority batches processed first and low-priority batches processed later. All reference images of a single batch are input into the multimodal image comparison model along with test images, and the second prompt word is loaded to perform in-depth comparison. After each batch of inference is completed, the second comparison results corresponding to all weak dimension devices in that batch can be stored separately.

[0141] In this way, the entire batch logic solves the scenario where the benchmark equipment still exceeds the limit after coarse screening. It not only meets the hardware input constraints of the large model, but also combines the priority scheduling calculation of defect business, ensuring the stability of the test without losing fine-grained comparison information.

[0142] S1024, take the second comparison result corresponding to the weak dimension device and the first comparison result corresponding to the target first device other than the weak dimension device as the comparison result corresponding to each target first device.

[0143] In this embodiment of the application, for the weak dimension device, the second comparison result with more complete information is used as the final comparison data of the device, while for other ordinary target reference devices, the first comparison result obtained by efficient coarse screening is retained as the final comparison data.

[0144] In this way, a hierarchical and comprehensive device image quality comparison archive can be built, which can not only realize rapid statistical analysis of all devices, but also retrieve detailed defect information of devices in different dimensions as needed, thus balancing processing efficiency and analysis depth.

[0145] In the above embodiments, a rapid dimensional comparison of all target first devices is first completed by batch coarse screening using lightweight first prompt words, quickly identifying the weak dimensions of the second devices and screening out devices with high benchmark value in weak dimensions. Then, a refined in-depth comparison of benchmark devices is carried out using second prompt words with regional detail analysis capabilities. Overall, while significantly reducing model computing power consumption and shortening comparison time, it not only achieves complete coverage statistics of the imaging situation of all reference devices, but also outputs fine-grained defect differences in typical areas of faces, skies, and text, improving testing efficiency while ensuring the accuracy of test results.

[0146] Figure 4 Flowchart of the device testing method provided in the embodiments of this application Figure 4 In the embodiments of this application, reference is made to... Figure 4 This application also provides a process for assessing the confidence level of the comparison results corresponding to each first device and incorporating them into the comparison results, which may include:

[0147] S401, determine the confidence level of the test results based on the comparison results of the first device for each target.

[0148] Among them, the result confidence level is used to characterize the credibility of the test results.

[0149] Here, the result confidence level is a quantitative value used to characterize the credibility and reliability of the conclusions on the superiority or inferiority of the reference image and the test image in the test results. The higher the value, the lower the judgment bias and the lower the probability of misjudgment.

[0150] It is understandable that when multimodal large models are affected by image blurring, strong light, texture loss, etc., they are prone to dimensional judgment bias and region comparison misjudgment. Simply outputting the superior or inferior conclusions cannot distinguish whether the conclusions are reliable, and cannot determine which comparison data can be directly used for optimization and which need to be verified. Therefore, the confidence level of the test results can be calculated based on each comparison result, and the test results can be updated based on the confidence level of the results.

[0151] Regarding the process of calculating the confidence level of the results, in one possible implementation, the comparison results can be directly input into a preset confidence level calculation model to obtain the confidence level of the model output.

[0152] In another possible implementation, the process of step S401, "determining the result confidence level corresponding to the test result based on the comparison results of each target first device," may include: determining a first confidence factor based on the degree of matching of scene features between the reference image and the test image in the image scene consistency comparison results of each comparison result; determining a second confidence factor based on the cosine similarity between the device visual features between the reference image and the test image in the image quality comparison results of each comparison result; determining a third confidence factor based on the dimensionality consistency comparison results at different model temperatures in each comparison result; and determining the result confidence level as the weighted sum of the first confidence factor, the second confidence factor, and the third confidence factor.

[0153] Here, the first confidence factor is used to measure whether the reference image and the test image belong to the same type of scene from the perspective of image content. The more similar the scenes are, the more valuable the image comparison is. It is understandable that if the two images have completely different content (one is a face, the other is just sky), the image quality comparison loses its fair benchmark, the model's judgment is prone to distortion, and the scene matching degree needs to be used to suppress the confidence.

[0154] Regarding the calculation process of the first confidence factor, in this embodiment of the application, the scene consistency score corresponding to each comparison result can be calculated first. For example, the scene consistency score is 1.0 when there is a perfect match, the scene consistency score is between 0.1 and 1 depending on the degree of matching when there is an approximate match, and 0.1 when there is a complete mismatch. Finally, the minimum scene consistency score can be selected as the value of the first confidence factor.

[0155] Here, the second confidence factor is used to quantify the similarity between the basic imaging capabilities of the second device and each of the first devices based on the feature vectors of the device's visual features. It is understandable that when the difference in basic imaging capabilities between the two devices is too large, the comparison of image quality has limited reference value, and the model's conclusions regarding the difference do not have optimization reference value, requiring a reduction in confidence.

[0156] Regarding the calculation process of the second confidence factor, in this embodiment, the visual consistency score corresponding to each comparison result can be calculated first. Here, the visual consistency score of each comparison result can be determined based on the cosine similarity between the visual features of the device between the reference image and the test image. After obtaining the visual consistency score corresponding to each comparison result, the mean of each visual consistency score can be calculated as the value of the second confidence factor.

[0157] Here, the third confidence factor is used to measure whether the output conclusions of the multimodal image comparison model are stable under different random parameters, reflecting the certainty of the model's own judgment. It is understandable that the model's output exhibits randomness when the temperature is high; if the conclusions change repeatedly in multiple inferences, it indicates that the image differences are blurred, the model cannot make stable judgments, and the reliability of the conclusions is low.

[0158] In the embodiments of this application, the calculation process of the third confidence factor involves determining the proportion of consistent image quality dimensions to the total number of image quality dimensions in the dimensional consistency comparison results under different model temperatures in each comparison result, obtaining the dimensional consistency score corresponding to each comparison result, and finally selecting the smallest dimensional consistency score as the third confidence factor.

[0159] After obtaining the first confidence factor, the second confidence factor, and the third confidence factor, the three can be combined and calculated, such as by weighted summation, and the final calculated value is the confidence level of the test result.

[0160] For example, the formula for calculating the confidence level of the result can be referred to as Formula 1 below:

[0161] Confidence = w1 × SCF + w2 × FMF + w3 × OSF Formula 1

[0162] Where Confidence is the confidence level of the result, SCF is the first confidence factor, w1 is the weight of the first confidence factor, FMF is the second confidence factor, w2 is the weight of the second confidence factor, OSF is the third confidence factor, and w3 is the weight of the third confidence factor.

[0163] Understandably, the values ​​of w1, w2, and w3 can be set based on empirical values, for example, w1=0.4, w2=0.3, and w3=0.3.

[0164] In this way, three types of confidence factors are generated sequentially from three independent dimensions: scene matching, image quality feature similarity, and model output stability. By using fixed weights to weight and fuse them, a standardized comprehensive confidence score is obtained and divided into three levels of confidence. This not only quantifies the risk of invalid comparison caused by scene misalignment and image quality interval gaps, but also identifies the judgment bias caused by random fluctuations in the model. It unifies multi-dimensional interference factors into calculable confidence values, greatly improving the objectivity and reliability of the test results.

[0165] S402, based on the result confidence level, update the test results of the second device.

[0166] In the embodiments of this application, the reliability of the test result can be determined based on the result confidence level, and the test result can be updated by giving the result confidence level.

[0167] Here, we can first map the level of credibility based on the confidence level of the results. For example, we can use a threshold division for mapping. When the confidence level of the results is greater than or equal to 0.85, it represents high confidence, the conclusion is reliable and can be directly used for image quality optimization. When the confidence level of the results is between 0.60 and 0.85, it represents medium confidence, the conclusion is basically credible, and we need to pay close attention to deviation points. When the confidence level of the results is less than 0.60, it represents low confidence, the uncertainty of the judgment is high, and we need to review the image and compare the results.

[0168] In one possible implementation, the confidence level of the result can be directly added to the test result, and a person can determine whether to refer to the content of the test result based on the confidence level of the result.

[0169] In another possible implementation, when the confidence level of the result is too low, representing low confidence, the test result can be regenerated by comparing the reference image of each first device with the test image of the second device.

[0170] In some embodiments, the model parameters of the multimodal image comparison large model can be fine-tuned based on the result confidence level to optimize the model performance.

[0171] In the above embodiments, the confidence level of the results representing the reliability of the judgment is generated by relying on the hierarchical comparison results of each first device, distinguishing between highly reliable and valid data and low-reliability data that is biased by image interference, and then updating the test results with the confidence level to improve the authenticity and reliability of the final test results.

[0172] Figure 5 This is a schematic diagram of the structure of the device testing apparatus provided in the embodiments of this application, such as... Figure 5 As shown, the device testing apparatus 500 provided in this embodiment includes:

[0173] The clustering and grouping module 501 is used to group each first device according to the reference images corresponding to multiple first devices, and determine at least one combination of first devices, wherein the combination of first devices includes at least two first devices with similar performance.

[0174] The comparison module 502 is used to compare the reference image corresponding to the target first device in each first device combination with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device. The comparison result includes at least the image scene consistency comparison result, the image quality comparison result, and the dimensionality consistency comparison result under different model temperatures.

[0175] The result generation module 503 is used to determine the test results of the second device based on the comparison results corresponding to each target first device. The test results include the comparison results between the second device and each first device.

[0176] In some embodiments, the clustering grouping module 501 includes:

[0177] The extraction unit is used to extract features from the reference image and obtain the image features corresponding to the reference image. The image features include device visual features, scene features, and color distribution features.

[0178] The clustering unit is used to cluster each first device according to image features based on a preset clustering algorithm to determine at least one group of first devices.

[0179] In some embodiments, the clustering grouping module 501 further includes:

[0180] The target determination unit is used to determine the cluster center corresponding to the first equipment combination; and to determine at least one target first equipment corresponding to the first equipment combination based on the cluster center of the first equipment combination.

[0181] In some embodiments, the comparison module 502 includes:

[0182] The first comparison unit is used to compare the reference image corresponding to the target first device with the test image corresponding to the second device in batches based on the first prompt word and through a multimodal image comparison large model, to determine the first comparison result corresponding to each target first device. In the image quality comparison process, the first prompt word only compares the differences between the test image and each reference image in multiple image quality dimensions, including sharpness, noise, color, and exposure.

[0183] The weakness determination unit is used to determine at least one weak dimension device from the target first device based on the first comparison result. The weak dimension device is the target first device with the best imaging performance in the image quality dimension where the second device has a weakness and is used for the second comparison.

[0184] The second comparison unit is used to compare the reference image corresponding to the weak dimension device with the test image corresponding to the second device based on the second prompt word through a multimodal image comparison model, and determine the second comparison result. In the image quality comparison process, in addition to comparing the differences between the test image and each reference image in multiple image quality dimensions, the second prompt word also compares the performance differences between the test image and each reference image in the same image area, which includes faces, sky and text.

[0185] The summarization unit is used to take the second comparison result corresponding to the weak dimension device and the first comparison result corresponding to the target first device other than the weak dimension device as the comparison result corresponding to each target first device.

[0186] In some embodiments, the second comparison unit is specifically used to perform the following: if the number of weak dimension devices is greater than the maximum number of comparisons in a single instance, the weak dimension devices are divided into multiple batches according to the image quality dimension corresponding to the weak dimension devices; based on the priority of the image quality dimension corresponding to each batch of weak dimension devices, and using a multimodal image comparison large model, a second comparison is performed on the reference image corresponding to the weak dimension device and the test image corresponding to the second device in batches.

[0187] In some embodiments, the result generation module 503 includes:

[0188] The reverse calculation unit is used to determine the comparison results corresponding to each first device in the first device combination to which the target first device belongs, based on the comparison results corresponding to each target first device;

[0189] The generation unit is used to determine the test results of the second device based on the comparison results corresponding to each first device.

[0190] In some embodiments, the device testing apparatus 500 further includes:

[0191] The results evaluation module is used to determine the result confidence level corresponding to the test results based on the comparison results of the first device for each target. The result confidence level is used to characterize the credibility of the test results.

[0192] The result update module is used to update the test results of the second device based on the result confidence level.

[0193] In some embodiments, the result evaluation module includes:

[0194] The confidence calculation unit is used to determine the first confidence factor based on the degree of matching of scene features between the reference image and the test image in the image scene consistency comparison results of each comparison result; to determine the second confidence factor based on the cosine similarity between the device visual features between the reference image and the test image in the image quality comparison results of each comparison result; to determine the third confidence factor based on the dimensionality consistency comparison results under different model temperatures in each comparison result; and to determine the result confidence value as the weighted sum of the first confidence factor, the second confidence factor, and the third confidence factor.

[0195] The device testing apparatus provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0196] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6As shown, the electronic device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus 604.

[0197] In a specific implementation, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to perform the above-described method.

[0198] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0199] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0200] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0201] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0202] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0203] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0204] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0205] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0206] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0207] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0208] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0209] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0210] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0211] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A device testing method, characterized in that, The method includes: Feature extraction is performed on reference images corresponding to multiple first devices to obtain image features corresponding to the reference images, including device visual features, scene features, and color distribution features; Based on a preset clustering algorithm, each of the first devices is clustered according to the image features to determine at least one group of first devices, wherein the first device group includes at least two first devices with similar performance. The reference image corresponding to the target first device in each of the first device combinations is compared with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device. This includes: based on a first prompt word, using a multimodal image comparison model, performing a first comparison on the reference image corresponding to the target first device in batches with the test image corresponding to the second device to determine the first comparison result for each target first device. In the image quality comparison process, the first prompt word only compares the differences between the test image and each reference image in multiple image quality dimensions, including sharpness, noise, color, and exposure. Based on the first comparison result, at least one weak dimension device is determined from the target first devices. The weak dimension device is the target first device with the best imaging performance in the image quality dimension where the second device has a weakness, and is used for the second comparison. Based on the second prompt word... A second comparison is performed between the reference image corresponding to the weak-dimensional device and the test image corresponding to the second device using a multimodal image comparison model to determine the second comparison result. During the image quality comparison process, the second prompt word, in addition to comparing the differences between the test image and each of the reference images in multiple image quality dimensions, also compares the performance differences between the test image and each of the reference images within the same image region. The same image region includes faces, sky, and text. The second comparison result corresponding to the weak-dimensional device and the first comparison result corresponding to the target first device (excluding the weak-dimensional device) are used as the comparison result for each target first device. The comparison result includes at least image scene consistency comparison result, image quality comparison result, and dimensional consistency comparison result under different model temperatures. The target first device is a representative device selected from each first device combination for comparison. The test result of the second device is determined based on the comparison results corresponding to each of the target first devices, and the test result includes the comparison results between the second device and each of the first devices.

2. The method according to claim 1, characterized in that, The method further includes: Determine the cluster center corresponding to the first device combination; At least one target first device corresponding to the first device combination is determined based on the cluster center of the first device combination.

3. The method according to claim 1, characterized in that, The second comparison, based on the second prompt word and using a multimodal image comparison model, involves comparing the reference image corresponding to the weaker dimension device with the test image corresponding to the second device. If the number of devices with weak dimensions is greater than the maximum number of comparisons in a single instance, the devices with weak dimensions are divided into multiple batches according to the image quality dimension corresponding to the devices with weak dimensions. Based on the priority of the image quality dimension corresponding to each batch of weak dimension devices, and using the second prompt word, a second comparison is performed on the reference image corresponding to the weak dimension device and the test image corresponding to the second device in batches through a multimodal image comparison model.

4. The method according to claim 1, characterized in that, The step of determining the test result of the second device based on the comparison results corresponding to each of the target first devices includes: Based on the comparison results corresponding to each of the target first devices, determine the comparison results corresponding to each first device in the first device combination to which the target first device belongs; Based on the comparison results corresponding to each of the first devices, the test results of the second device are determined.

5. The method according to claim 1, characterized in that, The method further includes: The confidence level of the test result is determined based on the comparison results of each of the target first devices, and the confidence level of the result is used to characterize the credibility of the test result; Based on the confidence level of the results, the test results of the second device are updated.

6. The method according to claim 5, characterized in that, The step of determining the result confidence level corresponding to the test result based on the comparison results of each of the target first devices, wherein the result confidence level is used to characterize the credibility of the test result, includes: Based on the degree of matching of scene features between the reference image and the test image in the image scene consistency comparison results of each comparison result, a first confidence factor is determined; Based on the cosine similarity between the device visual features of the reference image and the test image in the image quality comparison results of each comparison result, a second confidence factor is determined. The third confidence factor is determined based on the dimensionality consistency comparison results under different model temperatures in each of the comparison results. The weighted sum of the first confidence factor, the second confidence factor, and the third confidence factor is determined to be the result confidence level.

7. A device for testing equipment, characterized in that, The device includes: The clustering and grouping module is used to extract features from reference images corresponding to multiple first devices, and obtain image features corresponding to the reference images. The image features include device visual features, scene features, and color distribution features. Based on a preset clustering algorithm, each first device is clustered according to the image features to determine at least one group of first devices. The first device group includes at least two first devices with similar performance. The comparison module is used to compare the reference image corresponding to the target first device in each first device combination with the test image corresponding to the second device to obtain the comparison result between each target first device and the second device. The comparison result includes at least the image scene consistency comparison result, the image quality comparison result, and the dimensionality consistency comparison result under different model temperatures. The target first device is a representative device selected from each first device combination to participate in the comparison. The result generation module is used to determine the test result of the second device based on the comparison result corresponding to each of the target first devices, wherein the test result includes the comparison result between the second device and each of the first devices; The comparison module is specifically used to perform a first comparison on the reference images corresponding to the target first device and the test images corresponding to the second device in batches, based on a first prompt word and a multimodal image comparison model, to determine the first comparison result for each target first device. During the image quality comparison process, the first prompt word only compares the differences between the test images and each reference image in multiple image quality dimensions, including sharpness, noise, color, and exposure. Based on the first comparison result, at least one weak-dimensional device is identified from the target first devices. This weak-dimensional device is the one with the best imaging performance in the image quality dimension where the second device has a weakness, and is used for the second comparison. The device; based on the second prompt word, through a multimodal image comparison model, performs a second comparison between the reference image corresponding to the weak dimension device and the test image corresponding to the second device, and determines the second comparison result. In the image quality comparison process, in addition to comparing the differences between the test image and each of the reference images in multiple image quality dimensions, the second prompt word also compares the performance differences between the test image and each of the reference images in the same image area, which includes faces, sky, and text; the second comparison result corresponding to the weak dimension device and the first comparison result corresponding to the target first device other than the weak dimension device are used as the comparison result corresponding to each target first device.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-6.

Images