Image selection method and device

Abstract

The present application discloses an image selection method, which is applied to the field of artificial intelligence. The method includes: obtaining multiple candidate images, processing each candidate image respectively through a first image processing model and a second image processing model to obtain a first processed image and a second processed image corresponding to each candidate image, obtaining the image quality difference between the first processed image and the second processed image corresponding to each candidate image, and selecting M target images from multiple candidate images according to the image quality difference. The present application uses the image quality difference corresponding to each candidate image as the basis for selecting the test sample, ensuring that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, thereby helping image quality evaluators to more easily subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an image selection method and device. Background Art

[0002] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0003] When comparing the performance of image processing models, it is often impossible to obtain ideal reference images for real-world comparison test scenarios. As a result, it is impossible to use reference quality evaluation indicators (such as peak signal-to-noise ratio (PSNR), structural similarity (SSIM), etc.) for objective evaluation when evaluating the performance of models. Therefore, experts are often still needed to conduct subjective model performance evaluation.

[0004] When conducting subjective model performance evaluation, the models that need to be compared in performance need to process the same test sample and then perform performance evaluation based on the processing results. However, existing technologies cannot select test samples that can more accurately reflect the performance differences between models. Summary of the invention

[0005] In a first aspect, the present application provides an image selection method, the method comprising:

[0006] Acquire multiple candidate images;

[0007] Among them, multiple candidate images can be obtained, as well as a first image processing model and a second image processing model that need to be compared in performance, wherein the first image processing model and the second image processing model can be used to implement an image processing task, and the image enhancement task can be, for example, an image enhancement task, and the multiple candidate images can be a test sample set for the image enhancement task. Exemplarily, if the image enhancement task is defogging, the multiple candidate images can be a large-scale test sample set for the defogging scene.

[0008] Each candidate image is processed respectively by the first image processing model and the second image processing model to obtain the first processed image and the second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing by the first image processing model, and the second processed image is obtained by processing by the second image processing model; wherein the so-called "processing" can be understood as the reasoning process of the model.

[0009] Obtain an image quality difference between a first processed image and a second processed image corresponding to each candidate image, and select M target images from the multiple candidate images based on the image quality difference, wherein the M target images are used as test samples for performance comparison between the first image processing model and the second image processing model.

[0010] Among them, image quality can be used to characterize the degree to which an image restores a real scene, or it can be called the fidelity of an image. The closer the image is to the real scene, the higher the fidelity of the image, and thus the higher the image quality. In addition, image quality can also be used to characterize the readability of an image. Readability refers to the ability of an image to provide information to a person or machine. It is not only related to the application requirements of the image system, but also often related to the subjective feeling of the human eye. The higher the readability, the higher the image quality. In one implementation, the indicators used to evaluate image quality may include resolution, color depth, image distortion, and other aspects.

[0011] In order to enable experts or other personnel to more accurately compare the performance of the first image processing model and the second image processing model, it is necessary to ensure that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, and then the performance advantages and disadvantages between the first image processing model and the second image processing model can be accurately determined based on the quality of the output images.

[0012] The embodiment of the present application uses the image quality difference corresponding to each candidate image as the basis for selecting the test sample, ensuring that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, thereby helping image quality evaluators to more easily judge the performance advantages and disadvantages between the first image processing model and the second image processing model subjectively.

[0013] In a possible implementation, selecting M target images from the multiple candidate images according to the image quality difference includes:

[0014] According to the image quality difference corresponding to each candidate image, M target images whose corresponding image quality difference is greater than a first threshold are selected from the multiple candidate images.

[0015] In one implementation, M target images corresponding to the largest image quality differences may be selected from a plurality of candidate images.

[0016] The embodiments of the present application can ensure that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, thereby helping image quality evaluators to more easily subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

[0017] In a possible implementation, the method further includes:

[0018] Obtaining image content differences between each candidate image and other candidate images among the plurality of candidate images except itself;

[0019] The step of selecting M target images from the plurality of candidate images according to the image quality differences comprises:

[0020] M target images are selected from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image.

[0021] Among them, the image content difference can indicate the difference between the target subjects in the background area or the foreground area of ​​the image.

[0022] Taking the difference between target subjects in the foreground area of ​​image content difference as an example, the image content difference between images including people and images including non-human animals (or inanimate objects, such as buildings, etc.) is greater than the image content difference between images that both include people, and the image content difference between images including different people is greater than the image content difference between images including the same person.

[0023] In a possible implementation, traditional methods such as scale-invariant feature transform (SIFT) or convolutional neural network (CNN) can be used to extract image features of each candidate image, and obtain image feature vectors of each candidate image to characterize the image content, wherein the image features may include color features, texture features, shape features, and spatial relationship features of the image. Then, the distance metric between the image feature vector of each candidate image and the image feature vectors of other candidate images is calculated. The distance metric can express the content difference between the images. The greater the content difference, the greater the difference in the image content of the candidate image from the other candidate images.

[0024] It should be understood that the distance metric can also be called a metric similarity. By calculating the distance metric between two multidimensional data, the similarity between the two multidimensional data can be determined. Generally, the smaller the distance metric between the two multidimensional data, the higher the similarity between the two multidimensional data, and the smaller the difference; on the contrary, the larger the distance metric between the two multidimensional data, the smaller the similarity between the two multidimensional data, and the greater the difference. Exemplarily, the distance metric may include distances such as mean squared error (MES) distance, L1 distance, or mean squared error MSE. It should be understood that in addition to MES distance and L1 distance, the difference in image content can also be determined based on other distance metrics, and this embodiment does not specifically limit the distance metric method.

[0025] The embodiment of the present application uses the image content difference between each candidate image as the basis for selecting test samples, and can select test samples with greater content differences, thereby ensuring that the scenes covered by the sampling of the test samples are comprehensive enough, thereby ensuring the reliability of the test results.

[0026] In a possible implementation, selecting M target images from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image includes:

[0027] Determining a selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, wherein the selection weight is positively correlated with the image quality difference and the image content difference;

[0028] According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

[0029] In one implementation, the M target images with the largest selection weights may be selected from the multiple candidate images.

[0030] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0031] The weights corresponding to the image quality difference and the image content difference can be adjusted by the user as needed, depending on whether the user is more concerned about the quality difference or the diversity difference of the samples.

[0032] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0033] In an embodiment of the present application, the first image processing network and the second image processing network can be used to implement image enhancement tasks. The so-called image enhancement tasks can be understood as tasks used to enhance the quality of videos. For example, the image enhancement tasks can be video denoising tasks, video defogging tasks, super-resolution tasks, or high dynamic range tasks, etc., which are not limited here.

[0034] In a second aspect, the present application provides an image selection device, the device comprising:

[0035] An acquisition module, used for acquiring multiple candidate images;

[0036] An image processing module, used for processing each candidate image through a first image processing model and a second image processing model respectively, so as to obtain a first processed image and a second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model;

[0037] An image selection module is used to obtain the image quality difference between the first processed image and the second processed image corresponding to each candidate image, and select M target images from the multiple candidate images based on the image quality difference. The M target images are used as test samples for the first image processing model and the second image processing model when performing performance comparison.

[0038] In order to enable experts or other personnel to more accurately compare the performance of the first image processing model and the second image processing model, it is necessary to ensure that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, and then the performance advantages and disadvantages between the first image processing model and the second image processing model can be accurately determined based on the quality of the output images.

[0039] The embodiment of the present application uses the image quality difference corresponding to each candidate image as the basis for selecting the test sample, ensuring that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, thereby helping image quality evaluators to more easily judge the performance advantages and disadvantages between the first image processing model and the second image processing model subjectively.

[0040] In a possible implementation, the image selection module is used to select, according to the image quality difference corresponding to each candidate image, M target images whose corresponding image quality difference is greater than a first threshold from the multiple candidate images.

[0041] The embodiments of the present application can ensure that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, thereby helping image quality evaluators to more easily subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

[0042] In one implementation, M target images corresponding to the largest image quality differences may be selected from a plurality of candidate images.

[0043] In a possible implementation, the acquisition module is used to acquire image content differences between each candidate image and other candidate images among the multiple candidate images except itself;

[0044] The image selection module is used to select M target images from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image.

[0045] The image content difference may indicate the difference between the target subjects in the background area and / or the foreground area in the image.

[0046] Taking the difference between target subjects in the foreground area of ​​image content difference as an example, the image content difference between images including people and images including non-human animals (or inanimate objects, such as buildings, etc.) is greater than the image content difference between images that both include people, and the image content difference between images including different people is greater than the image content difference between images including the same person.

[0047] In a possible implementation, traditional methods such as scale-invariant feature transform (SIFT) or convolutional neural network (CNN) can be used to extract image features of each candidate image, and obtain image feature vectors of each candidate image to characterize the image content, wherein the image features may include color features, texture features, shape features, and spatial relationship features of the image. Then, the distance metric between the image feature vector of each candidate image and the image feature vectors of other candidate images is calculated. The distance metric can express the content difference between the images. The greater the content difference, the greater the difference in the image content of the candidate image from the other candidate images.

[0048] It should be understood that the distance metric can also be called a metric similarity. By calculating the distance metric between two multidimensional data, the similarity between the two multidimensional data can be determined. Generally, the smaller the distance metric between the two multidimensional data, the higher the similarity between the two multidimensional data, and the smaller the difference; on the contrary, the larger the distance metric between the two multidimensional data, the smaller the similarity between the two multidimensional data, and the greater the difference. Exemplarily, the distance metric may include distances such as mean squared error (MES) distance, L1 distance, or mean squared error MSE. It should be understood that in addition to MES distance and L1 distance, the difference in image content can also be determined based on other distance metrics, and this embodiment does not specifically limit the distance metric method.

[0049] The embodiment of the present application uses the image content difference between each candidate image as the basis for selecting test samples, and can select test samples with greater content differences, thereby ensuring that the scenes covered by the sampling of the test samples are comprehensive enough, thereby ensuring the reliability of the test results.

[0050] In a possible implementation, the image selection module is used to determine a selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, wherein the selection weight is positively correlated with the image quality difference and the image content difference;

[0051] According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

[0052] In one implementation, the M target images with the largest selection weights may be selected from the multiple candidate images.

[0053] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0054] The weights corresponding to the image quality difference and the image content difference can be adjusted by the user as needed, depending on whether the user is more concerned about the quality difference or the diversity difference of the samples.

[0055] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0056] In an embodiment of the present application, the first image processing network and the second image processing network can be used to implement image enhancement tasks. The so-called image enhancement tasks can be understood as tasks used to enhance the quality of videos. For example, the image enhancement tasks can be video denoising tasks, video defogging tasks, super-resolution tasks, or high dynamic range tasks, etc., which are not limited here.

[0057] In a third aspect, the present application provides an image selection method, the method comprising:

[0058] Acquire multiple candidate images;

[0059] Obtain image content differences between each candidate image and other candidate images other than itself in the plurality of candidate images; wherein the image content differences may indicate differences between target subjects in a background area and / or a foreground area in an image. Taking the difference between target subjects in a foreground area of ​​the image content difference as an example, the image content difference between an image including a person and an image including a non-human animal (or an inanimate object, such as a building, etc.) is greater than the image content difference between images including both persons, and the image content difference between images including different persons is greater than the image content difference between images including the same person.

[0060] According to the image content difference corresponding to each candidate image, M target images are selected from the multiple candidate images.

[0061] The embodiment of the present application uses the image content difference between each candidate image as the basis for selecting test samples, and can select test samples with greater content differences, thereby ensuring that the scenes covered by the sampling of the test samples are comprehensive enough, thereby ensuring the reliability of the test results.

[0062] In a possible implementation, selecting M target images from the multiple candidate images according to the image content difference corresponding to each candidate image includes:

[0063] According to the image content difference corresponding to each candidate image, M target images whose image content difference is greater than a third threshold are selected from the multiple candidate images.

[0064] In a possible implementation, the method further includes:

[0065] Process each candidate image by using the first image processing model and the second image processing model respectively to obtain a first processed image and a second processed image corresponding to each candidate image;

[0066] Obtaining an image quality difference between a first processed image and a second processed image corresponding to each candidate image;

[0067] The selecting M target images from the plurality of candidate images according to the image content difference corresponding to each candidate image comprises:

[0068] According to the image quality difference and the image content difference corresponding to each candidate image, M target images are selected from the multiple candidate images.

[0069] In a possible implementation, selecting M target images from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image includes:

[0070] Determining a selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, wherein the selection weight is positively correlated with the image quality difference and the image content difference;

[0071] According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

[0072] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0073] The weights corresponding to the image quality difference and the image content difference can be adjusted by the user as needed, depending on whether the user is more concerned about the quality difference or the diversity difference of the samples.

[0074] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0075] In an embodiment of the present application, the first image processing network and the second image processing network can be used to implement image enhancement tasks. The so-called image enhancement task can be understood as a task used to enhance the quality of a video. For example, the image enhancement task can be a video denoising task, a video defogging task, a super-resolution task or a high dynamic range task, etc., which is not limited here.

[0076] In a fourth aspect, the present application provides an image selection device, the device comprising:

[0077] An acquisition module, used for acquiring multiple candidate images;

[0078] Obtaining image content differences between each candidate image and other candidate images among the plurality of candidate images except itself;

[0079] The image selection module is used to select M target images from the multiple candidate images according to the image content difference corresponding to each candidate image.

[0080] The embodiment of the present application uses the image content difference between each candidate image as the basis for selecting test samples, and can select test samples with greater content differences, thereby ensuring that the scenes covered by the sampling of the test samples are comprehensive enough, thereby ensuring the reliability of the test results.

[0081] In a possible implementation, the image selection module is used to select M target images having image content differences greater than a third threshold from the multiple candidate images according to the image content difference corresponding to each candidate image.

[0082] In a possible implementation, the device further includes:

[0083] An image processing module, used to process each candidate image by using a first image processing model and a second image processing model respectively, so as to obtain a first processed image and a second processed image corresponding to each candidate image;

[0084] The acquisition module is used to acquire the image quality difference between the first processed image and the second processed image corresponding to each candidate image;

[0085] The image selection module is used to select M target images from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image.

[0086] In a possible implementation, the image selection module is used to determine a selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, wherein the selection weight is positively correlated with the image quality difference and the image content difference;

[0087] According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

[0088] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0089] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0090] In a fifth aspect, an embodiment of the present application provides a model testing method, the method comprising:

[0091] Obtain M target images;

[0092] Processing each target image respectively by a first image processing model and a second image processing model to obtain a first processed image and a second processed image corresponding to each target image; wherein the first image processing model and the second image processing model are used to implement the same image processing task, the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model; and the image quality difference between the first processed image and the second processed image corresponding to each target image is greater than a first threshold;

[0093] Obtain a model test result, where the model test result is used to represent a performance comparison result between the first image processing model and the second image processing model, and the model test result is determined based on the first processed image and the second processed image corresponding to each target image.

[0094] In the embodiment of the present application, there is a large difference in image quality between the output images obtained by the first image processing model and the second image processing model when processing the same target image, thereby helping image quality evaluators to more easily and subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

[0095] In one possible implementation, the selected weight of each target image in the M target images is greater than a second threshold, and the selected weight of each target image is positively correlated with the image quality difference and image content difference corresponding to each target image, wherein the image content difference corresponding to each target image is used to represent the image content difference between each target image and other target images among the multiple target images except itself.

[0096] In the embodiment of the present application, the image contents of the target images are quite different, that is, the scenes covered by the M target images are comprehensive enough, thus ensuring the reliability of the test results.

[0097] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0098] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0099] In a sixth aspect, an embodiment of the present application provides a model testing device, the device comprising:

[0100] An acquisition module, used for acquiring M target images;

[0101] An image processing module, used for processing each target image through a first image processing model and a second image processing model respectively, so as to obtain a first processed image and a second processed image corresponding to each target image; wherein the first image processing model and the second image processing model are used to implement the same image processing task, the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model; and the image quality difference between the first processed image and the second processed image corresponding to each target image is greater than a first threshold; the first processed image and the second processed image corresponding to each target image are used to perform a performance comparison between the first image processing model and the second image processing model;

[0102] The acquisition module is also used to obtain model test results, and the model test results are used to represent the performance comparison results between the first image processing model and the second image processing model. The model test results are determined based on the first processed image and the second processed image corresponding to each target image.

[0103] In the embodiment of the present application, there is a large difference in image quality between the output images obtained by the first image processing model and the second image processing model when processing the same target image, thereby helping image quality evaluators to more easily and subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

[0104] In one possible implementation, the selected weight of each target image in the M target images is greater than a second threshold, and the selected weight of each target image is positively correlated with the image quality difference and image content difference corresponding to each target image, wherein the image content difference corresponding to each target image is used to represent the image content difference between each target image and other target images among the multiple target images except itself.

[0105] In the embodiment of the present application, the image contents of the target images are quite different, that is, the scenes covered by the M target images are comprehensive enough, thus ensuring the reliability of the test results.

[0106] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0107] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0108] In the seventh aspect, an embodiment of the present application provides an image selection device, which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to execute the programs in the memory to perform the above-mentioned first aspect, second aspect, fifth aspect and any optional method.

[0109] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned first aspect, second aspect, fifth aspect and any optional method.

[0110] In a ninth aspect, an embodiment of the present application provides a computer program, including codes, which, when executed, are used to implement the above-mentioned first aspect, second aspect, fifth aspect and any optional method.

[0111] In a tenth aspect, the present application provides a chip system, which includes a processor for supporting an execution device or a training device to implement the functions involved in the above aspects, such as sending or processing the data involved in the above methods; or, information. In a possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the execution device or the training device. The chip system can be composed of chips, or it can include chips and other discrete devices.

[0112] The embodiment of the present application provides an image selection method, characterized in that the method includes: obtaining multiple candidate images; processing each candidate image by a first image processing model and a second image processing model respectively to obtain a first processed image and a second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model; obtaining the image quality difference between the first processed image and the second processed image corresponding to each candidate image, and selecting M target images from the multiple candidate images according to the image quality difference, and the M target images are used as test samples for the first image processing model and the second image processing model when performing performance comparison. In the above manner, the image quality difference corresponding to each candidate image is used as the basis for selecting the test sample, ensuring that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, thereby helping image quality evaluators to more easily subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

[0113] In addition, the image content differences between the candidate images are used as the basis for selecting test samples, so that test samples with greater content differences can be selected, thereby ensuring that the scenes covered by the sampling of the test samples are comprehensive enough, thereby ensuring the reliability of the test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Figure 1 A structural diagram of the main framework of artificial intelligence;

[0115] Figure 2 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0116] Figure 3 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0117] Figure 4 A schematic diagram of a convolutional neural network provided in an embodiment of the present application;

[0118] Figure 5 A schematic diagram of a convolutional neural network provided in an embodiment of the present application;

[0119] Figure 6 A schematic diagram of the structure of a system provided in an embodiment of the present application;

[0120] Figure 7 A schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0121] Figure 8 An illustration of an image selection method provided in an embodiment of the present application;

[0122] Fig. 9 This is a schematic diagram of the structure of an image processing model;

[0123] Fig.10a An illustration of an image selection method provided in an embodiment of the present application;

[0124] Fig.10b An illustration of an image selection method provided in an embodiment of the present application;

[0125] Fig.10c This is an illustration of the quality difference of the dehazing effect on the target image;

[0126] Fig.10d A schematic diagram of a model testing method provided in an embodiment of the present application;

[0127] Fig.11 A schematic diagram of an image selection device provided in an embodiment of the present application;

[0128] Fig.12 A schematic diagram of an image selection device provided in an embodiment of the present application;

[0129] Fig.13 A schematic diagram of a model testing device provided in an embodiment of the present application;

[0130] Fig.14 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0131] The embodiments of the present invention are described below in conjunction with the accompanying drawings in the embodiments of the present invention. The terms used in the implementation mode of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0132] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0133] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0134] First, the overall workflow of the artificial intelligence system is described. Figure 1 , Figure 1 The figure shows a structural diagram of the main framework of artificial intelligence. The following is an explanation of the above artificial intelligence theme framework from the two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). Among them, the "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has undergone a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecology process of the system.

[0135] (1) Infrastructure

[0136] Infrastructure provides computing power support for artificial intelligence systems, enables communication with the outside world, and supports it through the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips, such as central processing units (CPU), neural-network processing units (NPU), graphics processing units (GPU), application specific integrated circuits (ASIC) or field programmable gate arrays (FPGA) and other hardware acceleration chips; the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0137] (2) Data

[0138] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0139] (3) Data processing

[0140] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0141] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0142] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0143] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0144] (4) General ability

[0145] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0146] (5) Smart products and industry applications

[0147] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical applications. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0148] The image selection method provided in the embodiment of the present application can specifically select an image processing model that can more accurately evaluate the performance difference between the first image processing model and the second image processing model in the embodiment of the present application.

[0149] The first image processing model and the second image processing model in the embodiment of the present application can be applied to assisted driving and autonomous driving smart cars, and can also be applied to computer vision fields such as smart cities and smart terminals that require image enhancement. For example, the first image processing model and the second image processing model can be applied to video streaming scenarios and video surveillance scenarios. Figure 2 and Figure 3 This section briefly introduces video streaming transmission scenarios and video surveillance scenarios.

[0150] Video streaming scenarios:

[0151] For example, when a client uses a smart terminal (e.g., a mobile phone, car, robot, tablet computer, desktop computer, smart watch, virtual reality VR, augmented reality AR device, etc.) to play a video, in order to reduce the bandwidth requirement of the video stream, the server can transmit a downsampled, low-resolution, low-quality video stream to the client through the network. The client can then use the trained second video processing network to enhance the image in the low-quality video stream. For example, super-resolution, noise reduction and other operations are performed on the image in the video, and finally a high-quality image is presented to the user.

[0152] Video surveillance scenarios:

[0153] In the field of security, due to unfavorable conditions such as the installation location of surveillance cameras and limited storage space, the image quality of some video surveillance is poor, which will affect the accuracy of people or recognition algorithms in identifying targets. Therefore, the trained second video processing network provided in the embodiment of the present application can be used to convert low-quality video surveillance videos into high-quality high-definition videos, thereby effectively restoring a large number of details in the surveillance images and providing more effective and richer information for subsequent target recognition tasks.

[0154] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0155] (1) Neural Network

[0156] A neural network may be composed of neural units, and a neural unit may refer to an operation unit that takes xs (i.e., input data) and intercept 1 as input, and the output of the operation unit may be:

[0157]

[0158] Where s=1, 2, ...n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

[0159] (2) Convolutional neural network (CNN) is a deep neural network with a convolutional structure. Convolutional neural network contains a feature extractor composed of convolutional layer and subsampling layer, which can be regarded as a filter. Convolutional layer refers to the neuron layer that performs convolution processing on the input signal in convolutional neural network. In the convolutional layer of convolutional neural network, a neuron can only be connected to some neurons in the adjacent layer. A convolutional layer usually contains several feature planes, each of which can be composed of some rectangularly arranged neural units. The neural units in the same feature plane share weights, and the shared weights here are convolution kernels. Shared weights can be understood as the way of extracting features is independent of position. Convolution kernels can be formalized as matrices of random size, and convolution kernels can obtain reasonable weights through learning during the training process of convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connection between the layers of convolutional neural network, while reducing the risk of overfitting.

[0160] CNN is a very common neural network. Figure 4 The focus is on a detailed introduction to the structure of CNN. As mentioned in the previous basic concept introduction, a convolutional neural network is a deep neural network with a convolutional structure and a deep learning architecture. A deep learning architecture refers to multiple levels of learning at different levels of abstraction through machine learning algorithms. As a deep learning architecture, CNN is a feed-forward artificial neural network in which each neuron can respond to the image input into it.

[0161] like Figure 4 As shown, the convolutional neural network (CNN) 200 may include an input layer 210 , a convolutional layer / pooling layer 220 (wherein the pooling layer is optional), and a fully connected layer 230 .

[0162] Convolutional layer / pooling layer 220:

[0163] Convolutional Layer:

[0164] like Figure 4The convolution layer / pooling layer 220 shown may include layers 221-226, for example: in one implementation, layer 221 is a convolution layer, layer 222 is a pooling layer, layer 223 is a convolution layer, layer 224 is a pooling layer, layer 225 is a convolution layer, and layer 226 is a pooling layer; in another implementation, layers 221 and 222 are convolution layers, layer 223 is a pooling layer, layer 224 and 225 are convolution layers, and layer 226 is a pooling layer. That is, the output of a convolution layer can be used as the input of a subsequent pooling layer, or as the input of another convolution layer to continue the convolution operation.

[0165] The following will take the convolution layer 221 as an example to introduce the internal working principle of a convolution layer.

[0166] The convolution layer 221 may include a plurality of convolution operators, which are also called kernels. The convolution operator is equivalent to a filter that extracts specific information from the input image matrix in image processing. The convolution operator can be essentially a weight matrix, which is usually predefined. In the process of performing convolution operations on the image, the weight matrix is ​​usually processed one pixel after another (or two pixels after two pixels... depending on the value of the step length stride) in the horizontal direction on the input image, thereby completing the work of extracting specific features from the image. The size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image. In the process of performing convolution operations, the weight matrix will extend to the entire depth of the input image. Therefore, convolution with a single weight matrix will produce a convolution output with a single depth dimension, but in most cases, a single weight matrix is ​​not used, but multiple weight matrices of the same size (row × column), that is, multiple isotype matrices, are applied. The output of each weight matrix is ​​stacked to form the depth dimension of the convolution image, and the dimension here can be understood as being determined by the "multiple" mentioned above. Different weight matrices can be used to extract different features in the image, for example, one weight matrix is ​​used to extract image edge information, another weight matrix is ​​used to extract specific colors of the image, and another weight matrix is ​​used to blur unwanted noise in the image, etc. The multiple weight matrices have the same size (rows × columns), and the feature maps extracted by the multiple weight matrices of the same size are also the same size. The extracted feature maps of the same size are then merged to form the output of the convolution operation.

[0167] The weight values ​​in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input image, so that the convolutional neural network 200 can make correct predictions.

[0168] When the convolutional neural network 200 has multiple convolutional layers, the initial convolutional layer (for example, 221) often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network 200 increases, the features extracted by the later convolutional layers (for example, 226) become more and more complex, such as high-level semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0169] Pooling layer:

[0170] Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolution layer. Figure 4 Each layer 221-226 illustrated in 220 may be a convolution layer followed by a pooling layer, or may be multiple convolution layers followed by one or more pooling layers. In the image processing process, the only purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer may include an average pooling operator and / or a maximum pooling operator for sampling the input image to obtain an image of smaller size. The average pooling operator may calculate the pixel values ​​in the image within a specific range to generate an average value as the result of average pooling. The maximum pooling operator may take the pixel with the largest value in the range within a specific range as the result of maximum pooling. In addition, just as the size of the weight matrix used in the convolution layer should be related to the image size, the operator in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer may be smaller than the size of the image input to the pooling layer, and each pixel in the image output by the pooling layer represents the average value or maximum value of the corresponding sub-region of the image input to the pooling layer.

[0171] Fully connected layer 230:

[0172] After being processed by the convolution layer / pooling layer 220, the convolution neural network 200 is not sufficient to output the required output information. As mentioned above, the convolution layer / pooling layer 220 only extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (the required class information or other related information), the convolution neural network 200 needs to use the fully connected layer 230 to generate one or a group of outputs of the required number of classes. Therefore, the fully connected layer 230 may include multiple hidden layers (such as Figure 4 231, 232 to 23n), the parameters contained in the multi-layer hidden layer can be pre-trained according to relevant training data of a specific task type, for example, the task type may include image recognition, image classification, image super-resolution reconstruction, etc.

[0173] After the multiple hidden layers in the fully connected layer 230, that is, the last layer of the entire convolutional neural network 200 is the output layer 240, which has a loss function similar to the classification cross entropy, specifically used to calculate the prediction error. Once the forward propagation of the entire convolutional neural network 200 (such as Figure 4 The propagation from 210 to 240 is forward propagation), and the reverse propagation (such as Figure 4 The propagation from 240 to 210 is called back propagation) and then starts to update the weight values ​​and biases of the aforementioned layers to reduce the loss of the convolutional neural network 200 and the error between the result output by the convolutional neural network 200 through the output layer and the ideal result.

[0174] It should be noted that if Figure 4 The convolutional neural network 200 shown is only an example of a convolutional neural network. In a specific application, the convolutional neural network may also exist in the form of other network models, for example, only including Figure 4 A portion of the network structure shown in , for example, the convolutional neural network used in the embodiment of the present application may only include an input layer 210, a convolution layer / pooling layer 220 and an output layer 240.

[0175] It should be noted that if Figure 4 The convolutional neural network 100 shown is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, for example, Figure 5 The multiple convolutional layers / pooling layers shown are operated in parallel, and the features extracted respectively are input to the fully connected layer 230 for processing.

[0176] (3) Deep Neural Networks

[0177] Deep Neural Network (DNN), also known as multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here. From the position of different layers of DNN, the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since DNN has many layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Take coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscripts correspond to the output third layer index 2 and the input second layer index 4. In summary, the coefficients from the kth neuron in the L-1th layer to the jth neuron in the Lth layer are defined as It should be noted that the input layer does not have a W parameter. In a deep neural network, more hidden layers allow the network to better describe complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by many layers of vector W).

[0178] (4) Super-resolution

[0179] Super Resolution (SR) is an image enhancement technology that, given one or a group of low-resolution images, restores the high-frequency detail information of the image by learning the prior knowledge of the image, the similarity of the image itself, and the complementary information of multiple frames of images, thereby generating a target image with a higher resolution. In applications, super resolution can be divided into single-frame image super resolution and video super resolution according to the number of input images. Super resolution has important application value in the fields of high-definition television, monitoring equipment, satellite images, and medical imaging.

[0180] (5) Noise reduction

[0181] During the digitization and transmission process, images are often affected by the imaging device and the external environment, resulting in noise in the image. The process of reducing noise in an image is called image noise reduction, or sometimes image denoising.

[0182] (6) Image features

[0183] Image features mainly include color features, texture features, shape features, and spatial relationship features.

[0184] Color features are global features that describe the surface properties of the scene corresponding to an image or image region. Generally, color features are based on pixel features, in which case all pixels belonging to the image or image region have their own contributions. Since color is insensitive to changes in the direction, size, etc. of an image or image region, color features cannot capture the local features of an object in an image very well.

[0185] Texture features are also global features, which also describe the surface properties of the scene corresponding to the image or image area; however, since texture is only a characteristic of the surface of an object and cannot fully reflect the essential properties of the object, it is impossible to obtain high-level image content using only texture features. Unlike color features, texture features are not based on pixel points, and they need to be statistically calculated in an area containing multiple pixels.

[0186] There are two ways to represent shape features, one is contour features and the other is region features. The contour features of an image are mainly aimed at the outer boundaries of the object, while the region features of an image are related to the entire shape area.

[0187] Spatial relationship features refer to the spatial position or relative direction relationship between multiple objects segmented from an image. These relationships can also be divided into connection / adjacency relationships, overlap / overlap relationships, and inclusion / inclusion relationships. Generally, spatial position information can be divided into two categories: relative spatial position information and absolute spatial position information. The former relationship emphasizes the relative situation between objects, such as up, down, left, and right relationships, while the latter relationship emphasizes the distance and orientation between objects.

[0188] It should be noted that the above-listed image features can be used as some examples of features in an image. An image can also have other features, such as higher-level features: semantic features, which will not be expanded here.

[0189] (7) Image / Video Enhancement

[0190] Image / video enhancement refers to actions performed on images / videos that can improve the image quality. For example, enhancement processing includes super-resolution, noise reduction, sharpening, or demosaicing.

[0191] Combine the following Figure 6 The system architecture provided in the embodiments of the present application is introduced in detail. Figure 6 This is a schematic diagram of the system architecture provided by an embodiment of the present application. Figure 6 As shown, the system architecture 500 includes an execution device 510 , a training device 520 , a database 530 , a client device 540 , a data storage system 550 , and a data collection system 560 .

[0192] The execution device 510 includes a calculation module 511, an I / O interface 512, a preprocessing module 513 and a preprocessing module 514. The calculation module 511 may include a target model / rule 501, and the preprocessing module 513 and the preprocessing module 514 are optional.

[0193] The data acquisition device 560 is used to collect training data. The video sample can be a low-quality image, and the supervisory image is a high-quality image corresponding to the image sample acquired in advance before model training. The image sample can be, for example, a low-resolution image, and the supervisory image is a high-resolution image; or, the image sample can be, for example, a video containing fog or noise, and the supervisory image is an image with the fog or noise removed. After collecting the training data, the data acquisition device 560 stores the training data in the database 530, and the training device 520 obtains the target model / rule 501 through training based on the training data maintained in the database 530.

[0194] The above-mentioned target model / rule 501 (such as the first image processing model and the second image processing model in the embodiment of the present application) can be used to implement image enhancement tasks, that is, by inputting the image to be processed into the target model / rule 501, the processed enhanced image (such as the first processed image and the second processed image in the embodiment of the present application) can be obtained. It should be noted that in actual applications, the training data maintained in the database 530 does not necessarily all come from the collection of the data acquisition device 560, and may also be received from other devices. It should also be noted that the training device 520 does not necessarily train the target model / rule 501 entirely based on the training data maintained by the database 530, and it is also possible to obtain training data from the cloud or other places for model training. The above description should not be used as a limitation on the embodiments of the present application.

[0195] The target model / rule 501 obtained by training the training device 520 can be applied to different systems or devices, such as Figure 6 The execution device 510 shown in the figure may be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, an augmented reality (AR) / virtual reality (VR) device, a vehicle terminal, etc., or a server or a cloud. Figure 6 In the embodiment, the execution device 510 is configured with an input / output (I / O) interface 512 for data interaction with an external device. A user can input data to the I / O interface 512 through a client device 540 .

[0196] The preprocessing module 513 and the preprocessing module 514 are used to perform preprocessing according to the input data (such as multiple candidate images in the embodiment of the present application) received by the I / O interface 512. It should be understood that there may be no preprocessing module 513 and the preprocessing module 514 or only one preprocessing module. When there are no preprocessing modules 513 and the preprocessing module 514, the computing module 511 may be directly used to process the input data.

[0197] When the execution device 510 preprocesses the input data, or when the computing module 511 of the execution device 510 performs calculations and other related processing, the execution device 510 can call the data, code, etc. in the data storage system 550 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing into the data storage system 550.

[0198] Finally, the I / O interface 512 presents the processing results, such as the enhanced image obtained after processing (eg, the first processed image and the second processed image in the embodiment of the present application) to the client device 540, thereby providing them to the user.

[0199] It is worth noting that the training device 520 can generate corresponding target models / rules 501 based on different training data for different goals or different tasks, and the corresponding target models / rules 501 can be used to implement image enhancement tasks, thereby providing users with the desired results.

[0200] exist Figure 6 In the case shown, the user can manually give input data (such as multiple candidate images in the embodiment of the present application), and the "manually given input data" can be operated through the interface provided by the I / O interface 512. In another case, the client device 540 can automatically send input data to the I / O interface 512. If the client device 540 is required to automatically send input data and needs to obtain the user's authorization, the user can set the corresponding authority in the client device 540. The user can view the results output by the execution device 510 on the client device 540, and the specific presentation form can be a specific method such as display, sound, action, etc. The client device 540 can also be used as a data acquisition terminal to collect the input data of the input I / O interface 512 and the output results of the output I / O interface 512 as new sample data, and store them in the database 530. Of course, it is also possible not to collect through the client device 540, but the I / O interface 512 directly stores the input data of the input I / O interface 512 and the output results of the output I / O interface 512 as new sample data in the database 530.

[0201] In the embodiment of the present application, only the first processed image and the second processed image can be used as test samples for evaluating model performance from multiple candidate images.

[0202] It is worth noting that Figure 6 This is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 6 In the embodiment, the data storage system 550 is an external memory relative to the execution device 510. In other cases, the data storage system 550 can also be placed in the execution device 510.

[0203] The following introduces a chip hardware structure provided by an embodiment of the present application.

[0204] Figure 7 The chip hardware structure diagram provided in an embodiment of the present application includes a neural network processor 700. The chip can be set as follows Figure 6 The execution device 510 shown in FIG. 1 is used to complete the calculation work of the calculation module 511. The chip can also be set in Figure 6 The training device 520 shown in FIG. 5 is used to complete the training work of the training device 520 and output the target model / rule 501. Figure 6 The algorithms of each layer in the image processing model shown can be Figure 7 is implemented in the chip shown.

[0205] The neural processing unit (NPU) 700 is mounted on the host central processing unit (host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 703, and the controller 704 controls the operation circuit 703 to extract data from the memory (weight memory 702 or input memory 701) and perform operations.

[0206] In some implementations, the operation circuit 703 includes multiple processing units (process engines, PEs) inside. In some implementations, the operation circuit 703 is a two-dimensional systolic array. The operation circuit 703 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 703 is a general-purpose matrix processor.

[0207] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit 703 takes the corresponding data of the matrix B from the weight memory 702 and caches it on each PE in the operation circuit 703. The operation circuit 703 takes the matrix A data from the input memory 701 and performs a matrix operation with the matrix B, and the partial result or the final result of the matrix is ​​stored in the accumulator 708.

[0208] The vector calculation unit 707 can further process the output of the operation circuit 703, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector calculation unit 707 can be used for network calculations of non-convolutional / non-FC layers in a neural network, such as pooling, batch normalization, local response normalization, etc.

[0209] In some implementations, the vector calculation unit 707 can store the processed output vector to the unified memory 706. For example, the vector calculation unit 707 can apply a nonlinear function to the output of the operation circuit 703, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation unit 707 generates a normalized value, a merged value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 703, such as for use in a subsequent layer in a neural network.

[0210] The unified memory 706 is used to store input data and output data.

[0211] The weight data is directly transferred from the external memory to the input memory 701 and / or the unified memory 706 through the direct memory access controller (DMAC) 705, the weight data in the external memory is stored in the weight memory 702, and the data in the unified memory 706 is stored in the external memory.

[0212] The bus interface unit (BIU) 710 is used to implement the interaction between the main CPU, DMAC and instruction fetch memory 709 through the bus.

[0213] An instruction fetch buffer 709 connected to the controller 704 is used to store instructions used by the controller 704 .

[0214] The controller 704 is used to call the instructions cached in the instruction fetch memory 709 to control the working process of the computing accelerator.

[0215] Generally, the unified memory 706, the input memory 701, the weight memory 702 and the instruction fetch memory 709 are all on-chip memories, and the external memory is a memory outside the NPU, which can be a double data rate synchronous dynamic random access memory (DDR SDRAM), a high bandwidth memory (HBM) or other readable and writable memory.

[0216] Reference Figure 8 , Figure 8 An example of an image selection method provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, an image selection method provided by an embodiment of the present application includes:

[0217] 801. Obtain multiple candidate images.

[0218] In an embodiment of the present application, multiple candidate images can be obtained, as well as a first image processing model and a second image processing model that need to be compared in performance, wherein the first image processing model and the second image processing model can be used to implement an image processing task, and the image enhancement task can be, for example, an image enhancement task, and the multiple candidate images can be a test sample set for the image enhancement task. Exemplarily, if the image enhancement task is defogging, the multiple candidate images can be a large-scale test sample set for the defogging scene.

[0219] In an embodiment of the present application, the first image processing network and the second image processing network can be used to implement image enhancement tasks. The so-called image enhancement task can be understood as a task used to enhance the quality of a video. For example, the image enhancement task can be a video denoising task, a video defogging task, a super-resolution task or a high dynamic range task, etc., which is not limited here.

[0220] It should be understood that the first image processing network and the second image processing network are models for implementing the same image processing task, and the present application does not limit the specific type of image processing task. Taking the image enhancement task as a super-resolution task as an example, the following describes a schematic diagram of the network structure of the first image processing network and the second image processing network.

[0221] Reference Fig. 9 , Fig. 9 This is a schematic diagram of the structure of an image processing model. Fig. 9As shown, the image to be processed may be a low resolution image (LR), and the low resolution image frame may be processed by the feature extraction module to obtain image features, and then the feature map may be processed by multiple basic units, wherein the basic unit may be a network structure obtained by connecting the basic modules through the basic operations of the neural network, and the above network structure may include a pre-set basic operation or a combination of basic operations in the convolutional neural network, and these basic operations or combinations of basic operations may be collectively referred to as basic operations. For example, the basic operation may refer to a convolution operation, a pooling operation, a residual connection, etc., and the basic operations may connect the basic modules to obtain a network structure of the basic unit. The nonlinear transformation part is used to transform the image features of the image to be input, and map the image features to a high-dimensional feature space. Generally, the mapped high-dimensional space is easier to reconstruct the super-resolution image; the reconstruction part is used to upsample and convolve the image features output by the nonlinear change part to obtain a super-resolution image (LR, HR) corresponding to the image to be input.

[0222] 802. Process each candidate image by using a first image processing model and a second image processing model respectively to obtain a first processed image and a second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing with the first image processing model, and the second processed image is obtained by processing with the second image processing model.

[0223] In an embodiment of the present application, each candidate image can be processed by the first image processing model and the second image processing model respectively to obtain the first processed image and the second processed image corresponding to each candidate image, wherein the so-called "processing" can be understood as the reasoning process of the model.

[0224] In an embodiment of the present application, the first image processing model and the second image processing model are used to implement the same image processing task. Taking the first image processing network and the second image processing network as an example for implementing an image enhancement task, the first processed image is an enhanced image obtained after each candidate image is enhanced by the first image processing network, and the second processed image is an enhanced image obtained after each candidate image is enhanced by the second image processing network.

[0225] Taking defogging as an example of an image enhancement task, the first processed image and the second processed image are candidate images after defogging.

[0226] In an embodiment of the present application, each of the multiple candidate images can be traversed and processed in sequence, or the multiple candidate images can be processed in parallel to obtain a first processed image and a second processed image corresponding to each candidate image. The embodiment of the present application does not limit the chronological order of the first image processing model and the second image processing model when processing the candidate images.

[0227] 803. Obtain an image quality difference between a first processed image and a second processed image corresponding to each candidate image, and select M target images from the multiple candidate images based on the image quality difference, wherein the M target images are used as test samples for performance comparison between the first image processing model and the second image processing model.

[0228] In order to enable experts or other personnel to more accurately compare the performance of the first image processing model and the second image processing model, it is necessary to ensure that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, and then the performance advantages and disadvantages between the first image processing model and the second image processing model can be accurately determined based on the quality of the output images.

[0229] Among them, image quality can be used to characterize the degree to which an image restores a real scene, or it can be called the fidelity of an image. The closer the image is to the real scene, the higher the fidelity of the image, and thus the higher the image quality. In addition, image quality can also be used to characterize the readability of an image. Readability refers to the ability of an image to provide information to a person or machine. It is not only related to the application requirements of the image system, but also often related to the subjective feeling of the human eye. The higher the readability, the higher the image quality. In one implementation, the indicators used to evaluate image quality may include resolution, color depth, image distortion, and other aspects.

[0230] Next, we describe how to obtain the image quality difference between the first processed image and the second processed image corresponding to each candidate image:

[0231] In one implementation, the image quality difference can be calculated based on a non-convolutional neural network method, such as PSNR, SSIM and other methods, or the image quality difference can be calculated based on a convolutional neural network method, such as learned perceptual image patch similarity (LPIPS), DISTS and the like.

[0232] In an embodiment of the present application, the image quality difference between the first processed image and the second processed image corresponding to each candidate image is used as a factor in selecting test samples, and test samples are selected from multiple candidate samples for use as the first image processing model and the second image processing model when performing performance comparison.

[0233] Specifically, in one implementation, M target images whose corresponding image quality differences are greater than a first threshold value may be selected from the multiple candidate images according to the image quality differences corresponding to each candidate image. That is, a corresponding image quality difference may be calculated for each candidate image, and M target images may be selected from the multiple candidate images by comparing the sizes of the image quality differences corresponding to the respective candidate images.

[0234] In one implementation, M target images corresponding to the largest image quality differences may be selected from a plurality of candidate images.

[0235] Reference Fig.10a and Fig.10b , Fig.10a and Fig.10b The process diagram of an image selection method in an embodiment of the present application is as follows: for multiple candidate images, the first image processing model and the second image processing model process each candidate image respectively, and then obtain the first processed image and the second processed image corresponding to each candidate image, and select which candidate images as the target images required for model performance testing based on the image quality difference between the first processed image and the second processed image. Fig.10a As shown, for Fig.10a The candidate image shown, due to its corresponding first processed image ( Fig.10a The image on the upper right of the middle image) and the second processed image ( Fig.10a The image quality of the image in the lower right corner of the figure is quite different (specifically, Fig.10a The first processed image has richer texture details and sharper definition of the tree trunk and leaves than the second processed image, and the trees in the background area are brighter and have richer texture details), which can be used as a test sample for model performance testing. Fig.10b As shown, for Fig.10b The candidate image shown, due to its corresponding first processed image ( Fig.10b The image on the upper right of the middle image) and the second processed image ( Fig.10b The image quality difference between the two images (the image on the lower right in the figure) is small, and therefore cannot be used as test samples for model performance testing.

[0236] In one implementation, in addition to using the image quality difference as a selection factor for the test sample, the image content difference between the candidate images may also be used as a selection factor for the test sample.

[0237] In order to ensure that the test samples used in the model performance comparison can cover more scenes, the reliability of subsequent test results can be ensured when the scenes covered by the test samples are comprehensive enough. The so-called scenes covered by the test samples are comprehensive enough can also be understood as the image content of each test sample is quite different, for example, each test sample includes different target objects, such as test sample A includes people, test sample B includes dogs, test sample C includes cats, and so on.

[0238] Specifically, in an embodiment of the present application, the image content difference between each candidate image and other candidate images other than itself among the multiple candidate images can be obtained, and M target images can be selected from the multiple candidate images based on the image quality difference and the image content difference corresponding to each candidate image.

[0239] The image content difference may indicate the difference between the target subjects in the background area and / or the foreground area in the image.

[0240] Taking the difference between target subjects in the foreground area of ​​image content difference as an example, the image content difference between images including people and images including non-human animals (or inanimate objects, such as buildings, etc.) is greater than the image content difference between images that both include people, and the image content difference between images including different people is greater than the image content difference between images including the same person.

[0241] Next, we describe how to calculate the image content difference between each candidate image and other candidate images in the plurality of candidate images except itself:

[0242] In a possible implementation, traditional methods such as scale-invariant feature transform (SIFT) or convolutional neural network (CNN) can be used to extract image features of each candidate image to obtain image feature vectors of each candidate image for characterizing the image content. Then, the distance metric between the image feature vector of each candidate image and the image feature vectors of other candidate images is calculated. The distance metric can express the content difference between the images. The greater the content difference, the greater the difference between the image content of the candidate image and other candidate images.

[0243] It should be understood that the distance metric can also be called a metric similarity. By calculating the distance metric between two multidimensional data, the similarity between the two multidimensional data can be determined. Generally, the smaller the distance metric between the two multidimensional data, the higher the similarity between the two multidimensional data; on the contrary, the larger the distance metric between the two multidimensional data, the smaller the similarity between the two multidimensional data. Exemplarily, the distance metric may include distances such as mean squared error (MES) distance, L1 distance, or mean squared error MSE. It should be understood that in addition to MES distance and L1 distance, the difference in image content can also be determined based on other distance metrics, and this embodiment does not specifically limit the distance metric method.

[0244] Next, we introduce a structural diagram of a feature extraction network based on CNN to extract image features of each candidate image:

[0245] In an optional implementation, the feature extraction network may include a backbone network and a feature pyramid network (FPN). The backbone network is used to receive an input image, perform convolution processing on the input image, and output feature maps with different resolutions corresponding to the image; that is, output feature maps of different sizes corresponding to the image. The backbone network can perform a series of convolution processing on the input image to obtain feature maps at different scales. The backbone network can take a variety of forms, such as the visual geometry group (VGG), residual neural network (resnet), GoogLeNet's core structure (Inception-net), etc.

[0246] FPN is connected to the backbone network. FPN can perform convolution processing on multiple feature maps of different resolutions generated by the backbone network to construct a feature pyramid.

[0247] Among them, the feature map output by the feature pyramid can be used as the image feature vector of each candidate image.

[0248] In the embodiment of the present application, the selected weight of each candidate image can be determined according to the image quality difference and the image content difference corresponding to each candidate image, and the selected weight is positively correlated with the image quality difference and the image content difference, wherein the selected weight is used as a basis for selecting test samples. Specifically, according to the selected weight of each candidate image, M target images whose selected weights are greater than a second threshold value can be selected from the multiple candidate images. In one implementation, the M target images with the largest selected weights can be selected from the multiple candidate images.

[0249] In one implementation, the selected weight may be a weighted sum of the image quality difference and the image content difference, that is, the selected weight is the sum of a first weight and a second weight, the first weight being the product of the image quality difference and the corresponding weight, and the second weight being the product of the image content difference and the corresponding weight. The weights corresponding to the image quality difference and the image content difference may be adjusted by the user as required, depending on whether the user is more concerned with quality differences or sample diversity differences.

[0250] Specifically, in the process of selecting the target image, an image can be first randomly sampled from multiple candidate images and added to the candidate set of the test image, and the image quality difference corresponding to the candidate image can be calculated. Then, a candidate image can be sampled from multiple candidate images and added to the candidate set, and the image quality difference corresponding to the candidate image and the image content difference between the candidate image and the candidate images already added to the candidate set can be calculated, thereby obtaining the selection weight of the candidate image. Then, a candidate image can be sampled from multiple candidate images and added to the candidate set, and the image quality difference corresponding to the candidate image and the image content difference between the candidate image and the two candidate images already added to the candidate set can be calculated (specifically, it can be the average of the image content difference between the candidate image and each of the two candidate images in the candidate set), thereby obtaining the selection weight of the candidate image, and so on, until M candidate images are sampled, at which time the candidate set can include M candidate images, and the selection weight of each of the M candidate images is calculated. Then continue to randomly sample an image from multiple candidate images, and calculate the image quality difference corresponding to the candidate image, as well as the image content difference between the candidate image and the M candidate images that have been added to the candidate set (specifically, it can be the average of the image content difference between the candidate image and each of the M candidate images in the candidate set), and then obtain the selection weight of the candidate image, and then sort the selection weights of the sampled M+1 candidate images, and take the largest M selection weights as the new candidate set (that is, one of the original M candidate images is eliminated, or the latest sampled image is not added). And so on, until the sampling of all candidate images is completed, at which time the M candidate images in the candidate set can be the selected M target images, and the image quality difference corresponding to each of the M target images is large, and the content difference between the images is large.

[0251] Take the Shao20 algorithm and FFA-Net as examples for the first image processing model and the second image processing model. Fig.10c As shown, Fig.10c The selected target images have obvious quality differences in dehazing effects, and the image contents are quite different (one is a building and the other is a portrait), which meets the selection requirements of test samples.

[0252] After obtaining M target images used as test samples for performance comparison between the first image processing model and the second image processing model, the first image processing model and the second image processing model can process the M target images, and experts can perform subjective quality evaluation on the processing results of the first image processing model and the second image processing model, and compare the performance between the first image processing model and the second image processing model based on the subjective quality evaluation results.

[0253] It should be understood that in some scenarios, if only the comprehensiveness of the scene coverage between the test samples is considered, the image content difference between the candidate images can be used as the basis for selecting the test samples, instead of the image quality difference. Specifically, multiple candidate images can be obtained, and the image content difference between each candidate image and other candidate images among the multiple candidate images except itself is obtained. According to the image content difference corresponding to each candidate image, M target images are selected from the multiple candidate images.

[0254] The embodiment of the present application provides an image selection method, characterized in that the method includes: obtaining multiple candidate images; processing each candidate image by a first image processing model and a second image processing model respectively to obtain a first processed image and a second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model; obtaining the image quality difference between the first processed image and the second processed image corresponding to each candidate image, and selecting M target images from the multiple candidate images according to the image quality difference, and the M target images are used as test samples for the first image processing model and the second image processing model when performing performance comparison. In the above manner, the image quality difference corresponding to each candidate image is used as the basis for selecting the test sample, ensuring that the image quality difference between the output images obtained by the first image processing model and the second image processing model when processing the same test sample is large, thereby helping image quality evaluators to more easily subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

[0255] In addition, the image content differences between the candidate images are used as the basis for selecting test samples, so that test samples with greater content differences can be selected, thereby ensuring that the scenes covered by the sampling of the test samples are comprehensive enough, thereby ensuring the reliability of the test results.

[0256] The above introduces the process of selecting test samples. Next, the process of model performance testing is described in combination with the selected test samples.

[0257] Reference Fig.10d , Fig.10d A schematic diagram of a model testing method provided in an embodiment of the present application is shown in FIG. Fig.10d As shown, the method includes:

[0258] 1001. Obtain M target images;

[0259] Among them, the M target images are used as test samples when comparing the performance of the first image processing model and the second image processing model.

[0260] 1002. Process each target image by using a first image processing model and a second image processing model respectively to obtain a first processed image and a second processed image corresponding to each target image; wherein the first image processing model and the second image processing model are used to implement the same image processing task, the first processed image is obtained by processing by the first image processing model, and the second processed image is obtained by processing by the second image processing model; and the image quality difference between the first processed image and the second processed image corresponding to each target image is greater than a first threshold.

[0261] 1003. Obtain a model test result, where the model test result is used to represent a performance comparison result between the first image processing model and the second image processing model, and the model test result is determined based on the first processed image and the second processed image corresponding to each target image.

[0262] After obtaining the first processed image and the second processed image corresponding to each target image, the model tester can compare the performance between the first image processing model and the second image processing model based on the first processed image and the second processed image corresponding to each target image, and feedback the model performance comparison results, and then the model performance comparison results can be used as the model testing results.

[0263] In the embodiment of the present application, there is a large difference in image quality between the output images obtained by the first image processing model and the second image processing model when processing the same target image, thereby helping image quality evaluators to more easily and subjectively judge the performance advantages and disadvantages between the first image processing model and the second image processing model.

[0264] In one possible implementation, the selected weight of each target image in the M target images is greater than a second threshold, and the selected weight of each target image is positively correlated with the image quality difference and image content difference corresponding to each target image, wherein the image content difference corresponding to each target image is used to represent the image content difference between each target image and other target images among the multiple target images except itself.

[0265] In the embodiment of the present application, the image contents of the target images are quite different, that is, the scenes covered by the M target images are comprehensive enough, thus ensuring the reliability of the test results.

[0266] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0267] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0268] Reference Fig.11 , Fig.11 The structure of an image selection device 1100 provided in an embodiment of the present application is shown as follows: Fig.11 As shown, the device comprises:

[0269] An acquisition module 1101 is used to acquire multiple candidate images;

[0270] An image processing module 1102 is used to process each candidate image through a first image processing model and a second image processing model respectively, so as to obtain a first processed image and a second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model;

[0271] The image selection module 1103 is used to obtain the image quality difference between the first processed image and the second processed image corresponding to each candidate image, and select M target images from the multiple candidate images based on the image quality difference. The M target images are used as test samples for the first image processing model and the second image processing model when performing performance comparison.

[0272] In a possible implementation, the image selection module 1103 is configured to select, according to the image quality difference corresponding to each candidate image, M target images whose corresponding image quality difference is greater than a first threshold from the multiple candidate images.

[0273] In a possible implementation, the acquisition module is used to acquire image content differences between each candidate image and other candidate images among the multiple candidate images except itself;

[0274] The image selection module is used to select M target images from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image.

[0275] In a possible implementation, the image selection module 1103 is used to determine the selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, and the selection weight is positively correlated with the image quality difference and the image content difference;

[0276] According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

[0277] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0278] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0279] Reference Fig.12 , Fig.12 The structure of an image selection device 1200 provided in an embodiment of the present application is shown as follows: Fig.12 As shown, the device 1200 includes:

[0280] An acquisition module 1201 is used to acquire multiple candidate images;

[0281] Obtaining image content differences between each candidate image and other candidate images among the plurality of candidate images except itself;

[0282] The image selection module 1202 is configured to select M target images from the multiple candidate images according to the image content difference corresponding to each candidate image.

[0283] In a possible implementation, the image selection module 1202 is configured to select, from the multiple candidate images, M target images whose image content differences are greater than a third threshold value according to the image content difference corresponding to each candidate image.

[0284] In a possible implementation, the device further includes:

[0285] An image processing module, used to process each candidate image by using a first image processing model and a second image processing model respectively, so as to obtain a first processed image and a second processed image corresponding to each candidate image;

[0286] The acquisition module is used to acquire the image quality difference between the first processed image and the second processed image corresponding to each candidate image;

[0287] The image selection module is used to select M target images from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image.

[0288] In a possible implementation, the image selection module is used to determine a selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, wherein the selection weight is positively correlated with the image quality difference and the image content difference;

[0289] According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

[0290] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0291] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0292] Reference Fig.13 , Fig.13 A schematic diagram of the structure of a model testing device 1300 provided in an embodiment of the present application is shown as follows: Fig.13 As shown, the device 1300 includes:

[0293] An acquisition module 1301 is used to acquire M target images;

[0294] The image processing module 1302 is used to process each target image respectively through the first image processing model and the second image processing model to obtain the first processed image and the second processed image corresponding to each target image; wherein the first image processing model and the second image processing model are used to implement the same image processing task, the first processed image is obtained by processing by the first image processing model, and the second processed image is obtained by processing by the second image processing model; and the image quality difference between the first processed image and the second processed image corresponding to each target image is greater than a first threshold; the first processed image and the second processed image corresponding to each target image are used to perform performance comparison between the first image processing model and the second image processing model.

[0295] The acquisition module 1301 is also used to obtain model test results, which are used to represent the performance comparison results between the first image processing model and the second image processing model, and the model test results are determined based on the first processed image and the second processed image corresponding to each target image.

[0296] In one possible implementation, the selected weight of each target image in the M target images is greater than a second threshold, and the selected weight of each target image is positively correlated with the image quality difference and image content difference corresponding to each target image, wherein the image content difference corresponding to each target image is used to represent the image content difference between each target image and other target images among the multiple target images except itself.

[0297] In a possible implementation, the selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

[0298] In a possible implementation, the first image processing model and the second image processing model are used to implement an image enhancement task.

[0299] Next, an image processing device provided by an embodiment of the present application is introduced. Fig.14 , Fig.14 This is a schematic diagram of a structure of an image processing device provided in an embodiment of the present application. The image processing device 1400 can be specifically embodied as a mobile phone, a tablet, a laptop computer, a smart wearable device, a server, etc., which is not limited here. Among them, the image processing device 1400 can execute Figure 8 as well as Fig.10a The image selection method described in the corresponding embodiment, and Fig.10dThe model testing method described in the corresponding embodiment. Specifically, the image processing device 1400 includes: a receiver 1401, a transmitter 1402, a processor 1403 and a memory 1404 (wherein the number of the processor 1403 in the image selection device 1400 can be one or more, Fig.14 In the example of FIG. 1403 , the processor 1403 may include an application processor 14031 and a communication processor 14032. In some embodiments of the present application, the receiver 1401, the transmitter 1402, the processor 1403 and the memory 1404 may be connected via a bus or other means.

[0300] The memory 1404 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1403. A portion of the memory 1404 may also include a non-volatile random access memory (NVRAM). The memory 1404 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0301] The processor 1403 controls the operation of the execution device. In a specific application, the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0302] The method disclosed in the above embodiment of the present application can be applied to the processor 1403, or implemented by the processor 1403. The processor 1403 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware or software instructions in the processor 1403. The above-mentioned processor 1403 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and a vision processor (vision processing unit, VPU), a tensor processor (tensorprocessing unit, TPU) and other processors suitable for AI computing, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1403 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1404, and the processor 1403 reads the information in the memory 1404, and completes the steps of the above method in combination with its hardware.

[0303] The receiver 1401 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the execution device. The transmitter 1402 can be used to output digital or character information through the first interface; the transmitter 1402 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1402 can also include a display device such as a display screen. The execution device can execute Figure 8 as well as Fig.10a The image selection method described in the corresponding embodiment, and Fig.10d The model testing method is described in the corresponding embodiment.

[0304] Also provided in an embodiment of the present application is a computer program product which, when executed on a computer, enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0305] A computer-readable storage medium is also provided in an embodiment of the present application, which stores a program for signal processing. When the computer-readable storage medium is run on a computer, it enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0306] The execution device, training device or terminal device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0307] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0308] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0309] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0310] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. An image selection method, characterized in that: The method comprises: Acquire multiple candidate images; Processing each candidate image by a first image processing model and a second image processing model respectively to obtain a first processed image and a second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model; Obtain an image quality difference between a first processed image and a second processed image corresponding to each candidate image, and select M target images from the multiple candidate images based on the image quality difference, wherein the M target images are used as test samples for performance comparison between the first image processing model and the second image processing model.

2. The method according to claim 1, characterized in that The step of selecting M target images from the plurality of candidate images according to the image quality differences comprises: According to the image quality difference corresponding to each candidate image, M target images whose corresponding image quality difference is greater than a first threshold are selected from the multiple candidate images.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining image content differences between each candidate image and other candidate images among the plurality of candidate images except itself; The step of selecting M target images from the plurality of candidate images according to the image quality differences comprises: According to the image quality difference and the image content difference corresponding to each candidate image, M target images are selected from the multiple candidate images.

4. The method according to claim 3, characterized in that The selecting M target images from the plurality of candidate images according to the image quality difference and the image content difference corresponding to each candidate image comprises: Determining a selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, wherein the selection weight is positively correlated with the image quality difference and the image content difference; According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

5. The method according to claim 4, characterized in that The selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

6. The method according to any one of claims 1 to 5, characterized in that: The first image processing model and the second image processing model are used to implement image enhancement tasks.

7. An image selection device, characterized in that: The device comprises: An acquisition module, used for acquiring multiple candidate images; An image processing module, used for processing each candidate image through a first image processing model and a second image processing model respectively, so as to obtain a first processed image and a second processed image corresponding to each candidate image; wherein the first image processing model and the second image processing model are used to implement the same image processing task; the first processed image is obtained by processing the first image processing model, and the second processed image is obtained by processing the second image processing model; An image selection module is used to obtain the image quality difference between the first processed image and the second processed image corresponding to each candidate image, and select M target images from the multiple candidate images based on the image quality difference. The M target images are used as test samples for the first image processing model and the second image processing model when performing performance comparison.

8. The device according to claim 7, characterized in that The image selection module is used to select M target images whose corresponding image quality differences are greater than a first threshold from the multiple candidate images according to the image quality difference corresponding to each candidate image.

9. The device according to claim 7, characterized in that The acquisition module is used to acquire the image content difference between each candidate image and other candidate images among the multiple candidate images except itself; The image selection module is used to select M target images from the multiple candidate images according to the image quality difference and the image content difference corresponding to each candidate image.

10. The device according to claim 9, characterized in that The image selection module is used to determine the selection weight of each candidate image according to the image quality difference and the image content difference corresponding to each candidate image, wherein the selection weight is positively correlated with the image quality difference and the image content difference; According to the selected weight of each candidate image, M target images whose selected weight is greater than a second threshold are selected from the multiple candidate images.

11. The device according to claim 10, characterized in that The selected weight is the sum of a first weight and a second weight, wherein the first weight is the product of the image quality difference and the corresponding weight, and the second weight is the product of the image content difference and the corresponding weight.

12. The device according to any one of claims 7 to 11, characterized in that The first image processing model and the second image processing model are used to implement image enhancement tasks.

13. An image selection device, characterized in that: The device includes a memory and a processor; the memory stores codes, and the processor is configured to obtain the codes and execute the method according to any one of claims 1 to 6.

14. A computer storage medium, characterized in that: The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 6.

15. A computer product comprising code, characterized in that When the code is executed, it is used to implement the method according to any one of claims 1 to 6.

Images