Image processing chip parameter optimization method and device and electronic equipment

Abstract

The invention provides an image processing chip parameter optimization method and device and electronic equipment, and relates to the technical field of image processing.The method comprises the steps that a first image feature of a sample reference image and a second image feature of a sample processing image are extracted, the sample reference image is a noise-free image generated based on a Bayer image, and the second image feature of the sample processing image is a noise-free image generated based on a Bayer image; the sample processing image is an image generated after the Bayer image is processed by an image processing chip; determining an image quality evaluation index based on a feature difference degree between the first image feature and the second image feature; debugging parameters of the image processing chip by taking the image quality evaluation index as a training target by using a covariance adaptive evolution strategy to obtain optimized parameters; and based on the optimization parameters, controlling an image processing chip to carry out image processing. According to the method provided by the invention, the debugging efficiency of the image processing chip can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to a method, apparatus, and electronic device for optimizing image processing chip parameters. Background Art

[0002] An image signal processing (ISP) chip is a type of integrated circuit specifically designed for processing image data. It is widely used in electronic devices such as mobile phones, computers, and smart cameras. It can perform various processes on the captured images through AI (Artificial Intelligence) technologies, such as denoising, tone mapping, color correction, sharpening, automatic exposure, and automatic white balance. However, due to the coupling between various modules in the image processing chip and its high complexity, the current debugging of the image processing chip mainly relies on manually adjusting the image processing parameters in the algorithm module, which is inefficient and difficult to achieve the best image effect. Summary of the Invention

[0003] Embodiments of this application provide a method, apparatus, and electronic device for optimizing image processing chip parameters, which can improve the debugging efficiency of the image processing chip.

[0004] In a first aspect, embodiments of this application provide a method for optimizing image processing chip parameters, including:

[0005] Extracting a first image feature of a sample reference image and a second image feature of a sample processed image, where the sample reference image is a noise-free image generated based on a Bayer image, and the sample processed image is an image generated after the image processing chip processes the Bayer image;

[0006] Determining an image quality evaluation index based on the feature difference degree between the first image feature and the second image feature;

[0007] Using a covariance adaptive evolution strategy, taking the image quality evaluation index as a training target to debug the parameters of the image processing chip to obtain optimized parameters;

[0008] Based on the optimized parameters, controlling the image processing chip to perform image processing.

[0009] In some embodiments, the determining an image quality evaluation index based on the feature difference degree between the first image feature and the second image feature includes:

[0010] Mark a plurality of first interest points in the sample reference image, and mark a plurality of second interest points in the sample processed image, where the positions of the first interest points in the sample reference image are the same as the positions of the second interest points in the sample processed image, and both the first interest points and the second interest points are pixel points with unique features;

[0011] Determine the number of matching interest points whose Euclidean distance between the first interest points and the second interest points is less than a preset threshold;

[0012] Determine the image quality evaluation index based on the proportion of the number of the matching interest points in the total number of interest points.

[0013] In some embodiments, the determining the number of matching interest points whose Euclidean distance between the first interest points and the second interest points is less than a preset threshold includes:

[0014] Determine the Euclidean distance between the first interest point and the second interest point based on the distance between the first feature vector of the pixel points in the neighborhood of the first interest point and the second feature vector of the pixel points in the neighborhood of the second interest point.

[0015] In some embodiments, the determining the Euclidean distance between the first interest point and the second interest point based on the distance between the first feature vector of the pixel points in the neighborhood of the first interest point and the second feature vector of the pixel points in the neighborhood of the second interest point includes:

[0016] Select pixel points with structural similarity meeting a preset degree from the pixel points in the neighborhood of the first interest point and the pixel points in the neighborhood of the second interest point, where the structural similarity includes at least one of brightness, contrast, and structure;

[0017] Determine the Euclidean distance between the first interest point and the second interest point based on the distance between the first feature vector and the second feature vector of the selected pixel points.

[0018] In some embodiments, the using the covariance adaptive evolution strategy to debug the parameters of the image processing chip with the image quality evaluation index as the training target to obtain optimized parameters includes:

[0019] Select a plurality of sample technical parameters of the signal processing module within the value range corresponding to the technical parameters of the signal processing module in the image processing chip, where the signal processing module includes at least one of a brightness module and a sharpness module, and the value range is determined based on the initial variance of the covariance adaptive evolution strategy;

[0020] Based on multiple of the sample technical parameters, control the image processing chip to process the Bayer image to obtain multiple sample processed images;

[0021] Respectively compare the features of multiple sample processed images with the sample reference image to obtain multiple image processing evaluation metrics;

[0022] According to the degree of fit between multiple image processing evaluation metrics and the image quality evaluation metric, perform iterative training on the sample technical parameters to obtain optimized parameters.

[0023] In some embodiments, the performing iterative training on the sample technical parameters according to the degree of fit between multiple image processing evaluation metrics and the image quality evaluation metric to obtain optimized parameters includes:

[0024] Construct an optimization function according to the degree of fit between multiple image processing evaluation metrics and the image quality evaluation metric;

[0025] Use the covariance adaptive evolution strategy to process the optimization function to obtain the minimum value of the optimization function;

[0026] In the case where the minimum value is less than or equal to a preset value, and / or the iterative training reaches a preset number of times, select the corresponding sample technical parameter from multiple sample processed images based on the minimum value as the optimized parameter.

[0027] In some embodiments, the method for optimizing the parameters of the image processing chip further includes:

[0028] In each iterative training, generate multiple sample technical parameters by adjusting the weight coefficients corresponding to different technical parameters of the signal processing module, and the sum of the weight coefficients corresponding to the different technical parameters is 1.

[0029] In a second aspect, an embodiment of the present application proposes an apparatus for optimizing the parameters of an image processing chip, including:

[0030] An extraction unit, configured to extract a first image feature of a sample reference image and a second image feature of a sample processed image, where the sample reference image is a noise-free image generated based on a Bayer image, and the sample processed image is generated after the image processing chip processes the Bayer image;

[0031] A determination unit, configured to determine an image quality evaluation metric based on the feature difference degree between the first image feature and the second image feature,

[0032] A debugging unit, configured to use the covariance adaptive evolution strategy to debug the parameters of the image processing chip with the image quality evaluation index as the training target, so as to obtain optimized parameters;

[0033] A control unit, configured to control the image processing chip to perform image processing based on the optimized parameters.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0035] A memory, configured to store a computer program;

[0036] A processor, configured to execute the computer program to implement the steps of the method according to any one of the first aspect.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor is caused to execute the steps of the method according to any one of the first aspect.

[0038] For the method for optimizing the parameters of an image processing chip provided by the embodiment of the present application, the image feature difference between the image processed by the image processing chip and the reference image is compared, and the covariance adaptive evolution strategy is used to take the above image feature difference as the training target to debug the parameters of the image processing chip, so as to obtain optimized parameters. Based on the optimized parameters, the image processing chip is controlled to perform image processing. Compared with the traditional manual debugging of the image processing chip, using a gradient-free optimization algorithm to debug the parameters of the image processing chip can fully shorten the debugging cycle and improve the debugging efficiency.

[0039] The technical effects obtained in the above second aspect to fourth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the present application, the drawings required for the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0041] Figure 1 It is a schematic flowchart of a method for optimizing the parameters of an image processing chip provided by an embodiment of the present application;

[0042] Figure 2 It is an application schematic diagram of a method for optimizing the parameters of an image processing chip provided by an embodiment of the present application;

[0043] Figure 3Schematic structural diagram of an apparatus for optimizing parameters of an image processing chip proposed in an embodiment of the present application;

[0044] Figure 4 Schematic structural diagram of an electronic device proposed in an embodiment of the present application. Detailed implementation manners

[0045] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0046] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B; the "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first" and "second" do not limit the quantity and execution order, and the terms "first" and "second" do not necessarily mean different.

[0047] In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0048] An image processing chip can process the original Bayer image into an RGB image, which involves the coordinated cooperation of multiple internal modules, such as denoising, tone mapping, color correction, sharpening, automatic exposure, automatic white balance, etc.

[0049] However, currently, the debugging of the image processing chip mainly relies on manually debugging the image processing parameters in the algorithm module. Due to the coupling between the various modules in the image processing chip and the high complexity, in the related art, there are technical problems of low efficiency in debugging the image processing chip and it is difficult to achieve the best image effect.

[0050] In view of this, an embodiment of the present application proposes a method for optimizing the parameters of an image processing chip to at least solve the above technical problems. Figure 1 It is a schematic flowchart of a method for optimizing the parameters of an image processing chip proposed by an embodiment of the present application, as Figure 1 shown. The above method for optimizing the parameters of an image processing chip includes steps S110 to S140:

[0051] S110, extract the first image feature of the sample reference image and the second image feature of the sample processed image. The sample reference image is a noise-free image generated based on the Bayer image, and the sample processed image is an image generated after the image processing chip processes the Bayer image.

[0052] Figure 2 It is an application schematic diagram of a method for optimizing the parameters of an image processing chip proposed by an embodiment of the present application, as Figure 2 shown. The above sample reference image can be a noise-free image generated by processing the Bayer image. Specifically, it can generate an RGB image after processing such as demosaicing interpolation, color correction, and tone mapping. The above sample processed image is an RGB image generated after the above image processing chip (ISP) processes the Bayer image. It should be noted that the original image sources of the sample reference image and the sample processed image should be the same, that is, both are processed from the same above Bayer image to achieve the purpose of controlling variables.

[0053] It should be noted that the above sample reference image is an image for comparison, that is, a high-quality image.

[0054] To extract image features from the above sample reference image and the above sample processed image, an encoder can be used, such as a Transformer encoder or a U-Net encoder. The extracted image features can be features that can more intuitively reflect the external quality of the image, such as color features, texture features, shape features, etc.

[0055] S120, determine an image quality evaluation index based on the feature difference degree between the first image feature and the second image feature.

[0056] It should be noted that the above image quality evaluation index can be understood as an objective evaluation standard for evaluating image quality, which is determined by comparing the feature difference degree between the first image feature corresponding to the above sample reference image and the second image feature corresponding to the above sample processed image. The above feature difference degree can be understood as the gap between the sample processed image and the sample reference image in terms of image quality such as color, texture, shape, edge, as well as brightness, hue, contrast, etc. According to some gaps between the two in image quality, an evaluation standard is determined. For example, in terms of color, compare the intensity gaps of the R (red), G (green), and B (blue) color channels of the image pixel points; in terms of contrast, compare the peak signal-to-noise ratio; in terms of brightness, compare the gray values of the image pixel points, etc.

[0057] In some examples, step S120 can be carried out in the following manner:

[0058] Mark a plurality of first interest points in the sample reference image, and mark a plurality of second interest points in the sample processed image. The positions of the first interest points in the sample reference image are the same as the positions of the second interest points in the sample processed image. Both the first interest points and the second interest points are pixel points with unique features.

[0059] Exemplarily, mark a plurality of interest points at the same positions in the sample reference image and the sample processed image. The interest points are pixel points with unique features on the image, having the characteristics of rotational invariance and not changing with the illumination conditions.

[0060] It should be noted that when marking pixel points on the image, they can be marked dispersedly, covering as much of the entire image range as possible.

[0061] Determine the number of matching interest points whose Euclidean distance between the first interest points and the second interest points is less than a preset threshold;

[0062] Exemplarily, the Euclidean distance can be used to compare the differences between image feature points. By comparing the Euclidean distance between the first interest points and the second interest points at the same positions in the image, when the Euclidean distance of the interest points at a certain position is less than the preset threshold, then the interest point is the above-mentioned matching interest point. By comparing all the interest points, the number of all matching interest points can be obtained.

[0063] It should be noted that those skilled in the art can set the above preset threshold according to experience, requirements, and image characteristics, and the embodiments of the present application do not make specific limitations.

[0064] Determine the image quality evaluation index based on the proportion of the number of matching interest points in the number of interest points.

[0065] It should be noted that the more of the above-mentioned matching interest points, the higher the similarity between the sample reference image and the sample processed image can be proved, that is, the higher the quality of the image processed by the image processing chip. Therefore, the proportion of the number of matching interest points in the interest points can be used as an image quality evaluation index.

[0066] In the above example, the number of matching interest points whose Euclidean distance between the first interest point and the second interest point is less than the preset threshold may include:

[0067] Determine the Euclidean distance between the first interest point and the second interest point based on the distance between the first feature vector of the pixel points in the neighborhood of the first interest point and the second feature vector of the pixel points in the neighborhood of the second interest point.

[0068] Exemplarily, the neighborhood of an interest point refers to N×M pixel grids centered on the interest point, for example, it can be 16×16. To determine the Euclidean distance between interest points based on the distance between the first feature vector of the pixel points in the neighborhood of the first interest point and the second feature vector of the pixel points in the neighborhood of the second interest point, the following formula can be used:

[0069]

[0070] where d ab represents the Euclidean distance between the first interest point and the second interest point; d a (i) represents the first feature vector of the pixel points in the neighborhood of the first interest point; d b (i) represents the second feature vector of the pixel points in the neighborhood of the second interest point.

[0071] In the above example, determining the Euclidean distance between the first interest point and the second interest point based on the distance between the first feature vector of the pixel points in the neighborhood of the first interest point and the second feature vector of the pixel points in the neighborhood of the second interest point may include:

[0072] Select pixel points with structural similarity meeting the preset degree from the pixel points in the neighborhood of the first interest point and the pixel points in the neighborhood of the second interest point, and the structural similarity includes at least one of brightness, contrast, and structure.

[0073] Determine the Euclidean distance between the first interest point and the second interest point based on the distance between the first feature vector and the second feature vector of the selected pixel points.

[0074] It should be noted that the neighborhood of each point of interest includes multiple pixel points. Due to sampling technology, there are significant differences in the brightness, contrast, and structure of multiple pixel points. For some pixel points whose brightness, contrast, and structure differ greatly from the preset values, screening can be performed, and only the screened pixel points are used to calculate the above-mentioned Euclidean distance, thereby reducing the calculation error caused by external factors such as sampling technology.

[0075] Exemplarily, different preset values are set for different structural similarities. For example, in terms of brightness, overly dark pixel points can be removed, and pixel points with brightness meeting a certain value are retained; in terms of contrast, the light and dark differences of pixel points can be compared; in terms of structure, it can be determined based on the geometric structure of pixel points. Screening can be performed from multiple aspects such as brightness, contrast, and structure, or pixel points can be screened from only one aspect.

[0076] S130. Using the covariance adaptive evolution strategy, the image quality evaluation index is used as the training objective to debug the parameters of the image processing chip to obtain optimized parameters.

[0077] Exemplarily, as Figure 2 shown, an optimizer can be used to debug the parameters in the ISP with the above-mentioned image quality evaluation index as the training objective, and the strategy adopted by the optimizer is the covariance adaptive evolution strategy (CMA-ES). In the optimizer, the initial parameters of the ISP are debugged based on the variance set by CMA-ES until the preset effect is achieved. By using the CMA-ES strategy and adopting a gradient-free optimization algorithm to debug the parameters of the image processing chip, the debugging cycle can be fully shortened and the debugging efficiency can be improved.

[0078] In some examples, step S130 can be executed in the following manner:

[0079] Within the value range corresponding to the technical parameters of the signal processing module in the image processing chip, multiple sample technical parameters of the signal processing module are selected. The signal processing module includes at least one of a brightness module and a clarity module, and the value range is determined based on the initial variance of the covariance adaptive evolution strategy.

[0080] It should be noted that the signal processing module in the ISP is a functional module for processing images. Usually, it is divided into multiple sub-modules according to different processing angles. In the embodiments of the present application, considering the impact on image quality, at least the brightness module and the clarity module need to be debugged.

[0081] Exemplarily, the above-mentioned technical parameters of the signal processing module are the initial parameters. Within a certain value range, multiple sample technical parameters are selected, and this value range can be determined based on the initial variance of the covariance adaptive evolution strategy.

[0082] It should be noted that in this example, the initial parameters can be used as the initial seed nodes, and multiple child nodes can be selected within the range of the initial seed nodes, and in the next step of processing, new seed nodes are determined.

[0083] Based on multiple sample technical parameters, control the image processing chip to process the Bayer image to obtain multiple sample processed images.

[0084] Exemplarily, control the ISP to process the same Bayer image based on the multiple selected sample technical parameters, so that multiple corresponding sample processed images can be obtained.

[0085] Compare the multiple sample processed images with the sample reference image respectively to obtain multiple image processing evaluation indicators.

[0086] Exemplarily, compare the multiple obtained sample processed images with the sample reference image in the manner of step S120 above, so that corresponding image processing evaluation indicators can be obtained.

[0087] Iteratively train the sample technical parameters according to the degree of fit between the multiple image processing evaluation indicators and the image quality evaluation indicators to obtain optimized parameters.

[0088] Exemplarily, determine the child node closest to the image quality evaluation indicator as the new seed node. Based on the new seed node, determine a selection range, then select multiple child nodes. Based on the sample technical parameters corresponding to the multiple child nodes, control the image processing chip to process the Bayer image to generate multiple image processing evaluation indicators, and then further select a new seed node according to the degree of fit between the image processing evaluation indicators and the image quality evaluation indicators until the required technical parameters are found.

[0089] In the above example, the iterative training of the sample technical parameters according to the degree of fit between the multiple image processing evaluation indicators and the image quality evaluation indicators to obtain optimized parameters can be carried out in the following manner:

[0090] Construct an optimization function according to the degree of fit between the above multiple image processing evaluation indicators and the image quality evaluation indicators.

[0091] Exemplarily, the optimization function can be shown as follows:

[0092]

[0093] Among them, T represents the value of the optimization function; K represents the number of iterations, and K is greater than 1; p represents the image processing evaluation indicator; p′ represents the image quality evaluation indicator.

[0094] Using the covariance adaptive evolution strategy, the optimization function is processed to obtain the minimum value of the optimization function.

[0095] Exemplarily, in each iteration, the sample processing parameter corresponding to the minimum value of the optimization function among the above-mentioned multiple sample processing parameters is used as the new seed node.

[0096] In the case where the minimum value is less than or equal to the preset value and / or the iterative training reaches the preset number of times, the corresponding sample technical parameter is selected from multiple sample processing images based on the minimum value as the optimization parameter.

[0097] Exemplarily, a standard for stopping the optimization search is set. When the minimum value of the above optimization function reaches the preset value, or the number of iterations has been reached, or both are used as the standard for stopping the iteration. In the case of reaching the stop iteration, the sample parameter corresponding to the minimum value of the above optimization function in the last iteration is selected as the optimization parameter.

[0098] According to some embodiments, the image parameter optimization method provided by the embodiments of the present application further includes:

[0099] In each iterative training, by adjusting the weight coefficients corresponding to different technical parameters of the signal processing module, multiple sample technical parameters are generated, and the sum of the weight coefficients corresponding to different technical parameters is 1.

[0100] Exemplarily, each functional module in the ISP corresponds to multiple parameters, and the multiple parameters jointly affect the processing effect of the functional module. Taking the luminance module as an example, it includes three parameters a, b, and c. Initially, the ratio of a, b, and c is 1:2:3. When determining the sample processing parameters corresponding to the three parameters a, b, and c, it can be adjusted to multiple sets of ratio coefficients such as 2:1:3, 3:2:1, or 2.5:1.5:2 to form multiple sample technical parameters. By adjusting the weights between the parameters to select multiple sample processing parameters, it can better reflect the influence proportion of different parameters on the image quality, so as to better determine the debugging direction of the parameters and improve the debugging efficiency.

[0101] S140, based on the optimization parameter, control the image processing chip to perform image processing.

[0102] Exemplarily, based on the optimization parameter obtained in step S130, the ISP performs image processing and can obtain an image that best fits the image quality evaluation index.

[0103] In summary, the method for optimizing the parameters of the image processing chip proposed in the embodiments of the present application compares the image feature differences between the images processed by the image processing chip and the reference image, and uses the covariance adaptive evolution strategy to use the above image feature differences as the training target to debug the parameters of the image processing chip to obtain optimized parameters. Based on the optimized parameters, the image processing chip is controlled to perform image processing. Compared with the traditional manual debugging of the image processing chip, using a gradient-free optimization algorithm to debug the parameters of the image processing chip can fully shorten the debugging cycle and improve the debugging efficiency.

[0104] The above has given examples of the method embodiments according to the present application. Based on the method for optimizing the parameters of the image processing chip provided by the embodiments of the present application, the embodiments of the present application provide an apparatus for optimizing the parameters of the image processing chip. Figure 3 It is a schematic structural diagram of an apparatus for optimizing the parameters of an image processing chip provided by an embodiment of the present application. Referring to Figure 3 this, the apparatus includes the following units.

[0105] An extraction unit 310, configured to extract a first image feature of a sample reference image and a second image feature of a sample processed image, where the sample reference image is a noise-free image generated based on a Bayer image, and the sample processed image is generated after the image processing chip processes the Bayer image;

[0106] A determination unit 320, configured to determine an image quality evaluation index based on the feature difference degree between the first image feature and the second image feature;

[0107] A debugging unit 330, configured to use the covariance adaptive evolution strategy to use the image quality evaluation index as a training target to debug the parameters of the image processing chip to obtain optimized parameters;

[0108] A control unit 340, configured to control the image processing chip to perform image processing based on the optimized parameters.

[0109] The above has described the apparatus embodiments of the present application. Among them, for the detailed descriptions of data, terms, nouns, the specific execution processes of steps, technical problems and effects, alternative methods and combination methods, etc., please refer to the descriptions in the method embodiments, and will not be elaborated here.

[0110] The embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor is caused to execute the steps of the method according to any one of the first aspects.

[0111] The embodiments of the present application further provide an electronic device, including:

[0112] A memory for storing a computer program;

[0113] A processor for executing the computer program to implement the steps of the method described in any one of the first aspects.

[0114] For example, as Figure 4 shown, an electronic device includes a processor 410, at least one communication bus 420, a user interface 430, at least one external communication interface 440, and a memory 450. Among them, the communication bus 420 is configured to implement connection communication between these components. Among them, the user interface 430 may include a display screen, and the external communication interface 440 may include a standard wired interface and a wireless interface. Among them, a computer program is stored in the memory 500. Among them, the processor 410 is used to execute the computer program stored in the memory 450.

[0115] The descriptions of the above computer program products, computer-readable storage media, and electronic devices are similar to the descriptions of the above method embodiments, and have beneficial effects similar to those of the method embodiments. For the technical details not disclosed in the computer program products, computer-readable storage media, and electronic devices of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0116] The serial numbers of the embodiments of the present application or the order of introduction are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0117] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0118] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0120] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital versatile disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)). It should be noted that the computer-readable storage medium mentioned in the embodiments of the present application can be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0121] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the scene data of the current frame in the three-dimensional virtual scene, the device information of the client, and the scene interaction information involved in the embodiments of the present application are all obtained under sufficient authorization.

[0122] For the similar parts between the embodiments provided in this application, reference can be made to each other. The specific embodiments provided above are only several examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other embodiments extended based on the solution of this application without creative efforts fall within the protection scope of this application.

Claims

1. A method for optimizing image processing chip parameters, characterized in that: include: Extracting a first image feature of a sample reference image and a second image feature of a sample processed image, wherein the sample reference image is a noise-free image generated based on a Bayer image, and the sample processed image is an image generated after the image processing chip processes the Bayer image; Determining an image quality evaluation index based on a feature difference between the first image feature and the second image feature; Using a covariance adaptive evolution strategy, the image quality evaluation index is used as a training target to debug the parameters of the image processing chip to obtain optimized parameters; Based on the optimization parameters, the image processing chip is controlled to perform image processing.

2. The method according to claim 1, characterized in that The determining of the image quality evaluation index based on the feature difference between the first image feature and the second image feature includes: Marking a plurality of first interest points in the sample reference image, and marking a plurality of second interest points in the sample processed image, wherein the positions of the first interest points in the sample reference image and the positions of the second interest points in the sample processed image are the same, and both the first interest points and the second interest points are pixel points with unique features; Determining the number of matching interest points whose Euclidean distance between the first interest point and the second interest point is less than a preset threshold; The image quality evaluation index is determined based on the proportion of the number of the matching interest points in the interest points.

3. The method according to claim 2, characterized in that The determining the number of matching interest points whose Euclidean distance between the first interest point and the second interest point is less than a preset threshold comprises: The Euclidean distance between the first interest point and the second interest point is determined based on the distance between a first feature vector of pixels in the neighborhood of the first interest point and a second feature vector of pixels in the neighborhood of the second interest point.

4. The method according to claim 3, characterized in that The determining the Euclidean distance between the first interest point and the second interest point based on the distance between the first feature vector of the pixel points in the neighborhood of the first interest point and the second feature vector of the pixel points in the neighborhood of the second interest point comprises: Filtering out pixels whose structural similarity meets a preset degree from pixels in the neighborhood of the first point of interest and pixels in the neighborhood of the second point of interest, wherein the structural similarity includes at least one of brightness, contrast and structure; The Euclidean distance between the first interest point and the second interest point is determined based on the distance between the first feature vector and the second feature vector of the filtered pixel point.

5. The method according to any one of claims 1 to 4, characterized in that The method of using the covariance adaptive evolution strategy to debug the parameters of the image processing chip using the image quality evaluation index as a training target to obtain the optimized parameters includes: Selecting multiple sample technical parameters of the signal processing module within a value range corresponding to the technical parameters of the signal processing module in the image processing chip, wherein the signal processing module includes at least one of a brightness module and a clarity module, and the value range is determined based on the initial variance of the covariance adaptive evolution strategy; Based on the plurality of sample technical parameters, controlling the image processing chip to process the Bayer image to obtain a plurality of sample processed images; Performing feature comparison on the plurality of sample processed images and the sample reference image respectively to obtain a plurality of image processing evaluation indicators; According to the fit between the plurality of image processing evaluation indicators and the image quality evaluation indicator, the sample technical parameters are iteratively trained to obtain optimized parameters.

6. The method according to claim 5, characterized in that The iterative training of the sample technical parameters according to the fit between the plurality of image processing evaluation indicators and the image quality evaluation indicators to obtain the optimized parameters includes: constructing an optimization function according to the fit between the plurality of image processing evaluation indicators and the image quality evaluation indicator; Using the covariance adaptive evolution strategy, the optimization function is processed to obtain a minimum value of the optimization function; When the minimum value is less than or equal to a preset value, and / or the iterative training reaches a preset number of times, the corresponding sample technical parameters are selected from the plurality of sample processing images based on the minimum value as the optimization parameters.

7. The method according to claim 6, characterized in that Also includes: In each iterative training, a plurality of sample technical parameters are generated by adjusting weight coefficients corresponding to different technical parameters of the signal processing module, and the sum of the weight coefficients corresponding to the different technical parameters is 1.

8. An image processing chip parameter optimization device, characterized in that: include: an extraction unit, configured to extract a first image feature of a sample reference image, wherein the sample reference image is a noise-free image generated based on a Bayer image, and a second image feature of a sample processed image, wherein the sample reference image is generated by processing the Bayer image through the image processing chip; a determining unit, configured to determine an image quality evaluation index based on a feature difference between the first image feature and the second image feature, A debugging unit, used to use a covariance adaptive evolution strategy to debug the parameters of the image processing chip using the image quality evaluation index as a training target to obtain optimized parameters; A control unit is used to control the image processing chip to perform image processing based on the optimization parameters.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 7.

Images