ISP parameter determination method and device, electronic equipment and storage medium
By iteratively searching for the ISP parameter set in the search space and evaluating the RGB image quality based on the reference image, the problem of inefficient parameter optimization in device image processing is solved, and personalized image processing and dynamic adjustment of efficient image quality is realized.
Patent Information
- Application Number
- CN202410865757.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2025-07-25
AI Technical Summary
In the prior art, the device requires a large amount of sample data for parameter optimization during image processing, resulting in inefficiency and poor results.
By iteratively searching in the search space, the ISP parameter set is obtained, and the processed RGB image is evaluated based on the reference image, until the image quality requirements are met, the ISP parameters are dynamically adjusted.
Personalized image processing is realized according to the differences of terminal devices, improving image quality and processing efficiency, and meeting users' personalized needs for image quality.
Smart Images
Figure CN120372881A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technologies, and in particular, to a method, apparatus, electronic device, and storage medium for determining ISP parameters. Background Art
[0002] In recent years, with the development of artificial intelligence and deep learning technologies, the technology of local computing on devices has been increasingly widely used in image processing. For example, using a neural network model on a device for real-time image recognition, or using a local algorithm in a camera application for image enhancement and beautification processing, enables the device to process data and images more intelligently and efficiently, thus bringing a better experience to users. Summary of the Invention
[0003] The present disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for determining ISP parameters, so as to at least solve the problem that a large amount of sample data is required for testing and optimization in order to ensure that the device uses the best parameters. The technical solution of the present disclosure is as follows:
[0004] According to a first aspect of an embodiment of the present disclosure, a method for determining ISP parameters is provided, including: performing iterative search in a search space to obtain multiple ISP parameter sets for the i-th iteration, and sending the multiple ISP parameter sets to a terminal device, where i is a natural number greater than or equal to 1; receiving multiple RGB images obtained by processing a first original image based on the multiple ISP parameter sets sent by the terminal device, where R is red, G is green, and B is blue; performing quality evaluation on the multiple RGB images according to a reference image to obtain a target quality evaluation result; in response to the target quality evaluation result not meeting the image quality requirement, re-performing iterative search in the search space to obtain multiple ISP parameter sets for the (i + 1)-th iteration and performing the (i + 1)-th iteration until the iteration end condition is met to stop the iteration, obtaining a target ISP parameter set, and sending it to the terminal device.
[0005] According to a first aspect of an embodiment of the present disclosure, another method for determining ISP parameters is provided, including: receiving multiple ISP parameter sets for the i-th iteration sent by a server, where i is a natural number greater than or equal to 1; processing a first original image based on the multiple ISP parameter sets to obtain multiple RGB images, and sending the multiple RGB images to the server, where the multiple RGB images are used to determine a target quality evaluation result; receiving the target ISP parameter set sent by the server, where the target ISP parameter set is the ISP parameter set obtained by the server when the iteration ends.
[0006] According to a second aspect of the embodiments of the present disclosure, there is provided a device for determining ISP parameters, including: a first iteration module, configured to perform iterative search in a search space to obtain multiple ISP parameter sets for the i-th iteration, and send the multiple ISP parameter sets to a terminal device, where i is a natural number greater than or equal to 1; a receiving module, configured to receive multiple RGB images obtained by processing a first original image based on the multiple ISP parameter sets from the terminal device, where R is red, G is green, and B is blue; an evaluation module, configured to perform quality evaluation on the multiple RGB images according to a reference image to obtain a target quality evaluation result; a second iteration module, configured to, in response to the target quality evaluation result not meeting the image quality requirement, re-perform iterative search in the search space to obtain multiple ISP parameter sets for the (i + 1)-th iteration and perform the (i + 1)-th iteration until the iteration end condition is met to stop the iteration, obtain a target ISP parameter set, and send it to the terminal device.
[0007] According to a second aspect of the embodiments of the present disclosure, there is provided another device for determining ISP parameters, including: a first receiving module, configured to receive multiple ISP parameter sets for the i-th iteration sent by a server, where i is a natural number greater than or equal to 1; a processing module, configured to process a first original image based on the multiple ISP parameter sets to obtain multiple RGB images, and send the multiple RGB images to the server, where the multiple RGB images are used to determine a target quality evaluation result; a second receiving module, configured to receive the target ISP parameter set sent by the server, where the target ISP parameter set is the ISP parameter set obtained by the server when the iteration ends.
[0008] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to implement the steps of the method according to the first aspect of the embodiments of the present disclosure.
[0009] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the method according to the first aspect of the embodiments of the present disclosure are implemented.
[0010] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, characterized in that when the computer program is executed by a processor of an electronic device, the steps of the method according to the first aspect of the embodiments of the present disclosure are implemented.
[0011] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects: By iteratively searching for ISP parameters in the search space, an ISP parameter set is obtained, and the ISP parameter set is sent to the terminal device to obtain a plurality of RGB images sent by the terminal device. Furthermore, based on the reference image, the quality of the RGB images is evaluated to determine the target quality evaluation result of the RGB images, and it is judged whether the image quality requirements are met according to the target quality evaluation result. In the case where the image quality requirements are not met, iterative search is performed again until the iterative termination condition is met, and a target ISP parameter set is obtained. Further iterative optimization is performed according to the target quality evaluation result to achieve dynamic adjustment of the ISP parameters. By evaluating and optimizing the image quality based on the RGB images and the reference image to obtain a target ISP parameter set, the image effect of image processing using the target ISP parameter set can be improved.
[0012] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure and do not constitute an improper limitation of the present disclosure.
[0014] Figure 1 is a flowchart of a method for determining ISP parameters shown according to an exemplary embodiment.
[0015] Figure 2 is a flowchart of a method for determining ISP parameters shown according to another exemplary embodiment.
[0016] Figure 3 is a flowchart of a method for determining ISP parameters shown according to another exemplary embodiment.
[0017] Figure 4 is an interaction diagram of a method for determining ISP parameters shown according to an exemplary embodiment.
[0018] Figure 5 is a flowchart of obtaining a target ISP parameter set shown according to an exemplary embodiment.
[0019] Figure 6 is a block diagram of a device for determining ISP parameters shown according to an exemplary embodiment.
[0020] Figure 7 is a block diagram of a device for determining ISP parameters shown according to another exemplary embodiment.
[0021] Figure 8It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners
[0022] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0024] In the technical solutions of the present disclosure, the acquisition, storage, use, processing, etc. of data all comply with the provisions of relevant laws and regulations.
[0025] The application scenarios of the technical solutions of the present disclosure mainly focus on the field of determining the parameters of an Image Signal Processor (ISP), especially in the process of optimizing ISP parameters. The following are some specific application scenarios:
[0026] (1) Smartphone camera shooting optimization: Through the automatic debugging system for shooting parameters on the mobile phone side, the parameters can be adjusted according to the hardware characteristics of each mobile phone, thereby optimizing the shooting effect, improving the image quality, and ensuring that the photos taken by users on different mobile phones can achieve the best effect.
[0027] (2) Real-time image processing: Using the computing module on the mobile phone side, the original images taken can be processed in real time to improve the shooting experience. For example, during the shooting process, the images can be processed such as noise reduction, enhancement, color correction, etc. to obtain a clearer and more natural photo effect.
[0028] (3) Adaptive camera parameter adjustment: The system can dynamically adjust the camera parameters according to the feedback of the image quality evaluation module to adapt to different shooting scenarios and lighting conditions, thereby providing a more intelligent camera shooting experience.
[0029] (4) Personalized shooting experience: Through continuously optimized parameter adjustment and image quality evaluation, the system can provide a more personalized shooting experience according to the personalized needs and shooting habits of users, meeting the different needs of different users for image quality.
[0030] (5) Efficient image processing: The computing module on the mobile device side can quickly process the original image and batch output RGB images, thereby improving the efficiency and speed of image processing and ensuring that users can quickly obtain satisfactory photo effects.
[0031] These application scenarios can all be realized through the automatic debugging system of the shooting parameters on the mobile device side, thereby improving the user's shooting experience and meeting the image quality requirements of different users.
[0032] The following describes the method and device for determining ISP parameters according to the embodiments of the present disclosure with reference to the accompanying drawings.
[0033] Figure 1 is a flowchart of a method for determining ISP parameters shown according to an exemplary embodiment. As Figure 1 shown, the method for determining ISP parameters according to the embodiments of the present disclosure includes the following steps:
[0034] S101, perform iterative search in the search space to obtain multiple ISP parameter sets for the i-th iteration, and send the multiple ISP parameter sets to the terminal device, where i is a natural number greater than or equal to 1.
[0035] It should be noted that the execution subject of the method for determining ISP parameters according to the embodiments of the present disclosure is an electronic device, and the electronic device can be a server. Optionally, the server includes but is not limited to a web server, an application server, and can also be a server of a distributed system, or a server combined with a blockchain, etc. The method for determining ISP parameters according to the embodiments of the present disclosure can be executed by the device for determining ISP parameters according to the embodiments of the present disclosure, and the device for determining ISP parameters according to the embodiments of the present disclosure can be configured in any electronic device to execute the method for determining ISP parameters according to the embodiments of the present disclosure.
[0036] In some implementations, the server can pre-obtain the initial image signal processor (ISP) parameter set at the first iteration, and then perform iteration based on the initial parameter set to obtain multiple ISP parameter sets for the i-th iteration. Optionally, the set ISP parameter range can be obtained and the initial ISP parameter set can be based on this range.
[0037] Furthermore, send the multiple ISP parameter sets for the i-th iteration to the terminal device, and the terminal device processes the image based on the ISP parameter set. Optionally, data transmission can be performed according to the communication link between the server and the terminal device.
[0038] S102, receive multiple RGB images obtained by processing the first original image based on the multiple ISP parameter sets sent by the terminal device, where R is red, G is green, and B is blue.
[0039] In some implementations, the first original image is an image captured by a terminal device. The ISP parameter set can be used to process the first original image. After the server sends multiple ISP parameter sets to the terminal device, the terminal device processes the first original image based on the ISP parameter set, and then the server can receive multiple processed RGB images.
[0040] S103. Perform quality assessment on multiple RGB images according to a reference image to obtain a target quality assessment result.
[0041] In some implementations, the server can generate a high-quality reference image based on the first original image and perform quality assessment on the RGB images based on the reference image. Optionally, by determining the quality assessment value of each RGB image and the quality assessment value of the reference image, and then based on the quality assessment value, determine the target quality assessment result.
[0042] S104. In response to the target quality assessment result not meeting the image quality requirement, re-perform iterative search in the search space to obtain multiple ISP parameter sets at the (i + 1)-th iteration and perform the (i + 1)-th iteration until the iteration end condition is met and stop the iteration to obtain the target ISP parameter set, and send it to the terminal device.
[0043] In some implementations, it is possible to determine whether the target quality assessment result meets the image quality requirement by comparing the magnitude relationship between the target quality assessment result and a quality threshold. Optionally, it can be determined that the target quality assessment result meets the image quality requirement when the target quality assessment result is greater than or equal to a preset quality threshold. If the target quality assessment result is less than the quality threshold, it is determined that the target quality assessment result does not meet the image quality requirement.
[0044] Furthermore, based on the multiple ISP parameter sets at the i-th iteration, re-perform iterative search in the search space to obtain multiple ISP parameter sets at the (i + 1)-th iteration and perform the (i + 1)-th iteration until the iteration end condition is met and stop the iteration to obtain the target ISP parameter set, and send the target ISP parameter set to the terminal device.
[0045] Optionally, the iteration end condition can be that the number of iterations reaches a set value, or that the target quality assessment result meets the image quality requirement.
[0046] The method for determining ISP parameters provided by the embodiments of the present disclosure iteratively searches for ISP parameters in the search space to obtain an ISP parameter set, and sends the ISP parameter set to a terminal device to obtain a plurality of RGB images sent by the terminal device. Furthermore, based on a reference image, the quality of the RGB images is evaluated to determine the target quality evaluation result of the RGB images, and it is determined whether the image quality requirements are met according to the target quality evaluation result. In the case where the image quality requirements are not met, iterative search is performed again until the iterative termination condition is met, and a target ISP parameter set is obtained. Further iterative optimization is performed according to the target quality evaluation result to dynamically adjust the ISP parameters. By evaluating and optimizing the image quality based on the RGB images and the reference image to obtain a target ISP parameter set, the image effect of image processing using the target ISP parameter set can be improved.
[0047] Figure 2 is a flowchart of a method for determining ISP parameters shown according to an exemplary embodiment, as Figure 2 shown, the method for determining ISP parameters of the embodiments of the present disclosure includes the following steps:
[0048] S201, perform iterative search in the search space to obtain multiple ISP parameter sets for the i-th iteration, and send the multiple ISP parameter sets to the terminal device, where i is a natural number greater than or equal to 1.
[0049] S202, receive multiple RGB images obtained by processing a first raw image based on the multiple ISP parameter sets sent by the terminal device, where R is red, G is green, and B is blue.
[0050] For the relevant content of steps S201 - S202, reference can be made to the above embodiments, which will not be elaborated here.
[0051] S203, for each RGB image, obtain the similarity between the RGB image and the reference image, and determine the quality evaluation value of the RGB image based on the similarity.
[0052] Optionally, the reference image can be determined based on the first raw image. The first raw image sent by the terminal device can be received, and the reference image can be generated based on the first raw image.
[0053] In some implementations, the similarity between each RGB image and the reference image can be calculated respectively to obtain the similarity between the RGB image and the reference image. For each RGB image, the same similarity calculation method is used to obtain the similarity between the RGB image and the reference image. For example, a hash calculation method can be used to determine the similarity.
[0054] Further, the RGB images can be scored according to the similarity between the RGB images and the reference images to determine the quality evaluation value of each RGB image. For example, the RGB images can be quality-evaluated based on the similarity according to an image quality evaluation model.
[0055] S204. Obtain a target quality evaluation result according to the quality evaluation values of multiple RGB images.
[0056] Optionally, statistical calculations can be performed on the quality evaluation values of multiple RGB images to determine the target quality evaluation result. For example, the mean value of the quality evaluation values of multiple RGB images can be calculated, and the average value can be used as the target quality evaluation result. The minimum value in the quality evaluation values can also be determined and used as the target quality evaluation result.
[0057] S205. Determine whether the target quality evaluation result is greater than or equal to a set quality threshold.
[0058] In some implementations, the set quality threshold can be determined in advance, and based on the comparison between the set quality threshold and the target quality evaluation result, it can be determined whether the target quality evaluation result meets the image quality requirements. Optionally, the target quality evaluation result being greater than or equal to the set quality threshold can be regarded as meeting the image quality requirements; the target quality evaluation result being less than the set quality threshold can be regarded as not meeting the image quality requirements.
[0059] S206. In response to the target quality evaluation result being greater than or equal to the set quality threshold, determine that the target quality evaluation result meets the image quality requirements.
[0060] S207. In response to the target quality evaluation result being less than the set quality threshold, determine that the target quality evaluation result does not meet the image quality requirements.
[0061] In some implementations, by comparing the target quality evaluation result with the set quality threshold, it can be determined whether the target quality evaluation result meets the image quality requirements. That is, when the target quality evaluation result is greater than or equal to the set quality threshold, it can be determined that the target quality evaluation result meets the image quality requirements; when the target quality evaluation result is less than the set quality threshold, it can be determined that the target quality evaluation result does not meet the image quality requirements.
[0062] S208. Re-perform iterative search in the search space to obtain multiple ISP parameter sets at the (i + 1)-th iteration and perform the (i + 1)-th iteration until the iteration end condition is met and the iteration stops, obtain the target ISP parameter set, and send it to the terminal device.
[0063] For the relevant content of step S208, reference can be made to the above embodiments and will not be elaborated here.
[0064] The method for determining ISP parameters provided by the embodiments of the present disclosure iteratively searches for ISP parameters in a search space to obtain an ISP parameter set, and sends the ISP parameter set to a terminal device to obtain a plurality of RGB images sent by the terminal device. Furthermore, a reference image is determined based on a first original image, and the quality of the RGB images is evaluated to determine the target quality evaluation result of the RGB images, so as to realize personalized generation of reference images for different terminal devices. Further, it is judged whether the image quality requirement is met according to the target quality evaluation result, and if the image quality requirement is not met, iterative search is performed again until the iterative termination condition is met, and a target ISP parameter set is obtained. Further iterative optimization is performed according to the target quality evaluation result to realize dynamic adjustment of ISP parameters. According to the RGB images and the reference image, the image quality is evaluated and optimized to obtain a target ISP parameter set, which can improve the image effect of image processing using the target ISP parameter set and make the target ISP parameter set meet the personalized requirements of the terminal device.
[0065] Figure 3 is a flowchart of a method for determining ISP parameters shown according to an exemplary embodiment, as Figure 3 shown, the method for determining ISP parameters of the embodiments of the present disclosure includes the following steps:
[0066] S301, receive a plurality of ISP parameter sets for the i-th iteration sent by a server, where i is a natural number greater than or equal to 1.
[0067] It should be noted that the execution subject of the method for determining ISP parameters of the embodiments of the present disclosure is an electronic device, and the electronic device may be a terminal device. The method for determining ISP parameters of the embodiments of the present disclosure may be executed by the device for determining ISP parameters of the embodiments of the present disclosure, and the device for determining ISP parameters of the embodiments of the present disclosure may be configured in any electronic device to execute the method for determining ISP parameters of the embodiments of the present disclosure.
[0068] S302, process a first original image based on a plurality of ISP parameter sets to obtain a plurality of RGB images, and send the plurality of RGB images to the server, where the plurality of RGB images are used to determine a target quality evaluation result.
[0069] In some implementations, the terminal device receives a plurality of ISP parameter sets for the i-th iteration sent by the server based on a communication link with the server, processes a first original image based on the plurality of ISP parameter sets to obtain a plurality of RGB images, and then sends the plurality of RGB images to the server.
[0070] That is to say, the terminal device can convert the first original image into an RGB image based on the ISP parameter set. For example, if the first original image is in RAW format, the terminal device converts the RAW format image into an RGB format image based on the ISP parameter set to obtain multiple RGB images.
[0071] In some implementations, before receiving the multiple ISP parameter sets for the i-th iteration sent by the server, the terminal device can pre-collect the first original image and send the first original image to the server, where the first original image is used to generate a reference image.
[0072] Optionally, after collecting the first original image, the terminal device can cache the collected first original image and read the first original image from the cache after receiving the multiple ISP parameter sets, so as to quickly and accurately obtain the first original image.
[0073] S303. Receive the target ISP parameter set sent by the server, where the target ISP parameter set is the ISP parameter set obtained when the server finishes the iteration.
[0074] In some implementations, after sending the RGB image to the server, the terminal device can receive the target ISP parameter set returned by the server. The terminal device uses the target ISP parameter set to re-perform image acquisition, so that the quality of the image is improved. That is to say, the terminal device can process the second original image collected by the terminal device based on the target ISP parameter set and output a target RGB image, where the target RGB image's...
[0075] For the method for determining ISP parameters provided by the embodiments of the present disclosure, the terminal device receives an ISP parameter set, processes the first original image based on the ISP parameter set to obtain an RGB image, sends the RGB image to the server, the server determines the target ISP parameter set based on the RGB image, receives the target ISP parameter set sent by the server, and uses the target ISP parameter set for taking pictures, which can realize automatic adjustment of the parameters of the terminal device to adapt to the differences of different terminal devices and improve the image effect and image quality of image processing.
[0076] Figure 4 is an interaction diagram of a method for determining ISP parameters shown according to an exemplary embodiment. As Figure 4 shown, the method for determining ISP parameters in the embodiments of the present disclosure includes the following steps:
[0077] S401. The terminal device collects the first original image and sends the first original image to the server.
[0078] S402, the server iteratively searches in the search space to obtain multiple ISP parameter sets for the i-th iteration, and sends the multiple ISP parameter sets to the terminal device.
[0079] S403, the terminal device processes the first original image based on the multiple ISP parameter sets to obtain multiple RGB images, and sends the multiple RGB images to the server.
[0080] S404, the server generates a reference image based on the first original image.
[0081] S405, the server performs quality assessment on the multiple RGB images according to the reference image to obtain the target quality assessment result.
[0082] S406, the server determines that the target quality assessment result meets the image quality requirement, determines the target ISP parameter set, and sends it to the terminal device.
[0083] The method for determining ISP parameters provided by the embodiments of the present disclosure iteratively searches for ISP parameters in the search space, obtains the ISP parameter set, and sends the ISP parameter set to the terminal device to obtain multiple RGB images sent by the terminal device. Furthermore, based on the reference image, the quality of the RGB images is evaluated to determine the target quality assessment result of the RGB images, and it is judged whether the image quality requirement is met according to the target quality assessment result. If the image quality requirement is not met, iterative search is performed again until the iterative termination condition is met to obtain the target ISP parameter set. Further iterative optimization is performed according to the target quality assessment result to dynamically adjust the ISP parameters. According to the RGB images and the reference image, the image quality is evaluated and optimized to obtain the target ISP parameter set, which can improve the image effect of image processing using the target ISP parameter set.
[0084] As Figure 5 shown is the flowchart of obtaining the target ISP parameter set. Figure 5 Among them, the server includes an optimizer, a parameter sampler, a cloud large model, and an Image Quality Assessment (IQA) model. The terminal device is a smart phone, and the smart phone includes an image processor.
[0085] As Figure 5As shown, the parameter sampler of the server sends multiple ISP parameter sets of the i-th iteration, such as ISP set 1, ISP set 2, etc., to the image processor of the smartphone. The smartphone collects the first raw image and sends the first raw image to the cloud large model of the server to generate a reference image. At the same time, the first raw image is processed based on the ISP parameter set to obtain multiple RGB images. Then, based on the reference image, the IQA model is used to evaluate the quality of the RGB images, the IQA value of each RGB image is output, and the target quality evaluation result is determined according to the IQA value. The target quality evaluation result is transmitted to the optimizer to determine whether the image quality requirements are met. When the image quality requirements are met, the ISP parameter set of the i-th iteration is sent to the smartphone as the target ISP parameter set, and the smartphone uses the target ISP parameter set to take pictures to obtain the target RGB image.
[0086] Figure 6 is a block diagram of a device for determining ISP parameters shown according to an exemplary embodiment. Referring to Figure 6 In this regard, the device 600 for determining ISP parameters according to an embodiment of the present disclosure includes: a first iteration module 601, a receiving module 602, an evaluation module 603, and a second iteration module 604.
[0087] The first iteration module 601 is configured to perform iterative search in the search space to obtain multiple ISP parameter sets of the i-th iteration, and send the multiple ISP parameter sets to the terminal device, where i is a natural number greater than or equal to 1;
[0088] The receiving module 602 is configured to receive multiple RGB images obtained by processing the first raw image based on the multiple ISP parameter sets sent by the terminal device, where R is red, G is green, and B is blue;
[0089] The evaluation module 603 is configured to perform quality evaluation on the multiple RGB images according to the reference image to obtain a target quality evaluation result;
[0090] The second iteration module 604 is configured to, in response to the target quality evaluation result not meeting the image quality requirements, perform iterative search again in the search space to obtain multiple ISP parameter sets at the (i + 1)-th iteration and perform the (i + 1)-th iteration until the iteration end condition is met and the iteration stops, to obtain the target ISP parameter set, and send it to the terminal device.
[0091] In an embodiment of the present disclosure, the first iteration module 601 is further configured to: receive the first raw image sent by the terminal device, and generate the reference image based on the first raw image.
[0092] In one embodiment of the present disclosure, the evaluation module 603 is further configured to: determine whether the target quality evaluation result is greater than or equal to a set quality threshold; in response to the target quality evaluation result being greater than or equal to the set quality threshold, determine that the target quality evaluation result meets the image quality requirement; in response to the target quality evaluation result being less than the set quality threshold, determine that the target quality evaluation result does not meet the image quality requirement.
[0093] In one embodiment of the present disclosure, the evaluation module 603 is further configured to: for each of the RGB images, obtain the similarity between the RGB image and the reference image, and determine the quality evaluation value of the RGB image based on the similarity; obtain the target quality evaluation result according to the quality evaluation values of the multiple RGB images.
[0094] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.
[0095] The ISP parameter determination device provided by the embodiments of the present disclosure iteratively searches for ISP parameters in a search space to obtain an ISP parameter set, and sends the ISP parameter set to a terminal device to obtain multiple RGB images sent by the terminal device. Furthermore, the quality of the RGB images is evaluated based on a reference image to determine the target quality evaluation result of the RGB images, and it is determined whether the image quality requirement is met according to the target quality evaluation result. In the case where the image quality requirement is not met, iterative search is performed again until the iterative termination condition is met, and a target ISP parameter set is obtained. Further iterative optimization is performed according to the target quality evaluation result to dynamically adjust the ISP parameters. By evaluating and optimizing the image quality based on the RGB images and the reference image to obtain a target ISP parameter set, the image effect of image processing using the target ISP parameter set can be improved.
[0096] Figure 7 It is a block diagram of an ISP parameter determination device shown according to an exemplary embodiment. Referring to Figure 7 , the ISP parameter determination device 700 of the embodiments of the present disclosure includes: a first receiving module 701, a processing module 702, and a second receiving module 703.
[0097] The first receiving module 701 is configured to receive multiple ISP parameter sets of the i-th iteration sent by a server, where i is a natural number greater than or equal to 1;
[0098] A processing module 702, configured to process a first original image based on the multiple ISP parameter sets to obtain multiple RGB images, and send the multiple RGB images to the server, where the multiple RGB images are used to determine a target quality evaluation result;
[0099] A second receiving module 703, configured to receive a target ISP parameter set sent by the server, where the target ISP parameter set is the ISP parameter set obtained by the server at the end of iteration.
[0100] In an embodiment of the present disclosure, the first receiving module 701 is further configured to: collect the first original image and send the first original image to the server, where the first original image is used to generate a reference image.
[0101] In an embodiment of the present disclosure, the first receiving module 701 is further configured to: cache the collected first original image, and read the first original image from the cache after receiving the multiple ISP parameter sets.
[0102] In an embodiment of the present disclosure, the second receiving module 703 is further configured to: process a second original image collected by the terminal device based on the target ISP parameter set and output a target RGB image.
[0103] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.
[0104] The device for determining ISP parameters provided by the embodiments of the present disclosure enables a terminal device to receive an ISP parameter set, process a first original image based on the ISP parameter set to obtain an RGB image, send the RGB image to a server, have the server determine a target ISP parameter set based on the RGB image, receive the target ISP parameter set sent by the server, and use the target ISP parameter set for photographing, so as to achieve automatic adjustment of the parameters of the terminal device to adapt to the differences of different terminal devices and improve the image effect and image quality of image processing.
[0105] Figure 8 It is a block diagram of an electronic device shown according to an exemplary embodiment.
[0106] As Figure 8 shown, the above electronic device 800 includes:
[0107] A memory 801 and a processor 802, a bus 803 connecting different components (including the memory 801 and the processor 802), the memory 801 stores a computer program, and when the processor 802 executes the program, the method for determining ISP parameters described in the embodiments of the present disclosure is implemented.
[0108] The bus 803 represents one or more of several types of bus architectures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus architecture in a variety of bus structures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
[0109] The electronic device 800 typically includes a variety of electronic device-readable media. These media can be any available media that can be accessed by the electronic device 800, including volatile and non-volatile media, removable and non-removable media.
[0110] The memory 801 may further include a computer system-readable medium in the form of volatile memory, such as random access memory (RAM) 804 and / or cache memory 805. The electronic device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 806 may be used for reading and writing non-removable, non-volatile magnetic media ( Figure 8 not shown, commonly referred to as a "hard disk drive"). Although Figure 8 not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 803 through one or more data media interfaces. The memory 801 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present disclosure.
[0111] A program / utility 808 having a set (at least one) of program modules 807 may be stored, for example, in the memory 801. Such program modules 807 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules 807 generally perform the functions and / or methods in the embodiments described in the present disclosure.
[0112] The electronic device 800 can also communicate with one or more external devices 809 (such as a keyboard, a pointing device, a display 891, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 892. Moreover, the electronic device 800 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 893. As Figure 8 shown, the network adapter 893 communicates with other modules of the electronic device 800 through the bus 803. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0113] The processor 802 executes various functional applications and data processing by running the programs stored in the memory 801.
[0114] It should be noted that for the implementation process and technical principle of the electronic device in this embodiment, refer to the foregoing explanation of the method for determining ISP parameters in the embodiments of the present disclosure, and details are not described herein again.
[0115] To implement the above embodiments, the present disclosure also proposes a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the method for determining ISP parameters provided by the present disclosure are implemented.
[0116] Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0117] To implement the above embodiments, the present disclosure also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor of an electronic device, the method for determining ISP parameters as described above is implemented.
[0118] Those skilled in the art will readily think of other implementation schemes of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include well-known knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.
[0119] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.
Claims
1. A method for determining ISP parameters of an image signal processor, characterized in that, Executed by a server, the method includes: Performing iterative search in a search space to obtain multiple ISP parameter sets for the i-th iteration, and sending the multiple ISP parameter sets to a terminal device, where i is a natural number greater than or equal to 1; Receiving multiple RGB images obtained by processing a first raw image based on the multiple ISP parameter sets from the terminal device, where R is red, G is green, and B is blue; Performing quality assessment on the multiple RGB images according to a reference image to obtain a target quality assessment result; In response to the target quality assessment result not meeting the image quality requirement, re-performing iterative search in the search space to obtain multiple ISP parameter sets for the (i + 1)-th iteration and performing the (i + 1)-th iteration until the iteration end condition is met to stop the iteration, obtaining a target ISP parameter set, and sending it to the terminal device.
2. The method according to claim 1, characterized in that, Before performing iterative search in the search space to obtain multiple ISP parameter sets for the i-th iteration, it further includes: Receiving the first raw image sent by the terminal device and generating the reference image based on the first raw image.
3. The method according to claim 1 or 2, characterized in that The method further includes: Judging whether the target quality assessment result is greater than or equal to a set quality threshold; In response to the target quality assessment result being greater than or equal to the set quality threshold, determining that the target quality assessment result meets the image quality requirement; In response to the target quality assessment result being less than the set quality threshold, determining that the target quality assessment result does not meet the image quality requirement.
4. The method according to claim 3, characterized in that, The performing quality assessment on the multiple RGB images according to the reference image to obtain a target quality assessment result includes: For each of the RGB images, obtaining the similarity between the RGB image and the reference image, and determining the quality assessment value of the RGB image based on the similarity; Obtaining the target quality assessment result according to the quality assessment values of the multiple RGB images.
5. A method for determining ISP parameters, characterized in that, Executed by a terminal device, the method includes: Receiving multiple ISP parameter sets for the i-th iteration sent by the server, where i is a natural number greater than or equal to 1; Processing the first raw image based on the multiple ISP parameter sets to obtain multiple RGB images, and sending the multiple RGB images to the server, where the multiple RGB images are used to determine the target quality assessment result; Receiving the target ISP parameter set sent by the server, where the target ISP parameter set is the ISP parameter set obtained by the server when the iteration ends.
6. The method according to claim 5, wherein Before receiving the multiple ISP parameter sets for the i-th iteration sent by the server, it further includes: Collecting the first raw image and sending the first raw image to the server, where the first raw image is used to generate the reference image.
7. The method according to claim 6, wherein The method further includes: Caching the collected first raw image, and reading the first raw image from the cache after receiving the multiple ISP parameter sets.
8. The method according to claim 5, wherein After receiving the target ISP parameter set sent by the server, it further includes: Process the second original image collected by the terminal device based on the set of target ISP parameters, and output a target RGB image.
9. An apparatus for determining ISP parameters of an image signal processor, characterized in that, Applicable to a server, the device includes: A first iteration module, configured to perform iterative search in a search space to obtain multiple sets of ISP parameters for the i-th iteration, and send the multiple sets of ISP parameters to the terminal device, where i is a natural number greater than or equal to 1; A receiving module, configured to receive multiple RGB images obtained by processing the first original image based on the multiple sets of ISP parameters sent by the terminal device, where R is red, G is green, and B is blue; An evaluation module, configured to perform quality evaluation on the multiple RGB images according to a reference image to obtain a target quality evaluation result; A second iteration module, configured to, in response to the target quality evaluation result not meeting the image quality requirement, re-perform iterative search in the search space to obtain multiple sets of ISP parameters for the (i + 1)-th iteration and perform the (i + 1)-th iteration until the iteration end condition is met to stop the iteration, obtain a set of target ISP parameters, and send them to the terminal device.
10. An apparatus for determining ISP parameters, characterized in that Applicable to a terminal device, the device includes: A first receiving module, configured to receive multiple sets of ISP parameters for the i-th iteration sent by the server, where i is a natural number greater than or equal to 1; A processing module, configured to process the first original image based on the multiple sets of ISP parameters to obtain multiple RGB images, and send the multiple RGB images to the server, where the multiple RGB images are used to determine a target quality evaluation result; A second receiving module, configured to receive the set of target ISP parameters sent by the server, where the set of target ISP parameters is the set of ISP parameters obtained by the server when the iteration ends.
11. An electronic device, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to: Implement the steps of the method according to any one of claims 1-8.
12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, the steps of the method according to any one of claims 1-8 are implemented.