Simulation image acquisition method and device, electronic equipment and computer readable storage medium

Abstract

The application discloses an emulated image acquisition method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring an initial image and an output image, the initial image being an image collected by a camera, and the output image being an image obtained after image signal processing (ISP) is performed on the initial image, the output image being used for display on a display screen, and the output image being a nonlinear image; performing linear transformation on the output image to obtain a first linear image; converting the first linear image to a preset color space to obtain a first image; converting the initial image to the color space to obtain a second image; performing color fitting based on the first image and the second image to obtain a color fitting matrix; and performing simulation on a third image pre-acquired through the color fitting matrix to obtain an emulated image. Through the above steps, the accuracy of the emulated image can be improved, and the emulated image is closer to a real image obtained through a sensor.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a simulation image acquisition method and device, electronic equipment and a computer readable storage medium. BACKGROUND

[0002] Image Signal Process (ISP) is a unit mainly used for processing image signals output by a front-end image sensor, so as to match image sensors of different manufacturers. The ISP image processing unit contains multiple image processing modules, wherein a Color Correction Matrix (CCM) can convert RGB values captured by a camera sensor end to a unified color expression, and the CCM ensures that different cameras can display the same RGB values on a display screen when acquiring the same RGB values, thereby ensuring display effects.

[0003] In a controller hardware-in-the-loop (HIL) test, color information collected by a simulation image sensor needs to be simulated. A picture rendered by simulation software is an ideal image, which is similar to an image after ISP. The image rendered by the simulation software has no bad points and no white balance imbalance. The simulation software creates defects through post-processing, and simulates an image obtained by a sensor by inversely processing the ideal image.

[0004] Currently, when simulation software is used to simulate an image obtained by a sensor, the accuracy of the simulation image is low. SUMMARY

[0005] Embodiments of the present application provide a simulation image acquisition method and device, electronic equipment and a computer readable storage medium, which can improve the accuracy of the simulation image and make the simulation image closer to a real image obtained by a sensor.

[0006] In a first aspect, embodiments of the present application provide a simulation image acquisition method, comprising:

[0007] An initial image and an output image are acquired, the initial image is an image collected by a camera, the output image is an image obtained after image signal processing (ISP) is performed on the initial image, the output image is used for display on a display screen, and the output image is a nonlinear image;

[0008] A linear transformation is performed on the output image to obtain a first linear image;

[0009] The first linear image is converted to a preset color space to obtain a first image;

[0010] convert the initial image to the color space to obtain a second image;

[0011] perform color fitting based on the first image and the second image to obtain a color fitting matrix;

[0012] simulate a third image pre-acquired through the color fitting matrix to obtain a simulation image.

[0013] In a second aspect, an embodiment of the present application provides an apparatus for obtaining a simulation image, comprising:

[0014] a first obtaining module, configured to obtain an initial image and an output image, the initial image being an image collected through a camera, the output image being an image obtained after image signal processing (ISP) is performed on the initial image, the output image being used for display on a display screen, and the output image being a nonlinear image;

[0015] a second obtaining module, configured to perform linear transformation on the output image to obtain a first linear image;

[0016] a first conversion module, configured to convert the first linear image to a preset color space to obtain a first image;

[0017] a second conversion module, configured to convert the initial image to the color space to obtain a second image;

[0018] a color fitting module, configured to perform color fitting based on the first image and the second image to obtain a color fitting matrix;

[0019] a simulation module, configured to simulate a third image pre-acquired through the color fitting matrix to obtain a simulation image.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory having computer program instructions stored therein;

[0021] the processor, when executing the computer program instructions, implements the method in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, having computer program instructions stored thereon, and the computer program instructions, when executed by a processor, implement the method in the first aspect.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, and instructions in the computer program product, when executed by a processor of an electronic device, cause the electronic device to perform the method in the first aspect.

[0024] The simulation image acquisition method, device, electronic equipment and computer readable storage medium provided by the embodiment of the present application, wherein the method comprises: acquiring an initial image and an output image, the initial image being an image collected by a camera, the output image being an image obtained after image signal processing (ISP) is performed on the initial image, the output image being used for display on a display screen, and the output image being a nonlinear image; performing linear transformation on the output image to obtain a first linear image; converting the first linear image to a preset color space to obtain a first image; converting the initial image to the color space to obtain a second image; performing color fitting based on the first image and the second image to obtain a color fitting matrix; and performing simulation on a pre-acquired third image through the color fitting matrix to obtain a simulation image. In the above steps, the nonlinearity in the output image can be eliminated through linear transformation, which can effectively improve the effect of linear fitting, improve the determination of the color fitting matrix obtained through subsequent calculation, so that the accuracy of the simulation image can be improved when the image is simulated according to the color fitting matrix, and the simulation image is closer to the real image obtained through the sensor. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced. Those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0026] Figure 1 is a flowchart of a simulation image acquisition method provided by an embodiment of the present application;

[0027] Figure 2 is a schematic diagram of each processing module of ISP provided by an embodiment of the present application;

[0028] Figure 3 is a structural schematic diagram of a simulation image acquisition device provided by an embodiment of the present application;

[0029] Figure 4 is a structural schematic diagram of an electronic equipment provided by another embodiment of the present application. DETAILED DESCRIPTION

[0030] The features and exemplary embodiments of the various aspects of the present application will be described in detail below with reference to the drawings. For the purpose of clarity, the description is divided into the following sections: technical scheme, advantages, and specific embodiments. The technical scheme section describes the technical solutions of the present application. The advantages section describes the advantages of the present application. The specific embodiments section describes specific embodiments of the present application. The purpose of the above sections is to provide a better understanding of the present application. The specific embodiments described below are merely intended to explain the present application, and not to limit the present application. The present application can be implemented without some of the specific details described below. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.

[0031] It should be noted that, in this document, relational terms such as first and second, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprises... " does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0032] To solve the problems in the prior art, the embodiments of the present application provide a simulation image acquisition method, device, electronic equipment, medium and product. First, the simulation image acquisition method provided by the embodiments of the present application will be introduced.

[0033] Figure 1 The flowchart of the simulation image acquisition method provided by an embodiment of the present application is shown. As shown in Figure 1 The simulation image acquisition method provided by the embodiments of the present application includes the following steps 101-106, wherein:

[0034] Step 101, obtaining an initial image and an output image, the initial image being an image collected by a camera, the output image being an image obtained after image signal processing (ISP) on the initial image, the output image being used for display on a display screen, and the output image being a nonlinear image.

[0035] The camera records the image information of the photographed object in the form of electrical signals, and the photosensitive curves of the sensor ends of different cameras are different. Due to the different photosensitive curves, when different cameras are used to photograph the same scene, images with different pixel values will be obtained, and different colors will be presented on the display screen. The CCM in the ISP can correct the RGB deviation caused by the different photosensitive curves to the same RGB value, so as to ensure that the RGB values of the pixel points of the image displayed on the display are the same.

[0036] Generally, after the initial image is collected by the camera, the ISP process of the initial image is roughly as follows: high dynamic range imaging (HDR), color correction matrix (CCM) correction, gamma correction, etc., to obtain an output image, and the output image is displayed on the display screen. The CCM can convert the RGB values captured by the camera sensor end to a unified color expression, and the CCM ensures that different cameras can display the same RGB value on the display screen when obtaining the same RGB value, thereby ensuring the display effect. Gamma correction is a non-linear correction process, and through gamma correction, the dark area of the image can be strengthened to improve the dynamic range and dark area details of the picture and improve the image display effect. The image after gamma correction is a non-linear image.

[0037] Step 102, performing linear transformation on the output image to obtain a first linear image.

[0038] The linear transformation this time corresponds to the non-linear transformation used when the output image is obtained. For example, if the output image is a non-linear image obtained through gamma correction, then performing inverse gamma transformation on the output image to obtain the first linear image. If the output image is a non-linear image obtained through other processing, then performing inverse transformation corresponding to other processing on the output image to obtain the first linear image. Through linear transformation, the non-linearity in the output image can be eliminated, which can effectively improve the effect of linear fitting and improve the calculation accuracy.

[0039] Step 103, converting the first linear image to a preset color space to obtain a first image.

[0040] The color space can be Lab color space, which is composed of three elements of brightness (L) and related color a, b. L represents luminosity, a represents the range from magenta to green, and b represents the range from yellow to blue. The distance in Lab color space is closer to the color difference perceived by the human eye. After eliminating the gamma nonlinearity of the output image, the obtained first linear image can be converted to Lab color space and then color fitted. Converting the color space domain of the first linear image to Lab color space can make the subsequent solved color fitting matrix more accurate.

[0041] It should be noted that if the output image is an RGB image, the first linear image obtained by linearly transforming the output image is an image in XYZ color space. In this step, if the color space is Lab color space, the conversion of the first linear image to Lab color space can use the conversion method in the prior art.

[0042] Step 104, converting the initial image to the color space to obtain a second image.

[0043] The initial image is also converted to the color space, which facilitates subsequent color fitting with the first image.

[0044] It should be noted that if the initial image is an RGB image and the color space is Lab color space, the initial image cannot be directly converted to Lab color space. It needs to be first converted to XYZ color space and then from XYZ color space to Lab color space. The conversion of the initial image to XYZ color space and from XYZ color space to Lab color space can use the method in the prior art, which is not limited here.

[0045] Step 105, color fitting based on the first image and the second image to obtain a color fitting matrix.

[0046] This step can specifically include:

[0047] According to the pixel value of the first image, a first matrix is constructed, and the value of each element in the first matrix is determined according to the pixel value of the first image;

[0048] According to the pixel value of the second image, a second matrix is constructed, and the value of each element in the second matrix is determined according to the pixel value of the second image;

[0049] Based on the first matrix and the second matrix, a target function is constructed;

[0050] By solving the target function, the color fitting matrix is obtained.

[0051] Specifically, when color fitting is performed on the first image and the second image, a matrix corresponding to each of the first image and the second image can be obtained first, each component of a pixel value in the first image can correspond to an element in the first matrix, for example, if the pixel value of the first image is composed of three components of R, G and B, the first matrix can include three rows corresponding to the R, G and B components of each pixel point, and the column of the first matrix is the number of pixel values of the first image.

[0052] That is, the first matrix and the second matrix each include m rows and n columns, where m is the number of components of a pixel value of the first image or the second image, n is the number of pixel points of the first image or the second image, and m and n are positive integers. The first matrix and the second matrix are constructed in the same way.

[0053] The target function is a target function with a constraint condition, and the constraint condition is that the color fitting matrix is an m-row and m-column matrix, and the sum of elements in each row of the color fitting matrix is equal, or the sum of elements in each column is equal, and m is the number of rows of the first matrix or the second matrix. By solving the target function, the color fitting matrix can be obtained, which is an m-row and m-column matrix. Setting the constraint condition for the target function can ensure that the obtained color fitting matrix does not damage the white balance effect.

[0054] In step 106, the third image pre-acquired is simulated by the color fitting matrix to obtain a simulated image.

[0055] In the controller hardware-in-the-loop (HIL) test, the color information collected by the simulation sensor is needed, and the picture rendered by the simulation software is an ideal image, which is similar to the image after ISP. The image rendered by the simulation software has no bad points and no white balance imbalance. The simulation software creates defects through post-processing, and the ideal image obtained by simulation is inversely processed to achieve the conversion of the ideal image rendered by the simulation software to the image collected by the sensor. The third image can be an ideal image rendered by the simulation software, or an image after ISP.

[0056] Optionally, the above step 106 can specifically include the following steps:

[0057] Obtaining a third image, the third image being an image after ISP;

[0058] Performing linear transformation on the third image to obtain a second linear image;

[0059] Converting the second linear image to the color space to obtain a third matrix;

[0060] The third matrix is operated with the color fitting matrix to obtain a target matrix;

[0061] According to the target matrix, a simulation image is obtained, and pixel values of the simulation image are determined according to elements in the target matrix.

[0062] Specifically, the linear transformation of the third image is the same as the linear transformation used for the output image. The third image is linearly transformed to obtain a second linear image, and the second linear image is converted to a color space to obtain a fourth image. According to pixel values of the fourth image, a third matrix is constructed. The third matrix is constructed in a manner similar to the manner in which the first matrix is constructed. The value of each element in the third matrix is determined according to the pixel values of the fourth image. The third matrix, the first matrix, and the second matrix have the same number of rows and columns.

[0063] For example, the color fitting matrix The third matrix

[0064]

[0065] The target matrix P2 is obtained by the following expression:

[0066]

[0067] In the above formula, M 11 +M 12 +M 13 =M 21 +M 22 +M 23 =M 31 +M 32 +M 33 .

[0068] In the embodiments of the present application, an initial image and an output image are obtained. The initial image is an image collected by a camera, and the output image is an image obtained after image signal processing (ISP) is performed on the initial image. The output image is used for display on a display screen, and is a nonlinear image. Linear transformation is performed on the output image to obtain a first linear image. The first linear image is converted to a preset color space to obtain a first image. The initial image is converted to the color space to obtain a second image. Color fitting is performed based on the first image and the second image to obtain a color fitting matrix. A third image is simulated through the color fitting matrix to obtain a simulation image. In the above steps, the nonlinearity in the output image can be eliminated through linear transformation, which can effectively improve the effect of linear fitting, improve the determination of the color fitting matrix obtained through subsequent calculation, and make the simulation image more accurate and closer to the real image obtained through a sensor when the image is simulated according to the color fitting matrix.

[0069] The simulation image acquisition method provided in the application is used to simulate the image before ISP according to the image after ISP. As shown in the following formula, the ISP will sequentially correct the image acquired by the front camera through the HDR module, the Auto White Balance (AWB) module, the CCM, the Gamma module and other modules, and then output the corrected image. Figure 2

[0070] The embodiment of the application takes color as a fitting variable, considers the two modules Gamma and CCM in the ISP which affect the color change of the image, and fits a mathematical model reflecting the color relationship of the image before and after ISP by mathematical means. The mathematical model is used to replace the inverse process of the ISP. When the image is input into the fitted mathematical model, the image processed by the mathematical model is equivalent to the color inverse processing of the ISP, and the output image can be considered as an image without color correction, thereby realizing the simulation of the ISP input data.

[0071] The controller has the function of deriving the image before and after the controller. The image before the controller is the image acquired by the camera, which belongs to SensorRGB and is the RGB value related to the device, and will change with the change of the device. The image after the controller is the image processed by the ISP, which belongs to sRGB and is the RGB value independent of the device. The CCM module converts the SensorRGB related to the device into the linear sRGB independent of the device, and the sRGB independent of the device becomes the nonlinear sRGB independent of the device after the Gamma transformation.

[0072] The Gamma transformation will compress the linear color value into a nonlinear value, and the inverse transformation of Gamma will restore it to a linear value. In the application, the inverse transformation of Gamma is designed to convert the nonlinear sRGB into linear sRGB.

[0073] The difference between RGB and RGB is not close to the color difference perceived by the human eye. The distance in the Lab color space is more close to the color difference perceived by the human eye. At the same time, the CCM module is often after the AWB module in the ISP, and the white balance condition of the image needs to be ensured not to be damaged during color fitting. For the images before and after the controller which have eliminated the nonlinear relationship, the color space of the image is converted from RGB to Lab color space, and the mathematical model is designed as an optimization equation with a constraint condition to ensure that the color fitting matrix solved satisfies M 11 +M 12 +M 13 =M 21 +M 22 +M 23 =M​31 +M 32 +M 33 At the same time, a loss function is set in the equation solving and optimization process to determine whether to stop iteration.

[0074] The method provided by the application can solve and implement the inverse processing simulation of ISP images when color fitting matrix parameter values cannot be provided. The image obtained by the controller is subjected to inverse Gamma transformation to eliminate nonlinearity, which effectively improves the effect of linear fitting and improves the calculation accuracy; the color space domain of the image is converted to the Lab color space, so that the color fitting matrix can be solved with more accurate accuracy, and the added restriction condition can ensure that the obtained color fitting matrix does not damage the white balance effect.

[0075] Figure 3 The structure diagram of the simulation image acquisition device provided by the embodiment of the application is shown. As shown in Figure 3 The simulation image acquisition device 300 includes:

[0076] The first acquisition module 301 is configured to acquire an initial image and an output image, the initial image being an image collected by a camera, and the output image being an image obtained by performing image signal processing (ISP) on the initial image, the output image being used for display on a display screen, and the output image being a nonlinear image.

[0077] The second acquisition module 302 is configured to perform linear transformation on the output image to obtain a first linear image.

[0078] The first conversion module 303 is configured to convert the first linear image to a preset color space to obtain a first image.

[0079] The second conversion module 304 is configured to convert the initial image to the color space to obtain a second image.

[0080] The color fitting module 305 is configured to perform color fitting based on the first image and the second image to obtain a color fitting matrix.

[0081] The simulation module 306 is configured to simulate a third image pre-acquired through the color fitting matrix to obtain a simulation image.

[0082] Optionally, the color fitting module 305 includes:

[0083] The first construction submodule is configured to construct a first matrix according to the pixel value of the first image, the value of each element in the first matrix being determined according to the pixel value of the first image.

[0084] a second constructing sub-module, configured to construct a second matrix according to pixel values of the second image, wherein a value of each element in the second matrix is determined according to the pixel values of the second image;

[0085] a third constructing sub-module, configured to construct a target function based on the first matrix and the second matrix;

[0086] a first obtaining sub-module, configured to obtain the color fitting matrix by solving the target function.

[0087] Optionally, the target function is a target function with a constraint condition, and the constraint condition is that a sum of elements in each row of the color fitting matrix is equal or a sum of elements in each column is equal.

[0088] Optionally, the first matrix and the second matrix each include m rows and n columns, wherein m is a number of components of pixel values of the first image or the second image, and n is a number of pixel points of the first image or the second image.

[0089] Optionally, the simulation module 306 includes:

[0090] a second obtaining sub-module, configured to obtain a third image, wherein the third image is an image after ISP;

[0091] a third obtaining sub-module, configured to perform linear transformation on the third image to obtain a second linear image;

[0092] a converting sub-module, configured to convert the second linear image to the color space to obtain a third matrix;

[0093] a fourth obtaining sub-module, configured to perform operation on the third matrix and the color fitting matrix to obtain a target matrix;

[0094] a simulation sub-module, configured to obtain the simulation image according to the target matrix, wherein a pixel value of the simulation image is determined according to an element in the target matrix.

[0095] Optionally, the second obtaining module 302 is configured to perform inverse gamma transformation on the output image to obtain the first linear image.

[0096] Optionally, the color space is a Lab color space.

[0097] The simulation image obtaining apparatus 300 provided by the embodiment of the present application can realize each process of the simulation image obtaining method provided by the embodiment of the present application and achieve the same technical effects. To avoid repetition, details are not described herein.

[0098] Figure 4 A hardware structure schematic diagram of the simulation image obtaining method provided by the embodiment of the present application is shown.

[0099] The electronic device can include the processor 601 and the memory 602 having stored computer program instructions.

[0100] In particular, the processor 601 described above can include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or can be configured to implement one or more integrated circuits that embody the embodiments of the present application.

[0101] The memory 602 can include a mass storage that is used for data or instructions. By way of example, and not limitation, the memory 602 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. The memory 602 can include removable or non-removable (or fixed) media, where appropriate. The memory 602 can be internal or external to the integrated gateway disaster recovery device, as appropriate. In particular embodiments, the memory 602 is non-volatile, solid-state memory.

[0102] The memory can include read-only memory (ROM), random-access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (by one or more processors), is operable to perform operations described with reference to the methods according to the first aspect of the present disclosure.

[0103] The processor 601 implements the simulation image acquisition method in any of the embodiments described above by reading and executing the computer program instructions stored in the memory 602.

[0104] In one example, the electronic device can further include a communication interface 603 and a bus 610. As shown, the processor 601, the memory 602, and the communication interface 603 are connected through the bus 610 and complete communication with each other. Figure 4

[0105] The communication interface 603 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0106] ​Bus 610 includes hardware, software, or both, to couple components of the simulation image acquisition method to each other and to couple components to other components within the simulation image acquisition method. While in one embodiment bus 610 is illustrated as a single bus, alternative embodiments include any communication coupling(s) between components of the simulation image acquisition method. For example, while in one embodiment bus 610 is illustrated as a single bus, alternative embodiments include any communication coupling(s) between components of the simulation image acquisition method. In one embodiment, bus 610 includes an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 610 can include one or more buses of the same type or buses of different types. Although the simulation image acquisition method is described and shown in one embodiment as being a single bus, the simulation image acquisition method can include any suitable buses or interconnects.

[0107] In addition, in combination with the simulation image acquisition method in the above embodiments, the embodiments of the present application can provide a computer readable storage medium to implement. The computer readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by a processor to implement any one of the simulation image acquisition methods in the above embodiments.

[0108] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0109] The functional blocks shown in the above structural block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. The "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via a computer network such as the Internet, an intranet, etc.

[0110] It is also need to be explained that the example embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, can be different from the order in the embodiments, or several steps are performed simultaneously.

[0111] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0112] The above describes only the specific implementation of the present application, and those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, module and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described herein. It should be understood that the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements shall be covered within the protection scope of the present application.

Claims

1. A method for acquiring simulated images, characterized in that, The method includes: Acquire an initial image and an output image. The initial image is an image captured by a camera, and the output image is an image obtained after performing image signal processing (ISP) on the initial image. The output image is used to display on a display screen and is a non-linear image. A linear transformation is performed on the output image to obtain a first linear image; the linear transformation corresponds to the nonlinear transformation used when obtaining the output image. The first linear image is converted to a preset color space to obtain the first image; The initial image is converted to the color space to obtain the second image; A color fitting matrix is ​​obtained by performing color fitting based on the first image and the second image; The pre-acquired third image is simulated using the color fitting matrix to obtain a simulated image; the step of performing color fitting based on the first image and the second image to obtain a color fitting matrix includes: constructing a first matrix based on the pixel values ​​of the first image, wherein the value of each element in the first matrix is ​​determined based on the pixel values ​​of the first image; A second matrix is ​​constructed based on the pixel values ​​of the second image, wherein the value of each element in the second matrix is ​​determined based on the pixel values ​​of the second image; Based on the first matrix and the second matrix, construct the objective function; The color fitting matrix is ​​obtained by solving the objective function.

2. The method according to claim 1, characterized in that, The objective function is an objective function with constraints; The limiting condition is that the sum of the elements in each row of the color fitting matrix is ​​equal, or the sum of the elements in each column is equal.

3. The method according to claim 1 or 2, characterized in that, Both the first matrix and the second matrix comprise m rows and n columns, where m is the number of pixel components that make up the first image or the second image, and n is the number of pixels in the first image or the second image.

4. The method according to claim 1, characterized in that, The step of simulating the pre-acquired third image using the color fitting matrix to obtain a simulated image includes: Acquire a third image, which is the image after ISP processing; A second linear image is obtained by performing a linear transformation on the third image; The second linear image is converted to the color space to obtain the third matrix; The target matrix is ​​obtained by performing operations on the third matrix and the color fitting matrix; The simulated image is obtained based on the target matrix, and the pixel values ​​of the simulated image are determined based on the elements in the target matrix.

5. The method according to claim 1, characterized in that, The step of performing a linear transformation on the output image to obtain a first linear image includes: Perform an inverse gamma transform on the output image to obtain the first linear image.

6. The method according to claim 1, characterized in that, The color space is the Lab color space.

7. A simulated image acquisition device, characterized in that, The device includes: The first acquisition module is used to acquire an initial image and an output image. The initial image is an image captured by a camera, and the output image is an image obtained after performing image signal processing (ISP) on the initial image. The output image is used to display on a display screen and is a non-linear image. The second acquisition module is used to perform a linear transformation on the output image to obtain a first linear image; The first conversion module is used to convert the first linear image to a preset color space to obtain the first image; The second conversion module is used to convert the initial image to the color space to obtain the second image; A color fitting module is used to perform color fitting based on the first image and the second image to obtain a color fitting matrix; A simulation module is used to simulate the pre-acquired third image using the color fitting matrix to obtain a simulated image; the color fitting module includes: The first construction submodule is used to construct a first matrix based on the pixel values ​​of the first image, wherein the value of each element in the first matrix is ​​determined based on the pixel values ​​of the first image; The second construction submodule is used to construct a second matrix based on the pixel values ​​of the second image, wherein the value of each element in the second matrix is ​​determined based on the pixel values ​​of the second image; The third construction submodule is used to construct the objective function based on the first matrix and the second matrix; The first acquisition submodule is used to obtain the color fitting matrix by solving the objective function.

8. An electronic device, characterized in that, The device includes a processor and a memory storing computer program instructions, wherein the processor, when executing the computer program instructions, implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

Images