Color conversion method, device, electronic device and storage medium

Abstract

The present disclosure relates to a color conversion method, device, electronic device and storage medium, the method comprising: obtaining a first neural network, wherein the first neural network is used to simulate the color deviation in the shooting process; determining a second neural network according to the first neural network, wherein the second neural network is used to perform color correction on the color deviation in the shooting process; and performing color correction on the image in the virtual shooting process according to the second neural network. According to the embodiments of the present disclosure, the color correction can accurately meet the expected virtual shooting requirements.

Description

Technical Field

[0001] The present disclosure relates to the technical field of virtual shooting, and more specifically, to a color conversion method, a color conversion device, an electronic device, and a non-volatile computer-readable storage medium. Background Art

[0002] In recent years, digital background technology has been widely used in filming of film and television works. When using digital background technology for filming, images are displayed on the screen to simulate the real environment to be presented.

[0003] In order to make the color of the final photographed work meet the shooting requirements, it is usually necessary to perform color correction on the image during the virtual shooting process. Summary of the invention

[0004] One object of the present disclosure is to provide a new technical solution for color conversion.

[0005] According to a first aspect of the present disclosure, there is provided a color conversion method, comprising:

[0006] Acquire a first neural network, wherein the first neural network is used to simulate color deviation during shooting;

[0007] Determining a second neural network according to the first neural network, wherein the second neural network is used to perform color correction on color deviation during shooting;

[0008] Color correction is performed on the image in the virtual shooting process according to the second neural network.

[0009] In a possible implementation, obtaining the first neural network includes:

[0010] Acquire a first color lookup table; the first color lookup table is used to represent the mapping relationship between the initial color information of the captured image and the actual deviation color information after the color undergoes a deviation color change;

[0011] The first neural network is trained according to the initial color information and the actual deviation color information corresponding to the first color lookup table, and the network parameters of the first neural network are updated by minimizing the loss between the estimated deviation color information output by the first neural network according to the input initial color information and the corresponding actual deviation color information.

[0012] In a possible implementation, determining the second neural network according to the first neural network includes:

[0013] Based on the series network formed by the first neural network and the second neural network, the second neural network is trained by setting color information, and the loss between the output data and the input data of the series network is minimized as a training goal, and the network parameters of the second neural network are updated;

[0014] The input end of the second neural network is the input end of the series network, the output end of the second neural network is connected to the input end of the first neural network, and the output end of the first neural network is the output end of the series network.

[0015] In a possible implementation manner, performing color correction on the image in the virtual shooting process according to the second neural network includes:

[0016] Acquire a first image to be displayed during the virtual shooting process;

[0017] Performing color correction on the first image according to the second neural network to obtain a second image;

[0018] The method further comprises:

[0019] Rendering the second image to a screen for display;

[0020] The camera device is controlled to shoot the second image displayed on the screen to obtain a third image.

[0021] In a possible implementation manner, performing color correction on the first image to be displayed according to the second neural network to obtain the second image includes:

[0022] Acquire first color information corresponding to the first image;

[0023] performing color correction on the first color information according to the second neural network to obtain second color information corresponding to the first image;

[0024] The first color information corresponding to the first image is adjusted based on the second color information to obtain the second image.

[0025] In a possible implementation manner, performing color correction on the first image to be displayed according to the second neural network to obtain the second image includes:

[0026] According to the second neural network, a second color lookup table is obtained; the second color lookup table is used to represent the mapping relationship between the color information of the image to be displayed and the color information after color correction;

[0027] Color correction is performed on the first image according to the second color lookup table to obtain the second image.

[0028] In a possible implementation, determining the second neural network according to the first neural network includes:

[0029] Based on the series network formed by the first neural network and the second neural network, the second neural network is trained by setting color information, and the loss between the output data and the input data of the series network is minimized as a training goal, and the network parameters of the second neural network are updated;

[0030] The input end of the first neural network is the input end of the series network, the input end of the second neural network is connected to the output end of the first neural network, and the output end of the second neural network is the output end of the series network.

[0031] In a possible implementation manner, performing color correction on the image in the virtual shooting process according to the second neural network includes:

[0032] Acquire a first image to be displayed during the virtual shooting process;

[0033] The first image is rendered to a screen for display, and a camera device is controlled to shoot the first image displayed on the screen to obtain a fourth image; and color correction is performed on the fourth image according to the second neural network to obtain a fifth image.

[0034] According to a second aspect of the present disclosure, a color conversion device is provided, the device comprising:

[0035] A first network acquisition module, used to acquire a first neural network, wherein the first neural network is used to simulate color deviation during shooting;

[0036] A second network acquisition module, used to determine a second neural network according to the first neural network, wherein the second neural network is used to perform color correction on color deviation during shooting;

[0037] The color correction module is used to perform color correction on the image in the virtual shooting process according to the second neural network.

[0038] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0039] processor;

[0040] a memory for storing processor-executable instructions;

[0041] The processor is configured to implement the method described in the first aspect of the present disclosure when executing the instructions stored in the memory.

[0042] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect of the present disclosure is implemented.

[0043] Through the embodiments of the present disclosure, a second neural network is determined based on a first neural network used to simulate color deviations in a shooting process, and color correction is performed on the image in the virtual shooting process based on the second neural network, so that the image obtained by color correction can accurately meet the expected shooting requirements.

[0044] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0046] Figure 1 A schematic diagram showing application scenarios according to some embodiments of the present disclosure is shown;

[0047] Figure 2 A schematic diagram showing application scenarios according to some embodiments of the present disclosure is shown;

[0048] Figure 3 A schematic diagram of a process of a color conversion method according to some embodiments of the present disclosure is shown;

[0049] Figure 4 A schematic diagram showing a training objective of a second neural network according to some embodiments of the present disclosure;

[0050] Figure 5 A schematic diagram showing a training objective of a second neural network according to some embodiments of the present disclosure;

[0051] Figure 6 A structural block diagram of a color conversion device according to some embodiments of the present disclosure is shown;

[0052] Figure 7 A schematic diagram of the structure of an electronic device according to some embodiments of the present disclosure is shown;

[0053] Figure 8 A schematic structural diagram of an electronic device according to some other embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0054] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0055] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0056] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0057] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0058] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0059] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0060] <Application scenarios>

[0061] Figure 1 and Figure 2 It is a schematic diagram of an application scenario of an embodiment of the present disclosure.

[0062] In such Figure 1 In the virtual shooting scene shown, the color conversion device 1000 can perform color correction on the first image based on the second neural network to obtain a second image. The screen 3000 can display the second image. The camera device 2000 can shoot the second image displayed on the screen 3000 and the shooting object between the screen 3000 and the camera device 2000 to obtain a third image. The color information of the third image is consistent with the color information of the first image.

[0063] In such Figure 2In the virtual shooting scene shown, the screen 3000 can display the first image. The camera device 2000 can shoot the first image displayed on the screen 3000 and the shooting object between the screen 3000 and the camera device 2000 to obtain a fourth image. The color conversion device 1000 can perform color correction on the fourth image based on the second neural network to obtain a fifth image. The color information of the fifth image is consistent with the color information of the first image.

[0064] In the virtual shooting scene, the first image may be an image frame in a video, a static image or a dynamic image, or an image of a three-dimensional model from any viewing angle.

[0065] like Figure 1 and Figure 2 As shown, color correction can be implemented by a color conversion device 1000, wherein the color conversion device 1000 can be any electronic device that has the ability to support color correction, such as a PC, a notebook computer, a workstation, a server, etc. It can be understood by those skilled in the art that color correction and image display can also be implemented by the same device, which is not limited here.

[0066] <Method Example>

[0067] In this embodiment, a color conversion method is provided. The color conversion method can be implemented by an electronic device. The electronic device can be Figure 1 or Figure 2 The color conversion device 1000 is shown.

[0068] according to Figure 3 As shown, the color conversion method of this embodiment may include the following steps S3100 to S3300:

[0069] Step S3100, obtaining a first neural network, wherein the first neural network is used to simulate color deviation during shooting.

[0070] In some embodiments, the first neural network may be trained based on initial color information of a designated scene and actual deviation color information obtained after photographing the designated scene.

[0071] In this embodiment, initial color information and corresponding actual deviation color information of at least one pixel in a reference image may be obtained in advance, the obtained initial color information may be used as input of a first neural network, the corresponding actual deviation color information may be used as output of the first neural network, the first neural network may be trained, and the network parameters of the first neural network may be updated.

[0072] In some embodiments, obtaining a first neural network includes: obtaining a first color lookup table; training the first neural network according to the initial color information and deviation color information corresponding to the first color lookup table, taking minimizing the loss between the estimated deviation color information output by the first neural network according to the input initial color information and the corresponding actual deviation color information as the training goal, and updating the network parameters of the first neural network.

[0073] The first color lookup table is used to represent the mapping relationship between the initial color information of the captured image and the actual deviation color information after the color undergoes a deviation color change.

[0074] It can be understood that the input data of the first neural network is the initial color information, the output data of the first neural network is the estimated deviation color information, and the training goal of the first neural network is: the loss between the estimated deviation color information output by the first neural network based on the input initial color information and the actual deviation color information corresponding to the input initial color information in the first color lookup table is minimized.

[0075] The first color lookup table can be a three-dimensional mapping table (3D Look Up Table, 3D LUT for short). The core idea is to divide the source color space into regular cubes. The data of the eight vertices of each cube is known, and all the known points in the source color space form a three-dimensional mapping table. Generally, the three RGB channels are divided equally into N vertices to form N*N*N nodes. When in use, other nodes are directly interpolated. Commonly used three-dimensional mapping tables are 9*9*9, 17*17*17, 33*33*33, 65*65*65 and other sizes.

[0076] The first color lookup table in this embodiment may be obtained by sampling in advance according to initial color information of a designated scene and actual deviation color information obtained after photographing the designated scene.

[0077] In this embodiment, the corresponding initial color information and actual deviation color information used to train the first neural network may be the initial color information and actual deviation color information corresponding to any node in the first color lookup table.

[0078] The first neural network trained according to the corresponding initial color information and actual deviation color information in the first color lookup table can also represent the mapping relationship between the initial color information of the captured image and the actual deviation color information after the color has undergone a biased color change, so as to simulate the color deviation during the shooting process.

[0079] In the process of learning the complex nonlinear relationship of the first color lookup table through the first neural network in this embodiment, the training goal is to minimize the loss between the estimated deviation color information output by the first neural network based on the input initial color information and the corresponding actual deviation color information. This can enable the first neural network to more accurately reflect the mapping relationship between the initial color information of the captured image and the deviation color information after the color has undergone a deviation color change, thereby improving the accuracy of the color conversion of the second neural network obtained according to the first neural network.

[0080] Step S3200: determining a second neural network based on the first neural network, wherein the second neural network is used to perform color correction on color deviation during the shooting process.

[0081] In this embodiment, the second neural network is determined according to the first neural network, so that after any color information is input into the series network of the second neural network and the first neural network, the loss between the output color information and the input color information is minimized.

[0082] Furthermore, by determining the second neural network based on the first neural network, any one of the first neural network and the second neural network can reverse the color correction result of the other neural network.

[0083] In a possible implementation, the second neural network is determined based on the first neural network, including: based on the series network formed by the first neural network and the second neural network, the second neural network is trained by setting color information, and the loss between the output data and the input data of the series network is minimized as a training goal, and the network parameters of the second neural network are updated; wherein the input end of the second neural network is the input end of the series network, the output end of the second neural network is connected to the input end of the first neural network, and the output end of the first neural network is the output end of the series network, which can be specifically as follows Figure 4 shown.

[0084] During the training of the second neural network, the input data of the series network is the set color information, and the output data is the estimated color information. The training goal of the second neural network is to minimize the loss between the set color information input to the series network and the color information output by the series network. Under the ideal condition of the series network, the estimated color information output by the series network should also be the set color information.

[0085] By training the second neural network through this embodiment, the accuracy of the color conversion of the second neural network can be improved, so that the trained second neural network can accurately perform color correction on the color deviation during the shooting process. In addition, in the series network used in the training process, the second neural network is located before the first neural network, which can enable the second neural network to perform color correction on the image before rendering during the virtual shooting process to reduce the color deviation during the virtual shooting process, so that the image obtained by the virtual shooting can accurately meet the expected shooting requirements.

[0086] In another possible implementation, the second neural network is determined based on the first neural network, including: based on the series network formed by the first neural network and the second neural network, the second neural network is trained by setting color information, and the loss between the output data and the input data of the series network is minimized as a training goal, and the network parameters of the second neural network are updated; wherein the input end of the first neural network is the input end of the series network, the output end of the first neural network is connected to the input end of the second neural network, and the output end of the second neural network is the output end of the series network, which can be specifically as follows Figure 5 shown.

[0087] During the training of the second neural network, the input data of the series network is the set color information, and the output data is the estimated color information. The training goal of the second neural network is to minimize the loss between the set color information input to the series network and the color information output by the series network. Under the ideal condition of the series network, the estimated color information output by the series network should also be the set color information.

[0088] The set color information may be a plurality of color information set in advance according to application scenarios or specific requirements.

[0089] By training the second neural network using the method of this embodiment, the accuracy of the color conversion of the second neural network can be improved, so that the trained second neural network can accurately perform color correction on the color deviation during the shooting process. In addition, in the series network used in the training process, the second neural network is located after the first neural network, so that the second neural network can perform color correction on the image obtained by real shooting during the virtual shooting process to reduce the color deviation during the virtual shooting process, so that the image obtained by the virtual shooting can accurately meet the expected shooting requirements.

[0090] Step S3300: performing color correction on the image during the virtual shooting process according to the second neural network.

[0091] In this embodiment, the image in the virtual shooting process may be an image to be displayed and used for real shooting, or may be an image obtained by real shooting.

[0092] Through the embodiments of the present disclosure, a second neural network is determined based on a first neural network used to simulate color deviations in a shooting process, and color correction is performed on images in a virtual shooting process based on the second neural network, so that the color-corrected images can accurately meet the expected virtual shooting requirements.

[0093] Based on Figure 4 In the embodiment of training the second neural network through the series network shown, color correction of the first image according to the second neural network may include: obtaining the first image to be displayed during the virtual shooting process; color correction of the first image according to the second neural network to obtain the second image. On this basis, the method may also include: rendering the second image to the screen for display; controlling the camera device to shoot the second image displayed on the screen to obtain a third image.

[0094] The first image in this embodiment may be an image frame in a video, a static image or a dynamic image, or an image of a three-dimensional model from any viewing angle.

[0095] In this embodiment, the screen displaying the second image may be any one of an air imaging device, an air screen, a mobile terminal, a tablet computer, a PC computer, an LED display, an LCD display, an OLED display, a dot matrix display and a projection screen.

[0096] In the virtual shooting scene, after the second image is rendered to the screen for display, the camera device can be controlled to actually shoot the second image displayed on the screen and the object located between the camera device and the screen to achieve a shooting effect in which the object is located in the physical scene corresponding to the second image.

[0097] Correspondingly, the obtained third image may include at least part of the image content of the second image.

[0098] Through this embodiment, after color correction is performed on the first image according to the second neural network to obtain the second image, the second image is rendered to the screen for display, and the camera device is controlled to actually shoot the second image displayed on the screen to obtain the third image. The color information of the third image can be kept consistent with the color information of the first image to accurately meet the expected virtual shooting requirements.

[0099] In a possible implementation, color correcting the first image according to the second neural network to obtain the second image may include: acquiring first color information corresponding to the first image; color correcting the first color information according to the second neural network to obtain second color information corresponding to the first image; and adjusting the first color information corresponding to the first image based on the second color information to obtain the second image.

[0100] In this embodiment, the first color information corresponding to the first image may include color information of part or all of the pixels in the first image.

[0101] When the first color information is obtained, the first color information can be input into the second neural network to obtain corresponding second color information.

[0102] Through this embodiment, the first color information in the first image is color corrected according to the second neural network to obtain the corresponding second color information, and the first color information of the first image is adjusted based on the second color information to obtain the second image, so that the color deviation in the shooting process can be accurately color corrected. On the basis of color correcting the first image to be displayed according to the second neural network to obtain the second image, the second image is rendered to the screen for display, and the camera device is controlled to actually shoot the second image displayed on the screen to obtain the third image, so that the color information of the third image can be consistent with the color information of the first image to accurately meet the expected virtual shooting requirements.

[0103] In another possible implementation, color correcting the first image according to the second neural network to obtain the second image may include: obtaining a second color lookup table according to the second neural network; the second color lookup table is used to represent the mapping relationship between color information of the image to be displayed and color information after color correction; and color correcting the first image according to the second color lookup table to obtain the second image.

[0104] In this embodiment, the second color lookup table may be a three-dimensional mapping table (3D Look Up Table, 3D LUT for short). Specifically, the three RGB channels may be equally divided into N vertices to form N*N*N nodes, and the input color information corresponding to each node is input into the second neural network to obtain the output color information corresponding to each node, and then the second color lookup table is obtained according to the input color information and output color information corresponding to each node.

[0105] In some embodiments, color correction is performed on a first image according to a second color lookup table to obtain a second image, including: obtaining first color information corresponding to the first image; determining second color information corresponding to the first color information according to the second color lookup table; and adjusting the first color information corresponding to the first image based on the second color information to obtain the second image.

[0106] In this embodiment, the second color information corresponding to the first color information can be obtained by interpolating the data in the second color lookup table.

[0107] Specifically, the second color information corresponding to the first color information of each pixel in the first image may be determined according to the second color lookup table, and the first color information of each pixel in the first image may be adjusted based on the second color information of the pixel to obtain the second image.

[0108] Through this embodiment, a second color lookup table is generated according to the second neural network, and then the color correction of the first image to be displayed is performed based on the second color lookup table. Compared with the second neural network, the second color lookup table occupies less storage space, which can reduce the hardware storage requirements of the color conversion device. In addition, on the basis of performing color correction on the first image to be displayed according to the second color lookup table to obtain the second image, the second image is rendered to the screen for display, and the camera device is controlled to actually shoot the second image displayed on the screen to obtain the third image, so that the color information of the third image can be consistent with the color information of the first image to accurately meet the expected virtual shooting requirements.

[0109] Based on Figure 5 In the embodiment of training the second neural network through the series network shown, color correction of the first image according to the second neural network can include: obtaining the first image to be displayed in the virtual shooting process; rendering the first image to the screen for display, and controlling the camera device to shoot the first image displayed on the screen to obtain a fourth image; color correction of the fourth image according to the second neural network to obtain a fifth image, and the color information of the fifth image is consistent with the color information of the first image.

[0110] The first image in this embodiment may be an image frame in a video, a static image or a dynamic image, or an image of a three-dimensional model from any viewing angle.

[0111] Through the second neural network of this embodiment, the fourth image obtained by actual shooting can accurately meet the expected shooting requirements.

[0112] In this embodiment, the screen displaying the first image may be any one of an air imaging device, an air screen, a mobile terminal, a tablet computer, a PC computer, an LED display, an LCD display, an OLED display, a dot matrix display and a projection screen.

[0113] In the virtual shooting scene, after the first image is rendered to the screen for display, the camera device can be controlled to actually shoot the first image displayed on the screen and the object located between the camera device and the screen to achieve a shooting effect in which the object is located in the physical scene corresponding to the first image.

[0114] Correspondingly, the obtained fourth image may include at least part of the image content of the first image.

[0115] Through this embodiment, after the first image is rendered to the screen for display, the camera device is controlled to actually shoot the first image displayed on the screen to obtain a fourth image, and then the fourth image is color corrected according to the second neural network to obtain a fifth image. The color information of the fifth image can be made consistent with the color information of the first image to accurately meet the expected virtual shooting requirements.

[0116] In a possible implementation, color correcting the fourth image according to the second neural network to obtain the fifth image may include: acquiring third color information corresponding to the fourth image; color correcting the third color information according to the second neural network to obtain fourth color information corresponding to the fourth image; and adjusting the third color information corresponding to the fourth image based on the fourth color information to obtain the fifth image.

[0117] In this embodiment, the third color information corresponding to the fourth image may include color information of part or all of the pixels in the fourth image.

[0118] When the third color information is obtained, the third color information can be input into the second neural network to obtain corresponding fourth color information.

[0119] Through this embodiment, the third color information in the fourth image is color corrected according to the second neural network to obtain the corresponding fourth color information, and the third color information corresponding to the fourth image is adjusted based on the fourth color information to obtain the fifth image, so that the color deviation in the shooting process can be accurately color corrected. The fourth image obtained by actually shooting the first image displayed on the screen according to the second neural network is color corrected to obtain the fifth image, so that the color information of the fifth image can be consistent with the color information of the first image, so as to accurately meet the expected virtual shooting requirements.

[0120] In another possible implementation, color correcting the fourth image according to the second neural network to obtain the fifth image may include: obtaining a second color lookup table according to the second neural network; the second color lookup table is used to represent the mapping relationship between color information of the image to be displayed and color information after color correction; and color correcting the fourth image according to the second color lookup table to obtain the fifth image.

[0121] In this embodiment, the second color lookup table may be a three-dimensional mapping table (3D Look Up Table, 3D LUT for short). Specifically, the three RGB channels may be equally divided into N vertices to form N*N*N nodes, and the input color information corresponding to each node is input into the second neural network to obtain the output color information corresponding to each node, and then the second color lookup table is obtained according to the input color information and output color information corresponding to each node.

[0122] In some embodiments, color correction is performed on the fourth image according to the second color lookup table to obtain a fifth image, including: obtaining third color information corresponding to the fourth image; determining fourth color information corresponding to the third color information according to the second color lookup table; and adjusting the third color information corresponding to the fourth image based on the fourth color information to obtain the fifth image.

[0123] In this embodiment, fourth color information corresponding to the third color information can be obtained by interpolating data in the second color lookup table.

[0124] Specifically, the fourth color information corresponding to the third color information of each pixel in the fourth image can be determined according to the second color lookup table, and the third color information of each pixel in the fourth image can be adjusted based on the fourth color information of the pixel to obtain the fifth image.

[0125] Through this embodiment, a second color lookup table is generated according to the second neural network, and then the fourth image obtained by real shooting is color corrected based on the second color lookup table. Compared with the second neural network, the second color lookup table occupies less storage space, which can reduce the hardware storage requirements of the color conversion device. In addition, the first image is rendered to the screen for display, and the camera device is controlled to actually shoot the first image displayed on the screen to obtain the fourth image; the fourth image is color corrected according to the second neural network to obtain the fifth image, so that the color information of the fifth image can be consistent with the color information of the first image to accurately meet the expected virtual shooting requirements.

[0126] <Device Example>

[0127] The present disclosure also provides a color conversion device for implementing any of the above method embodiments. Figure 6 FIG. 4 shows a structural block diagram of a color conversion device according to some embodiments. Figure 6 As shown, the color conversion device 400 may include a first network acquisition module 410 , a second network acquisition module 420 and a color correction module 430 .

[0128] The first network acquisition module 410 is used to acquire a first neural network, wherein the first neural network is used to simulate color deviation during shooting.

[0129] The second network acquisition module 420 is used to determine a second neural network according to the first neural network, and the second neural network is used to perform color correction on color deviation during the shooting process.

[0130] The color correction module 430 is used to perform color correction on the image in the virtual shooting process according to the second neural network.

[0131] In some embodiments, when acquiring the first neural network, the first network acquisition module 410 is used to: acquire a first color lookup table; the first color lookup table is used to represent the mapping relationship between the initial color information of the captured image and the actual deviation color information after the color has undergone a deviation color change; train the first neural network according to the initial color information and the actual deviation color information corresponding to the first color lookup table, and take minimizing the loss between the estimated deviation color information output by the first neural network according to the input initial color information and the corresponding actual deviation color information as the training goal, and update the network parameters of the first neural network.

[0132] In some embodiments, when determining the second neural network based on the first neural network, the second network acquisition module 420 is used to: based on the series network formed by the first neural network and the second neural network, train the second neural network by setting color information, and take minimizing the loss between the output data and the input data of the series network as the training goal, and update the network parameters of the second neural network; wherein the input end of the second neural network is the input end of the series network, the output end of the second neural network is connected to the input end of the first neural network, and the output end of the first neural network is the output end of the series network.

[0133] In some embodiments, when the color correction module 430 performs color correction on the image during the virtual shooting process according to the second neural network, it is used to: obtain a first image to be displayed during the virtual shooting process; and perform color correction on the first image according to the second neural network to obtain a second image.

[0134] The color conversion device 400 may further include:

[0135] A display module, used for rendering the second image to a screen for display;

[0136] The shooting module is used to control the camera device to shoot the second image displayed on the screen to obtain a third image.

[0137] In some embodiments, when the color correction module 430 performs color correction on the first image to be displayed according to the second neural network to obtain the second image, it is used to: obtain first color information corresponding to the first image; perform color correction on the first color information according to the second neural network to obtain second color information corresponding to the first image; and adjust the first color information corresponding to the first image based on the second color information to obtain the second image.

[0138] In some embodiments, when the color correction module 430 performs color correction on the first image to be displayed according to the second neural network to obtain the second image, it is used to: obtain a second color lookup table according to the second neural network; the second color lookup table is used to represent the mapping relationship between the color information of the image to be displayed and the color information after color correction; perform color correction on the first image according to the second color lookup table to obtain the second image.

[0139] In some embodiments, when determining the second neural network based on the first neural network, the second network acquisition module 420 is used to: based on the series network formed by the first neural network and the second neural network, train the second neural network by setting color information, and take minimizing the loss between the output data and the input data of the series network as the training goal, and update the network parameters of the second neural network; wherein the input end of the first neural network is the input end of the series network, the input end of the second neural network is connected to the output end of the first neural network, and the output end of the second neural network is the output end of the series network.

[0140] In some embodiments, when the color correction module 430 performs color correction on the image in the virtual shooting process according to the second neural network, it is used to: obtain a first image to be displayed in the virtual shooting process; render the first image to the screen for display, and control the camera device to shoot the first image displayed on the screen to obtain a fourth image; perform color correction on the fourth image according to the second neural network to obtain a fifth image.

[0141] The present disclosure also provides an electronic device for implementing any of the above method embodiments. Figure 7 FIG. 4 shows a block diagram of an electronic device according to some embodiments. Figure 7 As shown, the electronic device 600 includes a processor 610 and a memory 620 for storing instructions executable by the processor 610. The processor 610 is configured to implement the color conversion method according to any embodiment of the present disclosure when executing the instructions stored in the memory 620.

[0142] The processor 610 is used to execute computer instructions, which can be written in an instruction set of an architecture such as x86, Arm, RISC, MIPS, SSE, etc. The memory 620 includes, for example, ROM (read-only memory), RAM (random access memory), a non-volatile memory such as a hard disk, etc., which are not limited here.

[0143] Figure 8 FIG. 2 shows a structural block diagram of an electronic device according to some other embodiments. Figure 8 As shown, the electronic device 700 may include a display device 730, an interface device 740, a communication device 750, an input device 760, etc. in addition to a processor 710 and a memory 720.

[0144] The interface device 740 includes, for example, a USB interface, a bus interface, a network interface, etc. The communication device 750 can, for example, perform wired or wireless communication. The communication device 750 may include at least one short-range communication module, such as any module for short-range wireless communication based on short-range wireless communication protocols such as Hilink protocol, WiFi (IEEE 802.11 protocol), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, LiFi, etc. The communication device 750 may also include a remote communication module, such as any module for WLAN, GPRS, 2G / 3G / 4G / 5G remote communication. The display device 730 may display a second image. The input device 760 may include a touch screen, a keyboard, a mouse, a microphone, a camera, etc., which are not limited here.

[0145] The present disclosure also provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the color conversion method according to any embodiment of the present disclosure.

[0146] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0147] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0148] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0149] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0150] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0151] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0152] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0153] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0154] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims

1. A color conversion method, characterized in that: include: Acquire a first neural network, wherein the first neural network is used to simulate color deviation during shooting; Determining a second neural network according to the first neural network, wherein the second neural network is used to perform color correction on color deviation during shooting; performing color correction on the image during the virtual shooting process according to the second neural network; Determining a second neural network according to the first neural network includes: Based on the series network formed by the first neural network and the second neural network, the second neural network is trained by setting color information, and the network parameters of the second neural network are updated by minimizing the loss between the output data and the input data of the series network as a training goal.

2. The method according to claim 1, characterized in that The obtaining of the first neural network comprises: Obtaining a first color lookup table; the first color lookup table is used to represent a mapping relationship between initial color information of a captured image and actual deviation color information after the color undergoes a deviation color change; The first neural network is trained according to the initial color information and the actual deviation color information corresponding to the first color lookup table, and the network parameters of the first neural network are updated by minimizing the loss between the estimated deviation color information output by the first neural network according to the input initial color information and the corresponding actual deviation color information.

3. The method according to claim 1, characterized in that The input end of the second neural network is the input end of the series network, the output end of the second neural network is connected to the input end of the first neural network, and the output end of the first neural network is the output end of the series network.

4. The method according to claim 3, characterized in that The performing color correction on the image in the virtual shooting process according to the second neural network includes: Acquire a first image to be displayed during the virtual shooting process; Performing color correction on the first image according to the second neural network to obtain a second image; The method further comprises: Rendering the second image to a screen for display; The camera device is controlled to shoot the second image displayed on the screen to obtain a third image.

5. The method according to claim 4, characterized in that The step of performing color correction on the first image to be displayed according to the second neural network to obtain the second image comprises: Acquire first color information corresponding to the first image; performing color correction on the first color information according to the second neural network to obtain second color information corresponding to the first image; The first color information corresponding to the first image is adjusted based on the second color information to obtain the second image.

6. The method according to claim 4, characterized in that The step of performing color correction on the first image to be displayed according to the second neural network to obtain the second image comprises: According to the second neural network, a second color lookup table is obtained; the second color lookup table is used to represent the mapping relationship between the color information of the image to be displayed and the color information after color correction; Color correction is performed on the first image according to the second color lookup table to obtain the second image.

7. The method according to claim 1, characterized in that The input end of the first neural network is the input end of the series network, the input end of the second neural network is connected to the output end of the first neural network, and the output end of the second neural network is the output end of the series network.

8. The method according to claim 7, characterized in that The performing color correction on the image in the virtual shooting process according to the second neural network includes: Acquire a first image to be displayed during the virtual shooting process; The first image is rendered to a screen for display, and a camera device is controlled to shoot the first image displayed on the screen to obtain a fourth image; and color correction is performed on the fourth image according to the second neural network to obtain a fifth image.

9. A color conversion device, characterized in that: The device comprises: A first network acquisition module, used to acquire a first neural network, wherein the first neural network is used to simulate color deviation during shooting; A second network acquisition module, used to determine a second neural network according to the first neural network, wherein the second neural network is used to perform color correction on color deviation during shooting; A color correction module, used for performing color correction on the image in the virtual shooting process according to the second neural network; Determining a second neural network according to the first neural network includes: Based on the series network formed by the first neural network and the second neural network, the second neural network is trained by setting color information, and the network parameters of the second neural network are updated by minimizing the loss between the output data and the input data of the series network as a training goal.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 8 when executing the instructions stored in the memory.

11. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images