Image processor, processing method, storage medium, and mixed reality display system

Abstract

The present invention provides an image processor, a processing method, a storage medium, and a mixed reality display system. The image processor comprises: a first color management module connected to an input terminal of a rendering module for acquiring virtual layer data from the rendering module and performing a first color processing; a second color management module connected to an input terminal of a camera module for acquiring real layer data from the camera module and performing a second color processing; a layer composite module connected to the input terminals of the first and second color management modules, respectively, for performing layer composite processing on the first color-processed virtual layer data and the second color-processed real layer data to determine composite image data; and a third color management module connected to an input terminal of a layer composite module for acquiring composite image data from the layer composite module, performing a third color processing, and outputting the third color-processed composite image data to a mixed reality display.

Claims

1. A rendering module is connected to the input terminal, and a first color management module is provided for acquiring virtual layer data from the rendering module and performing first color processing. A camera module is connected to the input terminal, and a second color management module is provided for acquiring real-world layer data from the camera module and performing a second color processing. The input terminals are connected to the first color management module and the second color management module, respectively, and a layer composite module is used to perform layer composite processing on the first color-processed virtual layer data and the second color-processed real layer data to determine composite image data. An image processor characterized by comprising: a layer composite module connected to its input terminal; a third color management module for acquiring the composite image data from the layer composite module, performing a third color processing, and outputting the third color-processed composite image data to a composite reality display.

2. The system further includes a color processing firmware unit and a communication interface to which each of the color management modules is connected. The first color management module is configured to acquire first register data from the color processing firmware unit via the communication interface in response to the demand for the first color processing, and to return first statistical information to the color processing firmware unit. The second color management module is configured to acquire second register data from the color processing firmware unit via the communication interface and return second statistical information to the color processing firmware unit in response to the demand for the second color processing. The image processor according to claim 1, characterized in that the third color management module is arranged to acquire third register data from the color processing firmware unit via the communication interface and return third statistical information to the color processing firmware unit in response to the demand for the third color processing.

3. Each of the aforementioned color management modules includes a color mapping unit, a color gamut mapping unit, and a color temperature processing unit. The image processor according to claim 2, characterized in that, in response to the demand for color processing, it is configured to acquire register data related to color tone mapping, color gamut mapping, and / or color temperature processing from the color processing firmware unit via the communication interface, and to return statistical information related to the color tone mapping, color gamut mapping, and / or color temperature processing to the color processing firmware unit.

4. Each of the aforementioned color management modules further comprises: Based on the register data relating to the color mapping, histogram equalization is performed on the corresponding virtual layer data, real layer data, or composite image data to determine the statistical information relating to the color mapping, and / or The image processor according to claim 3, which is configured to respond to the demand for color processing to transition from a wide color gamut to a narrow color gamut, and to perform stepwise mapping processing on corresponding virtual layer data, real layer data, or composite image data based on the register data related to the color gamut mapping, thereby determining statistical information related to the color gamut mapping.

5. Each of the aforementioned color management modules further includes a ReGamma unit and / or a DeGamma unit, The image processor according to claim 3, characterized in that, in response to the demand for the aforementioned color processing, it is configured to acquire register data related to ReGamma processing and / or DeGamma processing from the color processing firmware unit via the communication interface and to return statistical information related to the ReGamma processing and / or DeGamma processing to the color processing firmware unit.

6. The rendering module further inputs HDR metadata to the color processing firmware unit, and each of the color management modules further includes an HDR unit. The image processor according to claim 3, characterized in that, in response to the demand for the aforementioned color processing, it is configured to acquire register data related to HDR processing from the color processing firmware unit via the communication interface and return statistical information related to the HDR processing to the color processing firmware unit, and the register data related to HDR processing is updated and determined based on the HDR metadata and the returned statistical information related to the HDR processing.

7. The color processing firmware unit further comprises the color processing firmware unit, By acquiring external light parameter information, The image processor according to claim 3, characterized in that it is configured to update the register data by generating a color curve, generating a color gamut mapping function, performing a color temperature adjustment calculation and / or a brightness adjustment calculation based on the optical parameter information and statistical information returned from at least one of the color management modules.

8. The image processor according to claim 2, characterized in that the amount of data of the first register data and the second register data is smaller than the amount of data of the third register data, and the amount of data of the first statistical information and the second statistical information is smaller than the amount of data of the third statistical information.

9. The rendering module is further used to render and generate one or more UI layer data, and the first color management module is, One or more UI layer data are obtained from the rendering module, and one or more corresponding fourth color processing is performed. The image processor according to claim 1, characterized in that it is configured to input one or more of the fourth color-processed UI layer data into the layer composite module and perform layer composite processing to determine the corresponding composite image data.

10. The system further comprises an image signal processing module, the second color management module being connected to the camera module via the image signal processing module, and the image signal processing module is The video stream data captured from the aforementioned camera module is acquired, The image processor according to claim 1, further configured to perform AWB processing and / or AE processing on the video stream data to generate the reality layer data.

11. A rendering module for generating virtual layer data, A camera module for collecting real-world layer data, A composite reality display system comprising: an image processor according to any one of claims 1 to 10, which connects the rendering module and the camera module to acquire the virtual layer data and the reality layer data, performs image processing based on the virtual layer data and the reality layer data to output composite image data to be displayed.

12. The steps include obtaining virtual layer data from the rendering module and inputting it into the first color management module, The steps include acquiring real-world layer data from the camera module and inputting it into the second color management module, Steps to obtain the demand for color processing, The steps include controlling the first color management module to perform a first color processing on the virtual layer data and / or controlling the second color management module to perform a second color processing on the real layer data in response to the demand for the aforementioned color processing, The steps include: determining composite image data by performing layer composite processing on the layer data of the first color management module and the second color management module via the layer composite module; The steps include inputting the composite image data into a third color management module and executing the third color processing according to the demand for color processing, An image processing method characterized by comprising the step of transferring composite image data output by the third color management module to a mixed reality display.

13. A computer-readable storage medium in which computer commands are stored, characterized in that when the computer commands are executed by a processor, the image processing method described in claim 12 is performed.

Images