Interaction information processing method and device of mobile transparent display equipment

Abstract

The invention relates to the technical field of display equipment, and provides an interaction information processing method and device for mobile transparent display equipment, and the method comprises the steps: obtaining ambient light data information, the pose information of the mobile transparent display equipment, and the display parameter information of the mobile transparent display equipment; generating display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information comprises a brightness adjustment coefficient and a color correction matrix; generating a pixel projection matrix based on the pose information; correcting the color of a to-be-displayed image based on the color correction matrix to obtain a target to-be-displayed image; and adjusting a display parameter of the mobile transparent display device based on the brightness adjustment coefficient, and displaying the target to-be-displayed image on the mobile transparent display device based on the pixel projection matrix. According to the method, the mobile transparent display equipment can display clear and stable contents, and the visual experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a method and device for processing interactive information of a mobile transparent display device. Background Art

[0002] With the development of transparent display technology, transparent display devices are gradually applied to augmented reality, virtual reality and multi-scenario interactive systems. Traditional display devices can provide clear and stable display content under fixed light and viewing angle, but for mobile transparent display devices that need to support dynamic environments and variable postures, existing display processing technology is difficult to provide clear and stable display content, which seriously affects the user's visual experience. Summary of the invention

[0003] The present application provides a method and device for processing interactive information of a mobile transparent display device to solve the problems raised by the above background technology.

[0004] In a first aspect, the present application provides an interactive information processing method for a mobile transparent display device, comprising: Acquire ambient light data information, position information of the mobile transparent display device, and display parameter information of the mobile transparent display device; Generate display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information includes a brightness adjustment coefficient and a color correction matrix; Generate a pixel projection matrix based on the pose information; Correcting the color of the image to be displayed based on the color correction matrix to obtain a target image to be displayed; The display parameters of the mobile transparent display device are adjusted based on the brightness adjustment coefficient, and the target image to be displayed is displayed on the mobile transparent display device based on the pixel projection matrix.

[0005] In a possible implementation, the ambient light data information includes ambient light brightness and ambient light color temperature, the display parameter information includes display brightness and display color temperature, and generating display parameter adjustment information based on the ambient light data information and the display parameter information includes: generating a brightness adjustment coefficient based on the ambient light brightness and the display brightness; A color correction matrix is ​​generated based on the ambient light color temperature and the display color temperature.

[0006] In a possible implementation, the generating a brightness adjustment coefficient based on the ambient light brightness and the display brightness includes: The light adaptation brightness is calculated by formula (1); where is the brightness of light adaptation, is the reference brightness, is the ambient light brightness; (1); The brightness adjustment coefficient is calculated by formula (2); where: is the brightness adjustment coefficient, To display brightness; (2) .

[0007] In a possible implementation, generating a color correction matrix based on the ambient light color temperature and the display color temperature includes: The ratio of the ambient light color temperature to the display color temperature is used as the red channel correction coefficient ; Determine the green channel correction factor is 1; The ratio of the display color temperature to the ambient light color temperature is used as the blue channel correction coefficient ; Based on the red channel correction factor , Green channel correction factor and the blue channel correction factor Generate a color correction matrix as shown in formula (3); (3).

[0008] In a possible implementation, the posture information includes the position coordinates (x, y, z) and attitude angles α, β, and γ of the transparent display device, α is the roll angle of the mobile transparent display device around the x-axis, β is the pitch angle of the mobile transparent display device around the y-axis, and γ is the yaw angle of the mobile transparent display device around the z-axis. Generating a pixel projection matrix based on the posture information includes: Generate a translation vector based on the position coordinates (x, y, z) ; Generate the basic rotation matrix corresponding to the x-axis through formula (4) ; (4) Generate the basic rotation matrix corresponding to the y-axis through formula (5) ; (5) Generate the basic rotation matrix corresponding to the z-axis through formula (6) ; (6) Will , , and Multiply them in sequence to obtain the pixel projection matrix.

[0009] In a possible implementation manner, displaying the target image to be displayed on the mobile transparent display device based on the pixel projection matrix includes: For each target pixel of the target image to be displayed, the initial two-dimensional pixel coordinates corresponding to the target pixel on the display screen of the mobile transparent display device are determined, and the initial two-dimensional pixel coordinates are projected and transformed based on the pixel projection matrix to obtain the target two-dimensional pixel coordinates, and the target pixel is displayed on the display screen based on the target two-dimensional pixel coordinates.

[0010] In a possible implementation, performing a projection transformation on the initial two-dimensional pixel coordinates based on the pixel projection matrix to obtain target two-dimensional pixel coordinates includes: Constructing an inverse matrix of the pixel projection matrix, and constructing a two-dimensional homogeneous vector corresponding to the initial two-dimensional pixel coordinates; Multiplying the inverse matrix by the homogeneous vector to obtain the three-dimensional space coordinates corresponding to the initial two-dimensional pixel coordinates; Multiplying the pixel projection matrix by the vector corresponding to the three-dimensional space coordinates to obtain a two-dimensional homogeneous vector corresponding to the target two-dimensional pixel coordinates; The target two-dimensional pixel coordinates are generated based on the two-dimensional homogeneous vector corresponding to the target two-dimensional pixel coordinates.

[0011] In a second aspect, the present application provides an interactive information processing device for a mobile transparent display device, comprising: An acquisition module, used to acquire ambient light data information, position information of the mobile transparent display device, and display parameter information of the mobile transparent display device; A first generating module, configured to generate display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information includes a brightness adjustment coefficient and a color correction matrix; A second generation module, used to generate a pixel projection matrix based on the posture information; A correction module, used for correcting the color of the image to be displayed based on the color correction matrix to obtain a target image to be displayed; An adjustment module is used to adjust the display parameters of the mobile transparent display device based on the brightness adjustment coefficient, and display the target image to be displayed on the mobile transparent display device based on the pixel projection matrix.

[0012] The present application provides an interactive information processing method and device for a mobile transparent display device, the method comprising: acquiring ambient light data information, posture information of a mobile transparent display device, and display parameter information of the mobile transparent display device; generating display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information comprises a brightness adjustment coefficient and a color correction matrix; generating a pixel projection matrix based on the posture information; correcting the color of an image to be displayed based on the color correction matrix to obtain a target image to be displayed; adjusting the display parameters of the mobile transparent display device based on the brightness adjustment coefficient, and displaying the target image to be displayed on the mobile transparent display device based on the pixel projection matrix. The method, on the one hand, generates a brightness adjustment coefficient and a color correction matrix through the ambient light data information and the display parameter information, and corrects the color of the image to be displayed based on the color correction matrix, and adjusts the display parameters of the mobile transparent display device based on the brightness adjustment coefficient, so that the mobile transparent display device can adapt to variable ambient light conditions, ensure that the displayed content is clearly visible under different ambient light conditions, and avoid visual fatigue caused by improper brightness. On the other hand, by generating a pixel projection matrix based on the posture information, and displaying the target image to be displayed on the mobile transparent display device based on the pixel projection matrix, it is possible to correct the image geometric distortion caused by device movement or rotation, so that the displayed content is always consistent with the user's viewing angle, and improve the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0014] Figure 1 A schematic diagram of a flow chart of an interactive information processing method for a mobile transparent display device provided in an embodiment of the present application; Figure 2 A schematic block diagram of the structure of an interactive information processing device for a mobile transparent display device provided in an embodiment of the present application; Figure 3 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0017] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0018] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0019] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] See also Figure 1 , Figure 1 A flow chart of an interactive information processing method for a mobile transparent display device provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the interactive information processing method of the mobile transparent display device provided in the embodiment of the present application includes steps S1 to S5.

[0021] Step S1, obtaining ambient light data information, position information of the mobile transparent display device, and display parameter information of the mobile transparent display device.

[0022] Among them, the ambient light data information includes ambient light brightness and ambient light color temperature, the posture information includes the position coordinates (x, y, z) and attitude angles α, β, and γ of the transparent display device, α is the roll angle of the mobile transparent display device around the x-axis, β is the pitch angle of the mobile transparent display device around the y-axis, and γ is the yaw angle of the mobile transparent display device around the z-axis, and the display parameter information includes display brightness and display color temperature.

[0023] Step S2: generating display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information includes a brightness adjustment coefficient and a color correction matrix.

[0024] Specifically, step S2 includes the following steps: generating a brightness adjustment coefficient based on the ambient light brightness and the display brightness; A color correction matrix is ​​generated based on the ambient light color temperature and the display color temperature.

[0025] The step of generating a brightness adjustment coefficient based on the ambient light brightness and the display brightness includes: The light adaptation brightness is calculated by formula (1); where is the brightness of light adaptation, is the reference brightness, is the ambient light brightness; (1); The brightness adjustment coefficient is calculated by formula (2); where: is the brightness adjustment coefficient, To display brightness; (2) .

[0026] The step of generating a color correction matrix based on the ambient light color temperature and the display color temperature comprises the following steps: The ratio of the ambient light color temperature to the display color temperature is used as the red channel correction coefficient ; Determine the green channel correction factor is 1; The ratio of the display color temperature to the ambient light color temperature is used as the blue channel correction coefficient ; Based on the red channel correction factor , Green channel correction factor and the blue channel correction factor Generate a color correction matrix as shown in formula (3); (3).

[0027] Step S3: generating a pixel projection matrix based on the posture information.

[0028] Specifically, step S3 includes the following steps: Generate a translation vector based on the position coordinates (x, y, z) ; Generate the basic rotation matrix corresponding to the x-axis through formula (4) ; (4) Generate the basic rotation matrix corresponding to the y-axis through formula (5) ; (5) Generate the basic rotation matrix corresponding to the z-axis through formula (6) ; (6) Will , , and Multiply them in sequence to obtain the pixel projection matrix.

[0029] Step S4: Correct the color of the image to be displayed based on the color correction matrix to obtain a target image to be displayed.

[0030] Specifically, for each pixel of the image to be displayed, the red channel value, the green channel value and the blue channel value of the pixel are obtained, the red channel value, the green channel value and the blue channel value are arranged in sequence to obtain a color value vector, the color correction matrix is ​​multiplied by the color value vector to obtain a target color value vector of the pixel, and color correction is performed on the pixel based on the target color value vector; each element of the target color value vector is a target red channel value, a target green channel value and a target blue channel value in sequence.

[0031] Step S5: adjusting the display parameters of the mobile transparent display device based on the brightness adjustment coefficient, and displaying the target image to be displayed on the mobile transparent display device based on the pixel projection matrix.

[0032] Specifically, displaying the target image to be displayed on the mobile transparent display device based on the pixel projection matrix includes the following steps: For each target pixel of the target image to be displayed, the initial two-dimensional pixel coordinates corresponding to the target pixel on the display screen of the mobile transparent display device are determined, and the initial two-dimensional pixel coordinates are projected and transformed based on the pixel projection matrix to obtain the target two-dimensional pixel coordinates, and the target pixel is displayed on the display screen based on the target two-dimensional pixel coordinates.

[0033] The step of performing projection transformation on the initial two-dimensional pixel coordinates based on the pixel projection matrix to obtain target two-dimensional pixel coordinates includes the following steps: Constructing an inverse matrix of the pixel projection matrix, and constructing a two-dimensional homogeneous vector corresponding to the initial two-dimensional pixel coordinates; Multiplying the inverse matrix by the homogeneous vector to obtain the three-dimensional space coordinates corresponding to the initial two-dimensional pixel coordinates; Multiplying the pixel projection matrix by the vector corresponding to the three-dimensional space coordinates to obtain a two-dimensional homogeneous vector corresponding to the target two-dimensional pixel coordinates; The target two-dimensional pixel coordinates are generated based on the two-dimensional homogeneous vector corresponding to the target two-dimensional pixel coordinates.

[0034] Exemplarily, the pixel projection matrix is , the initial two-dimensional pixel coordinates are , the two-dimensional homogeneous vector corresponding to the initial two-dimensional pixel coordinate is , then the inverse matrix is , the three-dimensional space coordinates are , the two-dimensional homogeneous vector corresponding to the target two-dimensional pixel coordinate is , the target two-dimensional pixel coordinates are .

[0035] The method provided in this embodiment, on the one hand, generates a brightness adjustment coefficient and a color correction matrix through the ambient light data information and the display parameter information, and corrects the color of the image to be displayed based on the color correction matrix, and adjusts the display parameters of the mobile transparent display device based on the brightness adjustment coefficient, so that the mobile transparent display device can adapt to variable ambient light conditions, ensuring that the displayed content is clearly visible under different ambient light conditions, while avoiding visual fatigue caused by improper brightness; on the other hand, by generating a pixel projection matrix based on the posture information, and displaying the target image to be displayed on the mobile transparent display device based on the pixel projection matrix, it is possible to correct the image geometric distortion caused by device movement or rotation, so that the displayed content is always consistent with the user's perspective, thereby improving the user's visual experience.

[0036] See also Figure 2 , Figure 2 A schematic block diagram of the structure of the interactive information processing device 100 of the mobile transparent display device provided in the embodiment of the present application is shown in FIG. Figure 2 As shown, the interactive information processing device 100 of the mobile transparent display device provided in the embodiment of the present application includes: The acquisition module 110 is used to acquire ambient light data information, position information of the mobile transparent display device, and display parameter information of the mobile transparent display device.

[0037] The first generating module 120 is configured to generate display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information includes a brightness adjustment coefficient and a color correction matrix.

[0038] The second generating module 130 is used to generate a pixel projection matrix based on the posture information.

[0039] The correction module 140 is used to correct the color of the image to be displayed based on the color correction matrix to obtain a target image to be displayed.

[0040] The adjustment module 150 is used to adjust the display parameters of the mobile transparent display device based on the brightness adjustment coefficient, and display the target image to be displayed on the mobile transparent display device based on the pixel projection matrix.

[0041] It should be noted that technicians in the relevant technical field can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described device and each module can refer to the corresponding process in the embodiment of the interactive information processing method of the aforementioned mobile transparent display device, and will not be repeated here.

[0042] The interactive information processing device 100 of the mobile transparent display device provided in the above embodiment can be implemented in the form of a computer program. The computer program can be used in Figure 3 The system is run on the terminal device 200 shown.

[0043] See also Figure 3 , Figure 3 The present invention provides a schematic block diagram of the structure of a terminal device 200 according to an embodiment of the present application. The terminal device 200 includes a processor 201 and a memory 202. The processor 201 and the memory 202 are connected via a device bus 203, wherein the memory 202 may include a non-volatile storage medium and an internal memory.

[0044] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 201, the processor 201 can execute any of the above-mentioned interactive information processing methods for a mobile transparent display device.

[0045] The processor 201 is used to provide computing and control capabilities to support the operation of the entire terminal device 200.

[0046] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 201, the processor 201 can execute any of the above-mentioned interactive information processing methods of the mobile transparent display device.

[0047] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the terminal device 200 involved in the scheme of the present application. The specific terminal device 200 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0048] It should be understood that the processor 201 may be a central processing unit (CPU), and the processor 201 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0049] In some embodiments, the processor 201 is used to run a computer program stored in the memory to implement the following steps: Acquire ambient light data information, position information of the mobile transparent display device, and display parameter information of the mobile transparent display device; Generate display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information includes a brightness adjustment coefficient and a color correction matrix; Generate a pixel projection matrix based on the pose information; Correcting the color of the image to be displayed based on the color correction matrix to obtain a target image to be displayed; The display parameters of the mobile transparent display device are adjusted based on the brightness adjustment coefficient, and the target image to be displayed is displayed on the mobile transparent display device based on the pixel projection matrix.

[0050] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the terminal device 200 described above can refer to the corresponding process of the interactive information processing method of the aforementioned mobile transparent display device, and will not be repeated here.

[0051] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement the interactive information processing method of the mobile transparent display device provided in the embodiment of the present application.

[0052] The computer-readable storage medium may be an internal storage unit of the terminal device 200 in the aforementioned embodiment, such as a hard disk or memory of the terminal device 200. The computer-readable storage medium may also be an external storage device of the terminal device 200, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the terminal device 200.

[0053] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for processing interactive information of a mobile transparent display device, characterized in that: include: Acquire ambient light data information, position information of the mobile transparent display device, and display parameter information of the mobile transparent display device; Generate display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information includes a brightness adjustment coefficient and a color correction matrix; Generate a pixel projection matrix based on the pose information; Correcting the color of the image to be displayed based on the color correction matrix to obtain a target image to be displayed; The display parameters of the mobile transparent display device are adjusted based on the brightness adjustment coefficient, and the target image to be displayed is displayed on the mobile transparent display device based on the pixel projection matrix.

2. The interactive information processing method of the mobile transparent display device according to claim 1, characterized in that: The ambient light data information includes ambient light brightness and ambient light color temperature, the display parameter information includes display brightness and display color temperature, and the generating of display parameter adjustment information based on the ambient light data information and the display parameter information includes: generating a brightness adjustment coefficient based on the ambient light brightness and the display brightness; A color correction matrix is ​​generated based on the ambient light color temperature and the display color temperature.

3. The interactive information processing method of the mobile transparent display device according to claim 2, characterized in that: The generating a brightness adjustment coefficient based on the ambient light brightness and the display brightness includes: The light adaptation brightness is calculated by formula (1); where is the brightness of light adaptation, is the reference brightness, is the ambient light brightness; （1）； The brightness adjustment coefficient is calculated by formula (2); where: is the brightness adjustment coefficient, To display brightness; （2） 。 4. The interactive information processing method of the mobile transparent display device according to claim 2, characterized in that: The generating a color correction matrix based on the ambient light color temperature and the display color temperature comprises: The ratio of the ambient light color temperature to the display color temperature is used as the red channel correction coefficient ; Determine the green channel correction factor is 1; The ratio of the display color temperature to the ambient light color temperature is used as the blue channel correction coefficient ; Based on the red channel correction factor , Green channel correction factor and the blue channel correction factor Generate a color correction matrix as shown in formula (3); （3）。 5. The interactive information processing method of a mobile transparent display device according to claim 1, characterized in that: The posture information includes the position coordinates (x, y, z) and attitude angles α, β, and γ of the transparent display device, α is the roll angle of the mobile transparent display device around the x-axis, β is the pitch angle of the mobile transparent display device around the y-axis, and γ is the yaw angle of the mobile transparent display device around the z-axis. The pixel projection matrix is ​​generated based on the posture information, including: Generate a translation vector based on the position coordinates (x, y, z) ; Generate the basic rotation matrix corresponding to the x-axis through formula (4) ; （4） Generate the basic rotation matrix corresponding to the y-axis through formula (5) ; （5） Generate the basic rotation matrix corresponding to the z-axis through formula (6) ; （6） Will , , and Multiply them in sequence to obtain the pixel projection matrix.

6. The interactive information processing method of a mobile transparent display device according to claim 1, characterized in that: The displaying the target image to be displayed on the mobile transparent display device based on the pixel projection matrix includes: For each target pixel of the target image to be displayed, the initial two-dimensional pixel coordinates corresponding to the target pixel on the display screen of the mobile transparent display device are determined, and the initial two-dimensional pixel coordinates are projected and transformed based on the pixel projection matrix to obtain the target two-dimensional pixel coordinates, and the target pixel is displayed on the display screen based on the target two-dimensional pixel coordinates.

7. The interactive information processing method of the mobile transparent display device according to claim 6, characterized in that: The projecting transformation of the initial two-dimensional pixel coordinates based on the pixel projection matrix to obtain target two-dimensional pixel coordinates includes: Constructing an inverse matrix of the pixel projection matrix, and constructing a two-dimensional homogeneous vector corresponding to the initial two-dimensional pixel coordinates; Multiplying the inverse matrix by the homogeneous vector to obtain the three-dimensional space coordinates corresponding to the initial two-dimensional pixel coordinates; Multiplying the pixel projection matrix by the vector corresponding to the three-dimensional space coordinates to obtain a two-dimensional homogeneous vector corresponding to the target two-dimensional pixel coordinates; The target two-dimensional pixel coordinates are generated based on the two-dimensional homogeneous vector corresponding to the target two-dimensional pixel coordinates.

8. An interactive information processing device for a mobile transparent display device, characterized in that: include: An acquisition module, used to acquire ambient light data information, position information of the mobile transparent display device, and display parameter information of the mobile transparent display device; A first generating module, configured to generate display parameter adjustment information based on the ambient light data information and the display parameter information; wherein the display parameter adjustment information includes a brightness adjustment coefficient and a color correction matrix; A second generation module, used to generate a pixel projection matrix based on the posture information; A correction module, used for correcting the color of the image to be displayed based on the color correction matrix to obtain a target image to be displayed; An adjustment module is used to adjust the display parameters of the mobile transparent display device based on the brightness adjustment coefficient, and display the target image to be displayed on the mobile transparent display device based on the pixel projection matrix.

Images