User interface rendering method and device, storage medium and chip

Abstract

The invention relates to a user interface rendering method and device, a storage medium and a chip, and relates to the technical field of image data processing.The method comprises the steps that first image depth information corresponding to current user scene data is obtained, and a system interface image is obtained according to the first image depth information and current system interface data; and synthesizing the system interface image and a user scene image drawn based on the current user scene data to obtain a user interface. According to the method, data interference between the current system interface image and the current user scene image can be avoided, so that the current user scene image and the current system interface image can be smoothly synthesized together, a more natural user interface in which all data are fused together is obtained, and the user experience is improved. Notification information of a system interface can be displayed, and interference of the notification information on a user scene interface can be avoided as much as possible.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image data processing, and particularly to a user interface rendering method, apparatus, storage medium, and chip. Background Art

[0002] When a user opens a user scene interface, such as during an XR game, the system may receive notification information. For the display of this notification information, in the related art, the first method is to simply display the notification information on the system interface, and the corresponding notification information cannot be seen on the user scene interface. The user can only return to the system interface to view it. The second method is to display the notification information on the system interface over the entire user scene interface, which cannot be integrated with the current user application and appears as a piece of cloth covering the user application, causing occlusion of the user scene interface. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a user interface rendering method, apparatus, storage medium, and chip. By obtaining the first image depth information corresponding to the current user scene data, and based on the first image depth information and the current system interface data, obtaining a system interface image, and then synthesizing the system interface image and a user scene image drawn based on the current user scene data to obtain a user interface. By obtaining the first image depth information corresponding to the current user scene data and combining it with the current system interface data to draw the current system interface image, data interference between the current system interface image and the current user scene image can be avoided, so that the current user scene image and the current system interface image can be successfully synthesized together to obtain a more natural user interface in which all data is integrated, which can not only display the notification information on the system interface but also minimize the interference of the notification information on the user scene interface.

[0004] According to a first aspect of an embodiment of the present disclosure, a user interface rendering method is provided, including:

[0005] Obtaining first image depth information corresponding to current user scene data;

[0006] Based on the first image depth information and current system interface data, obtaining a system interface image;

[0007] Synthesizing the system interface image and a user scene image drawn based on the current user scene data to obtain the user interface.

[0008] Optionally, the obtaining of the first image depth information corresponding to the current user scene data includes:

[0009] Executing through a system interface process:

[0010] Obtain target user scenario data for a user scenario process to draw a user scenario image, and original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process;

[0011] Based on the target user scenario data, correct the original depth information to obtain first image depth information corresponding to the target scenario image, where the target scenario image is the user scenario image to be drawn by the user scenario process based on the current user scenario data.

[0012] Optionally, the target user scenario data includes first scenario data and second scenario data, where the first scenario data is the user scenario data corresponding to the most recent frame of the user scenario image drawn by the user scenario process, and the second scenario data is the current user scenario data;

[0013] The correcting the original depth information based on the target user scenario data to obtain first image depth information corresponding to the target scenario image includes:

[0014] Based on the first scenario data and the second scenario data, obtain an offset matrix;

[0015] According to the offset matrix, correct the original depth information to obtain first image depth information corresponding to the target scenario image.

[0016] Optionally, the system interface data includes pixel values corresponding to a plurality of first pixel points and second image depth information, where the second image depth information includes depth values corresponding to each first pixel point;

[0017] The obtaining a system interface image according to the first image depth information and current system interface data includes:

[0018] Execute through the system interface process:

[0019] Obtain second image depth information corresponding to the current system interface data;

[0020] Based on the first image depth information and the second image depth information, determine target pixel points from the plurality of first pixel points;

[0021] Based on the pixel values and depth values corresponding to the target pixel points, obtain the system interface image.

[0022] Optionally, the first image depth information includes depth values corresponding to a plurality of second pixel points;

[0023] Determining a target pixel point from the multiple first pixel points based on the first image depth information and the second image depth information includes:

[0024] Obtaining a first position coincidence point of the first pixel point and the second pixel point;

[0025] Determining the depth values of the first pixel point and the second pixel point at each first position coincidence point according to the first image depth information and the second image depth information;

[0026] Determining the first position coincidence point corresponding to the depth value of the first pixel point being less than that of the second pixel point as the target pixel point.

[0027] Optionally, the first image depth information includes depth values corresponding to multiple second pixel points;

[0028] Determining a target pixel point from the multiple first pixel points based on the first image depth information and the second image depth information includes:

[0029] Obtaining scene pixel points corresponding to the scene objects in the second pixel points;

[0030] Taking the least coincidence of the positions of the first pixel point and the scene pixel points as the target, performing an overall offset on the first pixel point to obtain offset pixel points;

[0031] Obtaining a second position coincidence point of the offset pixel points and the second pixel points;

[0032] Determining the depth values of the offset pixel points and the second pixel points at each second position coincidence point according to the first image depth information and the second image depth information;

[0033] Determining the second position coincidence point corresponding to the depth value of the offset pixel point being greater than that of the second pixel point as the target pixel point.

[0034] Optionally, obtaining target user scene data for the user scene process to draw a user scene image, and original depth information corresponding to the most recent frame of the user scene image drawn by the user scene process includes:

[0035] Obtaining, through a system sharing module, target user scene data for the user scene process to draw a user scene image, and original depth information corresponding to the most recent frame of the user scene image drawn by the user scene process.

[0036] According to a second aspect of the embodiments of the present disclosure, there is provided a user interface rendering device, including:

[0037] An acquisition module, configured to acquire first image depth information corresponding to current user scenario data;

[0038] An obtaining module, configured to obtain a system interface image according to the first image depth information and current system interface data;

[0039] A synthesis module, configured to synthesize the system interface image and a user scenario image drawn based on the current user scenario data to obtain the user interface.

[0040] According to a third aspect of the embodiments of the present disclosure, there is provided a user interface rendering device, including:

[0041] A processor;

[0042] A memory for storing processor-executable instructions;

[0043] Wherein, the processor is configured to implement the steps of the user interface rendering method provided in the first aspect of the present disclosure when executed.

[0044] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the user interface rendering method provided in the first aspect of the present disclosure are implemented.

[0045] According to a fifth aspect of the embodiments of the present disclosure, there is provided a chip, including a processor and an interface; the processor is used to read instructions to execute the user interface rendering method provided in the first aspect of the present disclosure.

[0046] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0047] By acquiring the first image depth information corresponding to the current user scenario data, and obtaining a system interface image according to the first image depth information and the current system interface data, and then synthesizing the system interface image and a user scenario image drawn based on the current user scenario data to obtain the user interface. By acquiring the first image depth information corresponding to the current user scenario data and combining the current system interface data to draw the current system interface image, so as to avoid data interference between the current system interface image and the current user scenario image, so that the current user scenario image and the current system interface image can be successfully synthesized together to obtain a more natural user interface in which all data is integrated, which can not only display the notification information of the system interface, but also avoid the interference of the notification information on the user scenario interface as much as possible.

[0048] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. Description of the Drawings

[0049] The drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure.

[0050] Figure 1 It is a schematic diagram of a method for displaying a user interface shown according to an exemplary embodiment.

[0051] Figure 2 It is a schematic diagram of another method for displaying a user interface shown according to an exemplary embodiment.

[0052] Figure 3 It is a schematic diagram of a method for rendering a user interface shown according to an exemplary embodiment.

[0053] Figure 4 It is a flowchart of a method for rendering a user interface shown according to an exemplary embodiment.

[0054] Figure 5 It is a flowchart of a method for obtaining depth information of a first image shown according to an exemplary embodiment.

[0055] Figure 6 It is a flowchart of a method for correcting original depth information shown according to an exemplary embodiment.

[0056] Figure 7 It is a flowchart of a method for obtaining a system interface image shown according to an exemplary embodiment.

[0057] Figure 8 It is a schematic diagram of another method for rendering a user interface shown according to an exemplary embodiment.

[0058] Figure 9 It is a schematic diagram of steps of a method for rendering a user interface shown according to an exemplary embodiment.

[0059] Figure 10 It is a block diagram of a device for rendering a user interface shown according to an exemplary embodiment.

[0060] Figure 11 It is a block diagram of a device for user interface rendering shown according to an exemplary embodiment. Detailed Description of the Invention

[0061] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0062] It should be noted that all actions of obtaining signals, information, or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where the location is located and obtaining authorization from the owner of the corresponding device.

[0063] Figure 1 is a schematic diagram of a method for displaying a user interface shown according to an exemplary embodiment. Figure 2 is a schematic diagram of another method for displaying a user interface shown according to an exemplary embodiment. When the user opens the user scenario interface, for example, when playing an XR game, the system may receive notification information. For the display of this notification information, in the related art, the first method is to simply display the notification information on the system interface, and the corresponding notification information cannot be seen on the user scenario interface. The user can only return to the system interface to see it, as shown in Figure 1 shown. The second method is to display the notification information on the system interface over the entire user scenario interface, which cannot be integrated with the current user application. It looks like a piece of cloth covering the user application, causing occlusion to the user scenario interface, as shown in Figure 2 shown.

[0064] Figure 3 is a schematic diagram of a method for rendering a user interface shown according to an exemplary embodiment. As shown in Figure 3 shown, to overcome the problems existing in the related art, the present disclosure provides a method, apparatus, storage medium, and chip for rendering a user interface. By obtaining the first image depth information corresponding to the current user scenario data, and based on the first image depth information and the current system interface data, obtaining a system interface image, and then synthesizing the system interface image and the user scenario image drawn based on the current user scenario data to obtain a user interface. By obtaining the first image depth information corresponding to the current user scenario data and combining the current system interface data to draw the current system interface image, data interference between the current system interface image and the current user scenario image can be avoided, so that the current user scenario image and the current system interface image can be smoothly synthesized together to obtain a more natural user interface in which all data is integrated, which can not only display the notification information on the system interface but also minimize the interference of the notification information on the user scenario interface.

[0065] Figure 4 is a flowchart of a user interface rendering method shown according to an exemplary embodiment. As Figure 4 shown, this method can be used in a terminal and may include the following steps.

[0066] In step S401, obtain the first image depth information corresponding to the current user scene data.

[0067] In this embodiment, for the user interface, it is necessary to obtain the current user scene image and the current system interface image to form the current user interface. Therefore, it is necessary to first draw the user scene image and the system interface image. Among them, for the user scene image, it is necessary to obtain the user scene data and draw the user scene image based on the user scene data. For the system interface image, it is necessary to obtain the system interface data and then draw the system interface image based on the system interface data. Among them, the user scene data can be the data corresponding to the application currently used by the user. For example, for an XR (Extended Reality) game, the user scene data can be the pose of the worn props, etc. The system interface data can be the data corresponding to notification information, etc.

[0068] Among them, if the user scene image and the system interface image are drawn using the same process, after the current user scene image is drawn using the current user scene data, the image depth information corresponding to the current user scene image is known, then the image depth information corresponding to the current user scene image drawn using the current user scene data can be directly obtained, that is, the first image depth information corresponding to the current user scene data can be directly obtained.

[0069] If the user scene image and the system interface image are drawn using different processes, the two processes perform image drawing in parallel. Before the current system interface image is drawn, the current user scene image is not drawn either. Therefore, the first image depth information corresponding to the current user scene data cannot be directly obtained, and the first image depth information corresponding to the current user scene data can be determined based on the image depth information corresponding to the last drawn frame of the user scene image. For example, the image depth information corresponding to the previous frame of the user scene image can be offset to obtain the first image depth information corresponding to the current user scene data.

[0070] In step S402, obtain the system interface image according to the first image depth information and the current system interface data.

[0071] In this embodiment, based on the current system interface data, the image depth information corresponding to the current system interface data can be determined. And based on the first image depth information and the image depth information corresponding to the current system interface data, it can be determined which data in the system interface data needs to be drawn and which data is covered. Thus, the system interface image corresponding to the current system interface data can be obtained, and interference between the system interface image and the user scene image can be avoided.

[0072] In step S403, the system interface image and the user scene image drawn based on the current user scene data are synthesized to obtain a user interface.

[0073] In this embodiment, based on the current user scene data, the current user scene image can be drawn, and the current user scene image is synthesized with the current system interface image, so that a more natural user interface in which the user scene data and the system interface data are integrated can be obtained. Moreover, since the system interface image is obtained based on the first image depth information and the current system interface data, the data of the system interface can be displayed through the user interface, and interference between the system interface image and the user scene image is also avoided.

[0074] Figure 5 is a flowchart of a method for obtaining first image depth information shown according to an exemplary embodiment, as Figure 5 shown. In a possible implementation, to obtain the first image depth information corresponding to the current user scene data, the following steps executed by the system interface process may be included:

[0075] In step S501, the target user scene data for the user scene process to draw the user scene image and the original depth information corresponding to the most recent frame of the user scene image drawn by the user scene process are obtained.

[0076] In this embodiment, the user scene image is drawn by the user scene process, and the system interface image is drawn by the system interface process. The target user scene data for the user scene process to draw the user scene image can be obtained through the system interface process. Moreover, at this time, the user scene process has completed the drawing of the most recent frame of the user scene image. Therefore, at this time, the depth information corresponding to the most recent frame of the user scene image is known, and the original depth information corresponding to the most recent frame of the user scene image drawn by the user scene process can be directly obtained.

[0077] In step S502, based on the target user scene data, the original depth information is corrected to obtain the first image depth information corresponding to the target scene image, where the target scene image is the user scene image to be drawn by the user scene process based on the current user scene data.

[0078] In this embodiment, the target scene image is the user scene image to be drawn from the current user scene data, that is, the current user scene image to be drawn. Since this current user scene image has not been drawn yet and the first image depth information corresponding to the current user scene image cannot be directly obtained, the original depth information corresponding to the previous frame of the user scene image can be corrected to obtain the first image depth information corresponding to the target scene image.

[0079] Figure 6 is a flowchart of a method for correcting original depth information shown according to an exemplary embodiment, as Figure 6 shown. In a possible implementation, the target user scene data includes first scene data and second scene data. The first scene data is the user scene data corresponding to the most recent frame of the user scene image drawn by the user scene process, and the second scene data is the current user scene data.

[0080] Based on the target user scene data, correcting the original depth information to obtain the first image depth information corresponding to the target scene image may include the following steps:

[0081] In step S601, an offset matrix is obtained based on the first scene data and the second scene data.

[0082] In this embodiment, the first scene data and the second scene data are different representations of the same batch of data at different times, that is, multiple sub-data in the first scene data correspond one-to-one with multiple sub-data in the second scene data. An offset between each sub-data in the second scene data and the corresponding sub-data in the first scene data can be obtained based on the first scene data and the second scene data to obtain the offset matrix.

[0083] In step S602, the original depth information is corrected according to the offset matrix to obtain the first image depth information corresponding to the target scene image.

[0084] In this embodiment, the original depth information can be offset based on the offset matrix to obtain the first image depth information corresponding to the target scene image, that is, the image depth information corresponding to the current user scene image to be drawn.

[0085] Figure 7 is a flowchart of a method for obtaining a system interface image shown according to an exemplary embodiment, as Figure 7 shown. In a possible implementation, the system interface data includes pixel values corresponding to multiple first pixel points and second image depth information, and the second image depth information includes depth values corresponding to each first pixel point.

[0086] Based on the first image depth information and the current system interface data, a system interface image is obtained, which may include the following steps executed by the system interface process:

[0087] In step S701, the second image depth information corresponding to the current system interface data is obtained.

[0088] In this embodiment, based on the current system interface data, the second image depth information corresponding to the current system interface data can be determined.

[0089] In step S702, based on the first image depth information and the second image depth information, target pixel points are determined from multiple first pixel points.

[0090] In this embodiment, the target pixel points are the pixel points that need to be assigned values. The target pixel points that need to be assigned values can be determined from multiple first pixel points by comparing the first image depth information and the second image depth information.

[0091] In step S703, based on the pixel value and the depth value corresponding to the target pixel point, a system interface image is obtained.

[0092] In this embodiment, based on the pixel value corresponding to the target pixel point, each target pixel point can be assigned a pixel value, and a system interface image can be rendered according to the depth value of each target pixel point.

[0093] In a possible implementation manner, the first image depth information includes the depth values corresponding to multiple second pixel points. The method for determining target pixel points from multiple first pixel points based on the first image depth information and the second image depth information may be: obtaining the first position coincidence points between the first pixel points and the second pixel points; determining the depth values of the first pixel points and the second pixel points at each first position coincidence point according to the first image depth information and the second image depth information; and determining the first position coincidence points where the depth value of the first pixel point is less than the depth value of the second pixel point as the target pixel points.

[0094] In this embodiment, in order to avoid interference between the user scene image and the system interface image, when rendering the system interface image, based on the first image depth information and the second image depth information, assignable points can be selected from multiple first pixel points corresponding to the system interface data for rendering the system interface image. Among them, the assignable points can be the first position coincidence points that coincide between the first pixel points and the second pixel points, and the first position coincidence points corresponding to the depth value of the first pixel point being less than the depth value of the second pixel point. At the same position coincidence point, the smaller the depth value corresponding to the pixel point, the higher it is. For example, at any position coincidence point, if the depth value of the first pixel point is less than the depth value of the second pixel point, then at this position coincidence point, the first pixel point is above the second pixel point, that is, the first pixel point will cover the second pixel point, and this position coincidence point can be used as the rendering point of the system interface image. If the depth value of the first pixel point is greater than the depth value of the second pixel point, then at this position coincidence point, the first pixel point is below the second pixel point, that is, the first pixel point will be covered by the second pixel point. Therefore, in order to avoid interference between the system interface data and the user scene data at this position coincidence point, the first pixel point at this position coincidence point can be discarded and not used for rendering the system interface image.

[0095] Figure 8 is a schematic diagram of another user interface rendering method shown according to an exemplary embodiment, as Figure 8 shown, in a possible implementation, the first image depth information includes depth values corresponding to multiple second pixel points. The method for determining the target pixel points from multiple first pixel points based on the first image depth information and the second image depth information can be: obtaining the scene pixel points corresponding to the scene objects in the second pixel points; taking the least coincidence of the positions of the first pixel points and the scene pixel points as the target, performing an overall offset on the first pixel points to obtain offset pixel points; obtaining the second position coincidence points between the offset pixel points and the second pixel points; determining the depth values of the offset pixel points and the second pixel points at each second position coincidence point according to the first image depth information and the second image depth information; and determining the second position coincidence points where the depth value of the offset pixel point is greater than the depth value of the second pixel point as the target pixel points.

[0096] In this embodiment, the first pixel points can be offset as a whole, that is, the system interface image to be drawn is offset as a whole so that the system interface image can be displayed at any position on the user interface. Among them, in order to reduce the interference between the system interface image and the scene objects in the user scene image, when offsetting the first pixel points as a whole, the scene pixel points corresponding to the scene objects in the second pixel points can be determined first, and then the first pixel points are offset as a whole with the goal of minimizing the coincidence of the positions of the first pixel points and the scene pixel points to obtain offset pixel points, so as to minimize the interference between the system interface image and the scene objects in the user scene image. And the target pixel points are determined according to the positional relationship and the depth value between the offset pixel points after offset and the second pixel points.

[0097] Among them, the assignable points can be multiple second position coincidence points among the offset pixel points and the second position coincidence points that coincide in the second pixel points, where the depth value of the offset pixel points is less than the depth value of the second pixel points. At the same position coincidence point, the smaller the depth value corresponding to the pixel point, the more above it is. For example, at any position coincidence point, if the depth value of the offset pixel point is less than the depth value of the second pixel point, then at this position coincidence point, the offset pixel point is above the second pixel point, that is, the offset pixel point will cover the second pixel point, and this position coincidence point can be used as the drawing point of the system interface image. If the depth value of the offset pixel point is greater than the depth value of the second pixel point, then at this position coincidence point, the offset pixel point is below the second pixel point, that is, the offset pixel point will be covered by the second pixel point. Therefore, in order to avoid the mutual interference between the system interface data and the user scene data at this position coincidence point, the offset pixel point at this position coincidence point can be discarded and not used for drawing the system interface image.

[0098] That is, the final obtained effect is as Figure 8 shown. On the user interface, the position occupied by the data corresponding to the system interface image is in the gap of the position occupied by the data corresponding to the scene objects in the user interface image, and there is a small amount of coverage or no coverage between the data of the system image and the data of the scene objects. Thus, the content of the system interface data can be displayed as much as possible without affecting the user scene data.

[0099] In a possible implementation manner, the terminal device may include a sharing module. The sharing module may store the acquired data or transmit it to a target process, that is, transmit it to the system interface process, based on the system shared memory of the terminal. The target user scenario data for the user scenario process to draw a user scenario image and the original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process may be acquired from the user scenario process through the system sharing module, and the acquired data may be transmitted to the system interface process through the system sharing module. Alternatively, the system interface process may acquire the target user scenario data for the user scenario process to draw a user scenario image and the original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process from the system sharing module. That is, through the sharing module, the transmission of data between different processes can be realized, and the target user scenario data for the user scenario process to draw a user scenario image and the original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process can be cross-process transmitted between the user scenario process and the system interface process through the system sharing module.

[0100] Figure 9 is a schematic diagram of the steps of a user interface rendering method shown according to an exemplary embodiment, as Figure 9 shown, the user scenario process and the system interface process process data in parallel. Among them, the user scenario process processes the user scenario data through a renderer to obtain a corresponding RGB (that is, the color representing the three channels of red, green, and blue) image and a corresponding depth information. The system sharing module acquires the target user scenario data for the user scenario process to draw a user scenario image and the original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process from the user scenario process, and transmits it to the system interface process. The system interface process corrects the original depth information based on the acquired target user scenario data to obtain the offset first image depth information, that is, the available depth information. The system interface process then renders based on the available depth information through a renderer to obtain a corresponding RGB image. The user scenario process transmits the user scenario image drawn from the current user scenario data to the system image synthesizer or the screen, and the system interface process transmits the current system interface image to the system image synthesizer or the screen, so that the system image synthesizer or the screen synthesizes the system interface image and the user scenario image drawn based on the current user scenario data to obtain a user interface.

[0101] Figure 10 is a block diagram of a user interface rendering device shown according to an exemplary embodiment. Referring to Figure 10 , the user interface rendering device 1000 includes an acquisition module 1001, an obtaining module 1002, and a synthesis module 1003.

[0102] The obtaining module 1001 is configured to obtain first image depth information corresponding to current user scenario data;

[0103] The obtaining module 1002 is configured to obtain a system interface image according to the first image depth information and current system interface data;

[0104] The synthesizing module 1003 is configured to synthesize the system interface image and a user scenario image drawn based on the current user scenario data to obtain the user interface.

[0105] Optionally, the obtaining module 1001 includes:

[0106] A first obtaining sub-module configured to obtain target user scenario data for a user scenario process to draw a user scenario image, and original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process;

[0107] A correction sub-module configured to correct the original depth information based on the target user scenario data to obtain first image depth information corresponding to a target scenario image, where the target scenario image is a user scenario image to be drawn by the user scenario process based on the current user scenario data.

[0108] Optionally, the target user scenario data includes first scenario data and second scenario data, where the first scenario data is user scenario data corresponding to the most recent frame of the user scenario image drawn by the user scenario process, and the second scenario data is the current user scenario data;

[0109] The correction sub-module includes:

[0110] A first obtaining unit configured to obtain an offset matrix based on the first scenario data and the second scenario data;

[0111] A second obtaining unit configured to correct the original depth information according to the offset matrix to obtain first image depth information corresponding to the target scenario image.

[0112] Optionally, the system interface data includes pixel values corresponding to a plurality of first pixel points and second image depth information, where the second image depth information includes depth values corresponding to each first pixel point;

[0113] The obtaining module 1002 includes:

[0114] A second obtaining sub-module configured to obtain second image depth information corresponding to current system interface data;

[0115] A determining sub-module, configured to determine target pixel points from the multiple first pixel points based on the first image depth information and the second image depth information;

[0116] An obtaining sub-module, configured to obtain the system interface image based on the pixel value and the depth value corresponding to the target pixel points.

[0117] Optionally, the first image depth information includes depth values corresponding to multiple second pixel points;

[0118] The determining sub-module includes:

[0119] A first obtaining unit, configured to obtain first position coincidence points among the first pixel points and the second pixel points;

[0120] A first determining unit, configured to determine the depth values of the first pixel points and the second pixel points at each first position coincidence point according to the first image depth information and the second image depth information;

[0121] A second determining unit, configured to determine the first position coincidence points where the depth value of the first pixel point is less than that of the second pixel point as the target pixel points.

[0122] Optionally, the first image depth information includes depth values corresponding to multiple second pixel points;

[0123] The determining sub-module includes:

[0124] A second obtaining unit, configured to obtain scene pixel points corresponding to scene objects among the second pixel points;

[0125] An offset unit, configured to perform an overall offset on the first pixel points with the goal of minimizing the position coincidence between the first pixel points and the scene pixel points, to obtain offset pixel points;

[0126] A third obtaining unit, configured to obtain second position coincidence points among the offset pixel points and the second pixel points;

[0127] A third determining unit, configured to determine the depth values of the offset pixel points and the second pixel points at each second position coincidence point according to the first image depth information and the second image depth information;

[0128] A fourth determining unit, configured to determine the second position coincidence points where the depth value of the offset pixel point is greater than that of the second pixel point as the target pixel points.

[0129] Optionally, the first obtaining sub-module includes:

[0130] A fourth acquisition unit, configured to acquire, through a system sharing module, target user scenario data for a user scenario process to draw a user scenario image, and original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process.

[0131] Regarding the user interface rendering device 1000 in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0132] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the user interface rendering method provided by the present disclosure are implemented.

[0133] Figure 11 It is a block diagram of a device for user interface rendering shown according to an exemplary embodiment. For example, the device 1100 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, etc.

[0134] Referring to Figure 11 , the device 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power supply component 1106, a multimedia component 1108, an audio component 1110, an input / output interface 1112, a sensor component 1114, and a communication component 1116.

[0135] The processing component 1102 generally controls the overall operation of the device 1100, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 1102 may include one or more processors 1120 to execute instructions to complete all or part of the steps of the above user interface rendering method. In addition, the processing component 1102 may include one or more modules to facilitate the interaction between the processing component 1102 and other components. For example, the processing component 1102 may include a multimedia module to facilitate the interaction between the multimedia component 1108 and the processing component 1102.

[0136] The memory 1104 is configured to store various types of data to support the operation of the device 1100. Examples of such data include instructions for any application or method operating on the device 1100, contact data, phone book data, messages, pictures, videos, etc. The memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0137] The power supply component 1106 provides power to various components of the device 1100. The power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1100.

[0138] The multimedia component 1108 includes a screen that provides an output interface between the device 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1108 includes a front camera and / or a rear camera. When the device 1100 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0139] The audio component 1110 is configured to output and / or input audio signals. For example, the audio component 1110 includes a microphone (MIC) that is configured to receive external audio signals when the device 1100 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 1104 or transmitted via the communication component 1116. In some embodiments, the audio component 1110 further includes a speaker for outputting audio signals.

[0140] The input / output interface 1112 provides an interface between the processing component 1102 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a start button, and a lock button.

[0141] The sensor assembly 1114 includes one or more sensors for providing a status assessment of various aspects of the device 1100. For example, the sensor assembly 1114 can detect the on / off state of the device 1100, the relative positioning of components, such as the display and keypad of the device 1100. The sensor assembly 1114 can also detect a change in the position of the device 1100 or a component of the device 1100, the presence or absence of user contact with the device 1100, the orientation or acceleration / deceleration of the device 1100, and the temperature change of the device 1100. The sensor assembly 1114 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1114 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1114 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0142] The communication component 1116 is configured to facilitate communication between the device 1100 and other devices in a wired or wireless manner. The device 1100 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1116 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1116 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0143] In an exemplary embodiment, the device 1100 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above user interface rendering method.

[0144] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1104 including instructions, and the above instructions can be executed by a processor 1120 of the device 1100 to complete the above user interface rendering method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0145] In addition to being an independent electronic device, the above device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip. The integrated circuit can be a single IC or a collection of multiple ICs. The chip can include, but is not limited to, the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip), etc. The above integrated circuit or chip can be used to execute executable instructions (or code) to implement the above user interface rendering method. The executable instructions can be stored in the integrated circuit or chip, or obtained from other devices or equipment. For example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instructions can be stored in the memory, and when the executable instructions are executed by the processor, the above user interface rendering method is implemented. Alternatively, the integrated circuit or chip can receive the executable instructions through the interface and transmit them to the processor for execution to implement the above user interface rendering method.

[0146] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above user interface rendering method when executed by the programmable device.

[0147] Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure aims to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only considered exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0148] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A user interface rendering method, characterized in that, Including: Obtaining first image depth information corresponding to current user scenario data; Obtaining a system interface image based on the first image depth information and current system interface data; Synthesizing the system interface image and a user scenario image drawn based on the current user scenario data to obtain the user interface.

2. The user interface rendering method according to claim 1, wherein the obtaining of the first image depth information corresponding to the current user scenario data includes: Executing through a system interface process: Obtaining target user scenario data for a user scenario process to draw a user scenario image, and original depth information corresponding to the most recent frame of the user scenario image drawn by the user scenario process; Based on the target user scenario data, correcting the original depth information to obtain first image depth information corresponding to a target scenario image, where the target scenario image is a user scenario image to be drawn by the user scenario process based on the current user scenario data.

3. The user interface rendering method according to claim 2, wherein The target user scenario data includes first scenario data and second scenario data, where the first scenario data is the user scenario data corresponding to the most recent frame of the user scenario image drawn by the user scenario process, and the second scenario data is the current user scenario data; the correcting of the original depth information based on the target user scenario data to obtain first image depth information corresponding to a target scenario image includes: Obtaining an offset matrix based on the first scenario data and the second scenario data; According to the offset matrix, correcting the original depth information to obtain first image depth information corresponding to the target scenario image.

4. The user interface rendering method according to claim 2, wherein The system interface data includes pixel values corresponding to a plurality of first pixel points and second image depth information, where the second image depth information includes depth values corresponding to each first pixel point; the obtaining of the system interface image based on the first image depth information and current system interface data includes: Executing through the system interface process: Obtaining second image depth information corresponding to current system interface data; Based on the first image depth information and the second image depth information, determining target pixel points from the plurality of first pixel points; Based on the pixel values and depth values corresponding to the target pixel points, obtaining the system interface image.

5. The user interface rendering method according to claim 4, wherein The first image depth information includes depth values corresponding to a plurality of second pixel points; the determining of the target pixel points from the plurality of first pixel points based on the first image depth information and the second image depth information includes: Obtaining first position coincidence points among the first pixel points and the second pixel points; According to the first image depth information and the second image depth information, determining depth values of the first pixel points and the second pixel points at each first position coincidence point; Determining the first position coincidence points where the depth value of the first pixel point is less than the depth value of the second pixel point as the target pixel points.

6. The user interface rendering method according to claim 4, characterized in that The first image depth information includes depth values corresponding to a plurality of second pixel points; Determining a target pixel point from the plurality of first pixel points based on the first image depth information and the second image depth information includes: Obtaining a scene pixel point corresponding to a scene object in the second pixel points; Taking the least coincidence of the positions of the first pixel points and the scene pixel points as a target, and performing an overall offset on the first pixel points to obtain offset pixel points; Obtaining a second position coincidence point between the offset pixel points and the second pixel points; Determining the depth values of the offset pixel points and the second pixel points at each second position coincidence point according to the first image depth information and the second image depth information; Determining the second position coincidence point where the depth value of the offset pixel point is greater than the depth value of the second pixel point as the target pixel point.

7. The user interface rendering method according to any one of claims 2 to 6, wherein Obtaining target user scene data for a user scene process to draw a user scene image, and original depth information corresponding to the most recent frame of the user scene image drawn by the user scene process includes: Obtaining, through a system sharing module, target user scene data for a user scene process to draw a user scene image, and original depth information corresponding to the most recent frame of the user scene image drawn by the user scene process.

8. A user interface rendering device, characterized in that, Including: An obtaining module configured to obtain first image depth information corresponding to current user scene data; An obtaining module configured to obtain a system interface image according to the first image depth information and current system interface data; A synthesizing module configured to synthesize the system interface image and a user scene image drawn based on the current user scene data to obtain the user interface.

9. A user interface rendering device, characterized in that, Including: A processor; A memory for storing processor-executable instructions; Wherein, the processor is configured to implement the steps of the user interface rendering method according to any one of claims 1 to 7 when executed.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, The program instructions implement the steps of the user interface rendering method according to any one of claims 1 to 7 when executed by the processor.

11. A chip, characterized in that, Including a processor and an interface; the processor is used to read instructions to execute the method for rendering a user interface according to any one of claims 1 to 7.

Images