Image drawing method, device, readable medium and electronic device

Abstract

The present disclosure relates to an image drawing method, device, readable medium and electronic device, including: obtaining first pose data; predicting second pose data corresponding to a moment after a first preset prediction time according to the first pose data; obtaining historical image frames and historical depth maps corresponding to the historical image frames; adjusting the historical image frames according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data. In this way, not only can the delay between the target image frame and the screen corresponding to the actual pose of the user when it is displayed in the display area of ​​the scene such as the AR glasses be reduced by predicting the second pose data, but also the target image frame drawn corresponding to the second pose data can be quickly obtained by adjusting the historical image frame, thereby reducing the first preset prediction time and improving the prediction accuracy of the target image frame.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to an image drawing method, device, readable medium and electronic device. Background Art

[0002] From the acquisition of real environment data to the display of display environment data, certain data calculation and data transmission are usually required. For example, the real environment picture displayed in AR glasses is obtained by acquiring display environment data, and then displayed in the AR glasses after certain data calculation and data transmission. The positioning of AR glasses is usually assisted by mobile terminals, that is, AR glasses send the collected data to the mobile terminal, which performs calculations, and then feeds back the calculated picture to be displayed to AR glasses for display. In this way, the transmission process may cause a delay in the display of the picture on the AR glasses, and since the calculation of the mobile terminal also takes time, the calculation time of the mobile terminal will also cause a certain degree of picture delay. In addition, after the posture is determined according to the collected data, the corresponding image frame needs to be drawn in the process of displaying, and drawing also takes a certain amount of time. In addition, in virtual reality scenes such as AR glasses, it is usually necessary to draw corresponding three-dimensional objects in the real scene to be displayed for display, and the process of drawing three-dimensional objects will also produce a certain delay, and the higher the requirements for the drawing effect, the more serious the delay. Summary of the invention

[0003] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, the present disclosure provides an image drawing method, the method comprising:

[0005] Get the first pose data;

[0006] Predicting, based on the first posture data, second posture data corresponding to a moment after a first preset prediction time;

[0007] Acquire a historical image frame and a historical depth map corresponding to the historical image frame, wherein the historical image frame is drawn according to historical posture data;

[0008] The historical image frame is adjusted according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data.

[0009] In a second aspect, the present disclosure provides an image drawing device, the device comprising:

[0010] A first acquisition module, used to acquire first position data;

[0011] A first prediction module, used to predict, based on the first posture data, second posture data corresponding to a moment after a first preset prediction time;

[0012] A second acquisition module is used to acquire a historical image frame and a historical depth map corresponding to the historical image frame, wherein the historical image frame is drawn according to the historical posture data;

[0013] The first drawing module is used to adjust the historical image frame according to the second posture data and the historical depth map to obtain a target image frame drawn according to the second posture data.

[0014] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0015] In a fourth aspect, the present disclosure provides an electronic device, including:

[0016] a storage device having a computer program stored thereon;

[0017] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0018] Through the above technical solution, after the second posture data is predicted based on the acquired first posture data, before determining to draw the image frame corresponding to the second posture data in the display area in the scene such as AR glasses, the historical image frame drawn corresponding to the historical posture data and the corresponding historical depth map can be obtained first, so that the historical image frame can be directly adjusted according to the change between the second posture data and the historical posture data through the historical depth map, and the target image frame corresponding to the second posture data can be quickly obtained, which greatly speeds up the speed of drawing the target image frame according to the second posture data. Not only can the delay between the target image frame and the screen corresponding to the actual posture of the user be reduced when the target image frame is displayed in the display area in the scene such as AR glasses by predicting the second posture data, but also the target image frame corresponding to the second posture data can be quickly obtained by adjusting the historical image frame without directly drawing the target image frame according to the second posture data, thereby reducing the first preset prediction time. When the prediction time is reduced, the prediction accuracy will also be improved accordingly, thereby improving the drawing accuracy of the target image frame.

[0019] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0021] Figure 1 The figure is a flowchart of an image drawing method according to an exemplary embodiment of the present disclosure.

[0022] Figure 2 is a flowchart of an image drawing method according to yet another exemplary embodiment of the present disclosure.

[0023] Figure 3 The flowchart of a method for determining a second bit setting prediction duration in an image drawing method according to another exemplary embodiment of the present disclosure is shown.

[0024] Figure 4 is a flowchart of an image drawing method according to yet another exemplary embodiment of the present disclosure.

[0025] Figure 5 The figure is a structural block diagram of an image drawing device according to an exemplary embodiment of the present disclosure.

[0026] Figure 6 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] Figure 1 FIG. 1 is a flowchart of an image drawing method according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the method includes steps 101 to 104.

[0034] In step 101, first position data is obtained.

[0035] The first pose data may be the current pose data acquired by any pose estimation sensor, for example, the pose data calculated by the IMU data acquired by the inertial measurement unit IMU (Inertial Measurement Unit), wherein the IMU data may include, for example, acceleration signals acquired by three accelerometers and angular velocity signals acquired by three gyroscopes, and the pose data may be determined by the IMU data. Alternatively, the pose data may be determined by an image signal acquired by a camera sensor such as a binocular camera or a depth camera. Alternatively, the pose data may be acquired by fusing the image signal acquired by the inertial measurement unit IMU and the camera sensor with the IMU data.

[0036] In step 102, second posture data corresponding to a moment after a first preset prediction time period is predicted based on the first posture data.

[0037] Among them, the first preset prediction duration can be pre-set, for example, predetermined, and the historical image frame is adjusted according to the second posture data and the historical depth map to obtain the duration required for the target image frame drawn according to the second posture data. For example, when determining the target image frame and drawing the target image frame in the display area of ​​a scene such as AR glasses, determining the target image frame and drawing it take time, which may cause the actual posture of the user of the AR glasses to change when the image determined according to the first posture data is displayed in the AR glasses.

[0038] After predetermining the time length required to adjust the historical image frame according to the second posture data and the historical depth map to obtain the target image frame drawn according to the second posture data, the time length is used as the first preset prediction time length to predict the second posture data after the first preset prediction time length based on the first posture data, so that when the target image frame determined according to the second posture data is displayed to the user, the posture data corresponding to the target image frame is closer to the actual posture data of the user.

[0039] Predicting the second posture data may be achieved through a preset prediction model, and the prediction model may be, for example, a Kalman filter model or other model capable of achieving a prediction function.

[0040] In step 103, a historical image frame and a historical depth map corresponding to the historical image frame are obtained, wherein the historical image frame is drawn according to the historical posture data.

[0041] The historical pose data may include all pose data corresponding to the historical image frames and historical depth maps obtained before the first pose data is obtained. The historical image frames corresponding to the historical pose data may be displayed image frames or undisplayed image frames. As long as the historical image frame has a corresponding historical depth map, the historical depth map is an image frame composed of the depth information corresponding to each pixel in the historical image frame.

[0042] In step 104, the historical image frame is adjusted according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data.

[0043] When the historical image frame and the historical depth map corresponding to the historical image frame are obtained, the historical image frame corresponding to the historical pose data can be adjusted accordingly based on the predicted changes between the second pose data and the historical pose data, so as to quickly draw the target image frame corresponding to the second pose data for display in the display area of ​​scenes such as AR glasses.

[0044] Through the above technical solution, after the second posture data is predicted based on the acquired first posture data, before determining to draw the image frame corresponding to the second posture data in the display area in the scene such as AR glasses, the drawn historical image frame corresponding to the historical posture data and the corresponding historical depth map can be obtained first, so that the historical image frame can be directly adjusted according to the change between the second posture data and the historical posture data through the historical depth map, and the target image frame corresponding to the second posture data can be quickly obtained, which greatly speeds up the speed of drawing the target image frame according to the second posture data. Therefore, not only can the delay between the target image frame and the screen corresponding to the actual posture of the user be reduced by predicting the second posture data, but also the target image frame corresponding to the second posture data can be quickly obtained by adjusting the historical image frame without directly drawing the target image frame according to the second posture data, thereby reducing the first preset prediction time. When the prediction time is reduced, the prediction accuracy will also be improved accordingly, thereby improving the drawing accuracy of the target image frame.

[0045] Figure 2 FIG. 1 is a flowchart of an image drawing method according to another exemplary embodiment of the present disclosure. Figure 2 As shown, the method further includes steps 201 to 203.

[0046] In step 201, third posture data and captured image data corresponding to the third posture data are obtained. The method for obtaining the third posture data may be the same as the method for obtaining the first posture data. The captured image data corresponding to the third posture data is also the image data captured when the third posture data is acquired. For example, when the third posture data is determined by IMU data acquired by an inertial measurement unit IMU and camera data captured by a camera, the captured image data may be the camera data.

[0047] In step 202, the fourth posture data corresponding to the moment after the second preset prediction time is predicted based on the third posture data. The process of predicting the fourth posture data after the second preset prediction time using the third posture data may be the same as the prediction method of predicting the second posture data based on the first posture data, or may be different. The prediction method is not limited in the present disclosure, as long as the prediction of the fourth posture data and the second posture data can be achieved.

[0048] The second predicted duration may be a preset calibration duration, or may be, for example, the time required to obtain the intermediate image frame by drawing according to the fourth posture data.

[0049] In step 203, an intermediate image frame and an intermediate depth map corresponding to the intermediate image frame are drawn according to the fourth pose data and the acquired image data, and the intermediate image frame and the intermediate depth map are determined as the historical image frame and the historical depth map.

[0050] That is, when the third pose data is obtained and the fourth pose data corresponding to the intermediate image frame when the drawing is completed is predicted according to the drawing time required, and then the intermediate image frame is drawn, the intermediate depth map corresponding to the intermediate image frame is also drawn. The intermediate depth map is the depth information corresponding to each pixel in the intermediate image frame.

[0051] Among them, the intermediate image frame and the intermediate depth map drawn will not actually be displayed in the display area of ​​the scene such as the AR glasses, but will be used as the historical image frame and the historical depth map to assist in determining the target image frame that should be drawn according to the second posture data.

[0052] The frequency of acquiring the third pose data and drawing the intermediate image frame and the intermediate depth map may be related to the frequency of acquiring the third pose data. For example, the frequency of acquiring the third pose data may be to acquire one third pose data every 1 ms. When the third pose data is acquired at the 0th ms moment, the fourth pose data is predicted based on the third pose data at the 0th ms moment, and the intermediate image frame and the intermediate depth map corresponding to the third pose data at the 0th ms moment are drawn; when the third pose data is acquired at the 1st ms moment, the fourth pose data is predicted based on the third pose data at the 1st ms moment, and the intermediate image frame and the intermediate depth map corresponding to the third pose data at the 1st ms moment are drawn; wherein, the time consumed for predicting the third pose data acquired every 1 ms and drawing the image frame may be greater than the interval for acquiring the third pose data, and the processing of the third pose data acquired every 1 ms may be processed in parallel.

[0053] Since the processing of the first pose data needs to be performed based on the historical image frame and the historical depth map, the first pose data can be obtained at the moment of drawing the first intermediate image frame and the first intermediate depth map. For example, the third pose data is obtained at the 0th ms moment, and the drawing of the intermediate image frame and the intermediate depth map corresponding to the third pose data obtained at the 0th ms moment is completed at the 7th ms moment, then the first pose data can be obtained from the 7th ms moment, and the first pose data and the third pose data can be obtained through the same device, that is, the pose data obtained at the 7th ms moment can be used as the 8th third pose data obtained from the 0th ms moment, or as the 1st first pose data obtained from the 7th ms moment.

[0054] When the intermediate image frame and the intermediate depth map have been drawn, the intermediate image frame and the intermediate depth map can be used to assist in adjusting the target image frame that should be drawn according to the second pose data, so as to display it in the display area of ​​scenes such as AR glasses based on the target image frame.

[0055] In a possible implementation, before determining the intermediate image frame and the intermediate depth map as the historical image frame and the historical depth map required for the first pose data, the drawing completion time corresponding to the intermediate image frame and the intermediate depth map can also be determined first; when the acquisition time of the first pose data is the same as the drawing completion time, the intermediate image frame and the intermediate depth map are determined as the historical image frame and the historical depth map. That is, for example, when the first pose data is acquired at the 7th ms, the intermediate image frame and the intermediate depth map completed at the 7th ms are determined as the historical image frame and the historical depth map to assist in the drawing of the target image frame corresponding to the first pose data acquired at the 7th ms. In this way, it can be ensured as much as possible that the fourth pose data corresponding to the historical image frame is close to the first pose data, and then the target image frame adjusted according to the historical depth map can be made more accurate.

[0056] Figure 3 FIG. 1 is a flowchart of a method for determining a second-position prediction duration in an image drawing method according to another exemplary embodiment of the present disclosure. Figure 3 As shown, the method includes steps 301 to 304.

[0057] In step 301, a first predicted duration required for rendering according to the collected image data is determined.

[0058] In step 302 , it is determined whether the first predicted duration is greater than the second predicted duration. If not, the process proceeds to step 303 . If yes, the process proceeds to step 304 .

[0059] In step 303, the first predicted duration is determined as the second preset predicted duration.

[0060] In step 304, the second predicted duration is determined as the second preset predicted duration.

[0061] Among them, the second prediction duration is a preset maximum prediction duration that can predict the third posture data. That is, when determining the second preset prediction duration, it can be determined based on the actual duration required for drawing the acquired image data, that is, the first prediction duration and the maximum prediction duration that can be predicted. In this way, the accuracy of the predicted fourth posture data can be guaranteed as much as possible.

[0062] Figure 4 FIG. 1 is a flowchart of an image drawing method according to another exemplary embodiment of the present disclosure. Figure 4 As shown, the method further includes step 401 and step 402.

[0063] In step 401, the position of each pixel in the historical image frame in the target image frame is determined according to the second pose data and the historical depth map.

[0064] In step 402, each pixel in the historical image frame is moved to a corresponding position in the target image frame, and the target image frame is inpainted to obtain the drawn target image frame. The target image frame may be inpainted by any image inpainting method.

[0065] Figure 5 FIG. 1 is a structural block diagram of an image drawing device according to an exemplary embodiment of the present disclosure. Figure 5 As shown, the device includes: a first acquisition module 10, used to acquire a first pose data; a first prediction module 20, used to predict, based on the first pose data, second pose data corresponding to a moment after a first preset prediction time; a second acquisition module 30, used to acquire a historical image frame and a historical depth map corresponding to the historical image frame, wherein the historical image frame is drawn based on the historical pose data; a first drawing module 40, used to adjust the historical image frame based on the second pose data and the historical depth map to obtain a target image frame drawn based on the second pose data.

[0066] Through the above technical solution, after the second posture data is predicted based on the acquired first posture data, before determining to draw the image frame corresponding to the second posture data in the display area in the scene such as AR glasses, the drawn historical image frame corresponding to the historical posture data and the corresponding historical depth map can be obtained first, so that the historical image frame can be directly adjusted according to the change between the second posture data and the historical posture data through the historical depth map, and the target image frame corresponding to the second posture data can be quickly obtained, which greatly speeds up the speed of drawing the target image frame according to the second posture data. Therefore, not only can the delay between the target image frame and the screen corresponding to the actual posture of the user be reduced by predicting the second posture data, but also the target image frame corresponding to the second posture data can be quickly obtained by adjusting the historical image frame without directly drawing the target image frame according to the second posture data, thereby reducing the first preset prediction time. When the prediction time is reduced, the prediction accuracy will also be improved accordingly, thereby improving the drawing accuracy of the target image frame.

[0067] In a possible implementation, the first preset prediction duration is the duration required for the first drawing module to adjust the historical image frame according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data.

[0068] In a possible embodiment, the device also includes: a third acquisition module, used to acquire third posture data and collected image data corresponding to the third posture data; a second prediction module, used to predict fourth posture data corresponding to a moment after a second preset prediction time period based on the third posture data; a second drawing module, used to draw an intermediate image frame and an intermediate depth map corresponding to the intermediate image frame based on the fourth posture data and the collected image data, and determine the intermediate image frame and the intermediate depth map as the historical image frame and the historical depth map.

[0069] In a possible implementation, the acquisition of historical image frames and historical depth maps corresponding to the historical image frames includes: a first determination submodule, used to determine the drawing completion time corresponding to the intermediate image frame and the intermediate depth map; and a second determination submodule, used to determine the intermediate image frame and the intermediate depth map as the historical image frame and the historical depth map when the acquisition time of the first pose data is the same as the drawing completion time.

[0070] In a possible implementation, the device also includes: a determination module, used to determine a first prediction duration required for drawing based on the acquired image data; a judgment module, used to determine the first prediction duration as the second preset prediction duration when the first prediction duration is not greater than the second prediction duration; the judgment module is also used to determine the second prediction duration as the second preset prediction duration when the first prediction duration is greater than the second prediction duration; wherein the second prediction duration is a pre-set maximum prediction duration that can predict the third posture data.

[0071] In a possible implementation, the first drawing module 40 includes: a third determination submodule, used to determine the position of each pixel in the historical image frame in the target image frame according to the second pose data and the historical depth map; a drawing submodule, used to move each pixel in the historical image frame to the corresponding position in the target image frame, and complete the target image frame to obtain the drawn target image frame.

[0072] Reference below Figure 6 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0073] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0074] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0075] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0076] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0077] In some embodiments, any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol) can be used for communication, and can be interconnected with any form or medium of digital data communication (e.g., communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), internets (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0078] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0079] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the first pose data; predicts the second pose data corresponding to the moment after the first preset prediction time length according to the first pose data; obtains the historical image frame and the historical depth map corresponding to the historical image frame, and the historical image frame is drawn according to the historical pose data; adjusts the historical image frame according to the second pose data and the historical depth map to obtain the target image frame drawn according to the second pose data.

[0080] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0081] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0082] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of the module does not limit the module itself in some cases. For example, the first acquisition module may also be described as a "module for acquiring the first pose data".

[0083] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0084] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0085] According to one or more embodiments of the present disclosure, Example 1 provides an image drawing method, the method comprising:

[0086] Get the first pose data;

[0087] Predicting, based on the first posture data, second posture data corresponding to a moment after a first preset prediction time;

[0088] Acquire a historical image frame and a historical depth map corresponding to the historical image frame, wherein the historical image frame is drawn according to historical posture data;

[0089] The historical image frame is adjusted according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data.

[0090] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the first preset prediction duration is to adjust the historical image frame according to the second pose data and the historical depth map to obtain the duration required for the target image frame drawn according to the second pose data.

[0091] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 1, wherein the method further includes:

[0092] Acquire third posture data and collected image data corresponding to the third posture data;

[0093] Predicting fourth posture data corresponding to a moment after a second preset prediction time period according to the third posture data;

[0094] An intermediate image frame and an intermediate depth map corresponding to the intermediate image frame are obtained by drawing according to the fourth posture data and the acquired image data, and the intermediate image frame and the intermediate depth map are determined as the historical image frame and the historical depth map.

[0095] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 3, wherein acquiring a historical image frame and a historical depth map corresponding to the historical image frame comprises:

[0096] Determining the rendering completion time corresponding to the intermediate image frame and the intermediate depth map;

[0097] In a case where the acquisition time of the first pose data is the same as the drawing completion time, the intermediate image frame and the intermediate depth map are determined as the historical image frame and the historical depth map.

[0098] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 3, wherein the method further includes:

[0099] Determining a first predicted duration required for rendering according to the collected image data;

[0100] If the first predicted duration is not greater than the second predicted duration, determining the first predicted duration as the second preset predicted duration;

[0101] When the first predicted duration is greater than the second predicted duration, determining the second predicted duration as the second preset predicted duration;

[0102] Among them, the second prediction duration is a pre-set maximum prediction duration that can predict the third posture data.

[0103] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 1, wherein adjusting the historical image frame according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data includes:

[0104] Determine the position of each pixel in the historical image frame in the target image frame according to the second pose data and the historical depth map;

[0105] Each pixel in the historical image frame is moved to a corresponding position in the target image frame, and the target image frame is completed to obtain the drawn target image frame.

[0106] According to one or more embodiments of the present disclosure, Example 7 provides an image drawing device, the device comprising:

[0107] A first acquisition module, used to acquire first position data;

[0108] A first prediction module, used to predict, based on the first posture data, second posture data corresponding to a moment after a first preset prediction time;

[0109] A second acquisition module is used to acquire a historical image frame and a historical depth map corresponding to the historical image frame, wherein the historical image frame is drawn according to the historical posture data;

[0110] The first drawing module is used to adjust the historical image frame according to the second posture data and the historical depth map to obtain a target image frame drawn according to the second posture data.

[0111] According to one or more embodiments of the present disclosure, Example 8 provides the device of Example 7, wherein the first preset prediction duration is the duration required for the first drawing module to adjust the historical image frame according to the second pose data and the historical depth map to obtain the target image frame drawn according to the second pose data.

[0112] According to one or more embodiments of the present disclosure, Example 9 provides a computer-readable medium having a computer program stored thereon, which implements the steps of any of the methods described in Examples 1-6 when executed by a processing device.

[0113] According to one or more embodiments of the present disclosure, Example 10 provides an electronic device, including:

[0114] a storage device having a computer program stored thereon;

[0115] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in any one of Examples 1-6.

[0116] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0117] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0118] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. An image drawing method, characterized in that: The method comprises: Get the first pose data; Predicting, based on the first posture data, second posture data corresponding to a moment after a first preset prediction time; Acquire a historical image frame and a historical depth map corresponding to the historical image frame, wherein the historical image frame is drawn according to historical posture data; According to the second posture data and the historical depth map, the historical image frame is adjusted to obtain a target image frame drawn according to the second posture data; The adjusting the historical image frame according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data includes: Determine the position of each pixel in the historical image frame in the target image frame according to the second pose data and the historical depth map; Each pixel in the historical image frame is moved to a corresponding position in the target image frame, and the target image frame is completed to obtain the drawn target image frame.

2. The method according to claim 1, characterized in that: The first preset prediction duration is the duration required for adjusting the historical image frame according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data.

3. The method according to claim 1, characterized in that: The method further comprises: Acquire third posture data and collected image data corresponding to the third posture data; Predicting fourth posture data corresponding to a moment after a second preset prediction time period according to the third posture data; An intermediate image frame and an intermediate depth map corresponding to the intermediate image frame are obtained according to the fourth posture data and the acquired image data, and the intermediate image frame and the intermediate depth map are determined as the historical image frame and the historical depth map.

4. The method according to claim 3, characterized in that The acquiring of the historical image frame and the historical depth map corresponding to the historical image frame comprises: Determining the rendering completion time corresponding to the intermediate image frame and the intermediate depth map; In a case where the acquisition time of the first pose data is the same as the drawing completion time, the intermediate image frame and the intermediate depth map are determined as the historical image frame and the historical depth map.

5. The method according to claim 3, characterized in that: The method further comprises: Determining a first predicted duration required for rendering according to the collected image data; If the first predicted duration is not greater than the second predicted duration, determining the first predicted duration as the second preset predicted duration; When the first predicted duration is greater than the second predicted duration, determining the second predicted duration as the second preset predicted duration; Among them, the second prediction duration is a pre-set maximum prediction duration that can predict the third posture data.

6. An image drawing device, characterized in that: The device comprises: A first acquisition module, used to acquire first position data; A first prediction module, used to predict, based on the first posture data, second posture data corresponding to a moment after a first preset prediction time; A second acquisition module is used to acquire a historical image frame and a historical depth map corresponding to the historical image frame, wherein the historical image frame is drawn according to the historical posture data; A first drawing module, configured to adjust the historical image frame according to the second posture data and the historical depth map, so as to obtain a target image frame drawn according to the second posture data; Wherein, the first drawing module includes: A third determination submodule, configured to determine a position of each pixel in the historical image frame in the target image frame according to the second posture data and the historical depth map; The drawing submodule is used to move each pixel in the historical image frame to a corresponding position in the target image frame, and complete the target image frame to obtain the drawn target image frame.

7. The device according to claim 6, characterized in that The first preset prediction duration is the duration required for the first drawing module to adjust the historical image frame according to the second pose data and the historical depth map to obtain a target image frame drawn according to the second pose data.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method described in any one of claims 1 to 5 are implemented.

9. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 5.

Images