Rendering Position Prediction Method and Apparatus, Electronic Device, and Storage Medium

Abstract

The present disclosure relates to a rendering position prediction method and apparatus, an electronic device, and a storage medium. The method includes: obtaining motion state information and inertial sensing information of a camera device at a first moment; predicting the motion state of the camera device at a second moment according to the motion state information of the camera device at the first moment and the inertial sensing information, to obtain first prediction information; performing filtering processing on the first prediction information to obtain second prediction information; and determining a rendering position of an augmented reality (AR) object at a target moment in an image captured by the camera device based on the second prediction information. Embodiments of the present disclosure can improve the accuracy of rendering position prediction.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a method and apparatus for predicting a rendering position, an electronic device, and a storage medium. Background Art

[0002] Augmented Reality (AR) technology is a new technology that combines real information and virtual information. Through AR technology, virtual information (visual information, sound information, tactile information) that originally does not exist in the real world can be superimposed on real information and perceived by people, so as to achieve a sensory experience beyond reality.

[0003] When combining virtual information with real information, virtual information can be superimposed on an image of a real scene through image rendering. However, when rendering virtual information in an image in real time, there will be a certain deviation in the determined rendering position. Summary of the Invention

[0004] The present disclosure provides a technical solution for predicting a rendering position.

[0005] According to one aspect of the present disclosure, there is provided a method for predicting a rendering position, including:

[0006] Obtaining motion state information and inertial sensing information of a camera device at a first moment; predicting the motion state of the camera device at a second moment according to the motion state information and the inertial sensing information of the camera device at the first moment to obtain first prediction information; performing filtering processing on the first prediction information to obtain second prediction information; and determining a rendering position of an Augmented Reality (AR) object at a target moment in an image collected by the camera device based on the second prediction information.

[0007] In some possible implementation manners, the predicting the motion state of the camera device at the second moment according to the motion state information and the inertial sensing information of the camera device at the first moment to obtain first prediction information includes: obtaining first estimation information for predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment; and updating the first estimation information based on the inertial sensing information to obtain the first prediction information.

[0008] In some possible implementation manners, the motion state information includes parameter values of a first motion parameter and a second motion parameter at the first moment, the first motion parameter and the second motion parameter satisfy a preset motion relationship, and the first estimation information includes the parameter value of the second motion parameter at the second moment; obtaining the first estimation information for predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment includes: determining the parameter value of the second motion parameter at the second moment based on the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment.

[0009] In some possible implementation manners, predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment and the inertial sensing information to obtain first prediction information includes: determining measurement information of the camera device at the second moment based on the inertial sensing information; inputting the motion state information of the camera device at the first moment and the measurement information into a first filter to obtain the first prediction information.

[0010] In some possible implementation manners, filtering the first prediction information to obtain second prediction information includes: inputting the first prediction information into a second filter to obtain the second prediction information.

[0011] In some possible implementation manners, inputting the first prediction information into a second filter to obtain the second prediction information includes: predicting the motion state of the camera device at the second moment by using the second filter according to the motion state information of the camera device at the first moment to obtain second estimation information; updating the second estimation information based on the first prediction information to obtain the second prediction information.

[0012] In some possible implementation manners, determining the rendering position of the AR object in the image collected by the camera device at the target moment based on the second prediction information includes: obtaining the pose information of the camera device at the target moment based on the second prediction information; determining the rendering position of the AR object in the image collected by the camera device at the target moment according to the pose information.

[0013] In some possible implementation manners, determining the rendering position of the AR object in the image collected by the camera device at the target moment according to the pose information includes: determining the position of the spatial point captured by the camera device according to the pose information of the camera device at the target moment; determining the projection position of the spatial point in the image according to the position of the spatial point; and determining the rendering position of the AR object in the image collected by the camera device at the target moment according to the projection position.

[0014] According to one aspect of the present disclosure, there is provided a rendering position prediction device, including:

[0015] An acquisition module, configured to acquire the motion state information and the inertial sensing information of the camera device at the first moment;

[0016] A prediction module, configured to predict the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment and the inertial sensing information, so as to obtain first prediction information;

[0017] A filtering module, configured to perform filtering processing on the first prediction information to obtain second prediction information;

[0018] A determination module, configured to determine the rendering position of the augmented reality (AR) object in the image collected by the camera device at the target moment based on the second prediction information.

[0019] In some possible implementation manners, the prediction module is configured to obtain first estimation information for predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment; and update the first estimation information based on the inertial sensing information to obtain the first prediction information.

[0020] In some possible implementation manners, the motion state information includes the parameter values of a first motion parameter and a second motion parameter at the first moment, the first motion parameter and the second motion parameter satisfy a preset motion relationship, and the first estimation information includes the parameter value of the second motion parameter at the second moment; the prediction module is configured to determine the parameter value of the second motion parameter at the second moment based on the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment.

[0021] In some possible implementation manners, the prediction module is configured to determine the measurement information of the camera device at the second moment based on the inertial sensing information; and input the motion state information of the camera device at the first moment and the measurement information into a first filter to obtain the first prediction information.

[0022] In some possible implementations, the filtering module is configured to input the first prediction information into a second filter to obtain the second prediction information.

[0023] In some possible implementations, the filtering module is configured to predict the motion state of the imaging device at a second moment by using the second filter according to the motion state information of the imaging device at a first moment, to obtain second estimation information; and update the second estimation information based on the first prediction information to obtain the second prediction information.

[0024] In some possible implementations, the determining module is configured to obtain the pose information of the imaging device at a target moment based on the second prediction information; and determine the rendering position of the AR object in the image collected by the imaging device at the target moment according to the pose information.

[0025] In some possible implementations, the determining module is configured to determine the position of a spatial point captured by the imaging device according to the pose information of the imaging device at the target moment; determine the projection position of the spatial point in the image according to the position of the spatial point; and determine the rendering position of the AR object in the image collected by the imaging device at the target moment according to the projection position.

[0026] According to one aspect of the present disclosure, there is provided an electronic device, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0027] According to one aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

[0028] In the embodiments of the present disclosure, the motion state information and inertial sensing information of the imaging device at a first moment can be obtained. Then, the motion state of the imaging device at a second moment can be predicted according to the motion state information and inertial sensing information of the imaging device at the first moment to obtain first prediction information, and then the first prediction information is filtered to obtain second prediction information, thereby reducing the jitter of the prediction result. Further, based on the second prediction information, the rendering position of the AR object in the image collected by the imaging device at the target moment is determined, so as to predict the rendering position of the AR object at the target moment and reduce the deviation of the rendering position.

[0029] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear according to the following detailed description of the exemplary embodiments with reference to the accompanying drawings. Description of the Drawings

[0030] The drawings herein are incorporated into and constitute a part of this specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0031] Figure 1 A flowchart showing a rendering position prediction method according to an embodiment of the present disclosure.

[0032] Figure 2 A flowchart showing a rendering position prediction method according to an embodiment of the present disclosure.

[0033] Figure 3 A block diagram showing a rendering position prediction device according to an embodiment of the present disclosure.

[0034] Figure 4 A block diagram showing an electronic device according to an embodiment of the present disclosure.

[0035] Figure 5 A block diagram showing an electronic device according to an embodiment of the present disclosure. Detailed Description of the Embodiments

[0036] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0037] The term "exemplary" used herein means "serving as an example, embodiment, or illustration". Any embodiment described herein as "exemplary" does not have to be construed as superior to or better than other embodiments.

[0038] The term "and / or" as used herein merely describes an association relationship between associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" as used herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0039] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed description of the embodiments. Those skilled in the art should understand that the present disclosure can be implemented without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0040] In the related technology, the rendering frame rate of most AR glasses is greater than 60HZ, and the image acquisition frame rate is only 30HZ. Due to the difference between the rendering frame rate and the acquisition frame rate, and the fact that image rendering also requires a certain amount of time, there will be a certain deviation in the rendering position determined based on the latest image, so that the image presented by the AR glasses to the user is poor.

[0041] The rendering position prediction scheme provided by the embodiments of the present disclosure can be applied to scenarios such as AR wearable devices, image rendering, and posture prediction. For example, when a user wears AR glasses, the rendering position at a future moment can be predicted in real time, and the AR object can be rendered in the image according to the rendering position. The rendering delay in image rendering can be considered, thereby reducing the possible deviation of the rendering position of the AR object rendered on the image, improving the authenticity of the picture presented by the AR glasses to the user, and improving the user experience.

[0042] The rendering position prediction method provided in the embodiment of the present disclosure can be executed by a terminal device, a server or other types of electronic devices, wherein the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the data processing method can be implemented by a processor calling a computer-readable instruction stored in a memory. Alternatively, the method can be executed by a server. For ease of description, the execution subject of the information prompt method is collectively referred to as a terminal hereinafter.

[0043] Figure 1 A flowchart of a rendering position prediction method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the rendering position prediction method includes:

[0044] Step S11, obtaining motion state information and inertial sensor information of the camera device at a first moment.

[0045] In the embodiments of the present disclosure, the terminal can obtain the motion state information and inertial sensing information of the imaging device in real time. The motion state information can be used to represent the motion state of the imaging device. The imaging device can be a device for capturing the scene where it is located. For example, the imaging device can be a camera, a video camera, or other capturing devices. The motion state of the imaging device can change in real time, and the motion state information of the imaging device at different moments can be different. The inertial sensing information can be the information measured by an inertial sensor. The inertial sensing information can include the acceleration of three axes and the angular velocity of three axes. Herein, the three axes refer to the x-axis, y-axis, and z-axis of the coordinate system established with the center of the imaging device as the origin. The inertial sensor can measure the inertial sensing information of the imaging device in real time. In some implementation manners, the imaging device and the inertial sensor can be configured in the terminal, and the terminal can obtain the motion state information and the inertial sensing information of the imaging device in real time. In some implementation manners, the imaging device and the inertial sensor can be configured in other devices, and the terminal can obtain the motion state information and the inertial sensing information of the imaging device from other devices. The first moment can be any known moment. For example, the first moment can be the current moment, and the terminal can obtain the motion state information and the inertial sensing information of the imaging device at the first moment.

[0046] In some implementation manners, the motion state information can include at least one of the following motion parameters: the acceleration of acceleration; acceleration; velocity; position; the acceleration of angular acceleration; the acceleration of angular velocity; angular velocity; azimuth angle. The imaging device can be in real-time motion, and the motion state of the imaging device can be described by one or more of the above motion parameters. The imaging device can not only move but also rotate. The motion parameters such as the acceleration of acceleration, acceleration, velocity, and position can describe the movement of the imaging device, and the motion parameters such as the acceleration of angular acceleration, the acceleration of angular velocity, angular velocity, and azimuth angle can describe the rotation of the imaging device.

[0047] In step S12, according to the motion state information of the imaging device at the first moment and the inertial sensing information, predict the motion state of the imaging device at the second moment to obtain the first prediction information.

[0048] In an embodiment of the present disclosure, the second moment may be the next moment after the first moment. The first moment and the second moment are relatively close, and the second moment may be equal to the first moment plus a preset duration. The preset duration may be an infinitesimal amount. The terminal may predict the motion state of the camera device at the second moment based on the motion state information and inertial sensing information of the camera device at the first moment, to obtain first prediction information. For example, the motion state information and inertial sensing information at the first moment may be integrated. For example, the acceleration of the acceleration at the first moment is obtained from the motion state information, and the acceleration at the second moment is obtained by integrating the acceleration of the acceleration at the first moment. The acceleration at the first moment is obtained from the inertial sensing information, and the velocity at the second moment is obtained by integrating the acceleration at the first moment. Alternatively, some optimization algorithms may also be used to obtain the first prediction information of the camera device from the motion state information and inertial sensing information of the camera device at the first moment. For example, the Newton method, the Gauss-Newton method, etc. may be used to perform batch optimization on the motion state information and inertial sensing information of the camera device at the first moment to obtain the first prediction information of the camera device. The first prediction information may be used to represent the motion state of the camera device at the second moment.

[0049] In some implementation manners, first estimation information for predicting the motion state of the camera device at the second moment may be obtained based on the motion state information of the camera device at the first moment. For example, the motion state information at the first moment may be integrated, or the motion state information at the first moment may be input into a first filter to obtain first estimation information for predicting the motion state at the second moment. The first prediction information is further updated based on the inertial sensing information at the first moment. For example, measurement information corresponding to the motion state of the camera device at the second moment may be obtained based on the inertial sensing information at the first moment, and the first estimation information is updated according to the measurement information at the second moment to obtain the first prediction information. For example, the inertial sensing information obtained at the first moment may be integrated to obtain measurement information corresponding to the motion state of the camera device at the second moment, and then the parameter values of the same motion parameters in the measurement information and the first estimation information may be weighted to obtain the first prediction information. Taking the motion parameter as velocity as an example, the acceleration may be obtained from the inertial sensing information at the first moment, and then the acceleration may be integrated from the first moment to the second moment to obtain the measured value of the velocity of the camera device at the second moment. Then, the predicted value of the velocity of the camera device at the second moment may be obtained from the first estimation information, and the measured value and the predicted value of the velocity may be further weighted and summed to obtain the updated value of the velocity in the first prediction information. In this way, the predicted value may be updated by the measured value of the motion state of the camera device, and the obtained first prediction information may more accurately describe the motion state of the camera device at the second moment.

[0050] In some implementations, the measurement information of the imaging device at the second moment can be determined based on the inertial sensing information at the first moment. For example, the inertial sensing information at the first moment can be integrated to obtain the measurement information of the imaging device at the second moment. Further, the motion state information of the imaging device at the first moment and the measurement information at the second moment can be input into a first filter, and the first filter can predict the motion state of the imaging device at the second moment according to the motion state information at the first moment and the measurement information at the second moment to obtain first prediction information. The first filter can be a Kalman filter. The first filter can consider the influence of noise, and the noise and interference in the motion state information and the inertial sensing information can be filtered through the first filter, so that relatively accurate first prediction information can be obtained.

[0051] Step S13: Perform filtering processing on the first prediction information to obtain second prediction information.

[0052] In the embodiments of the present disclosure, since the first prediction information is predicted based on the motion state information and the inertial sensing information at the first moment, there may be some jitters. Therefore, after obtaining the first prediction information, the first prediction information can be further filtered. For example, some filtering algorithms such as Kalman filtering, amplitude limiting filtering, median filtering, recursive average filtering, etc. can be used to filter the first pre-stored information, so as to reduce the phenomenon that the pre-stored result of the motion state has jitters. The second prediction information obtained after filtering processing can have a smooth transition.

[0053] In some implementations, the first prediction information can be input into a second filter to obtain second prediction information for predicting the motion state at the second moment. Here, the second filter can be a Kalman filter. The prediction process corresponding to the second filter can be the same as that of the above-mentioned first filter. The second filter can be used to correct the first prediction information again, so as to obtain second prediction information that is more accurate and smoother than the first prediction information. At the same time, when using the second filter to correct the second prediction information, the calculation amount is small and the speed is fast, which can improve the accuracy of predicting the motion state of the imaging device at the second moment.

[0054] Step S14: Based on the second prediction information, determine the rendering position of the target moment augmented reality (AR) object in the image collected by the imaging device.

[0055] In an embodiment of the present disclosure, after obtaining the second prediction information, the rendering position of the AR object in the image captured by the imaging device at the target moment can be determined based on the second prediction information. The target moment can be the upcoming rendering moment, and the target moment can be determined according to the refresh rate of the display. For example, according to the second prediction information, the moving distance and rotation angle of the imaging device from the first moment to the target moment can be determined, and then according to the correspondence between the distance and rotation angle in the real three-dimensional space and the pixel distance in the image, the pixel distance that the AR object moves in the image at the target moment can be determined. Further, the rendering position corresponding to the AR object at the target moment can be determined according to the pixel distance that the AR object moves in the image. Here, the AR object can be virtual information in the picture displayed on the terminal. For example, the AR object can be a virtual identifier, a virtual character, virtual text, etc.

[0056] In some implementation manners, after determining the rendering position corresponding to the AR object at the target moment, the AR object can be rendered in the image of the real scene captured by the imaging device according to the rendering position corresponding to the second moment, so as to implement superimposing the virtual information on the image of the real scene.

[0057] In some implementation manners, the terminal can also select the rendered AR object according to the user operation. For example, the target AR object displayed in the picture can be determined in the AR object library according to the user operation. In some implementation manners, the effect of the AR object can also be changed according to the user operation. For example, the effects such as the color and transparency of the AR object can be changed according to the user operation.

[0058] The rendering position prediction method provided by the embodiment of the present disclosure can predict the rendering position at a future moment after the first moment. First, the motion state at the second moment is predicted through the motion state information and inertial sensing information at the first moment, and then the first prediction information obtained by prediction is filtered to obtain the second prediction information that can more accurately describe the motion state at the second moment. The rendering position of the AR object at the target moment can be accurately determined through the second prediction information, thereby reducing the deviation and jitter of the rendering position.

[0059] In some implementations, when determining the rendering position of an AR object in an image captured by a camera device based on second prediction information, the pose information of the camera device at the target moment can also be obtained based on the second prediction information. Then, according to the pose information of the camera device at the target moment, the rendering position of the AR object in the image captured by the camera device at the target moment can be determined. For example, in the case where the target moment is the second moment, the pose information of the camera device at the target moment can be obtained directly from the second prediction information. In the case where the target moment is a future moment after the second moment, the second prediction information can be integrated from the second moment to the target time to obtain the motion state information corresponding to the target moment, and further, the pose information of the camera device at the target moment can be obtained from the motion state information corresponding to the target moment. Here, the pose information of the camera device at the target moment can include the position where the camera device is located and the azimuth angle it faces. According to the pose information of the camera device at the target moment, the real-world scene that the camera device can capture at the target moment can be determined. Furthermore, according to the position of the real-world object that the AR object is to be superimposed on the real-world scene, the rendering position of the AR object in the image of the real-world scene at the target moment can be determined. In this way, based on the pose information of the camera device at the target moment obtained from the second prediction information, the rendering position of the AR object at the target moment can be determined quickly and accurately.

[0060] In some implementations, when determining the rendering position of an AR object in an image captured by a camera device according to the pose information of the camera device at the target moment, the position of the spatial points captured by the camera device can be determined according to the pose information of the camera device at the target moment. For example, according to the pose information of the camera device at the target moment and the camera parameters of the camera device (such as camera parameters like focal length, shooting angle, etc.), the field of view range that the camera device can capture can be determined. According to the spatial position where the field of view range is located, the positions of multiple spatial points captured by the camera device can be determined. Further, according to the positions of the spatial points, the projection positions of the spatial points in the image can be determined. For example, according to the preset correspondence between three-dimensional spatial coordinates and image coordinates, the projection positions of the spatial points in the image can be determined. Then, according to the projection positions of the spatial points in the image, the rendering position of the AR object in the image captured by the camera device at the target moment can be determined. For example, the target spatial point to which the AR object is attached can be determined among the multiple spatial points, and then the projection position of the target spatial point can be determined as the rendering position of the AR object in the image. In this way, the rendering position of the AR object at the target moment can be determined quickly.

[0061] In the embodiments of the present disclosure, the motion state of the imaging device at the second moment can be predicted based on the motion state information and inertial sensing information of the imaging device at the first moment to obtain first prediction information, and then the first prediction information can be filtered to obtain second prediction information, thereby reducing the jitter of the prediction result. Further, based on the second prediction information, the rendering position of the AR object in the image collected by the imaging device at the target moment can be determined, so as to predict the rendering position of the AR object at the target moment and reduce the deviation of the rendering position. For example, the motion state information at the first moment and the measurement information at the second moment obtained based on the inertial sensing information at the first moment can be input into the first filter, and the first filter is used to obtain the first prediction information for predicting the motion state of the imaging device at the second moment. Then, the motion state information at the first moment and the first prediction information are input into the second filter, and the second filter is used to smooth the first prediction information to obtain the second prediction information. The second prediction information can represent the motion state of the imaging device more accurately. Further, the rendering position can be determined based on the second prediction information, so that the deviation and jitter of the rendering position can be reduced, and the user experience can be improved.

[0062] Here, when both the first filter and the second filter are Kalman filters, the process of estimating the motion state by each Kalman filter can include a prediction stage and an update stage. Among them, in the prediction stage, the motion state corresponding to the second moment can be predicted based on the motion state information at the first moment, and in the update stage, the measurement information at the second moment can be used to update the prediction information obtained in the prediction stage. For the first filter, the measurement information at the second moment obtained based on the inertial sensing information at the first moment can be used as the measurement information in the update stage. For the second filter, the first prediction information obtained by the first filter can be used as the measurement information in the update stage of the second filter.

[0063] In some implementation manners, inputting the first prediction information into the second filter to obtain the second prediction information may include: predicting the motion state of the imaging device at the second moment by using the second filter according to the motion state information of the imaging device at the first moment to obtain second estimation information. Then, the second prediction information is obtained by updating the second estimation information based on the first prediction information.

[0064] Here, in the case of obtaining the second prediction information by using the second filter, the second filter can first be used to predict the motion state of the imaging device at the second moment based on the preset motion relationship through the motion state information at the first moment, so as to obtain the second estimation information. For example, the motion state information at the first moment can be integrated from the first moment to the second moment to obtain the second estimation information corresponding to the second moment. Further, the second filter can be used to update the second estimation information based on the first prediction information. For example, the parameter values of the same motion parameters in the first prediction information and the second estimation information can be weighted to obtain the second prediction information. Through the second filter, the motion state of the imaging device at the second moment can first be predicted based on the motion state information at the first moment, and further, the second estimation information obtained by the prediction can be corrected by the first prediction information to obtain a more accurate second prediction information. In this way, the rendering position corresponding to the second moment obtained based on the second prediction information can be more accurate and smooth.

[0065] In some implementation manners, the first filter can have the same prediction model as the second filter. The prediction model can be understood as the motion relationship used to predict the motion device of the imaging device at the second moment based on the motion state information at the first moment. Taking the process of obtaining the first estimation information based on the motion state information at the first moment as an example, the prediction model will be described below.

[0066] In some implementation manners, the parameter values of the first motion parameter and the second motion parameter at the first moment can be determined from the motion state information at the first moment, where the first motion parameter and the second motion parameter satisfy the preset motion relationship. Then, based on the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment, the parameter value of the second motion parameter at the second moment can be determined. For example, the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment can be substituted into the preset motion relationship to obtain the parameter value of the second motion parameter at the second moment. The first estimation information includes the parameter value of the second motion parameter at the second moment.

[0067] For example, if the first motion parameter is the acceleration of acceleration and the second motion parameter is acceleration, the acceleration at the first moment can be used as the initial value, and the acceleration of acceleration at the first moment can be used as the change rate of acceleration. Substituting them into the linear motion relationship satisfied by acceleration and the acceleration of acceleration, the parameter value of acceleration at the second moment can be obtained.

[0068] Through the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment, the parameter value of the second motion parameter at the second moment can be predicted more accurately, thereby providing a basis for determining the rendering position corresponding to the second moment.

[0069] In some implementations, when determining the parameter value of the second motion parameter at the second moment based on the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment, the parameter value of the first motion parameter at the second moment can also be determined first. For example, the parameter value of the first motion parameter at the first moment can be added to a preset noise to obtain the parameter value of the first motion parameter at the second moment. Then, based on the parameter value of the first motion parameter at the first moment, the parameter value of the first motion parameter at the second moment, and the parameter value of the second motion parameter at the first moment, the parameter value of the second motion parameter at the second moment can be obtained. For example, the average value of the first motion parameter between the first moment and the second moment can be determined based on the parameter value of the first motion parameter at the first moment and the parameter value of the first motion parameter at the second moment, and then the parameter value of the second motion parameter at the second moment can be obtained based on the parameter value of the second motion parameter at the first moment and the average value of the first motion parameter.

[0070] For example, if the first motion parameter is acceleration and the second motion parameter is velocity, the average value of the acceleration at the first moment and the acceleration at the second moment can be calculated. Then, taking the velocity at the first moment as the initial velocity and the average value as the rate of change of velocity, and substituting them into the linear motion relationship satisfied by acceleration and velocity, the parameter value of the velocity at the second moment can be obtained.

[0071] In some implementations, during the prediction phases of the first filter and the second filter, the motion relationships satisfied by each motion parameter can be as shown in the following formulas:

[0072] b k+1 = b k + δw

[0073] a k+1 = a k + b k *Δt + δa

[0074] v k+1 = v k + 0.5*(a k+1 + a k )*Δt + δv

[0075] p k+1 = p k + v k *Δt + 0.5*(a k+1 + a k )*Δt*Δt + δp

[0076] wb k+1 = wb k + δwb

[0077] wa k+1 = wa k+wb k *Δt + δwa

[0078] w k+1 = w k + 0.5 * (wa k+1 + wa k ) * Δt + δw

[0079]

[0080] where b k is the acceleration of the acceleration at the first moment, b k+1 is the acceleration of the acceleration at the second moment, a k is the acceleration at the first moment, a k+1 is the acceleration at the second moment, v k is the velocity at the first moment, v k+1 is the velocity at the second moment, p k is the position at the first moment, p k+1 is the position at the second moment, wb k is the acceleration of the angular acceleration at the first moment, wb k+1 is the acceleration of the angular acceleration at the second moment, wa k is the angular acceleration at the first moment, wa k+1 is the angular acceleration at the second moment, w k is the angular velocity at the first moment, w k+1 is the angular velocity at the second moment, q k is the azimuth angle at the first moment, q k+1 is the azimuth angle at the second moment. Δt is the time difference between the second moment and the first moment, δw is the noise corresponding to the angular velocity, δa is the noise corresponding to the acceleration, δv is the noise corresponding to the velocity, δp is the noise corresponding to the position, δwb is the noise corresponding to the acceleration of the angular acceleration, δwa is the noise corresponding to the angular acceleration, and δq is the noise corresponding to the azimuth angle. These noises can be set according to the actual application scenario or requirements. For the motion parameters whose initial values cannot be directly determined among the above motion parameters, the initial values of the motion parameters can also be set according to the actual application scenario or requirements.

[0081] The following uses an example to exemplarily illustrate the rendering position prediction method provided by the embodiments of the present disclosure. Figure 2 The flowchart showing the rendering position prediction method according to the embodiments of the present disclosure is shown. In the example, the terminal can be an AR glasses, and the AR glasses are configured with a camera device and an inertial sensor. During the process of the user wearing the AR glasses, the AR glasses can provide the user with an AR picture combining the real scene and virtual information. The method includes the following steps:

[0082] Step S201: the AR glasses obtain motion state information and inertial sensing information at a first moment.

[0083] Here, the camera device may be configured in the AR glasses, so that the motion state information and inertial sensing information of the camera device may be considered as the motion state information and inertial sensing information of the AR glasses. The first moment may be the current moment.

[0084] Step S202: Integrate the inertial sensing information at the first moment to obtain measurement information of the motion state of the AR glasses at the second moment.

[0085] Here, the second moment may be a moment next to the first moment.

[0086] Step S203: input the motion state information and measurement information at the first moment into a first filter to obtain first prediction information.

[0087] Here, the first prediction information may be information for a preliminary estimation of the motion state of the AR glasses at the second moment, wherein the measurement information may be used as measurement information of the first filter in the update phase.

[0088] Step S204: input the motion state information at the first moment and the first prediction information into a second filter to obtain second prediction information.

[0089] Here, the second prediction information may be used as measurement information of the second filter in the update phase, that is, the measurement information corresponding to the first filter and the second filter is different.

[0090] Step S205: Acquire the position information of the AR glasses at the target time according to the second prediction information.

[0091] Here, the second prediction information can be integrated from the second moment to the target time to obtain the motion state information corresponding to the target moment, and further obtain the position and posture information of the camera device at the target moment from the motion state information corresponding to the target moment.

[0092] Step S206: Determine the rendering position of the AR object in the picture displayed by the AR glasses according to the position information of the AR glasses at the target time.

[0093] It should be noted that in the example, only the current moment is taken as the first moment to illustrate the process of predicting the rendering position. In practical applications, as time changes, the rendering position at future moments can be continuously predicted. That is, in the case where the current time changes to the second moment, the second prediction information obtained at the second moment can be used as the motion state information at the current moment, and the motion state at the next moment can be predicted based on the motion state information at the current moment and the inertial sensing information measured currently, so as to realize the prediction of the rendering position at the future target moment in real time.

[0094] In the example provided by the embodiments of the present disclosure, the rendering position of the AR object in the AR glasses screen at a future moment can be predicted by combining a filter with an inertial sensor. The predicted rendering position can be determined through multiple filtrations, which can make the rendering position smoother and more accurate, reduce the jitter of the AR glasses screen, improve the rendering effect of the AR glasses screen, and enhance the user experience. In addition, since the obtained rendering position corresponds to a future target moment, the dependence of the rendering position on the collected images can be reduced, and high-frame-rate rendering position prediction can be achieved.

[0095] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Due to space limitations, the present disclosure will not elaborate further. Those skilled in the art can understand that in the above methods of the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

[0096] In addition, the present disclosure also provides an information prompt device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any information prompt method provided by the present disclosure. For the corresponding technical solutions and descriptions, please refer to the corresponding records in the method part and will not be elaborated further.

[0097] Figure 3 The block diagram of the rendering position prediction device according to an embodiment of the present disclosure is shown as Figure 3 shown, and the device includes:

[0098] An acquisition module 31, configured to acquire the motion state information and inertial sensing information of the imaging device at the first moment;

[0099] A prediction module 32, configured to predict the motion state of the imaging device at the second moment according to the motion state information of the imaging device at the first moment and the inertial sensing information, so as to obtain first prediction information;

[0100] A filtering module 33, configured to perform filtering processing on the first prediction information to obtain second prediction information;

[0101] A determination module 34, configured to determine a rendering position of an augmented reality (AR) object at a target moment in an image captured by the imaging device based on the second prediction information.

[0102] In some possible implementation manners, the prediction module 32 is configured to obtain first estimation information for predicting a motion state of the imaging device at a second moment according to motion state information of the imaging device at a first moment; and update the first estimation information based on the inertial sensing information to obtain the first prediction information.

[0103] In some possible implementation manners, the motion state information includes parameter values of a first motion parameter and a second motion parameter at the first moment, the first motion parameter and the second motion parameter satisfy a preset motion relationship, and the first estimation information includes parameter values of the second motion parameter at the second moment; the prediction module 32 is configured to determine the parameter values of the second motion parameter at the second moment based on the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment.

[0104] In some possible implementation manners, the prediction module 32 is configured to determine measurement information of the imaging device at the second moment based on the inertial sensing information; and input the motion state information of the imaging device at the first moment and the measurement information into a first filter to obtain the first prediction information.

[0105] In some possible implementation manners, the filtering module 33 is configured to input the first prediction information into a second filter to obtain the second prediction information.

[0106] In some possible implementation manners, the filtering module 33 is configured to predict a motion state of the imaging device at the second moment by using the second filter according to the motion state information of the imaging device at the first moment to obtain second estimation information; and update the second estimation information based on the first prediction information to obtain the second prediction information.

[0107] In some possible implementation manners, the determination module 34 is configured to obtain pose information of the imaging device at the target moment based on the second prediction information; and determine a rendering position of the AR object in the image captured by the imaging device at the target moment according to the pose information.

[0108] In some possible implementation manners, the determining module 34 is configured to determine the position of a spatial point captured by the imaging device according to the pose information of the imaging device at a target moment; determine the projection position of the spatial point in the image according to the position of the spatial point; and determine the rendering position of the AR object in the image collected by the imaging device at the target moment according to the projection position.

[0109] In some embodiments, the functions or modules included in the device provided in the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments. The specific implementation may refer to the description of the above method embodiments. For the sake of brevity, it will not be described in detail here.

[0110] The embodiments of the present disclosure also propose a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above methods are implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0111] The embodiments of the present disclosure also propose an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above methods.

[0112] The embodiments of the present disclosure also provide a computer program product, including computer-readable code. When the computer-readable code runs on a device, the processor in the device executes instructions for implementing the rendering position prediction method provided in any of the above embodiments.

[0113] The embodiments of the present disclosure also provide another computer program product for storing computer-readable instructions. When the instructions are executed, the computer executes the operations of the rendering position prediction method provided in any of the above embodiments.

[0114] The electronic device may be provided as a terminal, a server, or other forms of devices.

[0115] Figure 4 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0116] Refer to Figure 4 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0117] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0118] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks.

[0119] The power component 806 provides power to various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0120] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each of the front camera and the rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0121] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0122] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0123] The sensor component 814 includes one or more sensors for providing an assessment of various aspects of the status of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. The sensor component 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 may also include a light sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0124] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a wireless network based on communication standards, such as a wireless local area network (WiFi), a second generation mobile communication technology (2G), or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0125] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0126] In an exemplary embodiment, a non - volatile computer - readable storage medium is also provided, such as a memory 804 including computer program instructions, and the computer program instructions can be executed by a processor 820 of the electronic device 800 to complete the above method.

[0127] Figure 5 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server. Referring to Figure 5 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0128] The electronic device 1900 may further include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Microsoft server operating system (Windows Server TM ), the graphical user interface - based operating system launched by Apple Inc. (Mac OS X TM ), the multi - user and multi - process computer operating system (Unix TM ), the free and open - source Unix - like operating system (Linux TM ), the open - source Unix - like operating system (FreeBSD TM ) or the like.

[0129] In an exemplary embodiment, a non - volatile computer - readable storage medium is also provided, such as a memory 1932 including computer program instructions, and the computer program instructions can be executed by the processing component 1922 of the electronic device 1900 to complete the above method.

[0130] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement aspects of the present disclosure.

[0131] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in a groove having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium as used herein is not to be construed as a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0132] The computer-readable program instructions described herein may be downloaded to respective computing / processing devices from a computer-readable storage medium or may be downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0133] The computer program instructions for performing the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of this disclosure.

[0134] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer - readable program instructions.

[0135] These computer - readable program instructions can be provided to a processor of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data - processing apparatus, create a means for implementing the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner, so that the computer - readable medium storing the instructions includes a manufacture that includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0136] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0137] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the boxes may occur in a different order than noted in the figures. For example, two consecutive boxes may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0138] The computer program product may be embodied specifically in hardware, software, or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium. In another alternative embodiment, the computer program product is embodied as a software product, such as a Software Development Kit (SDK), etc.

[0139] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skilled persons in the technical field to understand the embodiments disclosed herein.

Claims

1. A rendering position prediction method, characterized in that, it includes: Obtain the motion state information and inertial sensing information of the camera device at the first moment; Predict the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment and the inertial sensing information, and obtain first prediction information; Perform filtering processing on the first prediction information to obtain second prediction information; Based on the second prediction information, determine the rendering position of the augmented reality (AR) object in the image collected by the camera device at the target moment. The AR object includes virtual information in the picture displayed on the terminal, and the target moment is a future moment after the second moment; Wherein, the determining the rendering position of the AR object in the image collected by the camera device at the target moment based on the second prediction information includes: Based on the second prediction information, integrate the second prediction information from the second moment to the target moment to obtain the pose information of the camera device at the target moment; Determine the position of the spatial point captured by the camera device according to the pose information of the camera device at the target moment; Determine the projection position of the spatial point in the image according to the position of the spatial point; Determine the rendering position of the AR object in the image collected by the camera device at the target moment according to the projection position.

2. The method according to claim 1, characterized in that, the predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment and the inertial sensing information, and obtaining first prediction information includes: Obtain first estimation information for predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment; Update the first estimation information based on the inertial sensing information to obtain the first prediction information.

3. The method according to claim 2, characterized in that, the motion state information includes the parameter values of the first motion parameter and the second motion parameter at the first moment, the first motion parameter and the second motion parameter satisfy a preset motion relationship, and the first estimation information includes the parameter value of the second motion parameter at the second moment; the obtaining first estimation information for predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment includes: Determine the parameter value of the second motion parameter at the second moment based on the parameter value of the first motion parameter at the first moment and the parameter value of the second motion parameter at the first moment.

4. The method according to claim 1, characterized in that, the predicting the motion state of the camera device at the second moment according to the motion state information of the camera device at the first moment and the inertial sensing information, and obtaining first prediction information includes: Determine the measurement information of the camera device at the second moment based on the inertial sensing information; Input the motion state information of the imaging device at the first moment and the measurement information into a first filter to obtain the first prediction information.

5. The method according to any one of claims 1 to 4, wherein, the filtering the first prediction information to obtain a second prediction information includes: Inputting the first prediction information into a second filter to obtain the second prediction information.

6. The method according to claim 5, wherein, the inputting the first prediction information into a second filter to obtain the second prediction information includes: According to the motion state information of the imaging device at the first moment, using the second filter to predict the motion state of the imaging device at the second moment to obtain a second estimation information; Updating the second estimation information based on the first prediction information to obtain the second prediction information.

7. A rendering position prediction device, wherein, comprising: An acquisition module, configured to acquire the motion state information of the imaging device at the first moment and inertial sensing information; A prediction module, configured to predict the motion state of the imaging device at the second moment according to the motion state information of the imaging device at the first moment and the inertial sensing information to obtain a first prediction information; A filtering module, configured to filter the first prediction information to obtain a second prediction information; A determination module, configured to determine the rendering position of an augmented reality (AR) object in the image collected by the imaging device at a target moment based on the second prediction information, where the AR object includes virtual information in the picture displayed on the terminal, and the target moment is a future moment after the second moment; wherein, the determination module is configured to integrate the second prediction information from the second moment to the target moment based on the second prediction information to obtain the pose information of the imaging device at the target moment; Determine the position of the spatial point captured by the imaging device according to the pose information of the imaging device at the target moment; determine the projection position of the spatial point in the image according to the position of the spatial point; determine the rendering position of the AR object in the image collected by the imaging device at the target moment according to the projection position.

8. An electronic device, wherein, comprising: A processor; A memory for storing instructions executable by the processor; wherein, the processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, on which computer program instructions are stored, wherein, when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images