Information processing method and device and augmented reality equipment

Abstract

The invention discloses an information processing method and device and augmented reality equipment, and belongs to the technical field of virtual reality. The information processing method comprises the following steps: acquiring pose information of a camera; calculating a pixel offset value of a sampling point based on the pose information; and under the condition that the pixel deviation value is smaller than a preset deviation value, determining that the sampling information of the sampling point does not change, and controlling a rendering camera of the augmented reality equipment to filter the pose change of the camera.

Description

Technical Field

[0001] This application belongs to the field of virtual reality technology, and particularly relates to an information processing method, apparatus, and extended reality device. Background Art

[0002] Currently, in the conventional rendering process, the display image of the current pixel is determined by sampling points. However, when the information sampled by the sampling points changes compared with the information of the previous frame, flickering will occur in the user's eyes. For example, flickering will occur when rendering text, edges, or high-definition textures; especially in the field of Extended Reality (XR), due to the unique pose jitter of XR devices themselves, the information of the sampling points changes at all times, making the flickering phenomenon in the rendering process more easily noticed by users.

[0003] It can be seen that there is a problem of flickering in the rendering process of related technologies. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide an information processing method, apparatus, and extended reality device, which can solve the problem of flickering in the image rendering process of related technologies.

[0005] To solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, the embodiments of this application provide an information processing method, which is applied to an extended reality device including a camera. The method includes:

[0007] Obtain the pose information of the camera;

[0008] Based on the pose information, calculate the pixel offset value of the sampling point;

[0009] In the case where the pixel offset value is less than a preset offset value, determine that the sampling information of the sampling point has not changed, and control the rendering camera of the extended reality device to filter the pose change of the camera.

[0010] In a second aspect, the embodiments of this application provide an information processing apparatus, which is applied to an extended reality device including a camera. The apparatus includes:

[0011] An obtaining module, configured to obtain the pose information of the camera;

[0012] A calculating module, configured to calculate the pixel offset value of the sampling point based on the pose information;

[0013] A processing module, configured to, in the case where the pixel offset value is less than a preset offset value, determine that the sampling information of the sampling point has not changed, and control the rendering camera of the extended reality device to filter the pose change of the camera.

[0014] In a third aspect, an embodiment of the present application provides an extended reality device, which includes a processor and a memory. A program or instruction that can run on the processor is stored on the memory. When the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the method described in the first aspect is implemented.

[0018] In an embodiment of the present application, when the pixel offset value is less than a preset offset value, that is, the sampling information of the sampling point has not changed, the rendering camera of the extended reality device can be controlled to filter the pose change of the camera, that is, there is no need to synchronize the pose change of the camera to the rendering camera of the extended reality device. That is, by filtering the pose change or pose offset of the camera, the flickering problem caused by slight movement can be avoided, that is, the flickering caused by the in-pixel offset of the camera due to slight movement can be avoided. Description of the Drawings

[0019] Figure 1 is a flowchart of an information processing method provided by an embodiment of the present application;

[0020] Figure 2 is one of the sampling schematic diagrams provided by an embodiment of the present application;

[0021] Figure 3 is a flowchart of an information processing method provided by another embodiment of the present application;

[0022] Figure 4 is the second sampling schematic diagram provided by an embodiment of the present application;

[0023] Figure 5 is the third sampling schematic diagram provided by an embodiment of the present application;

[0024] Figure 6 is a structural diagram of an information processing device provided by an embodiment of the present application;

[0025] Figure 7It is a structural diagram of an extended reality device provided by an embodiment of the present application;

[0026] Figure 8 It is a structural diagram of an extended reality device provided by another embodiment of the present application. Detailed implementation manners

[0027] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0028] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data may be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order different from those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object may be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0029] To enable those skilled in the art to better understand the embodiments of the present application, the following explanations are provided first.

[0030] Intra-pixel movement: It refers to that the rendering object X is transformed to the screen space PA when the camera pose is A and to the screen space PB when the camera position is B. If PA and PB are within the same pixel, it is regarded as intra-pixel movement.

[0031] Cross-pixel movement: It refers to that the rendering object X is transformed to the screen space PA when the camera pose is A and to the screen space PB when the camera position is B. If PA and PB are in different pixels, it is regarded as cross-pixel movement.

[0032] Six degrees of freedom (6Dof): It refers to that an object has six degrees of freedom in space, namely the translational degrees of freedom along the three rectangular coordinate axes x, y, and z and the rotational degrees of freedom around these three coordinate axes.

[0033] Next, the information processing method provided by the embodiments of the present application will be described in detail in conjunction with the accompanying drawings through specific embodiments and their application scenarios.

[0034] See Figure 1 , Figure 1It is a flowchart of an information processing method provided by an embodiment of the present application. The information processing method provided by the embodiment of the present application can be applied to an extended reality device including a camera, and the extended reality device can be a virtual reality device, an augmented reality device, a mixed reality device, etc. For example, Figure 1 As shown, the information processing method includes the following steps:

[0035] Step 101, obtain the pose information of the camera.

[0036] In this step, the pose information of the extended reality device can be obtained through sensors such as the acceleration sensor and gyroscope of the extended display device, such as the attitude change of the extended reality device in the world coordinate system, and the pose information of the extended reality device can be determined as the pose information of the camera in the extended reality device.

[0037] For example, when the extended reality device is a wearable head-mounted display device, when the user's head makes a slight movement or rotation, the wearable head-mounted display device will move along with the user's head, causing the attitude information of the wearable head-mounted display device in the world coordinate system to change; that is, the position information of the camera can be obtained by obtaining the position information of the extended reality device.

[0038] Step 102, calculate the pixel offset value of the sampling point based on the pose information.

[0039] In this step, the camera matrix corresponding to the pose information can be obtained based on the conversion parameters between the world coordinate system and the pixel coordinate system, so as to calculate the pixel offset value of the sampling point based on the camera matrix corresponding to the pose information.

[0040] Step 103, when the pixel offset value is less than the preset offset value, determine that the sampling information of the sampling point has not changed, and control the rendering camera of the extended reality device to filter the pose change of the camera.

[0041] The above pixel offset value being less than the preset offset value can be understood as the sampling point having an in-pixel movement, that is, the sampling information of the sampling point has not changed. Therefore, the rendering camera of the extended reality device can be controlled to filter the pose change of the camera, that is, there is no need to synchronize the pose change of the camera to the rendering camera of the extended reality device. In this way, by filtering this pose change or pose offset, the flickering problem caused by slight movement can be avoided, that is, the flickering caused by the in-pixel offset of the camera pixels due to slight movement can be avoided, improving the rendering effect of the rendered object.

[0042] Among them, when the sampling information at the sampling point does not change, the pose change of the filtering camera of the extended reality device can be controlled, that is, there is no need to synchronize the pose change of the camera to the rendering camera of the extended reality device. For example, the rendering camera without synchronizing the above pose information can be used to render the rendering object to avoid the flicker caused by the in-pixel offset of the camera due to slight movement.

[0043] In some embodiments, after calculating the pixel offset value of the sampling point based on the pose information, the method further includes:

[0044] When the pixel offset value is greater than or equal to a preset offset value, it is determined that the sampling information at the sampling point has changed, the pose information is synchronized to the rendering camera of the extended reality device, and the rendering object is rendered based on the rendering camera synchronized with the pose information.

[0045] The above pixel offset value being greater than or equal to the preset offset value can be understood as the sampling point having moved across pixels, that is, the sampling information at the sampling point has changed; therefore, the pose information of the camera can be synchronized to the rendering camera of the extended reality device, and the rendering object can be rendered conventionally based on the rendering camera synchronized with the pose information of the camera to improve the rendering effect of the rendering object.

[0046] The above rendering of the rendering object can be understood as the rendering of the virtual scene.

[0047] Such as Figure 2 As shown, relative to the Nth frame, at the (N + 1)th frame, if the pose change of the camera causes a pixel offset of 0.5 units on the X-axis, it can be determined that the movement of the sampling point belongs to in-pixel movement, and there is no need to synchronize the pose information of the camera to the rendering camera of the extended reality device, that is, the pose change or pose offset can be filtered to avoid the flicker problem caused by slight movement, that is, the flicker caused by the in-pixel offset of the camera due to slight movement can be avoided; relative to the Nth frame, at the (N + 2)th frame, if the pose change of the camera causes a pixel offset of 1 unit on each of the X-axis / Y-axis, it can be determined that the movement of the sampling point belongs to cross-pixel movement, and the pose information of the camera can be synchronized to the rendering camera of the extended reality device, and the rendering object can be rendered conventionally based on the rendering camera synchronized with the pose information of the camera to improve the rendering effect of the rendering object.

[0048] Moreover, since there is no inter-pixel offset of the sampling point, it will not cause flicker during the rendering process. Therefore, to reduce the flicker phenomenon caused by the pose change of the camera, when the sampling point undergoes cross-pixel movement, the pose information of the camera can be synchronized to the rendering camera of the extended reality device, and the rendering object can be rendered based on the rendering camera synchronized with the pose information of the camera to improve the rendering effect of the rendering object.

[0049] In some embodiments, the camera matrix corresponding to the pose information may be calculated first, and then the offset information between the camera matrix corresponding to the pose information and the rendering camera matrix of the rendering camera may be determined based on parameters such as pixel density and camera field of view angle. When the offset information indicates that there is an offset between the camera matrix corresponding to the pose information and the rendering camera matrix of the rendering camera, the pixel offset value of the sampling point at the screen space level may be calculated based on the rendering object coordinates and the projection matrix information, so as to calculate the pixel offset value of the sampling point.

[0050] Among them, the above pixel density can be understood as the number of pixels contained per inch on the screen; the above camera field of view angle can be understood as the field of view range of the camera.

[0051] In this embodiment, by calculating the pixel offset value of the sampling point at the screen space level based on the rendering object coordinates and the projection matrix information, the accuracy of the calculation result of the pixel offset value of the sampling point can be improved.

[0052] See Figure 3 , Figure 3 is a flowchart of an information processing method provided by another embodiment of the present application. As Figure 3 shown, it includes the following steps:

[0053] Step 301, the pose of the extended reality device changes.

[0054] In this step, the change in the pose of the extended reality device can be understood as the change in the pose of the extended reality device in the world coordinate system when sampling adjacent frame information. As Figure 4 shown, when sampling the Nth frame information, the pose of the extended reality device is A; when sampling the N+1th information, the pose of the extended reality device is B; when sampling the N+2th information, the pose of the extended reality device is C.

[0055] Since the movement of the user, etc. directly acts on the camera. For example, when the user's head shakes slightly, it will cause the sampling points of the Nth frame, N+1th frame, and N+2th frame within the same pixel point to have a position offset change for the sampled area rendering object, resulting in different multi-frame sampling results. As Figure 5 shown, the same pixel point shows three colors of green, yellow, and red at the Nth frame, N+1th frame, and N+2th frame respectively, and then the structure seen by the user is the alternating appearance of green, yellow, and red, resulting in a flickering phenomenon.

[0056] Step 302, whether to perform filtering processing on the pose change of the extended reality device.

[0057] In this step, based on the pose change information of the extended reality device, the pixel offset value of the sampling point can be calculated. If the pixel offset value is greater than or equal to the preset offset value, the pose change of the extended reality device does not need to be filtered, and step 303 is executed; while if the pixel offset value is less than the preset offset value, it is determined that the pose change of the extended reality device needs to be filtered, and step 304 is executed.

[0058] Step 303: Synchronize the pose change information of the extended reality device to the virtual rendering camera, and perform rendering processing on the rendering object based on the virtual rendering camera synchronized with the pose change information of the extended reality device.

[0059] Step 304: Do not perform pose assignment to ensure the stability of sampling information.

[0060] In this step, not performing pose assignment can be understood as not synchronizing the pose change of the camera to the rendering camera of the extended reality device, that is, by filtering the pose change or pose offset of the camera to avoid the flicker problem caused by slight movement, that is, it can avoid the flicker caused by the in-pixel offset of the camera due to slight movement.

[0061] Step 305: Before display, perform layer compositing on the rendered image and perform pre-correction processing for lens distortion.

[0062] In some embodiments, considering the real movement requirements that may occur in reality, to distinguish this situation, the movement information of several frames of data can be recorded. Once movement occurs and both the movement position and angle are within the preset range, the pose change filtering is no longer performed, but instead, the pose information of the camera is directly synchronized to the rendering camera of the extended reality device, and the rendering object is rendered based on the virtual rendering camera synchronized with the pose change information of the extended reality device to improve the rendering effect of the rendering object.

[0063] For example, the 6Dof information can be statistically analyzed, and direction detection can be performed through multiple frames of data to determine whether there is movement or rotation in the same direction. If so, it is determined that the user is making continuous movement, and the pixel - to - pixel 6Dof filtering can be stopped. Until the inter - frame statistics are stable, then enter the filtering judgment state.

[0064] In some embodiments, after calculating the pixel offset value of the sampling point based on the pose information, the method further includes:

[0065] If the pixel offset values of consecutive multiple frames of data are all less than the preset offset value and the offset directions of the consecutive multiple frames of data are the same, it is determined that the user is making continuous movement, the pose information is synchronized to the rendering camera of the extended reality device, and the rendering object is rendered based on the rendering camera synchronized with the pose information.

[0066] In this embodiment, when it is determined that the user is continuously moving, the pose information of the camera can be synchronized to the rendering camera of the extended reality device, and the rendering object can be rendered based on the virtual rendering camera synchronized with the pose change information of the extended reality device, so as to reduce the influence of the user's real movement on the rendering result.

[0067] In some embodiments, when the pixel offset values of consecutive multiple frames of data are all less than the preset offset value and the offset directions of the consecutive multiple frames of data are the same, determining that the user is continuously moving includes:

[0068] When the pixel offset values of consecutive multiple frames of data are all less than the preset offset value and the offset directions of the consecutive multiple frames of data are the same, obtain the movement habit information of the user, where the movement habit information is used to characterize the movement direction of the user;

[0069] When the offset direction is consistent with the movement direction of the user, determine that the user is continuously moving.

[0070] In this embodiment, based on the movement habit of the user, it can be determined whether the user is continuously moving, so as to improve the accuracy of the determination result of the user's continuous movement.

[0071] The information processing method of the embodiment of the present application includes: obtaining the pose information of the camera; calculating the pixel offset value of the sampling point based on the pose information; when the pixel offset value is less than the preset offset value, determining that the sampling information of the sampling point has not changed, and controlling the rendering camera of the extended reality device to filter the pose change of the camera. In this way, by filtering the pose change or pose offset of the camera, the flicker problem caused by slight movement can be avoided, that is, the flicker caused by the in-pixel offset of the camera due to slight movement can be avoided.

[0072] For the information processing method provided by the embodiment of the present application, the execution subject may be an information processing device. In the embodiment of the present application, taking the information processing device executing the information processing method as an example, the information processing device provided by the embodiment of the present application is described.

[0073] See Figure 6 , Figure 6 is the structural diagram of the information processing device provided by an embodiment of the present application. As Figure 6 shown, the information processing device 600 can be applied to an extended reality device including a camera, and includes:

[0074] An obtaining module 601, configured to obtain the pose information of the camera;

[0075] A calculation module 602, configured to calculate a pixel offset value of a sampling point based on the pose information;

[0076] A processing module 603, configured to determine that the sampling information of the sampling point has not changed when the pixel offset value is less than a preset offset value, and control a rendering camera of the extended reality device to filter a pose change of the camera.

[0077] Optionally, the calculation module 602 is specifically configured to:

[0078] Calculate a camera matrix corresponding to the pose information;

[0079] Determine an offset information between the camera matrix corresponding to the pose information and a rendering camera matrix of the rendering camera based on a pixel density and a camera field of view angle;

[0080] When the offset information indicates an offset between the camera matrix corresponding to the pose information and the rendering camera matrix of the rendering camera, calculate a pixel offset value that occurs at the screen space level based on a rendering object coordinate and projection matrix information.

[0081] Optionally, the information processing device 600 further includes:

[0082] A first rendering module, configured to determine that a user is making a continuous movement when pixel offset values of consecutive multiple frames of data are all less than the preset offset value and offset directions of the consecutive multiple frames of data are the same, synchronize the pose information to a rendering camera of the extended reality device, and perform a rendering process on a rendering object based on the rendering camera synchronized with the pose information.

[0083] Optionally, the first rendering module is specifically configured to:

[0084] When pixel offset values of consecutive multiple frames of data are all less than the preset offset value and offset directions of the consecutive multiple frames of data are the same, obtain movement habit information of the user, where the movement habit information is used to characterize a movement direction of the user;

[0085] When the offset direction is consistent with the movement direction of the user, determine that the user is making a continuous movement.

[0086] Optionally, the information processing device 600 further includes:

[0087] A second rendering module, configured to determine that the sampling information of the sampling point has changed when the pixel offset value is greater than or equal to the preset offset value, synchronize the pose information to a rendering camera of the extended reality device, and perform a rendering process on a rendering object based on the rendering camera synchronized with the pose information.

[0088] The information processing device 600 in the embodiments of the present application may be an extended reality device or a component in an extended reality device, such as an integrated circuit or a chip. The extended reality device may be a terminal or other devices other than terminals. Exemplarily, the extended reality device may be an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, etc., and the embodiments of the present application do not make specific limitations.

[0089] The information processing device 600 in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, and the embodiments of the present application do not make specific limitations.

[0090] The information processing device 600 provided in the embodiments of the present application can implement Figure 1 each process implemented by the method embodiments. To avoid repetition, details are not described herein again.

[0091] Refer to Figure 7 , Figure 7 which is a structural diagram of an extended reality device provided in an embodiment of the present application. As Figure 7 shown, the embodiments of the present application also provide an extended reality device 700, including a processor 701 and a memory 702. A program or instruction that can run on the processor 701 is stored on the memory 702. When the program or instruction is executed by the processor 701, each step of the above information processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, details are not described herein again.

[0092] It should be noted that the extended reality devices in the embodiments of the present application include the above-mentioned mobile extended reality devices and non-mobile extended reality devices.

[0093] Refer to Figure 8 , Figure 8 which is a structural diagram of an extended reality device provided in another embodiment of the present application. As Figure 8 shown, the extended reality device 800 includes, but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810, etc.

[0094] Those skilled in the art can understand that the extended reality device 800 may further include a power source (such as a battery) for powering each component. The power source can be logically connected to the processor 810 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 8 The structure of the extended reality device shown in Figure 8 does not limit the extended reality device. The extended reality device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0095] Among them, the processor 810 is used for:

[0096] Obtain the pose information of the camera;

[0097] Based on the pose information, calculate the pixel offset value of the sampling point;

[0098] In the case where the pixel offset value is less than the preset offset value, determine that the sampling information of the sampling point has not changed, and control the rendering camera of the extended reality device to filter the pose change of the camera.

[0099] Optionally, the processor 810 is used for:

[0100] Calculate the camera matrix corresponding to the pose information;

[0101] Based on the pixel density and the camera field of view angle, determine the offset information between the camera matrix corresponding to the pose information and the rendering camera matrix of the rendering camera;

[0102] In the case where the offset information indicates that there is an offset between the camera matrix corresponding to the pose information and the rendering camera matrix of the rendering camera, calculate the pixel offset value of the sampling point occurring at the screen space level based on the rendered object coordinates and the projection matrix information.

[0103] Optionally, the processor 810 is used for:

[0104] In the case where the pixel offset values of consecutive multiple frames of data are all less than the preset offset value and the offset directions of the consecutive multiple frames of data are the same, it is determined that the user is making a continuous movement, synchronize the pose information to the rendering camera of the extended reality device, and perform rendering processing on the rendering object based on the rendering camera synchronized with the pose information.

[0105] Optionally, the processor 810 is used for:

[0106] In the case where the pixel offset values of consecutive multiple frames of data are all less than the preset offset value and the offset directions of the consecutive multiple frames of data are the same, obtain the movement habit information of the user, and the movement habit information is used to characterize the movement direction of the user;

[0107] When the offset direction is consistent with the moving direction of the user, it is determined that the user is making a continuous movement.

[0108] Optionally, the processor 810 is configured to:

[0109] When the pixel offset value is greater than or equal to a preset offset value, it is determined that the sampling information of the sampling point has changed, the pose information is synchronized to the rendering camera of the extended reality device, and the rendering object is rendered based on the rendering camera synchronized with the pose information.

[0110] It should be understood that in the embodiments of the present application, the input unit 804 may include a graphics processing unit (GPU) 8041 and a microphone 8042. The graphics processing unit 8041 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also referred to as a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. The other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0111] The memory 809 can be used to store software programs and various data. The memory 809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 809 can include volatile memory or non-volatile memory, or the memory 809 can include both volatile and non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 809 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0112] The processor 810 may include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor may not be integrated into the processor 810 either.

[0113] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above information processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0114] Among them, the processor is the processor in the extended reality device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs.

[0115] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above information processing method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0116] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0117] The embodiments of the present application provide a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement each process of the above information processing method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0118] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or device including that element. In addition, it should be pointed out that the methods and devices in the embodiments of the present application are not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0120] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

Claims

1. An information processing method, characterized in that: Applied to an extended reality device including a camera, the method comprises: Obtaining the position information of the camera; Based on the posture information, calculating the pixel offset value of the sampling point; When the pixel offset value is less than the preset offset value, it is determined that the sampling information of the sampling point has not changed, and the rendering camera of the extended reality device is controlled to filter the posture change of the camera.

2. The method according to claim 1, characterized in that The step of calculating the pixel offset value of the sampling point based on the posture information includes: Calculate the camera matrix corresponding to the pose information; Determine offset information between a camera matrix corresponding to the pose information and a rendering camera matrix of the rendering camera based on pixel density and camera field of view; When the offset information indicates that a camera matrix corresponding to the pose information is offset from a rendering camera matrix of the rendering camera, a pixel offset value of a sampling point occurring at a screen space level is calculated based on rendering object coordinates and projection matrix information.

3. The method according to claim 2, characterized in that After calculating the pixel offset value of the sampling point based on the posture information, the method further includes: When the pixel offset values ​​of multiple consecutive frames of data are all less than the preset offset value and the offset directions of the multiple consecutive frames of data are the same, it is determined that the user is moving continuously, the posture information is synchronized to the rendering camera of the extended reality device, and the rendering object is rendered based on the rendering camera synchronized with the posture information.

4. The method according to claim 3, characterized in that When the pixel offset values ​​of the continuous multiple frames of data are all smaller than the preset offset value and the offset directions of the continuous multiple frames of data are the same, it is determined that the user is performing continuous movement, including: When the pixel offset values ​​of the continuous multiple frames of data are all less than the preset offset value and the offset directions of the continuous multiple frames of data are the same, obtaining the user's movement habit information, where the movement habit information is used to characterize the user's movement direction; When the offset direction is consistent with the moving direction of the user, it is determined that the user is performing continuous movement.

5. The method according to any one of claims 1 to 4, characterized in that After calculating the pixel offset value of the sampling point based on the posture information, the method further includes: When the pixel offset value is greater than or equal to a preset offset value, it is determined that the sampling information of the sampling point has changed, the posture information is synchronized to the rendering camera of the extended reality device, and the rendering object is rendered based on the rendering camera synchronized with the posture information.

6. An information processing device, characterized in that: Applied to an extended reality device including a camera, the device comprises: An acquisition module, used to acquire the position and posture information of the camera; A calculation module, used for calculating the pixel offset value of the sampling point based on the posture information; A processing module is used to determine that the sampling information of the sampling point has not changed when the pixel offset value is less than a preset offset value, and control the rendering camera of the extended reality device to filter the posture change of the camera.

7. The device according to claim 6, characterized in that The computing module is specifically used for: Calculate the camera matrix corresponding to the pose information; Determine offset information between a camera matrix corresponding to the pose information and a rendering camera matrix of the rendering camera based on pixel density and camera field of view; When the offset information indicates that a camera matrix corresponding to the pose information is offset from a rendering camera matrix of the rendering camera, a pixel offset value of a sampling point occurring at a screen space level is calculated based on rendering object coordinates and projection matrix information.

8. The device according to claim 7, characterized in that The device also includes: The first rendering module is used to determine that the user is moving continuously when the pixel offset values ​​of multiple consecutive frames of data are all less than the preset offset value and the offset directions of the multiple consecutive frames of data are the same, synchronize the posture information to the rendering camera of the extended reality device, and render the rendering object based on the rendering camera synchronized with the posture information.

9. The device according to claim 8, characterized in that The first rendering module is specifically used for: When the pixel offset values ​​of the continuous multiple frames of data are all less than the preset offset value and the offset directions of the continuous multiple frames of data are the same, obtaining the user's movement habit information, where the movement habit information is used to characterize the user's movement direction; When the offset direction is consistent with the moving direction of the user, it is determined that the user is performing continuous movement.

10. The device according to any one of claims 6 to 9, characterized in that The device also includes: The second rendering module is used to determine that the sampling information of the sampling point has changed when the pixel offset value is greater than or equal to a preset offset value, synchronize the posture information to the rendering camera of the extended reality device, and render the rendering object based on the rendering camera synchronized with the posture information.

11. An extended reality device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the information processing method as claimed in any one of claims 1 to 5 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the information processing method according to any one of claims 1 to 5 are implemented.

13. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the information processing method according to any one of claims 1 to 5.

Images