Screen recording method and device of augmented reality equipment, equipment, medium and program

Abstract

The embodiment of the invention provides a screen recording method and device for an augmented reality device, equipment, a medium and a program, and the method comprises the steps: obtaining a target pose corresponding to an XR device at a t moment, the target pose being a pose at the t moment or a first predicted pose at a t + m moment, carrying out the pose stability augmentation processing of the target pose to obtain a stability augmentation pose, the pose stability augmentation processing comprises the following steps: performing reverse compensation on a roll angle of a target pose; and according to the stability augmentation pose, performing screen recording on the rendering result of the virtual scene to obtain a screen recording image at the moment corresponding to the target pose, and storing the screen recording image. According to the method, reverse compensation is carried out on the rolling angle of the target pose, shaking of the XR equipment in the rolling direction corresponding to the rolling angle is further reduced, therefore, shaking of a screen recording picture is reduced, and the screen recording quality is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of electronic devices, and in particular, to a method, apparatus, device, medium and program for recording screen of an extended reality device. Background Art

[0002] Extended Reality (XR) refers to the combination of reality and virtuality through computers to create a virtual environment for human-computer interaction. XR is also a general term for multiple technologies such as Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR). By integrating the visual interaction technologies of the three, it brings the experiencer an "immersive feeling" of seamless transition between the virtual world and the real world.

[0003] XR devices can record the screen to store the content displayed over a period of time in the form of a video locally for users to view later or send to other devices for playback. XR devices will inevitably shake during use. Screen recording usually captures and stores the screen displayed by the XR device. Therefore, the screen recording screen also has the problem of shaking, causing users to feel severe dizziness when watching the recorded screen. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device, medium and program for recording screen of an extended reality device, which can reduce the jitter of the recorded screen and improve the quality of the recorded screen.

[0005] In a first aspect, an embodiment of the present application provides a screen recording method for an extended reality device, the method comprising:

[0006] Obtain a target pose corresponding to the extended reality XR device at time t, where the target pose is the pose at time t or the first predicted pose at time t+m;

[0007] Performing posture stabilization processing on the target posture to obtain a stabilized posture, wherein the posture stabilization processing includes: performing reverse compensation on the roll angle of the target posture;

[0008] According to the stabilization posture, the rendering result of the virtual scene is recorded to obtain a screen recording image at the moment corresponding to the target posture, and the screen recording image is stored.

[0009] In some embodiments, the reverse compensation of the roll angle of the target posture includes:

[0010] Performing reverse compensation on the roll angle of the target posture so that the roll angle of the target posture after reverse compensation is a preset value;

[0011] In some embodiments, the reverse compensation for the roll angle of the target pose includes:

[0012] When the roll angle of the target pose is greater than or equal to a preset angle threshold, perform reverse compensation on the roll angle of the target pose, so that the value of the roll angle of the target pose after reverse compensation is a preset value or the angle threshold;

[0013] Or, when the roll angle of the target pose is greater than or equal to a preset angle threshold, perform reverse compensation on the roll angle of the target pose according to a preset compensation angle.

[0014] In some embodiments, the reverse compensation for the roll angle of the target pose includes:

[0015] Perform reverse compensation on the roll angle of the target pose according to a preset compensation ratio.

[0016] In some embodiments, the pose stabilization process further includes: performing filtering processing on the target pose.

[0017] In some embodiments, the filtering processing adopts any one of the following filtering methods: Bayesian filtering, wavelet filtering, Kalman filtering, Wiener filtering, Butterworth filtering, Chebyshev filtering, B-spline fitting, or multiple fitting.

[0018] In some embodiments, the performing filtering processing on the target pose includes:

[0019] Perform filtering on the pitch angle and / or yaw angle of the target pose.

[0020] In some embodiments, the method further includes:

[0021] Render the virtual scene according to the first predicted pose at the t + m moment to obtain a first rendering result;

[0022] The obtaining the screencap image corresponding to the moment of the target pose by screencapping the rendering result of the virtual scene according to the stabilized pose includes:

[0023] Perform screencapping according to the stabilized pose and the first rendering result to obtain the screencap image corresponding to the moment of the target pose.

[0024] In some embodiments, the performing screencapping according to the stabilized pose and the first rendering result to obtain the screencap image corresponding to the moment of the target pose includes:

[0025] Distort the first rendering result according to the stabilized pose to obtain a first distortion result;

[0026] Intercept the first warping result to obtain a screencapture image at the moment corresponding to the target pose.

[0027] In some embodiments, the method further includes:

[0028] Obtain the pose of the XR device at time t + n, and predict the second predicted pose of the XR device at time t + m according to the pose at time t + n, where the value of n is less than m;

[0029] Warp the first rendering result according to the second predicted pose to obtain a second warping result, and intercept and display the second warping result on the screen at time t + m.

[0030] In some embodiments, the step of obtaining a screencapture image at the moment corresponding to the target pose by recording the rendering result of the virtual scene according to the stabilized pose includes:

[0031] Render the virtual scene according to the stabilized pose to obtain a second rendering result;

[0032] Intercept the second rendering result to obtain a screencapture image at the moment corresponding to the target pose.

[0033] In some embodiments, the method further includes:

[0034] Render the virtual scene according to the first predicted pose at time t + m to obtain a first rendering result;

[0035] Obtain the pose of the XR device at time t + n, and predict the second predicted pose of the XR device at time t + m according to the pose at time t + n, where the value of n is less than m;

[0036] Warp the rendering result at time t + m according to the second predicted pose to obtain a second warping result, and intercept and display the second warping result on the screen at time t + m.

[0037] In some embodiments, the step of rendering the virtual scene according to the stabilized pose to obtain a second rendering result includes:

[0038] Render the virtual scene locally according to the stabilized pose to obtain the second rendering result.

[0039] In some embodiments, the step of rendering the virtual scene according to the first predicted pose at time t + m to obtain a first rendering result includes:

[0040] Perform global rendering on the virtual scene according to the first predicted pose to obtain the first rendering result.

[0041] In a second aspect, an embodiment of the present application provides a screen recording device for an extended reality device. The device includes:

[0042] A pose acquisition module, configured to acquire a target pose corresponding to an extended reality (XR) device at time t, where the target pose is the pose at time t or the first predicted pose at time t + m;

[0043] A stabilization processing module, configured to perform pose stabilization processing on the target pose to obtain a stabilized pose. The pose stabilization processing includes: performing reverse compensation on the roll angle of the target pose;

[0044] A screen recording module, configured to perform screen recording on the rendering result of the virtual scene according to the stabilized pose to obtain a screen recording image corresponding to the time of the target pose, and store the screen recording image.

[0045] In a third aspect, an embodiment of the present application provides an XR device. The XR device includes: a processor and a memory. The memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method described in any one of the above.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium is used to store a computer program, and the computer program causes a computer to execute the method described in any one of the above.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which when executed by a processor implements the method described in any one of the above.

[0048] The screen recording method, device, device, medium, and program for an extended reality device provided by the embodiments of the present application acquire a target pose corresponding to an XR device at time t. The target pose is the pose at time t or the first predicted pose at time t + m. Perform pose stabilization processing on the target pose to obtain a stabilized pose. The pose stabilization processing includes: performing reverse compensation on the roll angle of the target pose. According to the stabilized pose, perform screen recording on the rendering result of the virtual scene to obtain a screen recording image corresponding to the time of the target pose, and store the screen recording image. By performing reverse compensation on the roll angle of the target pose, the shaking of the XR device in the rolling direction corresponding to the roll angle is further reduced, thereby reducing the jitter of the screen recording video and improving the screen recording quality. Description of the Drawings

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0050] Figure 1 It is a schematic diagram of the principles of the rendering operation and the screen recording operation of an existing XR device;

[0051] Figure 2 It is a flowchart of the screen recording method for the XR device provided in the first embodiment of the present application;

[0052] Figure 3 It is a flowchart of the screen recording method for the XR device provided in the second embodiment of the present application;

[0053] Figure 4 It is a schematic diagram of the principles of the screen recording method for the XR device provided in the second embodiment of the present application;

[0054] Figure 5 It is a flowchart of the screen recording method for the XR device provided in the third embodiment of the present application;

[0055] Figure 6 It is a schematic diagram of the principles of the screen recording method for the XR device provided in the third embodiment of the present application;

[0056] Figure 7 It is a schematic diagram of the structure of the screen recording device for the extended reality device provided in the fourth embodiment of the present application;

[0057] Figure 8 It is a schematic diagram of a structure of the XR device provided in the fifth embodiment of the present application. Detailed implementation manners

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0059] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0060] The embodiment of the present application provides a screen recording method for an extended reality device, which can be used for screen recording of XR devices. The XR devices include, but are not limited to, VR devices, AR or MR devices. The method of this embodiment can be executed by an XR device, or by a server or other device that renders a virtual scene.

[0061] VR: A technology for creating and experiencing virtual worlds, which computationally generates a virtual environment. It is a multi-source information (the virtual reality mentioned in this article includes at least visual perception, and may also include auditory perception, tactile perception, motion perception, and even taste perception, olfactory perception, etc.). It realizes the integration of a virtual environment, an interactive three-dimensional dynamic visual scene and the simulation of entity behavior, enabling users to immerse themselves in the simulated virtual reality environment and realizing applications in various virtual environments such as maps, games, videos, education, medical care, simulation, collaborative training, sales, assisting manufacturing, maintenance and repair.

[0062] AR: An AR scene refers to a simulated scene in which at least one virtual object is superimposed on a physical scene or its representation. For example, an electronic system may have an opaque display and at least one imaging sensor for capturing images or videos of the physical scene, which are representations of the physical scene. The system combines the images or videos with virtual objects and displays the combination on the opaque display. An individual uses the system to indirectly view the physical scene via the images or videos of the physical scene and observes the virtual objects superimposed on the physical scene. When the system uses one or more image sensors to capture images of the physical scene and uses those images to present an AR scene on the opaque display, the displayed images are called video pass-through. Alternatively, an electronic system for displaying an AR scene may have a transparent or translucent display through which an individual can directly view the physical scene. The system can display virtual objects on the transparent or translucent display such that an individual uses the system to observe the virtual objects superimposed on the physical scene. For another example, the system may include a projection system that projects virtual objects into the physical scene. The virtual objects can be projected, for example, onto a physical surface or as a hologram such that an individual uses the system to observe the virtual objects superimposed on the physical scene. Specifically, it is a technology that calculates the camera pose parameters of a camera in the real world (or three-dimensional world, real world) in real time during the process of the camera capturing images, and adds virtual elements to the images captured by the camera according to the camera pose parameters. The virtual elements include, but are not limited to: images, videos, and 3D models. The goal of AR technology is to interact by overlaying the virtual world on the real world on the screen.

[0063] MR: By presenting virtual scene information in a real-world scenario, an interactive feedback information loop is established among the real world, the virtual world, and the user to enhance the realism of the user experience. For example, integrating sensory inputs created by a computer (such as virtual objects) with sensory inputs from a physical scene or its representation in a simulated scene. In some MR scenes, the sensory inputs created by the computer can adapt to changes in the sensory inputs from the physical scene. Additionally, some electronic systems for presenting MR scenes can monitor the orientation and / or position relative to the physical scene so that virtual objects can interact with real objects (i.e., physical elements from the physical scene or their representations). For example, the system can monitor movement such that a virtual plant appears stationary relative to a physical building.

[0064] A virtual reality device refers to a terminal that realizes virtual reality effects and can usually be provided in the form of glasses, a head-mounted display (abbreviated as HMD), contact lenses, etc. for realizing visual perception and other forms of perception. Of course, the form in which the virtual reality device is realized is not limited to this and can be further miniaturized or enlarged according to actual needs.

[0065] Optionally, the virtual reality devices (i.e., XR devices) described in the embodiments of the present application may include, but are not limited to, the following types:

[0066] (1) Mobile virtual reality devices, which support setting a mobile terminal (such as a smartphone) in various ways (such as a head-mounted display with a dedicated card slot), and through a wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations for virtual reality functions and outputs data to the mobile virtual reality device. For example, watch virtual reality videos through the APP of the mobile terminal.

[0067] (2) All-in-one virtual reality devices, which have a processor for performing relevant calculations for virtual functions, and thus have independent virtual reality input and output functions, do not need to be connected to a PC or a mobile terminal, and have a high degree of freedom of use.

[0068] (3) Personal computer VR (PCVR) devices, which use the PC to perform relevant calculations for virtual reality functions and data output, and the external PCVR device uses the data output by the PC to achieve the virtual reality effect.

[0069] The rendering operation of the XR device needs to be performed based on the pose of the XR device at the display time. The pose of the XR device is a 6 degrees of freedom (6DOF) data, including the position and orientation of the XR device. The rendering operation of the XR device refers to rendering the virtual scene. The rendering operation of the XR device takes a certain amount of time. Therefore, it is necessary to perform the rendering operation in advance for a period of time before the display time. The pose used for performing the rendering operation in advance is the predicted pose of the XR device at the future display time. Usually, the pose of the XR device at the current time, or the pose of the XR device at the current time and the previous time is used to predict the pose of the XR device at the future display time.

[0070] Figure 1 For the schematic diagrams of the rendering operation and the screen recording operation of the existing XR device, as Figure 1 shown, the XR device starts rendering the image to be displayed at the (t + m)-th moment at the t-th moment. Relative to the t-th moment, the (t + m)-th moment is the future display time, that is, the XR device performs the rendering operation m moments in advance. The pose used for performing the rendering operation in advance is the pose at the (t + m)-th moment (denoted as Pt + m) predicted according to the pose of the XR device at the current time. After predicting P t+m and then, according to P t+m the virtual scene is rendered to obtain the rendering result (denoted as R t+m ).

[0071] Since the pose used for rendering is not the true pose at the (t + m)-th moment but a predicted pose obtained in advance, and the predicted pose result is not accurate, after the rendering is completed, the XR device re-predicts the pose of the XR device at the (t + m)-th moment (P′ t+m ) according to the pose at the end of the rendering (denoted as t + n). Among them, the true pose of the XR device at a certain moment refers to the pose of the XR device detected by the positioning module of the XR device.

[0072] According to Figure 1 the timing relationship shown, the (t + n)-th moment is closer to the (t + m)-th moment, so the predicted P′ t+m is more accurate than P t+m . Before being displayed on the screen, the XR device uses P′ t+m to warp the previous rendering result R t+m , and the warping result is W′ t+m . After intercepting (or called cropping) the warping result W′ t+m , it is displayed on the screen at the (t + m)-th moment.

[0073] The existing screen recording function encodes and stores the image displayed on the screen, that is, the intercepted data of the warping result W′ t+m is used for both screen display and encoding and storage. During the use of the XR device, any head rotation of the user will cause the displayed picture of the XR device to change accordingly. When the user's head shakes frequently, the displayed picture and the screen recording picture of the XR device will shake accordingly, resulting in a large shake of the screen recording picture and a low screen recording quality. This causes a serious sense of dizziness when the user watches the screen recording picture and a poor user experience.

[0074] Figure 1 The time unit of the time series shown is milliseconds (ms). The difference between the t-th moment and the (t + m)-th moment is m milliseconds, and the difference between the t-th moment and the (t + n)-th moment is n milliseconds. It can be understood that the time unit can be smaller than milliseconds. For example, the time unit can be microseconds.

[0075] To solve the problems of the existing technology, the embodiment of the present application provides a screen recording method for an extended reality device, performs pose stabilization processing on the pose of the XR device, and uses the pose after the stabilization processing to record the rendering result of the virtual scene, so that the screen recording picture is more stable, without an obvious sense of shake, and reduces the sense of dizziness when the user watches the screen recording picture.

[0076] Combined with Figure 1 the timing relationship shown, the screen recording method of the XR device provided by the embodiment of the present application is described. Figure 2The flowchart of the screen recording method for the XR device provided in the first embodiment of this application. The method of this embodiment can be executed by the XR device, such as Figure 2 shown. The method provided in this embodiment includes the following steps.

[0077] S101. Obtain the target pose corresponding to the XR device at time t, where the target pose is the pose at time t or the first predicted pose at time t + m.

[0078] The pose at time t is the pose obtained by positioning the positioning module of the XR device, which is the actual pose of the XR device. t + m is a future time after time t. The actual pose at time t + m that cannot be obtained by the positioning module of the XR device at time t needs to predict the pose of the XR device at time t + m based on the actually measured poses. The predicted pose of the XR device at time t + m is called the first predicted pose.

[0079] The XR device can predict the pose at time t + m based on the pose at time t, or can also predict the pose at time t + m based on the pose at time t and the poses at multiple times before time t. Exemplarily, a neural network model can be used to predict the pose at time t + m, or a fitting method can be used to determine the pose at time t + m. This is only an example here, and other methods can also be used to predict the pose at time t + m. The embodiments of this application do not limit this.

[0080] Taking the neural network model as an example, input the pose at time t, or the pose at time t and the poses at multiple times before time t into the neural network model, and the output of the neural network model is the first predicted pose at time t + m.

[0081] Optionally, the positioning module includes but is not limited to a Simultaneous Localization and Mapping (SLAM) module, an Inertial Measurement Unit (IMU), a gyroscope, a laser, an accelerometer, an infrared sensor, etc. Among them, the SLAM module can perform positioning based on the images captured by the camera module and the IMU data.

[0082] In this embodiment, the time unit can be milliseconds, and the difference between time t and time t + m is m milliseconds. The time unit can also be a unit smaller than milliseconds. For example, the time unit can be microseconds.

[0083] S102. Perform pose stabilization processing on the target pose to obtain a stabilized pose. The pose stabilization processing includes: performing reverse compensation on the roll angle of the target pose.

[0084] The pose of the XR device includes position and orientation. Among them, the position refers to the values of the XR device on the X, Y, and Z axes, and the orientation refers to the rotation angles of the XR device on the X, Y, and Z axes. The pose of the XR device includes three angles: yaw angle, pitch angle, and roll angle. Among them, the angle of rotation around the X axis is called the pitch angle, the angle of rotation around the Y axis is called the yaw angle, and the angle of rotation around the Z axis is called the roll angle, and the roll angle is also called the bank angle. Usually, when the user wears the XR device, the horizontal direction is the X-axis direction, the vertical direction (i.e., the direction of gravity) is the Y-axis direction, and the front-back direction is the Z-axis direction.

[0085] The rotation of the roll angle has a greater impact on the jitter of the screen recording. Therefore, in this embodiment, the roll angle of the target pose is compensated in the reverse direction. Reverse compensation means compensating in the opposite direction of the rotation direction of the roll angle to reduce the rotation angle of the roll angle of the target pose, so that the change of the target pose for screen recording is relatively gentle.

[0086] Exemplarily, the roll angle of the target pose can be compensated in the reverse direction in the following several ways:

[0087] The first implementation method is to compensate the roll angle of the target pose in the reverse direction so that the value of the roll angle of the target pose after reverse compensation is a preset value.

[0088] This preset value can be 0, or it can be other values. For example, in some scenarios, the preset value is set to 90 degrees. In this method, after starting the screen recording, no matter how the XR device rotates, the value of the roll angle of the XR device is the preset value, that is, the roll angle of the XR device always remains unchanged, thereby reducing the jitter of the screen recording.

[0089] Among them, the preset value of 0 means that the rotation angle of the roll angle is 0, that is, the rotation angle of the roll angle of the XR device from the previous moment to the current moment is 0, that is, the X device does not rotate on the Z axis. In this method, after starting the screen recording, no matter how the XR device rotates, the value of the roll angle of the XR device is 0, that is, the roll angle of the XR device always remains unchanged, thereby reducing the jitter of the screen recording.

[0090] The second implementation method is to compensate the roll angle of the target pose in the reverse direction when the roll angle of the target pose is greater than or equal to a preset angle threshold, so that the value of the roll angle of the target pose after reverse compensation is a preset value.

[0091] The preset value can be 0 or other values. The value of the angle threshold should be such that when the roll angle of the target pose is less than the preset angle threshold, the rotation of the roll angle has no or little impact on the jitter of the screen recording. Therefore, in this method, when the roll angle of the target pose is less than the preset angle threshold, no reverse compensation is performed on the roll angle of the target pose. When the roll angle of the target pose is greater than or equal to the angle threshold, reverse compensation is performed on the roll angle of the target pose, and the value of the roll angle of the target pose after reverse compensation is the preset value. Exemplarily, the angle threshold is 1 degree, 2 degrees, or 5 degrees, etc.

[0092] In the third implementation method, when the roll angle of the target pose is greater than or equal to the preset angle threshold, reverse compensation is performed on the roll angle of the target pose, so that the value of the roll angle of the target pose after reverse compensation is the angle threshold.

[0093] Exemplarily, the angle threshold is 1 degree, 2 degrees, or 5 degrees, etc. When the roll angle of the target pose is greater than or equal to the preset angle threshold, by reverse compensating the value of the roll angle of the target pose to the angle threshold, it is avoided that the reverse compensation angle is too large, resulting in the jump of the screen recording image, thereby making the screen recording image more continuous and stable.

[0094] In the fourth implementation method, when the roll angle of the target pose is greater than or equal to the preset angle threshold, reverse compensation is performed on the roll angle of the target pose according to the preset compensation angle.

[0095] In this method, compensation is performed according to the preset compensation angle. The compensation angle can be less than, greater than, or equal to the preset angle threshold. Exemplarily, the compensation angle is 2 degrees, 3 degrees, or 5 degrees, etc. The value of the roll angle of the target pose after compensation using this method may be 0 or may not be 0.

[0096] In the fifth implementation method, reverse compensation is performed on the roll angle of the target pose according to the preset compensation ratio.

[0097] In this method, reverse compensation is performed according to the preset compensation ratio. The compensation ratio is greater than 0 and less than 1. Exemplarily, the compensation ratio is 1 / 2 or 2 / 3. When the value of the roll angle of the target pose is 6 and the compensation ratio is 2 / 3, the reverse compensation angle is 6*(2 / 3) = 4, and the value of the roll angle after reverse compensation is 2.

[0098] The above several implementation methods are only examples. This embodiment can also use other methods for reverse compensation, and the value of the roll angle after reverse compensation is reduced.

[0099] Optionally, the pose stabilization process further includes filtering the target pose to smooth the target pose through the filtering process, making the change of the stabilized pose smoother. This filtering process is also called low-pass filtering, and any existing filtering method can be used for filtering, which is not limited in this embodiment.

[0100] The filtering process adopts any one of the following filtering methods: Bayesian filtering, wavelet filtering, Kalman filtering, Wiener filtering, Butterworth filtering, Chebyshev filtering, B-spline fitting, or multiple fittings. The filtering parameters can be adjusted according to different actual usage scenarios and devices, so that the filtering effect meets the user's expectations.

[0101] Optionally, the target pose can be filtered first, and then the roll angle after the filtering process can be compensated in reverse. Or, the roll angle of the target pose can be compensated in reverse first, and then the target pose after the reverse compensation can be filtered.

[0102] Optionally, when filtering the target pose or the target pose after reverse compensation, only the yaw angle and / or pitch angle of the target pose or the target pose after reverse compensation can be filtered, and the roll angle is not filtered.

[0103] Optionally, the roll angle of the stabilized pose is a preset value, which can be 0 for example. For example, when only the yaw angle and / or pitch angle of the target pose are filtered and the roll angle is not filtered, since the roll angle value of the target pose after reverse compensation is the preset value, correspondingly, the roll angle value of the stabilized pose is the preset value.

[0104] S103. According to the stabilized pose, record the screen of the rendering result of the virtual scene, and store the screen recording image corresponding to the target pose at that moment.

[0105] In the embodiment of the present application, the rendering operation and the on-screen display of the XR device can still be performed using existing methods, and only the screen recording process is improved. Therefore, the screen recording process is mainly described in this embodiment.

[0106] In one implementation manner, according to the stabilized target pose, the first rendering result corresponding to the t + m moment is distorted to obtain a first distortion result, and the first distortion result is intercepted to obtain the screen recording image corresponding to the target pose at that moment. The first rendering result is the rendering result obtained by the XR device rendering the virtual scene according to the first predicted pose at the t + m moment.

[0107] Distorting the first rendering result can be understood as correcting or calibrating the first rendering result. This is because the stabilized pose has changed relative to the first predicted pose used for rendering the first rendering result. After the pose changes, the rendering result will also change accordingly. However, re-rendering takes a relatively long time, while the distortion operation is a 2D operation and takes a very short time. Therefore, the distortion operation is used to correct the first rendering result, and the corrected rendering result (i.e., the first distorted result) matches the stabilized pose.

[0108] Optionally, the XR device can use Asynchronous Timewarp (ATW) or Asynchronous Spacewarp (ASW) to distort the first rendering result.

[0109] The first distorted result obtained by distorting the first rendering result is a spherical scene. However, the display screen of the XR device (i.e., the screen that can be seen within the user's view) is only a part of the spherical scene. Therefore, a frame of image can be intercepted from the spherical scene according to the size of the display screen as the screen recording image, and the XR device encodes and stores the intercepted screen recording image.

[0110] In another implementation, according to the stabilized pose, the virtual scene is rendered to obtain a second rendering result, and the second rendering result is intercepted to obtain the screen recording image at the moment corresponding to the target pose.

[0111] In this implementation, two-way rendering is required. One-way rendering is for normal screen display, which can refer to Figure 1 the rendering operation and the screen display process shown. The other-way rendering is for screen recording. According to the stabilized pose, the virtual scene is rendered to obtain a second rendering result, the screen recording image is obtained using the second rendering result, and the screen recording image is stored.

[0112] Optionally, before the XR device stores the screen recording image, operations such as encoding the screen recording image are performed to encode the image into the required format. Correspondingly, when playing the screen recording image, the screen recording image is decoded and then played. The XR device can store the screen recording image locally. Specifically, the screen recording image can be stored on a disk or a hard disk, or can be stored in the memory first.

[0113] Optionally, the XR device can also store the screen recording image on other devices. For example, the XR device stores the encoded screen recording image in a mobile phone or a computer connected to the XR device.

[0114] It can be understood that this embodiment takes the recording of a single frame image as an example. In the XR scenario, multiple frames of images are usually captured continuously. The continuously captured recorded images can be encoded and stored as a recorded video, and the user can subsequently view the stored recorded video.

[0115] In this embodiment, the target posture corresponding to the XR device at time t is obtained, and the target posture is the posture at time t or the first predicted posture at time t+m, and the target posture is subjected to posture stabilization processing to obtain a stabilized posture, and the posture stabilization processing includes: reverse compensation for the roll angle of the target posture; according to the stabilized posture, the rendering result of the virtual scene is recorded to obtain a recorded screen image at the time corresponding to the target posture, and the recorded screen image is stored. By reversely compensating for the roll angle of the target posture, the shaking of the XR device in the roll direction corresponding to the roll angle is further reduced, thereby reducing the jitter of the recorded screen and improving the quality of the recorded screen.

[0116] Based on the first embodiment, the second embodiment of the present application provides a screen recording method for an XR device. In this embodiment, the virtual scene is rendered one way, and the screen recording process and the screen display process use different postures to distort the rendering results. The screen recording process uses a stabilization posture to distort the rendering results. Figure 3 This is a flowchart of the screen recording method of the XR device provided in the second embodiment of the present application. Figure 4 This is a schematic diagram of the principle of the screen recording method of the XR device provided in the second embodiment of the present application, referring to Figure 3 and Figure 4 , the method provided in this embodiment includes the following steps.

[0117] S201. Obtain the posture of the XR device at time t, and predict a first predicted posture at time t+m based on the posture at time t.

[0118] Reference Figure 4 As shown, the pose at time t is represented by P t , the first predicted pose at time t+m is represented as P t+m At time t, the XR device detects the posture at time t through its own positioning module, and predicts the first predicted posture at time t+m based on the posture at time t.

[0119] S202: Render the virtual scene according to the first predicted posture at time t+m to obtain a first rendering result.

[0120] S203, performing posture stabilization processing on the target posture to obtain a stabilized posture, wherein the posture stabilization processing includes: reverse compensation for the roll angle of the target posture, and filtering processing on the target posture, wherein the target posture is the posture at time t or the first predicted posture at time t+m.

[0121] Figure 4 The target posture at time t is used as an example to illustrate. Figure 4 As shown, there is no order in which step S202 and step S203 are executed, and they can also be executed simultaneously. The first rendering result obtained by rendering step S202 is Figure 4 The rendering result in R t+m .

[0122] The specific process of posture stabilization processing refers to the relevant description of Example 1 and will not be repeated here.

[0123] S204: distort the first rendering result according to the stabilization posture to obtain a first distorted result.

[0124] S205. The first distortion result is intercepted to obtain a screen recording image at a time corresponding to the target posture, and the screen recording image is encoded and stored.

[0125] Figure 4 The target posture is taken as the posture at time t as an example, and the first distortion result obtained in step S204 is Figure 4 W″ t Correspondingly, the screen recording result is the screen recording image at time t. After encoding and other operations are performed on the screen recording image at time t, it is stored in the screen recording file.

[0126] In this embodiment, after obtaining the first rendering result, the screen is recorded according to the stabilization posture and the first rendering result to obtain a screen recording image at the time corresponding to the target posture. In one implementation, the screen recording operation is performed through the above steps S204 and S205. Optionally, in other implementations, the recording frame can also be distorted according to the stabilization posture to obtain a target recording frame, and the first rendering result is cut using the target recording frame to obtain a screen recording image at the time corresponding to the target posture.

[0127] Step S206: Obtain the posture of the XR device at time t+n, and predict a second predicted posture of the XR device at time t+m based on the posture at time t+n, where the value of n is less than m.

[0128] The posture of the XR device at time t+n is the posture detected by the positioning module of the XR device. The value of n is less than m, that is, time t+n is the time before time t+m. Time t+n is the time corresponding to the distortion operation before the screen display. Time t+n is between the rendering end time and the screen display time. Time t+n may be equal to the rendering end time or later than the rendering end time.

[0129] According to the posture at time t+n, the second predicted posture at time t+m can be predicted. The prediction method of the second predicted posture is the same as the prediction method of the first predicted posture, which will not be repeated here.

[0130] S207. Distort the first rendering result according to the second predicted pose to obtain a second distorted result.

[0131] S208. At time t + m, intercept the second distorted result and display it on the screen.

[0132] Reference Figure 4 , the second predicted pose at time t + m is Figure 4 P' in t+m , the second distorted result is Figure 4 W' in t+m . At time t + m, intercept the second distorted result according to the size of the display screen to obtain the display image at time t + m, and display the display image at time t + m on the screen. Reference Figure 4 According to the timing relationship shown, it can be seen that steps S203 - S205 and steps S206 - S208 are executed in parallel.

[0133] In this embodiment, the stabilized pose is obtained by performing pose stabilization processing on the target pose. The first rendering result at time t + m is distorted using the stabilized pose to obtain a first distorted result. The screencapture image corresponding to the target pose is obtained by intercepting the first distorted result, and the screencapture image is encoded and stored. The stabilized pose is smoother, eliminating or greatly reducing the jitter in the roll direction, thereby reducing the jitter of the screencapture video.

[0134] Based on Embodiment 1, Embodiment 3 of the present application provides a screencapture method for an XR device. In this embodiment, two - way rendering of the virtual scene is performed. One rendering result is used for on - screen display, and one rendering result is used for screencapture. The screencapture process uses the pose after stabilization processing for rendering. Figure 5 is the flowchart of the screencapture method for the XR device provided in Embodiment 3 of the present application, Figure 6 is the schematic diagram of the principle of the screencapture method for the XR device provided in Embodiment 3 of the present application. Referring to Figure 5 and Figure 6 , the method provided in this embodiment includes the following steps.

[0135] S301. Obtain the pose of the XR device at time t, and predict the first predicted pose at time t + m according to the pose at time t.

[0136] S302. Perform pose stabilization processing on the target pose to obtain a stabilized pose. This pose stabilization processing includes: performing reverse compensation on the roll angle of the target pose and performing filtering processing on the target pose. The target pose is the pose at time t or the first predicted pose at time t + m.

[0137] S303. Render the virtual scene according to the stabilized pose to obtain a second rendering result.

[0138] S304. Crop the second rendering result to obtain a screencast image corresponding to the moment of the target pose, and store the screencast image.

[0139] Reference Figure 6 As shown Figure 6 In, the target pose is taken as the pose at time t for illustration. Correspondingly, the second rendering result obtained in step S303 is Figure 6 R in t , the screencast result is the screencast image at time t. After operations such as encoding the screencast image at time t, it is stored in the screencast file.

[0140] S305. Render the virtual scene according to the first predicted pose at time t + m to obtain a first rendering result.

[0141] S306. Obtain the pose of the XR device at time t + n, and predict the second predicted pose of the XR device at time t + m according to the pose at time t + n, where the value of n is less than m.

[0142] S307. Distort the first rendering result according to the second predicted pose to obtain a second distortion result.

[0143] S308. Crop the second distortion result at time t + m and display it on the screen.

[0144] Reference Figure 6 , the second predicted pose at time t + m is Figure 4 P' in t+m , the second distortion result is Figure 6 W' in t+m , at time t + m, crop the second distortion result according to the size of the display screen to obtain a display image at time t + m, and display the display image at time t + m on the screen. Referring to Figure 6 As shown in the timing relationship, it can be seen that steps S302 - S304 and steps S305 - S308 are executed in parallel.

[0145] In this embodiment, the screencast process and the screen display process are rendered separately. Among them, the rendering result for screencasting does not need to be distorted. The second rendering result is obtained by rendering using the stabilized pose, and operations such as cropping, encoding, and storing are directly performed on the second rendering result. Since the second rendering result for screencasting does not need to be distorted, therefore, when rendering the virtual scene, optionally, the virtual scene can be locally rendered according to the stabilized pose to obtain the second rendering result.

[0146] Local rendering is relative to global rendering. Global rendering refers to rendering a spherical area centered on the pose of the XR device, that is, the rendering result is a spherical scene. The rendering area of local rendering is smaller than the spherical rendering area, and it can render a hemispherical area, a 1 / 4 spherical area, a 1 / 3 spherical area, a 1 / 2 spherical area, etc. The size of the local rendering area is not limited in this embodiment, and the local rendering area is greater than or equal to the size of the display area of the XR device.

[0147] It can be understood that the larger the rendering area, the greater the resources and time required for rendering, and the smaller the rendering area, the smaller the resources and time required for rendering. Compared with global rendering, local rendering can save computing resources and reduce the rendering time.

[0148] In this embodiment, the screen recording process can adopt local rendering, and the screen display process adopts global rendering, so as to save the computing resources for rendering and reduce the power consumption of the XR device.

[0149] Optionally, in one implementation, both the screen recording process and the screen display process can adopt global rendering, and this embodiment does not limit this.

[0150] In this embodiment, by performing pose stabilization processing on the target pose, using the obtained stabilized pose to render the virtual scene, obtaining a second rendering result, intercepting the second rendering result to obtain the screen recording image corresponding to the target pose at a certain moment, and encoding and storing the screen recording image. The stabilized pose is smoother than the target pose, eliminating or greatly reducing the jitter in the roll direction, thereby reducing the jitter of the screen recording video.

[0151] It should be noted that Figure 4 and Figure 6 the target pose used for screen recording in is the actual pose at time t. In other embodiments of this application, the target pose can also be replaced with the first predicted pose at time t + m. Whether the pose at time t or the first predicted pose at time t + m is used as the target pose, the screen recording process and principle are the same, and will not be repeated here.

[0152] To facilitate better implementation of the screen recording method for the extended reality device in the embodiments of this application, the embodiments of this application also provide a screen recording device for an extended reality device. Figure 7 For the structural schematic diagram of the screen recording device for the extended reality device provided in the fourth embodiment of this application, as Figure 7 shown, the screen recording device 100 of this extended reality device may include:

[0153] A pose acquisition module 11, configured to acquire the target pose corresponding to the extended reality (XR) device at time t, where the target pose is the pose at time t or the first predicted pose at time t + m;

[0154] The stabilization processing module 12 is configured to perform pose stabilization processing on the target pose to obtain a stabilized pose. The pose stabilization processing includes: performing reverse compensation on the roll angle of the target pose;

[0155] The screen recording module 13 is configured to perform screen recording on the rendering result of the virtual scene according to the stabilized pose to obtain a screen recording image corresponding to the moment of the target pose, and store the screen recording image.

[0156] In some embodiments, the stabilization processing module 12 is specifically configured to:

[0157] Perform reverse compensation on the roll angle of the target pose so that the value of the roll angle of the target pose after reverse compensation is a preset value;

[0158] In some embodiments, the stabilization processing module 12 is specifically configured to:

[0159] When the roll angle of the target pose is greater than or equal to a preset angle threshold, perform reverse compensation on the roll angle of the target pose so that the value of the roll angle of the target pose after reverse compensation is a preset value or the angle threshold;

[0160] Or, when the roll angle of the target pose is greater than or equal to a preset angle threshold, perform reverse compensation on the roll angle of the target pose according to a preset compensation angle.

[0161] In some embodiments, the stabilization processing module 12 is specifically configured to:

[0162] Perform reverse compensation on the roll angle of the target pose according to a preset compensation ratio.

[0163] In some embodiments, the pose stabilization processing further includes: performing filtering processing on the target pose.

[0164] In some embodiments, the filtering processing adopts any one of the following filtering methods: Bayesian filtering, wavelet filtering, Kalman filtering, Wiener filtering, Butterworth filtering, Chebyshev filtering, B-spline fitting, or multiple fitting.

[0165] In some embodiments, performing the filtering processing on the target pose includes:

[0166] Performing filtering on the pitch angle and / or yaw angle of the target pose.

[0167] In some embodiments, the device further includes a rendering module;

[0168] The rendering module is configured to render the virtual scene according to the first predicted pose at the t + m moment to obtain a first rendering result;

[0169] The screen recording module 13 is specifically configured to: perform screen recording based on the stabilized pose and the first rendering result to obtain a screen recording image at the moment corresponding to the target pose.

[0170] In some embodiments, the screen recording module 13 is specifically configured to: distort the first rendering result according to the stabilized pose to obtain a first distortion result;

[0171] Intercept the first distortion result to obtain a screen recording image at the moment corresponding to the target pose.

[0172] In some embodiments, the device further includes a screen display module;

[0173] The pose acquisition module 11 is further configured to acquire the pose of the XR device at the moment of t + n, and predict the second predicted pose of the XR device at the moment of t + m according to the pose at the moment of t + n, where the value of n is less than m;

[0174] The screen display module is configured to distort the first rendering result according to the second predicted pose to obtain a second distortion result, and perform screen display after intercepting the second distortion result at the moment of t + m.

[0175] In some embodiments, the screen recording module 13 is specifically configured to:

[0176] Render the virtual scene according to the stabilized pose to obtain a second rendering result;

[0177] Intercept the second rendering result to obtain a screen recording image at the moment corresponding to the target pose.

[0178] In some embodiments, the device further includes a rendering module and a screen display module;

[0179] The rendering module is configured to render the virtual scene according to the first predicted pose at the moment of t + m to obtain a first rendering result;

[0180] The pose acquisition module 11 is further configured to acquire the pose of the XR device at the moment of t + n, and predict the second predicted pose of the XR device at the moment of t + m according to the pose at the moment of t + n, where the value of n is less than m;

[0181] The screen display module is configured to distort the first rendering result according to the second predicted pose to obtain a second distortion result, and perform screen display after intercepting the second distortion result at the moment of t + m.

[0182] In some embodiments, the screen recording module 13 is specifically configured to: perform local rendering on the virtual scene according to the stabilized pose to obtain the second rendering result.

[0183] In other embodiments, the screen recording module 13 is specifically configured to: perform global rendering on the virtual scene according to the first predicted pose to obtain the first rendering result.

[0184] It should be understood that the apparatus embodiments and the method embodiments can correspond to each other, and similar descriptions can refer to the method embodiments. To avoid repetition, it will not be elaborated here.

[0185] In the foregoing, the apparatus 100 of the embodiments of the present application has been described from the perspective of functional modules with reference to the accompanying drawings. It should be understood that the functional modules can be implemented in the form of hardware, or in the form of instructions in software, or in the form of a combination of hardware and software modules. Specifically, the steps of the method embodiments in the present application can be completed by the integrated logic circuit in the hardware in the processor and / or instructions in software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps in the foregoing method embodiments.

[0186] The embodiments of the present application further provide an XR device. Figure 8 FIG. is a schematic structural diagram of the XR device provided in the fifth embodiment of the present application, as Figure 8 shown, the XR device 200 may include:

[0187] A memory 21 and a processor 22. The memory 21 is used to store a computer program and transmit the program code to the processor 22. In other words, the processor 22 can call and run the computer program from the memory 21 to implement the method in the embodiments of the present application.

[0188] For example, the processor 22 can be used to execute the foregoing method embodiments according to the instructions in the computer program.

[0189] In some embodiments of the present application, the processor 22 may include, but is not limited to: a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on.

[0190] In some embodiments of the present application, the memory 21 includes, but is not limited to: volatile memory and / or non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0191] In some embodiments of the present application, the computer program may be divided into one or more modules, and the one or more modules are stored in the memory 21 and executed by the processor 22 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the XR device.

[0192] Such as Figure 8As shown, the XR device 200 may further include: a transceiver 23, which may be connected to the processor 22 or the memory 21.

[0193] Among them, the processor 22 can control the transceiver 23 to communicate with other devices. Specifically, it can send information or data to other devices, or receive information or data sent by other devices. The transceiver 23 may include a transmitter and a receiver. The transceiver 23 may further include an antenna, and the number of antennas may be one or more.

[0194] It can be understood that although Figure 8 not shown in the figure, the XR device 200 may further include a camera module, a Wi-Fi module, a positioning module, a Bluetooth module, a display, a controller, etc., which will not be elaborated here.

[0195] It should be understood that each component in the XR device is connected through a bus system. Among them, the bus system includes not only a data bus, but also a power bus, a control bus, and a status signal bus.

[0196] This application also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by the computer, the computer can execute the method in the above method embodiment. Or rather, this application embodiment also provides a computer program product containing instructions. When the instructions are executed by the computer, the computer executes the method in the above method embodiment.

[0197] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of the XR device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the XR device executes the corresponding process in the method embodiment. For the sake of brevity, it will not be elaborated here.

[0198] In several embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0199] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, in each embodiment of this application, the functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0200] The above are only specific embodiments of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for screen recording of an extended reality device, characterized in that, it includes: Obtain the target pose corresponding to the extended reality (XR) device at time t, where the target pose is the pose at time t or the first predicted pose at time t + m; Perform pose stabilization processing on the target pose to obtain a stabilized pose, and the pose stabilization processing includes: performing reverse compensation on the roll angle of the target pose; According to the stabilized pose, perform screen recording on the rendering result of the virtual scene to obtain a screen recording image corresponding to the time of the target pose, and store the screen recording image.

2. The method according to claim 1, characterized in that, the performing reverse compensation on the roll angle of the target pose includes: Performing reverse compensation on the roll angle of the target pose so that the value of the roll angle of the target pose after reverse compensation is a preset value.

3. The method according to claim 1, characterized in that, the performing reverse compensation on the roll angle of the target pose includes: When the roll angle of the target pose is greater than or equal to a preset angle threshold, perform reverse compensation on the roll angle of the target pose so that the value of the roll angle of the target pose after reverse compensation is a preset value or the angle threshold; Or, when the roll angle of the target pose is greater than or equal to a preset angle threshold, perform reverse compensation on the roll angle of the target pose according to a preset compensation angle.

4. The method according to claim 1, characterized in that, the performing reverse compensation on the roll angle of the target pose includes: Performing reverse compensation on the roll angle of the target pose according to a preset compensation ratio.

5. The method according to claim 1, characterized in that, the pose stabilization processing further includes: performing filtering processing on the target pose; The filtering processing adopts any one of the following filtering methods: Bayesian filtering, wavelet filtering, Kalman filtering, Wiener filtering, Butterworth filtering, Chebyshev filtering, B-spline fitting or multiple fitting.

6. The method according to claim 5, characterized in that, the performing filtering processing on the target pose includes: Performing filtering on the pitch angle and / or yaw angle of the target pose.

7. The method according to any one of claims 1-6, characterized in that, it further includes: According to the first predicted pose at time t + m, render the virtual scene to obtain a first rendering result; The performing screen recording on the rendering result of the virtual scene according to the stabilized pose to obtain a screen recording image corresponding to the time of the target pose includes: Performing screen recording according to the stabilized pose and the first rendering result to obtain a screen recording image corresponding to the time of the target pose.

8. The method according to claim 7, the performing screen recording according to the stabilized pose and the first rendering result to obtain a screen recording image corresponding to the time of the target pose, includes: According to the stabilized pose, distort the first rendering result to obtain a first distortion result; Perform cropping on the first distortion result to obtain a screen recording image corresponding to the time of the target pose.

9. The method according to claim 7, characterized in that, it further includes: Obtain the pose of the XR device at time t + n, and predict the second predicted pose of the XR device at time t + m according to the pose at time t + n, where the value of n is less than m; Distort the first rendering result according to the second predicted pose to obtain a second distorted result, and perform cropping on the second distorted result at time t + m and then display it on the screen.

10. The method according to any one of claims 1 - 6, characterized in that The step of obtaining the screencapture image corresponding to the target pose by performing screencapture on the rendering result of the virtual scene according to the stabilized pose includes: Render the virtual scene according to the stabilized pose to obtain a second rendering result; Perform cropping on the second rendering result to obtain the screencapture image corresponding to the target pose.

11. The method according to claim 10, characterized in that further comprising: Render the virtual scene according to the first predicted pose at time t + m to obtain a first rendering result; Obtain the pose of the XR device at time t + n, and predict the second predicted pose of the XR device at time t + m according to the pose at time t + n, where the value of n is less than m; Distort the first rendering result according to the second predicted pose to obtain a second distorted result, and perform cropping on the second distorted result at time t + m and then display it on the screen.

12. The method according to claim 10, characterized in that The step of rendering the virtual scene according to the stabilized pose to obtain a second rendering result includes: Perform local rendering on the virtual scene according to the stabilized pose to obtain the second rendering result.

13. The method according to claim 7, characterized in that The step of rendering the virtual scene according to the first predicted pose at time t + m to obtain a first rendering result includes: Perform global rendering on the virtual scene according to the first predicted pose to obtain the first rendering result.

14. A screencapture device for an extended reality device, characterized in that comprising: A pose acquisition module, configured to acquire the target pose corresponding to the extended reality (XR) device at time t, where the target pose is the pose at time t or the first predicted pose at time t + m; A stabilization processing module, configured to perform pose stabilization processing on the target pose to obtain a stabilized pose, and the pose stabilization processing includes: performing reverse compensation on the roll angle of the target pose; A screencapture module, configured to perform screencapture on the rendering result of the virtual scene according to the stabilized pose to obtain the screencapture image corresponding to the target pose, and store the screencapture image.

15. An extended reality device, characterized in that comprising: A processor and a memory, where the memory is used to store a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that used to store a computer program, and the computer program causes a computer to execute the method according to any one of claims 1 to 13.

17. A computer program product comprising a computer program, wherein, when the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images