A method and device for generating a screen recording video

Abstract

The application provides a screen recording video generation method and device, which is used for generating a screen recording video according to a specified encoding frame rate. The method comprises the following steps: every target time length, a target video image in each video image in a virtual display screen is obtained, wherein each video image is obtained by frame extraction on a video image displayed by a first device based on a screen refresh rate of the first device, the target time length is obtained based on the specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate; the target video image is rendered to a target texture corresponding to the virtual display screen; the target video image on the target texture is rendered to a target Surface through an OpenGL ES rendering thread; the target video image is sent to a media codec thread through the target Surface, so that the target video image is encoded through the media codec thread; and a screen recording video is obtained according to each encoded video image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and device for generating screen-recorded videos. Background Technology

[0002] Current screen recording technology utilizes three standard APIs (Application Programming Interfaces) within the Android SDK (Software Development Kit): MediaProjection, VirtualDisplay, and MediaCodec. MediaProjection is a dedicated class in the Android SDK for screen recording, allowing the creation of VirtualDisplay objects. MediaCodec is another class in the Android SDK used for audio and video data encoding and decoding; for encoding, it takes raw audio and video data as input and outputs encoded audio and video data.

[0003] like Figure 1 The diagram illustrates the relationship between the three. A VirtualDisplay object is created using MediaProjection. This VirtualDisplay object contains a Surface, and the image displayed on the screen will be automatically rendered onto this OutSurface at the aforementioned frame rate. This Surface then serves as the MediaCodec's InputSurface, which is the data source for image encoding. Thus, the image data displayed on the screen directly serves as the encoder's data source, which in turn determines the frame rate of the encoder's input / output data—a process that cannot be controlled. Therefore, the frame rate of the screen-recorded video using this method is the same as the screen refresh rate, resulting in a fixed and unchangeable frame rate. However, this method may cause the receiving end to fail to display the video recorded by the mobile device correctly. For example, a screen-recorded video recorded by a mobile device at a high screen refresh rate may not be displayed correctly due to insufficient decoding capabilities on the receiving end. Summary of the Invention

[0004] This application provides a method and device for generating screen recording videos, which generates screen recording videos according to a specified encoding frame rate, rather than based on the screen refresh rate, so that the receiving end can decode the screen recording video and ensure that the receiving end can display the recorded video normally.

[0005] In a first aspect, embodiments of this application provide a method for generating screen-recorded videos, the method comprising:

[0006] At each target duration, a target video image is acquired from each video image stored in the virtual display screen of the first device. Each video image is obtained by extracting frames from the video image displayed by the first device based on the screen refresh rate of the first device after receiving the screen recording instruction. The target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate.

[0007] The target video image is rendered onto a target texture corresponding to the virtual display screen;

[0008] The target video image on the target texture is rendered into the target surface using the OpenGL ES rendering thread;

[0009] The target video image is sent to the media codec thread via the target Surface, so that the target video image can be encoded by the media codec thread to obtain the encoded video image;

[0010] Upon receiving the screen recording end command, the screen recording video is obtained based on each encoded video image, and the recorded video is sent to the second device for display.

[0011] A second aspect of this application provides a first device, including a processor and a memory, wherein the processor and the memory are connected via a bus;

[0012] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0013] At each target duration, a target video image is acquired from each video image stored in the virtual display screen of the first device. Each video image is obtained by extracting frames from the video image displayed by the first device based on the screen refresh rate of the first device after receiving the screen recording instruction. The target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate.

[0014] The target video image is rendered onto a target texture corresponding to the virtual display screen;

[0015] The target video image on the target texture is rendered into the target Surface using the OpenGL ES rendering thread;

[0016] The target video image is sent to the media codec thread via the target Surface, so that the target video image can be encoded by the media codec thread to obtain the encoded video image;

[0017] Upon receiving the screen recording end command, the screen recording video is obtained based on each encoded video image, and the recorded video is sent to the second device for display.

[0018] According to a third aspect of the present invention, a computer storage medium is provided, the computer storage medium storing a computer program for performing the method as described in the first aspect.

[0019] In the above embodiments of this application, a target video image is obtained by extracting frames from the video image displayed on the first device at regular intervals based on the screen refresh rate of the first device, stored in the virtual display screen of the first device. The target duration is based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate. The target video image is then rendered onto a target texture corresponding to the virtual display screen. The OpenGL ES rendering thread then renders the target video image from the target texture onto a target Surface. The target video image is then sent to a media codec thread via the target Surface for encoding, resulting in an encoded video image. Upon receiving a screen recording end command, the screen recording video is obtained based on each encoded video image and sent to a second device for display. Therefore, in this embodiment, an OpenGL ES rendering thread is inserted between the virtual display screen and the media codec thread. This rendering thread renders the video image from the target texture bound to the virtual display screen onto the target Surface of the media codec thread. The frequency of this rendering process is controllable, i.e., the specified encoding frequency. Therefore, in this embodiment, the screen recording video is generated according to the specified encoding frame rate, rather than based on the screen refresh rate. The specified encoding frame rate can be set according to the specific actual situation so that the receiving end can decode the screen recording video and ensure that the receiving end can display the recorded video normally. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 An exemplary diagram illustrates the generation of screen recording videos in the prior art provided in this application embodiment;

[0022] Figure 2 An exemplary diagram illustrating an application scenario provided in an embodiment of this application is shown;

[0023] Figure 3 An exemplary flowchart of the screen recording video generation method provided in an embodiment of this application is shown;

[0024] Figure 4 An exemplary schematic diagram of the process for acquiring a target video image provided in an embodiment of this application is shown;

[0025] Figure 5 An exemplary schematic diagram of one of the flowcharts of the method for determining multiple prediction images provided in an embodiment of this application is shown;

[0026] Figure 6 A second schematic flowchart of the method for determining multiple prediction images provided in an embodiment of this application is illustrated.

[0027] Figure 7 An exemplary schematic diagram of the surface provided in an embodiment of this application is shown;

[0028] Figure 8 An exemplary illustration shows the relationship between the OpenGLES rendering thread and the media codec thread provided in an embodiment of this application. Figure 2

[0029] Figure 9 An exemplary diagram of the screen recording video generation apparatus provided in an embodiment of this application is shown;

[0030] Figure 10 A schematic diagram of the device provided in an embodiment of this application is shown as an example. Detailed Implementation

[0031] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0032] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation on its own.

[0033] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0034] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to be omnipresent but not exclusive; for example, a product or device comprising a series of components is not necessarily limited to those explicitly listed, but may include other components not explicitly listed or inherent to such product or device.

[0035] As used in this application, the term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0036] The following is an overview of the ideas behind the embodiments of this application.

[0037] The existing method of encoding video images using a fixed screen refresh rate to obtain screen recording videos may result in the receiving end being unable to display the video recorded by the mobile device correctly. This application proposes a method for generating screen recording videos. The method involves obtaining target video images from the video images displayed on the first device's virtual display screen at target durations, based on the first device's own screen refresh rate. The target duration is based on a specified encoding frame rate, and the target video image is determined based on both the screen refresh rate and the specified encoding frame rate. The target video image is then rendered onto a target texture corresponding to the virtual display screen. An OpenGLES rendering thread then renders the target video image from the target texture onto a target Surface. The target video image is then sent to a media codec thread via the target Surface for encoding, resulting in an encoded video image. Upon receiving a screen recording end command, the screen recording video is obtained based on each encoded video image and sent to a second device for display. Therefore, in this embodiment, an OpenGL ES rendering thread is inserted between the virtual display screen and the media codec thread. This rendering thread renders the video image on the target texture bound to the virtual display screen onto the target Surface of the media codec thread. The frequency of this rendering thread is controllable, i.e., a specified encoding frequency. Therefore, this embodiment generates screen recording video according to a specified encoding frame rate, rather than based on the screen refresh rate. This specified encoding frame rate can be set according to specific actual conditions to enable the receiving end to decode the screen recording video and ensure that the receiving end can display the recorded video normally.

[0038] like Figure 2 As shown, an application scenario for a screen recording video generation method is presented, which includes a first device 201 and a second device 202.

[0039] In one possible application scenario, after receiving a screen recording command, the first device 201 extracts frames from the video images displayed on its own screen refresh rate to obtain various video images, and stores each video image in its own virtual display screen. Every target duration, the first device 201 acquires the target video image from the various video images stored in its virtual display screen, where the target duration is based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate. Then, the first device 201 renders the target video image onto a target texture corresponding to the virtual display screen; and renders the target video image on the target texture onto a target Surface using an OpenGLES rendering thread. The first device 201 then sends the target video image to a media codec thread through the target Surface, so that the target video image can be encoded by the media codec thread to obtain an encoded video image. When the first device 201 receives a screen recording end command, it obtains the screen recording video based on each encoded video image and sends the recorded video to the second device 202 for display.

[0040] in, Figure 2 The first device 201 and the second device 202 can exchange information through a communication network. The communication network can be either wireless or wired.

[0041] For example, the first device 201 can access the network via cellular mobile communication technology and communicate with the second device 202, wherein the cellular mobile communication technology includes, for example, 5th generation mobile networks (5G) technology.

[0042] Optionally, the first device 201 can access the network and communicate with the second device 202 via short-range wireless communication, wherein the short-range wireless communication method includes, for example, Wireless Fidelity (Wi-Fi) technology.

[0043] In this application, only a single first device 201 and a single second device 202 are described in detail. However, those skilled in the art should understand that the illustrated first device 201 and the single second device 202 are intended to illustrate the operation of the first device 201 and the second device 202 involved in the technical solution of this application, and do not imply any limitation on the number, type, or location of the first device 201 and the second device 202. It should be noted that adding additional modules to or removing individual modules from the illustrated environment will not change the underlying concept of the exemplary embodiments of this application.

[0044] It should be noted that the screen recording video generation method proposed in this application is not only applicable to... Figure 2 The application scenarios shown can also be applied to any device that generates screen recording videos.

[0045] The following describes the screen recording video generation method in an exemplary embodiment of this application, in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the methods and principles of this application, and the implementation of this application is not limited in any way in this respect.

[0046] The method for generating screen recording videos in the embodiments of this application will now be described with reference to the accompanying drawings. Figure 3 The diagram shown illustrates the process of generating screen-recorded videos, which may include the following steps:

[0047] Step 301: At each target duration, acquire the target video image from each video image stored in the virtual display screen of the first device, wherein each video image is obtained by extracting frames from the video image displayed by the first device based on the screen refresh rate of the first device after receiving the screen recording instruction, and the target duration is obtained based on the specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate.

[0048] For example, if the screen refresh rate of the first device is 120 FPS, then the first device needs to extract 120 frames of video images per second.

[0049] In one embodiment, the target duration is obtained by dividing a preset value by the specified encoding frame rate.

[0050] The preset value in this embodiment is 1, but this is not a limitation on the preset value. The preset value in this embodiment can be set according to the specific actual situation. The specified encoding frame rate in this embodiment can be set according to the actual situation and can be changed, but this embodiment does not limit the specified encoding frame rate.

[0051] The method for acquiring the target video image in step 301 is explained below. For example... Figure 4 The diagram shown illustrates the process of acquiring a target video image, which may include the following steps:

[0052] Step 401: Determine whether the specified encoding frame rate is less than the screen refresh rate. If yes, proceed to step 402; otherwise, proceed to step 403.

[0053] Step 402: Determine the target video image based on the screen refresh rate and the specified encoding frame rate;

[0054] In one embodiment, step 402 can be specifically implemented as follows: dividing the screen refresh rate by the specified encoding frame rate to obtain the interval frame number; multiplying the interval frame number by the number of times the target video image is acquired to obtain the target video image frame number; and determining the target video image based on the target video image frame number, wherein the interval frame number is the difference in the number of frames between two adjacent target video images. The interval frame number can be obtained using formula (1):

[0055] N = P1 / P2......(1);

[0056] Wherein, N is the number of interval frames, P1 is the screen refresh rate, and P2 is the specified encoding frame rate.

[0057] For example, if the interval between frames is 4, and the first frame corresponding to the target video image is 4, then the fourth frame is determined to be the target video image acquired in the first acquisition. If the second frame corresponding to the target video image is 8, then the eighth frame is determined to be the target video image acquired in the second acquisition.

[0058] Although the target video image and the video images preceding the target video image are deleted each time the encoded video image is obtained in this embodiment, the number of frames corresponding to each video image stored in this embodiment is fixed. For example, if the video images of the 1st to 10th frames are deleted, the number of frames of each remaining video image is still 5 to 10.

[0059] Step 403: Determine whether the specified encoding frame rate is equal to the screen refresh rate. If yes, end; otherwise, proceed to step 404.

[0060] Step 404: Perform image prediction using the video images to obtain multiple predicted images, and determine the video image with the target frame number as the target video image. The total number of the multiple predicted images and the video images is equal to the number of frames corresponding to the specified encoding frame rate, and the target frame number is obtained based on the specified encoding frame rate.

[0061] In this embodiment, the target frame number is the same as the specified encoding frame rate. For example, if the specified encoding frame rate is 120 FPS, then the target frame number is 120, meaning the 120th frame of the stored video images is determined as the target video image.

[0062] This application provides two methods for determining multiple prediction images in its embodiments. For example... Figure 5The diagram shown illustrates one of the flowcharts for determining multiple prediction images, which may include the following steps:

[0063] Step 501: Copy each video image to obtain multiple prediction images;

[0064] Step 502: Determine whether the number of the multiple predicted images is greater than the target number. If not, proceed to step 503; if yes, end.

[0065] The target number is obtained based on the specified encoding frame rate and the number of each video image;

[0066] In one embodiment, the target number is obtained by subtracting the value corresponding to the specified encoding frame rate from the number of each video image.

[0067] Step 503: After updating each video image using the multiple predicted images, return to step 501.

[0068] In one embodiment, step 503 may be specifically implemented as: determining the plurality of predicted images as video images.

[0069] In the embodiments of this application, the updated video images include multiple predicted images and each video image.

[0070] Figure 6 This is a second schematic diagram of the process for determining multiple predicted images in this application embodiment, which may specifically include the following steps:

[0071] Step 601: For any frame of video image in each video image, use a pre-trained image prediction model to perform image prediction on the arbitrary frame of video image to obtain a predicted image;

[0072] The image prediction model in this application embodiment can be set according to the specific actual situation, and this application embodiment does not limit the image prediction model.

[0073] Step 602: Determine whether the number of each predicted image is greater than the target number. If not, proceed to step 603; if yes, end.

[0074] The method for determining the target quantity in this embodiment is the same as the method for determining the target quantity in step 502 described above, and will not be repeated here.

[0075] The target number is obtained based on the specified encoding frame rate and the number of each video image;

[0076] Step 603: After updating each video image using the predicted images, return to step 601.

[0077] The method for updating each video image in this embodiment is the same as that in step 503 described above, and will not be repeated here.

[0078] Step 302: Render the target video image onto the target texture corresponding to the virtual display screen;

[0079] In this embodiment, the video image is stored in a Surface corresponding to a virtual display screen; and the Surface corresponding to the virtual display screen includes the Surface Texture function. For example... Figure 7 The image shown is a schematic diagram of a Surface. Figure 7 As can be seen from the embodiments of this application, the Surface includes a SurfaceTexture function, which is bound to a target texture. Thus, a relationship is established between the Surface and the target texture. Therefore, the SurfaceTexture function acts as a link between the Surface and the target texture, assisting the application in transferring video images from the Surface to the target texture.

[0080] In one embodiment, step 302 may be specifically implemented as: rendering the target video image onto the target texture corresponding to the Surface Texture function through the Surface Texture function.

[0081] In this embodiment, the video image stored in the Surface of the virtual display screen exists in the form of a GraphicsBuffer, with the Surface acting as the producer of the GraphicsBuffer. The SurfaceTexture function receives the GraphicsBuffer from the Surface and stores it in a queue. Every target duration, the GraphicsBuffer (i.e., the target video image) in the queue is consumed. As described above, the SurfaceTexture function binds to a target texture. After consuming the GraphicsBuffer, the video image is updated on the target texture.

[0082] Step 303: Render the target video image on the target texture to the target Surface using the OpenGL ES rendering thread;

[0083] The data source for the MediaCode thread is still an input surface (the target surface in this embodiment), but this input surface is no longer associated with the virtual display screen. Instead, it serves as the target surface for rendering by the OpenGL ES rendering thread. In other words, the OpenGL ES rendering thread will render the video image onto the target surface.

[0084] In this embodiment, the OpenGL ES rendering thread needs to create a target Surface at runtime. This target Surface is then bound to the input surface of the MediaCodec thread (i.e., the target surface in this embodiment). Thus, the input surface of the MediaCodec thread serves as the target Surface for rendering by the OpenGL ES rendering thread. Specifically... Figure 8 As shown, the target Surface of the OpenGL ES rendering thread and the input surface of the MediaCodec thread are the same Surface. Thus, the OpenGL ES rendering thread transforms the target video image from the target texture to the target surface. Since the target surface and the MediaCodec thread's input surface are the same, the target video image is transformed from the target texture to the MediaCodec thread's input surface.

[0085] In this embodiment, rendering the target video image on the target texture to the target Surface via the OpenGL ES rendering thread is a function of the OpenGL ES rendering thread itself, and will not be described in detail here.

[0086] Step 304: Send the target video image to the media codec thread through the target Surface, so that the target video image can be encoded by the media codec thread to obtain the encoded video image;

[0087] The method by which the media codec thread encodes the target video image in this embodiment is not limited here; the specific method can be set according to the actual situation.

[0088] The number of times the media codec thread executes per second depends on the number of times the OpenGL ES rendering thread executes per second. Each time the OpenGL ES rendering thread executes, the media codec thread encodes once. Therefore, by controlling the number of times the OpenGL ES rendering thread executes, the MediaCodec encoding frame rate can be controlled; that is, by setting a specified encoding frame rate, the target duration can be controlled, thus achieving control over the corresponding OpenGL ES rendering thread.

[0089] To save storage resources, in one embodiment, after performing step 304, the target video image and all video frames preceding the target video image in the video images stored in the virtual display screen are deleted.

[0090] For example, if the video images include images 1 to 10, and the target video image is image 6, then images 1 to 6 will be deleted.

[0091] Step 305: Upon receiving the screen recording end command, obtain the screen recording video based on each encoded video image, and send the recorded video to the second device for display.

[0092] Based on the same inventive concept, the screen recording video generation method described above can also be implemented by a screen recording video generation device. The effect of this device is similar to that of the aforementioned method, and will not be repeated here.

[0093] Figure 9 This is a schematic diagram of the structure of a screen recording video generation apparatus according to an embodiment of the present disclosure.

[0094] like Figure 9 As shown, the screen recording video generation device 900 disclosed herein may include an acquisition module 910, a first rendering module 920, a second rendering module 930, an image encoding module 940, and a screen recording video generation module 950.

[0095] The acquisition module 910 is used to acquire a target video image from each video image stored in the virtual display screen of the first device at every target duration. The video images are obtained by extracting frames from the video images displayed by the first device based on the screen refresh rate of the first device after receiving the screen recording instruction. The target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate.

[0096] The first rendering module 920 is used to render the target video image onto a target texture corresponding to the virtual display screen;

[0097] The second rendering module 930 is used to render the target video image on the target texture to the target Surface through the OpenGL ES rendering thread;

[0098] The image encoding module 940 is used to send the target video image to the media codec thread through the target Surface, so that the target video image can be encoded by the media codec thread to obtain an encoded video image;

[0099] The screen recording video generation module 950 is used to obtain the screen recording video based on each encoded video image after receiving the screen recording end command, and send the recorded video to the second device for display.

[0100] In one embodiment, each video image is stored in a Surface corresponding to a virtual display screen; and the Surface corresponding to the virtual display screen includes the Surface Texture function;

[0101] The first rendering module 920 is specifically used for:

[0102] The target video image is rendered onto the target texture corresponding to the SurfaceTexture function using the SurfaceTexture function.

[0103] In one embodiment, the acquisition module 910 is specifically used for:

[0104] If the specified encoding frame rate is less than the screen refresh rate, then the target video image is determined based on the screen refresh rate and the specified encoding frame rate; or,

[0105] If the specified encoding frame rate is greater than the screen refresh rate, then image prediction is performed using each video image to obtain multiple predicted images, and the video image with the target frame number is determined as the target video image. The total number of the multiple predicted images and each video image is equal to the number of frames corresponding to the specified encoding frame rate, and the target frame number is obtained based on the specified encoding frame rate.

[0106] In one embodiment, the acquisition module 910 is further configured to:

[0107] Each video image is copied to obtain multiple predicted images;

[0108] Determine whether the number of the multiple predicted images is greater than the target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image;

[0109] If not, the process will update each video image using the multiple predicted images, then return to the step of copying each video image to obtain multiple predicted images, until the number of multiple predicted images exceeds the target number, at which point the process ends.

[0110] In one embodiment, the acquisition module 910 is further configured to:

[0111] For any frame of video image in each of the video images, a pre-trained image prediction model is used to predict the image of any frame of video image to obtain a predicted image;

[0112] Determine whether the number of each predicted image is greater than the target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image;

[0113] If not, then after updating each video image using the predicted images, return to the step of using a pre-trained image prediction model to predict the image of any one frame of the video image, until the number of the multiple predicted images is greater than the target number, then end.

[0114] In one embodiment, the apparatus further includes:

[0115] The target duration determination module 960 is used to obtain the target duration in the following manner:

[0116] The target duration is obtained by dividing the preset value by the specified encoding frame rate.

[0117] In one embodiment, the apparatus further includes:

[0118] The deletion module 970 is used to send the target video image to the media codec thread through the target Surface so that the target video image can be encoded by the media codec thread to obtain the encoded video image, and then delete the target video image and the video images preceding the target video image from the video images stored in the virtual display screen.

[0119] After introducing a screen recording video generation method and apparatus according to an exemplary embodiment of the present invention, the following describes a device according to another exemplary embodiment of the present invention, which may be a first device and a second device.

[0120] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as "circuit", "module", or "system".

[0121] In some possible implementations, the device according to the invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code that, when executed by the processor, causes the processor to perform the steps in the screen recording video generation method according to various exemplary embodiments of the invention described above. For example, the processor may perform actions such as... Figure 3 Steps 301-305 are shown in the diagram.

[0122] The following reference Figure 10 The device 1000 according to this embodiment of the present invention will be described. Figure 10 The device 1000 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0123] like Figure 10 As shown, device 1000 is represented in the form of a general-purpose ΔR device. The components of device 1000 may include, but are not limited to: at least one processor 1001, at least one computer storage medium 1002, and a bus 1003 connecting different system components (including computer storage medium 1002 and processor 1001).

[0124] Bus 1003 represents one or more of several bus structures, including computer storage media bus or computer storage media controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0125] Computer storage medium 1002 may include readable media in the form of volatile computer storage media, such as random access computer storage medium (RAM) 1021 and / or cache storage medium 1022, and may further include read-only computer storage medium (ROM) 1023.

[0126] The computer storage medium 1002 may also include a program / utility 1025 having a set (at least one) of program modules 1024, such program modules 1024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0127] Device 1000 can also communicate with one or more external devices 1004 (e.g., keyboard, pointing device, etc.), one or more devices that enable a user to interact with device 1000, and / or any device that enables device 1000 to communicate with one or more other AR devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1005. Furthermore, device 1000 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1006. As shown, network adapter 1006 communicates with other modules used for device 1000 via bus 1003. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with device 1000, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0128] In some possible implementations, various aspects of the screen recording video generation method provided by the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the screen recording video generation method according to various exemplary embodiments of the present invention described above.

[0129] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for generating screen-recorded videos, characterized in that, The method includes: At each target duration, a target video image is acquired from each video image stored in the virtual display screen of the first device. Each video image is obtained by extracting frames from the video image displayed by the first device based on the screen refresh rate of the first device after receiving the screen recording instruction. The target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate. The target video image is rendered onto a target texture corresponding to the virtual display screen; The target video image on the target texture is rendered into the target Surface using the OpenGL ES rendering thread; The target video image is sent to the media codec thread via the target Surface, so that the target video image can be encoded by the media codec thread to obtain the encoded video image; Upon receiving the screen recording end command, the screen recording video is obtained based on each encoded video image, and the recorded video is sent to the second device for display.

2. The method according to claim 1, characterized in that, Each video image is stored in a Surface corresponding to the virtual display screen; and the Surface corresponding to the virtual display screen includes the interface texture Surface Texture function. The step of rendering the target video image onto a target texture corresponding to the virtual display screen includes: The target video image is rendered onto the target texture corresponding to the Surface Texture function using the Surface Texture function.

3. The method according to claim 1, characterized in that, The target video image among the video images stored in the virtual display screen of the acquisition device includes: If the specified encoding frame rate is less than the screen refresh rate, then the target video image is determined based on the screen refresh rate and the specified encoding frame rate; or, If the specified encoding frame rate is greater than the screen refresh rate, then image prediction is performed using each video image to obtain multiple predicted images, and the video image with the target frame number is determined as the target video image. The total number of the multiple predicted images and each video image is equal to the number of frames corresponding to the specified encoding frame rate, and the target frame number is obtained based on the specified encoding frame rate.

4. The method according to claim 3, characterized in that, The process of using the video images to perform image prediction to obtain multiple predicted images includes: Each video image is copied to obtain multiple predicted images; Determine whether the number of the multiple predicted images is greater than the target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image; If not, the process will update each video image using the multiple predicted images, then return to the step of copying each video image to obtain multiple predicted images, until the number of multiple predicted images exceeds the target number, at which point the process ends.

5. The method according to claim 3, characterized in that, The process of using the video images to perform image prediction to obtain multiple predicted images includes: For any frame of video image in each of the video images, a pre-trained image prediction model is used to predict the image of any frame of video image to obtain a predicted image; Determine whether the number of each predicted image is greater than the target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image; If not, then after updating each video image using the predicted images, return to the step of using a pre-trained image prediction model to predict the image of any one frame of the video image, until the number of the multiple predicted images is greater than the target number, then end.

6. The method according to claim 1, characterized in that, The target duration is obtained in the following way: The target duration is obtained by dividing the preset value by the specified encoding frame rate.

7. The method according to claim 1, characterized in that, After sending the target video image to the media codec thread via the target Surface to encode the target video image and obtain the encoded video image, the method further includes: The target video image and all video frames preceding the target video image are deleted from the video images stored in the virtual display screen.

8. A first device, characterized in that, It includes a processor and a memory, which are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: At each target duration, a target video image is acquired from each video image stored in the virtual display screen of the first device. Each video image is obtained by extracting frames from the video image displayed by the first device based on the screen refresh rate of the first device after receiving the screen recording instruction. The target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate. The target video image is rendered onto a target texture corresponding to the virtual display screen; The target video image on the target texture is rendered into the target Surface using the OpenGL ES rendering thread; The target video image is sent to the media codec thread via the target Surface, so that the target video image can be encoded by the media codec thread to obtain the encoded video image; Upon receiving the screen recording end command, the screen recording video is obtained based on each encoded video image, and the recorded video is sent to the second device for display.

9. The first device according to claim 8, characterized in that, Each video image is stored in a Surface corresponding to the virtual display screen; and the Surface corresponding to the virtual display screen includes the interface texture Surface Texture function. The processor is specifically configured to render the target video image onto a target texture corresponding to the virtual display screen, as follows: The target video image is rendered onto the target texture corresponding to the Surface Texture function using the Surface Texture function.

10. The first device according to claim 8, characterized in that, The processor is also configured to: The target duration is obtained in the following way: The target duration is obtained by dividing the preset value by the specified encoding frame rate.

Images