Method and device for generating screen recording video

Abstract

The invention provides a method and equipment for generating a screen recording video. The method and the equipment are used for generating the screen recording video according to a specified coding frame rate. Comprising the steps that target video images in all video images in a virtual display screen are obtained every target duration, all the video images are obtained by conducting frame extraction on the video images displayed by first equipment on the basis of the screen refresh rate of the first equipment, and the target duration is obtained on the basis of a specified coding frame rate; the target video image is determined based on a screen refresh rate and a specified coding frame rate; rendering the target video image to a target texture corresponding to the virtual display screen; the target video image on the target texture is rendered to the target Surface through an OpenGL ES rendering thread; the target video image is sent to a media coder-decoder thread through the target Surface, so that the target video image is coded through the media coder-decoder thread; and obtaining a screen recording video according to each coded video image.

Description

Background Art

[0002] Screen recording in the prior art is achieved through three standard APIs (Application Programming Interface) in the Android SDK (Software Development Kit), namely MediaProjection, VirtualDisplay and MediaCodec. MediaProjection is a function class in the Android SDK specifically used for screen recording, through which a VirtualDisplay object can be created. MediaCodec is a function class in the Android SDK used for encoding and decoding audio and video data. For encoding, it inputs the original audio and video data and outputs the encoded audio and video data.

[0003] like Figure 1 As shown in the figure, the relationship between the three is that a VirtualDisplay object is created through MediaProjection. This VirtualDisplay object contains a Surface. The image displayed on the screen will be automatically rendered to this OutSurface according to the above frame rate, and this Surface is used as the lnputSurface of MediaCodec, which is the data source of image encoding. In this way, the image data displayed on the screen is directly used as the data source of the encoder, that is, as the frame rate of the encoder input / output data, which cannot be controlled. Therefore, the frame rate of the video recorded on the screen in this way is the same as the screen refresh rate, resulting in the frame rate of the recorded video being fixed and cannot be changed according to one's own needs. However, this method may cause the receiving end to be unable to display the video recorded by the mobile device normally. For example, the mobile end records a screen video when its own screen refresh rate is high, but the receiving end may not be able to decode the screen video due to insufficient decoding capabilities, resulting in the inability to display the screen video recorded on the mobile end normally. Summary of the invention

[0004] The present application provides a method and device for generating a screen recording video, which is used to generate a screen recording video according to a specified encoding frame rate, rather than generating a screen recording video based on a screen refresh rate, so that a receiving end can decode the screen recording video and ensure that the receiving end can display the recorded video normally.

[0005] In a first aspect, an embodiment of the present application provides a method for generating a screen recording video, the method comprising:

[0006] At every target duration, a target video image among the video images stored in the virtual display screen of the first device is obtained, wherein the video images are obtained by extracting frames of the video images displayed by the first device based on the screen refresh rate of the first device itself after receiving the screen recording instruction, and the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate;

[0007] Rendering the target video image onto a target texture corresponding to the virtual display screen;

[0008] Rendering the target video image on the target texture into the target Surface through the OpenGL ES rendering thread;

[0009] Sending the target video image to a media codec thread through the target Surface, so as to encode the target video image through the media codec thread to obtain an encoded video image;

[0010] When the screen recording end instruction is received, the screen recording video is obtained according to each encoded video image, and the recorded video is sent to the second device for display.

[0011] A second aspect of the present application provides a first device, including a processor and a memory, wherein the processor and the memory are connected via a bus;

[0012] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0013] At every target duration, a target video image among the video images stored in the virtual display screen of the first device is obtained, wherein the video images are obtained by extracting frames of the video images displayed by the first device based on the screen refresh rate of the first device itself after receiving the screen recording instruction, and the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate;

[0014] Rendering the target video image onto a target texture corresponding to the virtual display screen;

[0015] Rendering the target video image on the target texture to the target Surface through the OpenGL ES rendering thread;

[0016] Sending the target video image to a media codec thread through the target Surface, so as to encode the target video image through the media codec thread to obtain an encoded video image;

[0017] When the screen recording end instruction is received, the screen recording video is obtained according to each encoded video image, and the recorded video is sent to the second device for display.

[0018] According to a third aspect provided by an embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a computer program, and the computer program is used to execute the method as described in the first aspect.

[0019] In the above embodiment of the present application, the target video image is obtained by extracting frames of the video image displayed by the first device based on the screen refresh rate of the first device itself stored in the virtual display screen of the first device at target durations, wherein the target duration is obtained based on the specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate; and the target video image is rendered to the target texture corresponding to the virtual display screen; then the target video image on the target texture is rendered to the target Surface through the OpenGL ES rendering thread; and the target video image is sent to the media codec thread through the target Surface, so that the target video image is encoded by the media codec thread to obtain the encoded video image; when the screen recording end instruction is received, the screen recording video is obtained according to each encoded video image, and the recorded video is sent to the second device for display. Therefore, in the embodiment of the present application, the rendering link of the OpenGLES rendering thread is inserted between the virtual display screen and the media codec thread, and the rendering link is to render the video image on the target texture bound to the virtual display screen to the target Surface of the media codec thread. The frequency corresponding to the rendering link can be controlled, that is, the specified encoding frequency, so the embodiment of the present application realizes the generation of screen recording video according to the specified encoding frame rate, and does not generate screen recording video based on the screen refresh rate. The specified encoding frame rate can be set according to the specific actual situation, so that the receiving end can decode the screen recording video and ensure that the receiving end can display the recorded video normally. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A schematic diagram of screen recording video generation in the prior art provided by an embodiment of the present application is exemplified;

[0022] Figure 2 The following is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0023] Figure 3 The following is a flow chart showing a method for generating a screen recording video according to an embodiment of the present application;

[0024] Figure 4 The following is a schematic diagram of a process for obtaining a target video image provided by an embodiment of the present application;

[0025] Figure 5 One of the flowcharts of the method for determining multiple prediction images provided in an embodiment of the present application is exemplarily shown;

[0026] Figure 6 The second flowchart of the method for determining multiple prediction images provided in the embodiment of the present application is exemplarily shown;

[0027] Figure 7 The schematic diagram of Surface provided in the embodiment of the present application is exemplarily shown;

[0028] Figure 8 The schematic diagram of the connection between the OpenGLES rendering thread and the media codec thread provided by the embodiment of the present application is shown in FIG. Figure 2

[0029] Fig. 9 A schematic diagram of a device for generating a screen recording video provided in an embodiment of the present application is exemplarily shown;

[0030] Fig.10 The schematic diagram of the device provided in the embodiment of the present application is exemplarily shown. DETAILED DESCRIPTION

[0031] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0032] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.

[0033] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and common meanings.

[0034] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a series of components is not necessarily limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0035] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0036] The idea of ​​the embodiments of the present application is summarized below.

[0037] The method of encoding video images by a fixed screen refresh rate in the prior art may cause the receiving end to be unable to display the video recorded by the mobile device normally. The present application proposes a method for generating a screen recording video, which obtains a target video image by extracting frames of the video image displayed by the first device based on the screen refresh rate of the first device itself stored in the virtual display screen of the first device at target durations, wherein the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate; and the target video image is rendered to a target texture corresponding to the virtual display screen; then the target video image on the target texture is rendered to the target Surface through the OpenGLES rendering thread; and the target video image is sent to the media codec thread through the target Surface, so that the target video image is encoded by the media codec thread to obtain the encoded video image; when the screen recording end instruction is received, the screen recording video is obtained according to each encoded video image, and the recorded video is sent to the second device for display. Therefore, in the embodiment of the present application, the rendering link of the OpenGL ES rendering thread is inserted between the virtual display screen and the media codec thread, and the rendering link is to render the video image on the target texture bound to the virtual display screen to the target Surface of the media codec thread. The frequency corresponding to the rendering link can be controlled, that is, the specified encoding frequency, so the embodiment of the present application realizes the generation of the screen recording video according to the specified encoding frame rate, and does not generate the screen recording video based on the screen refresh rate. The specified encoding frame rate can be set according to the specific actual situation, so that the receiving end can decode the screen recording video and ensure that the receiving end can display the recorded video normally.

[0038] like Figure 2 As shown, an application scenario of a method for generating a screen recording video includes a first device 201 and a second device 202.

[0039] In a possible application scenario, after receiving the screen recording instruction, the first device 201 extracts frames of the video images displayed by the first device based on the screen refresh rate of the first device itself to obtain each video image, and stores each video image in its own virtual display screen. The first device 201 obtains the target video image from each video image stored in its own virtual display screen at every target duration, wherein the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate; then the first device 201 renders the target video image to a target texture corresponding to the virtual display screen; and renders the target video image on the target texture to a target Surface through an OpenGLES rendering thread; then the first device 201 sends the target video image to a media codec thread through the target Surface, so that the target video image is encoded through the media codec thread to obtain an encoded video image; when the first device 201 receives the screen recording end instruction, it obtains a screen recording video according to each encoded video image, and sends the recorded video to the second device 202 for display.

[0040] in, Figure 2 The first device 201 and the second device 202 can exchange information via a communication network, wherein the communication mode adopted by the communication network can be divided into a wireless communication mode or a wired communication mode.

[0041] Exemplarily, the first device 201 can access the network through cellular mobile communication technology to communicate with the second device 202, wherein the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th Generation Mobile Networks, 5G) technology.

[0042] Optionally, the first device 201 may access the network and communicate with the second device 202 via a short-range wireless communication method, wherein the short-range wireless communication method, for example, includes Wireless Fidelity (Wi-Fi) technology.

[0043] Among them, only a single first device 201 and a single second device 202 are described in detail in the description of this application, but those skilled in the art should understand that the first device 201 and the single second device 202 shown are intended to represent the operations of the first device 201 and the second device 202 involved in the technical solution of this application. It does not imply any limitation on the number, type or location of the first device 201 and the second device 202. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed therefrom, the underlying concepts of the exemplary embodiments of this application will not be changed.

[0044] It should be noted that the method for generating screen recording videos proposed in this application is not only applicable to Figure 2 The application scenario shown is also applicable to any device that can generate screen recording videos.

[0045] The following describes the method of generating screen recording video in an exemplary implementation of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the methods and principles of the present application, and the implementation of the present application is not limited in this regard.

[0046] The following is an introduction to the method for generating a screen recording video in an embodiment of the present application in conjunction with the accompanying drawings. Figure 3 FIG. 1 is a flow chart of a method for generating a screen recording video, which may specifically include the following steps:

[0047] Step 301: acquiring a target video image from among the video images stored in the virtual display screen of the first device at every target duration, wherein the video images are obtained by extracting frames of the video images displayed by the first device based on the screen refresh rate of the first device itself after receiving the screen recording instruction, and the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate;

[0048] For example, if the screen refresh rate of the first device itself is 120 FPS, the first device needs to extract 120 frames of video images in 1 second.

[0049] In one embodiment, the target duration is obtained by: dividing a preset value by the specified encoding frame rate to obtain the target duration.

[0050] The preset value in the embodiment of the present application is 1, but the preset value in the embodiment of the present application is not limited, and the preset value in the embodiment of the present application can be set according to the specific actual situation. The specified encoding frame rate in the embodiment of the present application can be set according to the actual situation and can be changed, but the implementation of the present application does not limit the specified encoding frame rate.

[0051] The following is an explanation of the method of obtaining the target video image in step 301. Figure 4 As shown, it is a schematic diagram of the process of obtaining a target video image, which may specifically include the following steps:

[0052] Step 401: determine whether the specified encoding frame rate is less than the screen refresh rate, if so, execute step 402, if not, execute step 403;

[0053] Step 402: determining the target video image according to the screen refresh rate and the specified encoding frame rate;

[0054] In one embodiment, step 402 can be specifically implemented as follows: dividing the screen refresh rate by the specified encoding frame rate to obtain an interval frame number, multiplying the interval frame number by the number of times the target video image is acquired to obtain a target video image frame number, and determining the target video image according to the target video image frame number, wherein the interval frame number is the difference in the frame number between two adjacent target video images. The interval frame number can be obtained by formula (1):

[0055] N = P1 / P2......(1);

[0056] Wherein, N is the interval frame number, P1 is the screen refresh rate, and P2 is the specified encoding frame rate.

[0057] For example, if the interval frame number is 4, the frame number corresponding to the first acquisition of the target video image is 4, then the fourth frame image is determined to be the first acquisition of the target video image. The frame number corresponding to the second acquisition of the target video image is 8, then the eighth frame image is determined to be the second acquisition of the target video image.

[0058] Although in the embodiment of the present application, the target video image and the frames of video images before the target video image are deleted each time the encoded video image is obtained, the corresponding frame numbers of the video images stored in the embodiment of the present application are fixed. For example, for the stored video images of frames 1-10, if the 1-4 frames of video images are deleted, the frame numbers of the remaining frames of the video images are still 5-10 frames.

[0059] Step 403: determine whether the specified encoding frame rate is equal to the screen refresh rate, if yes, then end, if no, then execute step 404;

[0060] Step 404: Perform image prediction using the video images to obtain multiple predicted images, and determine the video image with a target frame number as the target video image, wherein the total number of the multiple predicted images and the video images is equal to the number of frames corresponding to the specified encoding frame rate, and the target frame number is obtained based on the specified encoding frame rate.

[0061] The target frame number in the embodiment of the present application is the same as the value of the specified encoding frame rate. For example, if the specified encoding frame rate is 120 FPS, the target frame number is 120, that is, the 120th frame of each stored video image is determined as the target video image.

[0062] In the embodiment of the present application, two methods for determining multiple prediction images are provided. Figure 5FIG. 1 is a flow chart of determining a plurality of prediction images, which may specifically include the following steps:

[0063] Step 501: copying each of the video images respectively to obtain a plurality of predicted images;

[0064] Step 502: determine whether the number of the plurality of predicted images is greater than the target number, if not, execute step 503, if yes, end;

[0065] Wherein, the target number is obtained based on the specified encoding frame rate and the number of each video image;

[0066] In one embodiment, the target number is obtained by: subtracting a value corresponding to the specified encoding frame rate from the number of each video image to obtain the target number.

[0067] Step 503: After updating each of the video images using the multiple predicted images, the process returns to step 501.

[0068] In one embodiment, step 503 may be specifically implemented as: determining the multiple predicted images as video images.

[0069] In the embodiment of the present application, each updated video image includes multiple predicted images and each video image.

[0070] Figure 6 This is a second flow chart of determining multiple prediction images in an embodiment of the present application, which may specifically include the following steps:

[0071] Step 601: for any frame of video images among the video images, use a pre-trained image prediction model to perform image prediction on the any frame of video images to obtain a predicted image;

[0072] The image prediction model in the embodiments of the present application can be set according to specific actual conditions, and the embodiments of the present application do not limit the image prediction model.

[0073] Step 602: determine whether the number of each predicted image is greater than the target number, if not, execute step 603, if yes, end;

[0074] The method for determining the target number in the embodiment of the present application is the same as the method for determining the target number in step 502 described above, and the embodiment of the present application will not be repeated here.

[0075] Wherein, the target number is obtained based on the specified encoding frame rate and the number of each video image;

[0076] Step 603: After updating the video images using the predicted images, the process returns to step 601.

[0077] The method of updating each video image in the embodiment of the present application is the same as that in step 503 described above, and the embodiment of the present application will not be repeated here.

[0078] Step 302: Rendering the target video image onto a target texture corresponding to the virtual display screen;

[0079] The video image in the embodiment of the present application is stored in the Surface corresponding to the virtual display screen; and the Surface corresponding to the virtual display screen includes a Surface Texture function. Figure 7 As shown, it is a schematic diagram of Surface. Figure 7 It can be seen that the Surface in the embodiment of the present application includes a SurfaceTexture function, and this SurfaceTexture function is bound to a target texture Texture. Thus, the relationship between the Surface and the target texture Texture is established. Therefore, the SurfaceTexture function is equivalent to the link between the Surface and the target texture Texture, which can assist the application in transferring the video image from the Surface to the target texture Texture.

[0080] In one embodiment, step 302 may be specifically implemented as: rendering the target video image onto a target texture corresponding to the Surface Texture function through the Surface Texture function.

[0081] In the embodiment of the present application, the video image stored in the Surface of the virtual display screen exists in the form of GraphicsBuffer, and Surface acts as the producer of the image GraphicsBuffer; the SurfaceTexture function receives the GriphicsBuffer from the Surface and stores it in a queue. Every target duration, the GraphicsBuffer (i.e., the target video image) in the queue will be consumed; based on the above, the SurfaceTexture function will bind a target texture Textrue, and after the SurfaceTexture function consumes the GraphicsBuffer, the video image has been updated to the target texture Texture.

[0082] Step 303: Rendering the target video image on the target texture to the target Surface through the OpenGL ES rendering thread;

[0083] The data source of the media codec thread MediaCode is still an input surface (the target surface in the embodiment of the present application), but this input surface is no longer connected to the virtual display screen, but serves as the target Surface for rendering by the OpenGL ES rendering thread, that is, the OpenGL ES rendering thread will render the video image onto the target Surface.

[0084] The OpenGL ES rendering thread in the embodiment of the present application needs to create a target Surface when running, and the target Surface will be bound to the input surface of the above media codec thread MediaCodec (i.e., the target surface in the embodiment of the present application). In this way, the input surface of the media codec thread MediaCodec serves as the target Surface for rendering by the OpenGL ES rendering thread. Figure 8 As shown in the figure, the target Surface of the OpenGL ES rendering thread is the same Surface as the input surface of the media codec thread MediaCodec. In this way, the OpenGL ES rendering thread will convert the target video image from the target texture to the target surface. Since the target surface is the same as the input surface of the media codec thread MediaCodec, the target video image is converted from the target texture to the input surface of the media codec thread.

[0085] In the embodiment of the present application, rendering the target video image on the target texture to the target Surface through the OpenGL ES rendering thread is a function of the OpenGL ES rendering thread itself, and the embodiment of the present application will not be described in detail here.

[0086] Step 304: sending the target video image to the media codec thread through the target Surface, so as to encode the target video image through the media codec thread to obtain an encoded video image;

[0087] The manner in which the media codec thread in the embodiment of the present application encodes the target video image is not limited in the embodiment of the present application, and the specific manner can be set according to actual conditions.

[0088] The number of times the media codec thread is executed in one second depends on the number of times the OpenGL ES rendering thread is executed in one second. Each time the OpenGL ES rendering thread is executed, the media codec thread will encode once. In this way, by controlling the number of executions of the OpenGL ES rendering thread, the purpose of controlling the MediaCodec encoding frame rate can be achieved, that is, by setting the specified encoding frame rate to control the target duration, the corresponding OpenGL ES rendering thread can be controlled.

[0089] In order to save storage resources, in one embodiment, after executing step 304, the target video image and each frame of video image located before the target video image among the video images stored in the virtual display screen are deleted.

[0090] For example, the video image includes image 1 to image 10. If the target video image is image 6, then image 1 to image 6 are deleted.

[0091] Step 305: After receiving the screen recording end instruction, a screen recording video is obtained according to each encoded video image, and the recorded video is sent to the second device for display.

[0092] Based on the same inventive concept, the screen recording video generation method disclosed above can also be implemented by a screen recording video generation device. The effect of the screen recording video generation device is similar to that of the aforementioned method, and will not be repeated here.

[0093] Fig. 9 The present invention is a schematic diagram of the structure of a device for generating screen recording videos according to an embodiment of the present invention.

[0094] like Fig. 9 As shown, the device 900 for generating a screen recording video of the present disclosure may include an acquisition module 910 , a first rendering module 920 , a second rendering module 930 , an image encoding module 940 and a screen recording video generating module 950 .

[0095] An acquisition module 910 is used to acquire a target video image from among the video images stored in the virtual display screen of the first device at target durations, wherein the video images are obtained by extracting frames of the video images displayed by the first device based on the screen refresh rate of the first device itself after receiving a screen recording instruction, and the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate;

[0096] A first rendering module 920, configured to render the target video image onto a target texture corresponding to the virtual display screen;

[0097] A second rendering module 930, configured to render the target video image on the target texture into a target Surface through an OpenGL ES rendering thread;

[0098] An image encoding module 940 is used to send the target video image to the media codec thread through the target Surface, so as to encode the target video image through the media codec thread to obtain an encoded video image;

[0099] The screen recording video generation module 950 is used to obtain the screen recording video according to each encoded video image after receiving the screen recording end instruction, and send the recorded video to the second device for display.

[0100] In one embodiment, each video image is stored in a Surface corresponding to a virtual display screen; and the Surface corresponding to the virtual display screen includes a Surface Texture function;

[0101] The first rendering module 920 is specifically used for:

[0102] The target video image is rendered onto a target texture corresponding to the SurfaceTexture function through the Surface Texture function.

[0103] In one embodiment, the acquisition module 910 is specifically configured to:

[0104] If the specified encoding frame rate is less than the screen refresh rate, determining the target video image according to the screen refresh rate and the specified encoding frame rate; or,

[0105] If the specified encoding frame rate is greater than the screen refresh rate, image prediction is performed using the video images to obtain multiple predicted images, and the video image with a target frame number is determined as the target video image, wherein the total number of the multiple predicted images and the video images is equal to the number of frames corresponding to the specified encoding frame rate, and the target frame number is obtained based on the specified encoding frame rate.

[0106] In one embodiment, the acquisition module 910 is further configured to:

[0107] Copying each of the video images respectively to obtain a plurality of prediction images;

[0108] Determining whether the number of the plurality of predicted images is greater than a target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image;

[0109] If not, after updating each of the video images using the multiple predicted images, return to the step of copying each of the video images separately to obtain multiple predicted images, until the number of the multiple predicted images is greater than the target number, and then end.

[0110] In one embodiment, the acquisition module 910 is further configured to:

[0111] For any one frame of the video images, use a pre-trained image prediction model to perform image prediction on the any one frame of the video images to obtain a predicted image;

[0112] Determining whether the number of each predicted image is greater than a target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image;

[0113] If not, after updating the video images using the predicted images, return to any one frame of the video images, use the pre-trained image prediction model to perform image prediction on the any one frame of the video images, and obtain the predicted image step, until the number of the multiple predicted images is greater than the target number, then end.

[0114] In one embodiment, the apparatus further comprises:

[0115] The target duration determination module 960 is used to obtain the target duration by:

[0116] The target duration is obtained by dividing the preset value by the specified encoding frame rate.

[0117] In one embodiment, the apparatus further comprises:

[0118] The deletion module 970 is used to send the target video image to the media codec thread through the target Surface, so as to encode the target video image through the media codec thread, and after obtaining the encoded video image, delete the target video image in each video image stored in the virtual display screen and each frame of video image located before the target video image.

[0119] After introducing a method and apparatus for generating a screen recording video according to an exemplary embodiment of the present invention, next, a device according to another exemplary embodiment of the present invention is introduced, and the device can be a first device and a second device.

[0120] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0121] In some possible implementations, the device according to the present invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the method for generating a screen recording video according to various exemplary embodiments of the present invention described above in this specification. For example, the processor may execute the following steps: Figure 3 Steps 301-305 shown in .

[0122] Refer to the following Fig.10 1000 according to this embodiment of the present invention will be described. Fig.10 The device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0123] like Fig.10 As shown, the device 1000 is in the form of a general ΔR device. The components of the device 1000 may include but are not limited to: the at least one processor 1001, the at least one computer storage medium 1002, and a bus 1003 connecting different system components (including the computer storage medium 1002 and the processor 1001).

[0124] Bus 1003 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0125] The computer storage medium 1002 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1021 and / or cache storage media 1022 , and may further include read-only computer storage media (ROM) 1023 .

[0126] The computer storage medium 1002 may also include a program / utility 1025 having a set (at least one) of program modules 1024, such program modules 1024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0127] The device 1000 may also communicate with one or more external devices 1004 (e.g., keyboards, pointing devices, etc.), one or more devices that enable a user to interact with the device 1000, and / or any device that enables the device 1000 to communicate with one or more other AR devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 1005. In addition, the device 1000 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 1006. As shown, the network adapter 1006 communicates with other modules for the device 1000 through a bus 1003. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the device 1000, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0128] In some possible implementations, various aspects of a method for generating a screen recording video provided by the present invention may also be implemented in the form of a program product, which includes a program code. When the program product runs on a computer device, the program code is used to enable the computer device to execute the steps of the method for generating a screen recording video according to various exemplary embodiments of the present invention described above in this specification.

[0129] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for generating a screen recording video, characterized in that: The method comprises: At every target duration, a target video image among the video images stored in the virtual display screen of the first device is obtained, wherein the video images are obtained by extracting frames of the video images displayed by the first device based on the screen refresh rate of the first device itself after receiving the screen recording instruction, and the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate; Rendering the target video image onto a target texture corresponding to the virtual display screen; Rendering the target video image on the target texture into the target Surface through the OpenGL ES rendering thread; Sending the target video image to a media codec thread through the target Surface, so as to encode the target video image through the media codec thread to obtain an encoded video image; When the screen recording end instruction is received, the screen recording video is obtained according to each encoded video image, and the recorded video is sent to the second device for display.

2. The method according to claim 1, characterized in that Each video image is stored in a Surface corresponding to the virtual display screen; and the Surface corresponding to the virtual display screen includes an interface texture Surface Texture function; The rendering of the target video image onto a target texture corresponding to the virtual display screen includes: The target video image is rendered onto a target texture corresponding to the Surface Texture function through the Surface Texture function.

3. The method according to claim 1, characterized in that The target video image among the video images stored in the virtual display screen of the acquisition device includes: If the specified encoding frame rate is less than the screen refresh rate, determining the target video image according to the screen refresh rate and the specified encoding frame rate; or, If the specified encoding frame rate is greater than the screen refresh rate, image prediction is performed using the video images to obtain multiple predicted images, and the video image with a target frame number is determined as the target video image, wherein the total number of the multiple predicted images and the video images is equal to the number of frames corresponding to the specified encoding frame rate, and the target frame number is obtained based on the specified encoding frame rate.

4. The method according to claim 3, characterized in that The method of using the video images to perform image prediction to obtain a plurality of predicted images includes: Copying each of the video images respectively to obtain a plurality of prediction images; Determining whether the number of the plurality of predicted images is greater than a target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image; If not, after updating each of the video images using the multiple predicted images, return to the step of copying each of the video images separately to obtain multiple predicted images, until the number of the multiple predicted images is greater than the target number, and then end.

5. The method according to claim 3, characterized in that: The method of using the video images to perform image prediction to obtain a plurality of predicted images includes: For any one frame of the video images, use a pre-trained image prediction model to perform image prediction on the any one frame of the video images to obtain a predicted image; Determining whether the number of each predicted image is greater than a target number, wherein the target number is obtained based on the specified encoding frame rate and the number of each video image; If not, after updating the video images using the predicted images, return to any one frame of the video images, use the pre-trained image prediction model to perform image prediction on the any one frame of the video images, and obtain the predicted image step, until the number of the multiple predicted images is greater than the target number, then end.

6. The method according to claim 1, characterized in that The target duration is obtained by: The target duration is obtained by dividing the preset value by the specified encoding frame rate.

7. The method according to claim 1, characterized in that The method further comprises: sending the target video image to the media codec thread through the target Surface so as to encode the target video image through the media codec thread to obtain the encoded video image. The target video image and each frame of video image located before the target video image in each video image stored in the virtual display screen are deleted.

8. A first device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: At every target duration, a target video image among the video images stored in the virtual display screen of the first device is obtained, wherein the video images are obtained by extracting frames of the video images displayed by the first device based on the screen refresh rate of the first device itself after receiving the screen recording instruction, and the target duration is obtained based on a specified encoding frame rate, and the target video image is determined based on the screen refresh rate and the specified encoding frame rate; Rendering the target video image onto a target texture corresponding to the virtual display screen; Rendering the target video image on the target texture into the target Surface through the OpenGL ES rendering thread; Sending the target video image to a media codec thread through the target Surface, so as to encode the target video image through the media codec thread to obtain an encoded video image; When the screen recording end instruction is received, the screen recording video is obtained according to each encoded video image, and the recorded video is sent to the second device for display.

9. The first device according to claim 8, characterized in that Each video image is stored in a Surface corresponding to a virtual display screen; and the Surface corresponding to the virtual display screen includes an interface texture Surface Texture function; The processor performs the step of rendering the target video image to a target texture corresponding to the virtual display screen, and is specifically configured as follows: The target video image is rendered onto a target texture corresponding to the Surface Texture function through the Surface Texture function.

10. The first device according to claim 8, characterized in that The processor is further configured to: The target duration is obtained by: The target duration is obtained by dividing the preset value by the specified encoding frame rate.

Images