Screen recording method and related device

Abstract

The invention discloses a screen recording method and a related device, the method is applied to a target application process corresponding to a target application, the target application process comprises a main rendering thread, a secondary rendering thread and a coding thread, and the secondary rendering thread and the coding thread are started in response to a screen recording operation triggered by the target application; the method comprises the following steps: copying rendering data stored in a backward buffer area in a main rendering thread to a bridging buffer area through the main rendering thread; the bridging buffer area is bound with a shared texture identifier; obtaining rendering data from the bridging buffer area through a secondary rendering thread based on the shared texture identifier; based on the rendering data, drawing an input rendering surface provided by the encoding thread to obtain an output rendering surface, and transmitting the output rendering surface to the encoding thread; and carrying out coding processing based on the output rendering surface through the coding thread to obtain recorded video frame data. According to the method, the actual screen recording requirement of the user can be met, and the screen recording efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a screen recording method and related devices. Background Art

[0002] Screen recording refers to the function of a terminal device recording the content displayed on its screen and generating a corresponding video. With the development of diversified Internet services, the screen recording function has been widely used in various scenarios, such as game live broadcasts and online meetings.

[0003] In the related art, applications on terminal devices that have screen recording requirements (such as game applications, conference applications, etc.) need to use the screen recording capabilities provided by the terminal device's system to record the display content. Specifically, the display content is recorded by calling the screen rendering process in the system.

[0004] However, the above recording method usually has the following problems: calling the screen rendering process in the system to record the display content will record all the display content on the screen; in this way, the recorded content not only includes the display content of the application with screen recording requirements required by the user, but also includes the display content or notification content of other applications on the terminal device that the user does not need, which does not meet the user's actual screen recording needs; and, since all the display content on the screen is recorded, there is a need to obtain the display content of other applications. Therefore, the application with screen recording requirements also needs to apply for system permissions before screen recording, and then perform screen recording after obtaining authorization, which is cumbersome. Summary of the invention

[0005] The embodiments of the present application provide a screen recording method and related devices, which can effectively meet the actual screen recording needs of users, simplify the operation process of screen recording, and improve the efficiency of implementing the screen recording function.

[0006] In view of this, the first aspect of the present application provides a screen recording method, which is applied to a target application process corresponding to a target application, wherein the target application process includes a main rendering thread, a secondary rendering thread, and an encoding thread, wherein the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by the target application; the method includes:

[0007] By means of the main rendering thread, the rendering data stored in the back buffer of the main rendering thread is copied to the bridge buffer; the bridge buffer is bound with a shared texture identifier;

[0008] Obtaining the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; drawing the input rendering surface provided by the encoding thread based on the rendering data to obtain an output rendering surface, and transmitting the output rendering surface to the encoding thread;

[0009] The encoding thread performs encoding processing based on the output rendering surface to obtain recorded video frame data.

[0010] A second aspect of the present application provides a screen recording device, the device supports the operation of a target application process corresponding to a target application, the target application process includes a main rendering thread, a secondary rendering thread and an encoding thread, the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by the target application; the device includes:

[0011] A data copy module, used for copying the rendering data stored in the back buffer of the main rendering thread to the bridge buffer through the main rendering thread; the bridge buffer is bound with a shared texture identifier;

[0012] A rendering module, configured to obtain the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; draw the input rendering surface provided by the encoding thread based on the rendering data to obtain an output rendering surface, and transmit the output rendering surface to the encoding thread;

[0013] The encoding module is used to perform encoding processing based on the output rendering surface through the encoding thread to obtain the recorded video frame data.

[0014] A third aspect of the present application provides a computer device, the device comprising a processor and a memory:

[0015] The memory is used to store computer programs;

[0016] The processor is used to execute the steps of the screen recording method as described in the first aspect according to the computer program.

[0017] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the steps of the screen recording method described in the first aspect above.

[0018] In a fifth aspect, the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device performs the steps of the screen recording method described in the first aspect above.

[0019] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0020] The embodiment of the present application provides a screen recording method, which is applied to a target application process corresponding to a target application, and the target application process includes a main rendering thread, a secondary rendering thread, and an encoding thread, wherein the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by a target application. In the screen recording method, the rendering data stored in the back buffer of the main rendering thread is copied to a bridge buffer through the main rendering thread, and the bridge buffer is bound to a shared texture identifier. The secondary rendering thread obtains the rendering data from the bridge buffer based on the shared texture identifier, and based on the rendering data, draws the input rendering surface provided by the encoding thread to obtain the output rendering surface, and transmits the output rendering surface to the encoding thread; the output rendering surface is encoded by the encoding thread to obtain the recorded video frame data. In the above-mentioned screen recording method, the recording of the display screen of the target application itself is realized through the main rendering thread, secondary rendering thread and encoding thread in the target application process, and does not involve the use of the screen rendering process in the system. Therefore, the display content or notification content of other applications unrelated to the target application will not be recorded, which effectively meets the actual screen recording needs of users. In addition, since the above-mentioned screen recording method is implemented through the target application process corresponding to the target application, the rendering data can be directly obtained and used in the target application process. Without involving the acquisition of data in other applications, there is no need to apply for system permissions before screen recording, which simplifies the operation process of screen recording and improves the implementation efficiency of the screen recording function. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a screen recording method provided for related technologies;

[0022] Figure 2 A specific flow chart for obtaining video frame data through the system API provided for related technologies;

[0023] Figure 3 A schematic diagram of an application scenario of the screen recording method provided in an embodiment of the present application;

[0024] Figure 4A schematic diagram of the rendering logic of the target application provided in the embodiment of the present application on the terminal device;

[0025] Figure 5 A method flow chart of a screen recording method provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of the interaction between various threads in a target application process provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a screen recording device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] In the related art, when an application on a terminal device that has a screen recording requirement, such as a game application, a conference application, etc., performs screen recording, the display content is recorded by calling a screen rendering process in the system.

[0033] See also Figure 1 , Figure 1The following is a schematic diagram of the screen recording method in the above-mentioned related technology. When the user triggers the screen recording process through the screen recording application (which can be any application with screen recording function) in the terminal device, the terminal device will record all the contents displayed on the screen, involving the display contents in multiple applications in the terminal device. Therefore, before starting the screen recording, the screen recording application needs to apply for screen recording permission from the system. After the screen recording application receives the result of the permission application from the system agreeing to perform screen recording, the screen recording application can call the system's application programming interface (Application Programming Interface, API) to start recording the screen display contents in the terminal device. After starting the screen recording, the screen recording application obtains each video frame obtained by recording through the system API, and transmits the video frame to the encoder. The encoder encodes the video frame to obtain the video frame data obtained by the screen recording. According to the actual scene requirements, the video frame data is transmitted to the server or stored in the local video file of the terminal device.

[0034] For the specific process of obtaining video frame data through the system API, see Figure 2 .exist Figure 2 In the process, the screen recording application process, as the recording demander, completes the screen recording permission application, initiates screen recording through the screen recording API provided by the system, and creates an encoder at the same time. The encoder provides a rendering surface (surface) for carrying the screen recording content, and transmits the rendering surface to the system screen rendering and management process through the screen recording API. The system screen rendering and management process collects the drawing instructions and content rendering surfaces from all application processes on the terminal device screen, synthesizes the final screen to be displayed according to the drawing instructions and content rendering surfaces corresponding to the content rendering surfaces in each application process, and displays the screen on the terminal device screen; after starting the screen recording, the system screen rendering and management process writes the collected rendering content into the rendering surface provided by the encoder, and returns the rendering surface with the rendering content written to the encoder in the screen recording application. The encoder encodes the rendering surface with the rendering content written to it, and finally generates the video frame data to be obtained by the screen recording. According to the actual recording requirements, the video frame data is written into the local video file or pushed to the server.

[0035] It can be seen that in the screen recording method described in the relevant technology, each video frame in the screen recording is obtained through the screen rendering process in the system - the system screen rendering and management process, and by calling the screen rendering process in the system to record the display content, all the display content on the screen will be recorded. In this way, the recorded content not only includes the display content of the application with screen recording requirements (i.e., the screen recording application) required by the user, but also includes the display content or notification content of other applications on the terminal device that the user does not need, which does not meet the user's actual screen recording needs and causes unnecessary waste of data processing resources; and, since all the display content on the screen is recorded, there is the acquisition of the display content of other applications. Therefore, the application with screen recording requirements also needs to apply for system permissions before screen recording, and then perform screen recording after obtaining authorization, which is cumbersome.

[0036] At the same time, when calling the screen rendering process in the system to record the display content, cross-process data interaction is required, that is, the rendering surface is interactively transmitted between the screen recording application process and the screen rendering process in the system. When a large number of video frames need to be transmitted across processes, certain performance problems will arise.

[0037] In order to solve the above problems, an embodiment of the present application provides a screen recording method, which is applied to a target application process corresponding to a target application, including a main rendering thread, a secondary rendering thread and an encoding thread, wherein the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by a target application. In the screen recording method, the rendering data stored in the back buffer of the main rendering thread is copied to a bridge buffer through the main rendering thread, and the bridge buffer is bound to a shared texture identifier. The secondary rendering thread obtains the rendering data from the bridge buffer based on the shared texture identifier, and based on the rendering data, draws the input rendering surface provided by the encoding thread to obtain the output rendering surface, and transmits the output rendering surface to the encoding thread; the output rendering surface is encoded by the encoding thread to obtain the recorded video frame data.

[0038] In the above-mentioned screen recording method, the recording of the display screen of the target application itself is realized through the main rendering thread, secondary rendering thread and encoding thread in the target application process, and does not involve the use of the screen rendering process in the system. Therefore, the display content or notification content of other applications unrelated to the target application will not be recorded, which effectively meets the actual screen recording needs of users. In addition, since the above-mentioned screen recording method is implemented through the target application process corresponding to the target application, the rendering data can be directly obtained and used in the target application process. Without involving the acquisition of data in other applications, there is no need to apply for system permissions before screen recording, which simplifies the operation process of screen recording and improves the implementation efficiency of the screen recording function.

[0039] At the same time, the screen recording method provided in the embodiment of the present application only performs data interaction within the target application process, that is, data is transmitted between the main rendering thread, the secondary rendering thread and the encoding thread within the target application process. Compared with the related technology of cross-process data transmission between the system and the screen recording application, the screen recording method provided in the embodiment of the present application avoids related performance issues.

[0040] The screen recording method provided in the embodiment of the present application can be implemented by a computer device, which can be a terminal device. The terminal device includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, etc. The terminal device can be directly or indirectly connected to the server via wired or wireless communication, and the present application does not limit this.

[0041] To facilitate understanding of the screen recording method provided in the embodiment of the present application, the following uses a terminal device as an example in which the execution subject of the screen recording method is an example to exemplarily introduce the application scenario of the screen recording method.

[0042] See also Figure 3 , Figure 3 A schematic diagram of an application scenario of the screen recording method provided in an embodiment of the present application. Figure 3 In the application scenario shown, the terminal device 300 is used as an example of the computer device for executing the screen recording method provided in the embodiment of the present application.

[0043] See also Figure 3 As shown, the target application can be any application with a screen recording function, exemplarily a game application, or other types of applications. After the screen recording process is triggered in the target application, it is necessary to obtain the display content of the target application on the screen of the terminal device 300, and realize the screen recording of the target application by saving the display content in real time.

[0044] The target application process of the target application includes a main rendering thread, a secondary rendering thread, and an encoding thread. Data transmission between threads in the target application process is performed in the target application process. In the state where screen recording is not performed, the rendering logic of the target application on the terminal device 300 is as follows: Figure 4 As shown, the main rendering thread is used to render the content that needs to be displayed in the target application onto the screen of the terminal device 300 for the user to watch. When the main rendering thread performs the rendering operation, the drawing engine or other screen rendering framework produces drawing instructions based on the model data (such as models of people, vehicles, buildings, etc.) and texture data in the target application, generates a specific rendering function based on the drawing instructions, and executes the rendering function through the main rendering thread to render the model data and texture data into the screen, thereby finally realizing the on-screen display of the rendered data.

[0045] After the screen recording starts, the video frame being displayed on the screen of the terminal device 300 is the current video frame. In the back buffer of the main rendering thread, the rendering data corresponding to the next video frame after the current video frame is stored. As the video frame on the screen of the terminal device 300 is played, by obtaining the rendering data in the back buffer, the real-time acquisition of the content displayed in the target application can be achieved. Therefore, after detecting the start of screen recording, the main rendering thread creates a shared texture identifier for sharing rendering data and a bridge buffer for storing rendering data, and binds the shared texture identifier to the bridge buffer. The rendering data in the back buffer is copied to the bridge buffer by the main rendering thread. After the secondary rendering thread obtains the shared texture identifier, it can obtain the rendering data in the bridge buffer bound to it according to the shared texture identifier, that is, the secondary rendering thread can obtain the rendering data in the back buffer. At the same time, since the content corresponding to the shared texture identifier is the rendering data in the bridge buffer, when the rendering data is copied from the back buffer to the bridge buffer, the content corresponding to the shared texture identifier will also be updated accordingly, realizing the real-time acquisition of the rendering data in the back buffer by the secondary rendering thread, and screen recording is achieved through these real-time acquired rendering data.

[0046] In the target application process, the encoding thread is used to encode the video screen obtained by the secondary rendering thread to obtain the video frame data of the screen recording. During screen recording, the encoding thread will create an input rendering surface and provide the input rendering surface to the secondary rendering thread to carry the rendering content generated by the secondary rendering thread; in the secondary rendering thread, after obtaining the rendering data, the input rendering surface provided by the encoding thread will be drawn based on the rendering data. After the drawing is completed, the output rendering surface corresponding to the input rendering surface is obtained. The output rendering surface is a video frame displayed on the screen of the terminal device 300 during the screen recording process. Since the video frame is generated based on the rendering data in the target application, only the display content in the target application is included in the video frame, which meets the user's screen recording needs on the target application. The output rendering surface drawn in the secondary rendering thread is transmitted to the encoding thread. When the encoding thread receives the output rendering surface, it encodes the output rendering surface. Each output rendering surface after encoding forms the video frame data in this screen recording process. In the video frame data, each video frame is arranged in time order. The video frame data is stored frame by frame, which is the video generated by the screen recording.

[0047] That is to say, when screen recording is performed in the target application, rendering data can be obtained and used in the target application through the target application process, avoiding cross-process transmission of video frames; the rendering data of the back buffer is obtained through the main rendering thread and copied to the bridge buffer, and the secondary rendering thread can obtain the rendering data in the bridge buffer bound to the shared texture identifier based on the shared texture identifier created by the main rendering thread, and draw the rendering data on the input rendering surface provided by the encoding thread, and transmit the output rendering surface obtained after the drawing is completed to the encoding thread for encoding, and finally obtain the video frame data of this screen recording, and only a simple rendering data copy operation is performed in the main rendering thread, and the secondary rendering of the rendering data in the target application is performed by the secondary rendering thread, which reduces the impact of screen recording on the main rendering thread; at the same time, the recording of the display screen of the target application itself is realized through the target application process, and the rendering content and data in other applications are not involved. There is no need to apply for system permissions before screen recording, which simplifies the operation process of screen recording and effectively meets the user's needs for screen recording in the target application.

[0048] See also Figure 5 , Figure 5 A method flow chart of a screen recording method provided in an embodiment of the present application, the method can be executed by a terminal device as the aforementioned computer device, the method is applied to a target application process corresponding to a target application, the target application process includes a main rendering thread, a secondary rendering thread and an encoding thread, the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by the target application; Figure 5 As shown, the method specifically comprises the following steps:

[0049] Step 501: The rendering data stored in the back buffer of the main rendering thread is copied to the bridge buffer through the main rendering thread; the bridge buffer is bound with a shared texture identifier.

[0050] Rendering refers to the process of generating images from models by software of various electronic devices such as computers, mobile terminals, and game consoles. Models are descriptions of three-dimensional objects or virtual scenes that are strictly defined in language or data structures, including information such as geometry, viewpoint, texture, lighting, and shadows. Textures are usually mapped on the surface of three-dimensional models to make the surface of the model look more real and detailed. It should be noted that this application does not limit the type of terminal device that executes the screen recording method. The terminal device can be a mobile phone, computer, etc.

[0051] In the embodiment of the present application, the process of generating a specific scene screen according to the abstract model data and texture data is rendering, for example, the process of generating different display screens corresponding to each control in the game application according to the operation parameters such as the change mode, change angle, and change speed of each control saved in the game application. In the terminal device, the main rendering thread can use an open graphics programming interface (Open Graphics Library, OpenGL), which is a cross-language, cross-platform application program interface for rendering two-dimensional or three-dimensional vector graphics. It is widely used in various image rendering scenarios, including but not limited to game rendering, computer-aided design drawing, etc., to realize the rendering of various graphics rendering scenes in terminal device applications.

[0052] Both the back buffer and the bridge buffer belong to the frame buffer. The frame buffer can be regarded as a memory buffer for storing pixel data of rendering results, including information such as color, depth, and template.

[0053] Among them, the main rendering thread renders the display content in the target application, and the obtained rendering data is saved in the back buffer of the main rendering thread. The rendering data is a kind of pixel data. By drawing the pixel data, the display screen of the target application on the terminal device screen can be obtained. It should be noted that this application does not impose any restrictions on the type of target application. For example, the target application can be a game application, a conference application, a live broadcast application, etc.

[0054] The bridge buffer is a buffer created by the main rendering thread for sharing rendering data. In the embodiment of the present application, the rendering data generated in the main rendering thread is not only stored in the back buffer, but also copied to the bridge buffer for storage. Since the bridge buffer is bound with a shared texture identifier, which is a unique identifier for identifying a texture object and can be used to reference the texture object during rendering, when the bridge buffer is bound with a shared texture identifier, the shared texture identifier can be used to reference the texture object in the bridge buffer and obtain the rendering data copied from the back buffer by the bridge buffer.

[0055] When the target application is running normally, the main rendering thread is mainly used to render the display content in the target application to generate a display screen. When the screen is recorded, the embodiment of the present application only performs the copy operation of the rendering data in the main rendering thread, and the operation of copying the rendering data from the back buffer to the bridge buffer in the main rendering thread is run on the cache of the graphics processing unit (GPU), which processes a small amount of data and executes at a fast speed, thereby reducing the impact on the frame rate and rendering efficiency when the main rendering thread is running normally.

[0056] In a possible implementation, the screen recording method described in the present application may further include the following steps:

[0057] A secondary graphics program context corresponding to the secondary rendering thread is created through the secondary rendering thread according to the primary graphics program context corresponding to the primary rendering thread, and the primary graphics program context is determined as a parent graphics program context of the secondary graphics program context.

[0058] The shared texture identifier created in the main rendering thread corresponding to the parent graphics program context supports use by the secondary rendering thread corresponding to the secondary graphics program context.

[0059] In the OpenGL application programming interface, the graphics program context (OpenGL Context) represents the current state and environment of OpenGL, and contains all OpenGL state information, such as the current rendering mode, buffer objects, texture objects, shader programs, etc. In OpenGL, all rendering operations are performed in the current OpenGL context.

[0060] In the embodiment of the present application, when the main rendering thread is running, the corresponding main graphics program context contains the program running status record and running data of the main rendering thread. The main rendering thread provides its own main graphics program context to the secondary rendering thread, and the secondary rendering thread can create a secondary graphics program context corresponding to itself based on the main graphics program context, and the secondary graphics program context includes the program running record and running data of the secondary rendering thread.

[0061] In the process of creating a secondary graphics program context according to the main graphics program context, the secondary rendering thread can also specify the corresponding parent graphics program context for the secondary graphics program context. After specifying the parent graphics program context, the shared texture identifier in the thread corresponding to the parent graphics program context can be provided to the thread corresponding to the child graphics program context for use. In an embodiment of the present application, the main graphics program context corresponding to the main rendering thread is determined as the parent graphics program context of the secondary graphics program context, and the secondary graphics program context is the child graphics program context of the main graphics program context. In this way, the shared texture identifier created in the main rendering thread corresponding to the parent graphics program context can be used by the secondary rendering thread corresponding to the child graphics program context, forming the basis for sharing rendering data between the main rendering thread and the secondary rendering thread. The secondary rendering thread can access the bridge buffer bound to the shared texture identifier based on the shared texture identifier and obtain rendering data from the bridge buffer.

[0062] A secondary graphics program context corresponding to the secondary rendering thread is created through the main graphics program context, and the main graphics program context is determined as the parent graphics program context of the secondary graphics program context, so that the secondary rendering thread can use the shared texture identifier in the main rendering thread to realize the sharing of rendering data between the main rendering thread and the secondary rendering thread, thereby improving the efficiency of the secondary rendering thread in obtaining rendering data.

[0063] In a possible implementation, the screen recording method described in the present application may further include the following steps:

[0064] Step 11: Detect whether the shared texture data has been initialized through the main rendering thread.

[0065] In the main rendering thread, by detecting whether there is a shared texture identifier and a bridge buffer bound to the shared texture identifier, it is determined whether the shared texture data in the main rendering thread has been initialized; when the shared texture identifier and the bridge buffer bound to the shared texture identifier already exist in the main rendering thread, it is considered that the shared texture data in the main rendering thread has been initialized; otherwise, it is considered that the shared texture data in the main rendering thread has not been initialized.

[0066] In an embodiment of the present application, the purpose of detecting whether the shared texture data has been initialized is to determine whether there is a corresponding buffer to save the copied rendering data after a copy operation is performed on the rendering data in the back buffer of the main rendering thread; if so, the copy operation can be performed directly on the data in the back buffer; if not, it is necessary to create a corresponding shared texture identifier and a bridge buffer to save the rendering data copied from the back buffer.

[0067] Step 12: If not, create the shared texture identifier and the bridge buffer through the main rendering thread, and bind the shared texture identifier and the bridge buffer; execute the main rendering thread to copy the rendering data stored in the back buffer in the main rendering thread to the bridge buffer.

[0068] When the shared texture data in the main rendering thread is not initialized, the main rendering thread needs to create a shared texture identifier and a bridge buffer and bind the two. The bridge buffer is used to store the rendering data copied from the back buffer. Based on the aforementioned copy operation of the rendering data and the binding relationship between the bridge buffer and the shared texture identifier, the shared texture identifier is used to map the rendering data in the bridge buffer. Based on this mapping, the secondary rendering thread can obtain the same rendering data as in the back buffer of the main rendering thread through the shared texture identifier; and when new rendering data is copied and the rendering data in the bridge buffer is updated, the mapping between the shared texture identifier and the rendering data will also be updated accordingly.

[0069] Step 13: If yes, then execute the process of copying the rendering data stored in the back buffer of the main rendering thread to the bridge buffer through the main rendering thread.

[0070] When the shared texture data in the main rendering thread has been initialized, based on the shared texture identifier created by the main rendering thread when the shared texture data is initialized and the bridge buffer bound to the shared texture identifier, a copy operation is performed on the rendering data stored in the back buffer of the main rendering thread, and the aforementioned rendering data is copied from the back buffer to the bridge buffer. Since the bridge buffer is bound to the shared texture identifier, the sharing of the rendering data in the bridge buffer can be achieved based on the shared texture identifier, thereby achieving the sharing of the rendering data in the back buffer of the main rendering thread.

[0071] The shared texture data is initialized through the main rendering thread, a shared texture identifier and a bridge buffer bound to the shared texture identifier are created, rendering data copied from the back buffer is saved in the bridge buffer, and sharing of rendering data is achieved based on the shared texture identifier bound to the bridge buffer.

[0072] In a possible implementation, step 501 is implemented as follows:

[0073] Step 21: through the main rendering thread, setting the bridge buffer as the current write buffer in the main rendering thread, and setting the back buffer as the current read buffer in the main rendering thread.

[0074] During the operation of the main rendering thread, a current read buffer for reading data and a current write buffer for writing data are set, and the data read from the current read buffer will be written into the current write buffer.

[0075] In an embodiment of the present application, when it is necessary to copy the rendering data in the back buffer of the main rendering thread, the back buffer will be set as the current read buffer so that the rendering data stored therein can be read from the current read buffer. At the same time, the bridge buffer will be used as the current write buffer in the main rendering thread for writing the aforementioned rendering data.

[0076] Step 22: Execute a buffer copy function through the main rendering thread to copy the rendering data from the current read buffer to the current write buffer.

[0077] The buffer copy function is a function that can copy the contents of one buffer to another buffer, such as the glBlitFramebuffer function in OpenGL. The buffer copy function is used when copying the rendering data of the back buffer. The buffer copy function is executed by the main rendering thread, which will copy the rendering data in the current read buffer to the current write buffer, that is, copy the rendering data in the back buffer to the bridge buffer.

[0078] By setting the current write buffer and the current read buffer in the main rendering thread and executing the buffer copy function, the rendering data sharing between the back buffer and the bridge buffer is realized.

[0079] In a possible implementation, step 22 is implemented as follows:

[0080] The buffer copy function is executed by the main rendering thread to copy the color data stored in the color buffer in the current read buffer to the current write buffer.

[0081] In the main rendering thread, the rendering data stored in the current read buffer is a kind of pixel data, which includes information such as color, depth and template. Among them, the buffer storing color is the color buffer. The buffer that reads pixel data in OpenGL is usually the color buffer. The pixel data is the pixel value corresponding to each pixel on the display screen of the terminal device. The color of the corresponding pixel can be determined by the pixel value. When reading rendering data from the current read buffer, it can be set to read data from the color buffer of the current read buffer, read the color data stored in the color buffer, and copy the color data to the current write buffer, so as to realize the sharing of color data between the back buffer and the bridge buffer.

[0082] As an embodiment, based on step 21 and step 22, after the screen recording is completed, the following steps may also be included:

[0083] In response to a screen recording stop operation triggered by the target application, the read frame buffer target and the write frame buffer target in the main rendering thread are restored through the main rendering thread, and the read frame buffer target and the write frame buffer target are restored to the values ​​before responding to the screen recording operation.

[0084] The read frame buffer target is used to indicate the current read buffer, and the write frame buffer target is used to indicate the current write buffer.

[0085] During the screen recording process, the current read buffer pointed to by the read frame buffer target in the main rendering thread is the back buffer, and the current write buffer pointed to by the write frame buffer target is the bridge buffer. This setting is a specific setting in the screen recording scenario. After the screen recording is completed, this specific setting may affect the normal operation of the main rendering thread. Therefore, in response to the screen recording stop operation triggered by the target application, when the screen recording ends, the read frame buffer target and the write frame buffer target in the main rendering thread need to be restored to the values ​​before the screen recording starts, so that the main rendering thread can be restored to the working state before the screen recording starts and continue to perform the rendering tasks before the screen recording starts.

[0086] After the screen recording stops, the read frame buffer target and the write frame buffer target in the main rendering thread are restored to the values ​​before responding to the screen recording operation, thereby reducing the impact of the screen recording on the main rendering thread in the target application.

[0087] Step 502: Obtain the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; draw the input rendering surface provided by the encoding thread based on the rendering data to obtain an output rendering surface, and transmit the output rendering surface to the encoding thread.

[0088] The secondary rendering thread is a thread that runs independently of the main rendering thread in the target application process. It is used to perform secondary rendering on the rendering data, restore the rendering content drawn by the main rendering thread according to the rendering data, and perform screen recording based on the rendering content.

[0089] The secondary rendering thread can access the bridge buffer bound to the shared texture identifier according to the shared texture identifier, and then reference the corresponding texture object from the bridge buffer. That is, the rendering data copied from the back buffer can be obtained in the bridge buffer according to the shared texture identifier. The sharing of the rendering data in the back buffer is achieved through the shared texture identifier and the bridge buffer, so that the secondary rendering thread can draw the rendering content consistent with the main rendering thread.

[0090] It should be noted that in the embodiment of the present application, the rendering data is pixel data collected by the main rendering thread, which records the pixel value corresponding to each pixel point in the display screen, and the rendering content is the specific content drawn according to the rendering data.

[0091] After obtaining these rendering data, the secondary rendering thread needs to perform secondary rendering on the rendering data in the target application, and draw the rendering content obtained by the secondary rendering to the input rendering surface provided by the encoding thread, wherein the input rendering surface is a blank surface created by the encoding thread for carrying the rendering content. After the secondary rendering thread completes drawing the input rendering surface, it obtains the corresponding output rendering surface, which is a video frame obtained by screen recording. The secondary rendering thread then transmits the output rendering surface to the encoding thread, which is subsequently processed in the encoding thread.

[0092] In one possible implementation, the secondary rendering thread draws the input rendering surface through the following steps:

[0093] Step 31: constructing a shader through the secondary rendering thread;

[0094] Step 32: Drawing the rendering result based on the mapping relationship between the vertex coordinates of the input rendering surface and the texture coordinates of the rendering data by the shader;

[0095] Step 33: output the rendering result to the input rendering surface to obtain the output rendering surface.

[0096] A shader is a drawing tool that draws graphics based on rendering data. When creating a shader in the secondary rendering thread, a vertex shader and a fragment shader can be created through the rendering function in OpenGL. The vertex shader is used to draw output vertices, and the fragment shader is used to draw output colors. By linking the vertex shader and the fragment shader, the program object corresponding to the shader can be obtained.

[0097] After the secondary rendering thread obtains the rendering data and the input rendering surface, it needs to match the texture coordinates of the rendering data with the vertex coordinates of the input rendering surface so that it can draw the content corresponding to the rendering data into the input rendering surface and restore the rendering content drawn by the main rendering thread based on the rendering data.

[0098] When rendering using a shader, the rendering result corresponding to the rendering data is drawn according to the mapping relationship between the vertex coordinates of the input rendering surface and the texture coordinates of the rendering data. Specifically, during the rendering process, the input rendering surface can be divided into two parts according to the diagonal of the rectangle corresponding to the input rendering surface, each part is a triangle, and the triangles of the two parts are drawn respectively according to the aforementioned mapping relationship, so as to complete the drawing of the input rendering surface.

[0099] The rendering result obtained based on the rendering data is drawn on the input rendering surface provided by the encoding thread. For example, the rendering result can be drawn on the input rendering surface using the swapbuffer function in OpenGL. In the secondary rendering thread, the input rendering surface is the output object of the rendering result of the secondary rendering thread.

[0100] Step 503: Perform encoding processing based on the output rendering surface through the encoding thread to obtain recorded video frame data.

[0101] After being processed by the main rendering thread and the secondary rendering thread, the output rendering surface corresponding to each frame displayed by the target application during the screen recording process is output to the encoding thread for encoding processing. The encoding processing is the process of encoding and sorting each output rendering surface to obtain a video frame sequence. After the encoding thread obtains the output rendering surface, it encodes the video frame images corresponding to each output rendering surface in a preset order to generate video frame data, wherein the preset order can be the time order corresponding to the recording; the video frame data can be a video frame combination sequence arranged in a preset order, and by playing the video frame combination sequence, all video frames displayed by the target application during the screen recording process can be viewed, thereby obtaining the video corresponding to this screen recording.

[0102] In an embodiment of the present application, the video frame data finally generated by the encoding thread can be written into a local video file of the terminal device for viewing, or uploaded to a server for backup, thereby meeting the user's different screen recording requirements for target applications.

[0103] It should be noted that in the embodiments of the present application, the video frame data can be output in a variety of different forms, and the present application does not make any specific limitations on this.

[0104] In a possible implementation, the encoding thread encodes the output rendering surface through the following steps:

[0105] Step 41: Create a media encoder through the encoding thread;

[0106] Step 42: creating the input rendering surface through the media encoder, and transmitting the input rendering surface to the secondary rendering thread;

[0107] At this time, step 503 can be implemented in the following manner:

[0108] The media encoder performs encoding processing based on the output rendering surface to obtain the video frame data.

[0109] In OpenGL, the media encoder is used to encode the original video frame image data according to international standards so that it can be efficiently transmitted and stored on the Internet. In the embodiment of the present application, the media encoder is used to encode the output rendering surface output by the secondary rendering thread.

[0110] When a media encoder is created through an encoding thread, the media encoder is provided by the system platform of the terminal device, and the media encoder can be created by calling a system-wide API. It should be noted that this application does not specifically limit the system platform type of the terminal device.

[0111] After the media encoder is created, the media encoder can create a blank input rendering surface and transmit the blank input rendering surface to the secondary rendering thread. After being processed by the secondary rendering thread, the input rendering surface can be used as input data of the media encoder in the encoding thread.

[0112] The data input source of the media encoder is the secondary rendering thread, and the corresponding input data is the output rendering surface drawn by the secondary rendering thread based on the rendering data. The media encoder encodes these output rendering surfaces and arranges each output rendering surface in a preset order to obtain the video frame data of this screen recording. The video frame data can be transmitted to the server or stored in the local video file of the terminal device according to the user's needs. The video frame data includes all the display screens of the target application on the terminal device screen during this screen recording process.

[0113] The embodiment of the present application provides a screen recording method, which is applied to a target application process corresponding to a target application, including a main rendering thread, a secondary rendering thread and an encoding thread, wherein the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by a target application. In the screen recording method, the rendering data stored in the back buffer of the main rendering thread is copied to a bridge buffer through the main rendering thread, and the bridge buffer is bound to a shared texture identifier. The secondary rendering thread obtains the rendering data from the bridge buffer based on the shared texture identifier, and based on the rendering data, draws the input rendering surface provided by the encoding thread to obtain the output rendering surface, and transmits the output rendering surface to the encoding thread; the output rendering surface is encoded by the encoding thread to obtain the recorded video frame data. It can be seen that in the above-mentioned screen recording method, the recording of the display screen of the target application itself is realized through the main rendering thread, secondary rendering thread and encoding thread in the target application process, and does not involve the use of the screen rendering process in the system. Therefore, the display content or notification content of other applications not related to the target application will not be recorded, which effectively meets the actual screen recording needs of users. In addition, since the above-mentioned screen recording method is implemented through the target application process corresponding to the target application, the rendering data can be directly obtained and used in the target application process. Without involving the acquisition of data in other applications, there is no need to apply for system permissions before screen recording, which simplifies the operation process of screen recording and improves the implementation efficiency of the screen recording function.

[0114] Furthermore, during the screen recording process, the size of the video obtained by the screen recording can be set according to the actual needs of the user, which is specifically achieved through the following steps:

[0115] Step 51: Obtaining a recording video size set by the target application; the recording video size is used to indicate the size of a video frame obtained by screen recording;

[0116] Step 52: Create the input rendering surface corresponding to the size of the recorded video through the encoding thread, and provide the input rendering surface to the secondary rendering thread to obtain the video frame data corresponding to the size of the recorded video based on the input rendering surface.

[0117] When performing screen recording in the target application, users can be supported to set the recording video size. When the media encoder in the encoding thread creates the input rendering surface, it will create an input rendering surface that meets the recording video size set by the user. The secondary rendering thread draws the rendering content on the input rendering surface, and the obtained output rendering surface also meets the recording video size. Then, in the video frame data output by the encoding thread, each video frame also meets the recording video size, thereby realizing the determination of the size of the output video frame of this screen recording by the recording video size, meeting the different needs of users for the screen recording video size.

[0118] Furthermore, during the screen recording process, the frame rate of the video obtained by the screen recording can be set according to the actual needs of the user, which can be achieved in the following ways:

[0119] Obtaining a recording video frame rate set by the target application; the recording video frame rate is used to indicate a frame rate of a video obtained by screen recording;

[0120] At this point, the secondary rendering thread draws the input rendering surface through the following steps:

[0121] The input rendering surface is rendered based on the rendering data according to the rendering cycle corresponding to the recorded video frame rate through the secondary rendering thread to obtain the output rendering surface.

[0122] During screen recording, the recording video frame rate can be set. When the secondary rendering thread draws the input rendering surface, it draws the input rendering surface based on the drawing cycle corresponding to the recording video frame rate. For example, when the recording video frame rate is 30 frames and the frame rate of the target application main rendering thread itself is 60 frames, the secondary rendering thread draws the input rendering surface once every video frame. After the output rendering surface is encoded by the encoding thread, the frame rate corresponding to the video frame data conforms to the recording video frame rate.

[0123] When the recorded video frame rate is higher than the frame rate of the target application's main rendering thread, the secondary rendering thread will draw duplicate frames when drawing the input rendering surface. For example, if the recorded video frame rate is 60 frames and the main rendering thread's frame rate is 30 frames, each video frame corresponding to the rendering data will be drawn twice. The frame rate of the video obtained by screen recording can be increased by increasing the number of duplicate frames.

[0124] By setting the recording video frame rate, screen recording video frame data corresponding to different frame rates can be obtained, effectively meeting users' different requirements for screen recording video frame rates.

[0125] See also Figure 6 , Figure 6A schematic diagram of the interaction of various threads in a target application process provided in an embodiment of the present application, wherein the target application process includes a main rendering thread, a secondary rendering thread, and an encoding thread.

[0126] In the main rendering thread, by receiving the drawing instruction, executing the specific rendering function corresponding to the drawing instruction, and determining whether the shared texture data has been initialized, if so, directly copying the rendering data in the back buffer to the bridge buffer bound to the shared texture identifier; if not, creating a shared texture identifier and a bridge buffer, and binding the shared texture identifier to the bridge buffer, and then copying the rendering data in the back buffer to the bridge buffer bound to the shared texture identifier; when the main rendering thread runs normally, the rendering data in the back buffer will be rendered and displayed on the screen for the user to watch.

[0127] In the secondary rendering thread, after the secondary rendering thread is started, a secondary graphics program context corresponding to the secondary rendering thread is created based on the main graphics program context provided by the main rendering thread, and the main graphics program context is confirmed as the parent graphics program context of the secondary graphics program context, and the secondary graphics program context is determined to be the child graphics program context of the main graphics program context; the secondary rendering thread obtains the input rendering surface created by the encoding thread, and based on the shared texture identifier, obtains the rendering data in the bridge buffer, performs secondary rendering based on the rendering data, draws the rendering result onto the input rendering surface, obtains the output rendering surface after the drawing is completed, and provides the output rendering surface to the encoding thread for encoding processing.

[0128] After the encoding thread is started, a media encoder is first created, and then the media encoder creates an input rendering surface, which is transferred to the secondary rendering thread for drawing. When the secondary rendering thread completes drawing, the encoding thread receives the output rendering surface carrying the rendering results from the secondary rendering thread, and encodes the output rendering surface to obtain the video frame data of this screen recording.

[0129] See also Figure 7 As shown, Figure 7 A structural diagram of a screen recording device 700 provided in an embodiment of the present application, wherein the device supports the operation of a target application process corresponding to a target application, wherein the target application process includes a main rendering thread, a secondary rendering thread, and an encoding thread, wherein the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by the target application; Figure 7 As shown, the screen recording device 700 includes:

[0130] The data copy module 701 is used to copy the rendering data stored in the back buffer of the main rendering thread to the bridge buffer through the main rendering thread; the bridge buffer is bound with a shared texture identifier;

[0131] The rendering module 702 is used to obtain the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; based on the rendering data, draw the input rendering surface provided by the encoding thread to obtain an output rendering surface, and transmit the output rendering surface to the encoding thread;

[0132] The encoding module 703 is used to perform encoding processing based on the output rendering surface through the encoding thread to obtain the recorded video frame data.

[0133] Optionally, the device further comprises a graphics program context creation module, configured to:

[0134] Creating, by the secondary rendering thread, a secondary graphics program context corresponding to the secondary rendering thread according to the primary graphics program context corresponding to the primary rendering thread, and determining the primary graphics program context as a parent graphics program context of the secondary graphics program context;

[0135] The shared texture identifier created in the main rendering thread corresponding to the parent graphics program context supports use by the secondary rendering thread corresponding to the secondary graphics program context.

[0136] Optionally, the rendering module 702 is further used for:

[0137] Detecting, by the main rendering thread, whether the shared texture data has been initialized;

[0138] If not, creating the shared texture identifier and the bridge buffer through the main rendering thread, and binding the shared texture identifier and the bridge buffer; executing the process of copying the rendering data stored in the back buffer in the main rendering thread to the bridge buffer through the main rendering thread;

[0139] If so, the main rendering thread is executed to copy the rendering data stored in the back buffer in the main rendering thread to the bridge buffer.

[0140] Optionally, the data replication module 701 is specifically used for:

[0141] By means of the main rendering thread, setting the bridge buffer as a current write buffer in the main rendering thread, and setting the back buffer as a current read buffer in the main rendering thread;

[0142] The main rendering thread executes a buffer copy function to copy the rendering data from the current read buffer to the current write buffer.

[0143] Optionally, the data replication module 701 is specifically used for:

[0144] The buffer copy function is executed by the main rendering thread to copy the color data stored in the color buffer in the current read buffer to the current write buffer.

[0145] Optionally, the device further includes a parameter restoration module, configured to:

[0146] In response to a screen recording stop operation triggered by the target application, restoring a read frame buffer target and a write frame buffer target in the main rendering thread through the main rendering thread, and restoring the read frame buffer target and the write frame buffer target to values ​​before responding to the screen recording operation;

[0147] The read frame buffer target is used to indicate the current read buffer, and the write frame buffer target is used to indicate the current write buffer.

[0148] Optionally, the rendering module 702 is specifically used for:

[0149] Building a shader through the secondary rendering thread;

[0150] Drawing the rendering result based on the mapping relationship between the vertex coordinates of the input rendering surface and the texture coordinates of the rendering data by the shader;

[0151] The rendering result is output to the input rendering surface to obtain the output rendering surface.

[0152] Optionally, the encoding module 703 is further used for:

[0153] Creating a media encoder through the encoding thread;

[0154] Creating the input rendering surface through the media encoder, and transmitting the input rendering surface to the secondary rendering thread;

[0155] The encoding module 703 is specifically used for:

[0156] The media encoder performs encoding processing based on the output rendering surface to obtain the video frame data.

[0157] Optionally, the device further comprises a size setting module, configured to:

[0158] Obtaining a recording video size set by the target application; the recording video size is used to indicate the size of a video frame obtained by screen recording;

[0159] The input rendering surface corresponding to the size of the recorded video is created through the encoding thread, and the input rendering surface is provided to the secondary rendering thread, so as to obtain the video frame data corresponding to the size of the recorded video based on the input rendering surface recording.

[0160] Optionally, the device further includes a frame rate setting module, configured to:

[0161] Obtaining a recording video frame rate set by the target application; the recording video frame rate is used to indicate a frame rate of a video obtained by screen recording;

[0162] The step of drawing the input rendering surface provided by the encoding thread based on the rendering data to obtain the output rendering surface includes:

[0163] The input rendering surface is rendered based on the rendering data according to the rendering cycle corresponding to the recorded video frame rate through the secondary rendering thread to obtain the output rendering surface.

[0164] The screen recording device provided in the embodiment of the present application realizes the recording of the display screen of the target application itself through the main rendering thread, secondary rendering thread and encoding thread in the target application process, and does not involve the use of the screen rendering process in the system. Therefore, the display content or notification content of other applications unrelated to the target application will not be recorded, which effectively meets the actual screen recording needs of the user. In addition, since the above-mentioned screen recording device is implemented through the target application process corresponding to the target application, the rendering data can be directly obtained and used in the target application process. Without involving the acquisition of data in other applications, there is no need to apply for system permissions before screen recording, which simplifies the operation process of screen recording and improves the implementation efficiency of the screen recording function.

[0165] The embodiment of the present application also provides a computer device, which may specifically be a terminal device or a server. The terminal device and server provided in the embodiment of the present application will be introduced below from the perspective of hardware entity.

[0166] See also Figure 8 , Figure 8 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 8For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the terminal as a computer as an example:

[0167] Figure 8 FIG. 1 is a block diagram showing a partial structure of a computer related to a terminal provided in an embodiment of the present application. Figure 8 The computer includes: a radio frequency (RF) circuit 810, a memory 820, an input unit 830 (including a touch panel 831 and other input devices 832), a display unit 840 (including a display panel 841), a sensor 850, an audio circuit 860 (which is connected to a speaker 861 and a microphone 862), a wireless fidelity (WiFi) module 870, a processor 880, and a power supply 890. Those skilled in the art will understand that Figure 8 The computer structure shown in the figure does not constitute a limitation of the computer, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0168] The memory 820 can be used to store software programs and modules. The processor 880 executes various functional applications and data processing of the computer by running the software programs and modules stored in the memory 820. The memory 820 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer (such as audio data, a phone book, etc.), etc. In addition, the memory 820 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0169] The processor 880 is the control center of the computer. It uses various interfaces and lines to connect various parts of the entire computer. It executes various functions of the computer and processes data by running or executing software programs and / or modules stored in the memory 820 and calling data stored in the memory 820. Optionally, the processor 880 may include one or more processing units; preferably, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 880.

[0170] In the embodiment of the present application, the processor 880 included in the terminal also has the following functions:

[0171] By means of the main rendering thread, the rendering data stored in the back buffer of the main rendering thread is copied to the bridge buffer; the bridge buffer is bound with a shared texture identifier;

[0172] Obtaining the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; drawing the input rendering surface provided by the encoding thread based on the rendering data to obtain an output rendering surface, and transmitting the output rendering surface to the encoding thread;

[0173] The encoding thread performs encoding processing based on the output rendering surface to obtain recorded video frame data.

[0174] Optionally, the processor 880 is also used to execute the steps of any implementation method of the screen recording method provided in the embodiments of the present application.

[0175] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of a server 900 provided for an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 922 (for example, one or more processors) and a memory 932, and one or more storage media 930 (for example, one or more mass storage devices) storing application programs 942 or data 944. Among them, the memory 932 and the storage medium 930 may be temporary storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 922 may be configured to communicate with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the server 900.

[0176] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems, such as Windows Server 2000. TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.

[0177] The steps performed by the server in the above embodiment can be based on the Fig. 9 The server structure shown.

[0178] The CPU 922 is used to execute the following steps:

[0179] By means of the main rendering thread, the rendering data stored in the back buffer of the main rendering thread is copied to the bridge buffer; the bridge buffer is bound with a shared texture identifier;

[0180] Obtaining the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; drawing the input rendering surface provided by the encoding thread based on the rendering data to obtain an output rendering surface, and transmitting the output rendering surface to the encoding thread;

[0181] The encoding thread performs encoding processing based on the output rendering surface to obtain recorded video frame data.

[0182] Optionally, the CPU 922 can also be used to execute the steps of any implementation method of the screen recording method provided in the embodiments of the present application.

[0183] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program, wherein the computer program is used to execute any one of the implementation methods of a screen recording method described in the aforementioned embodiments.

[0184] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes any one of the implementations of the screen recording method described in the above embodiments.

[0185] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0186] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store computer programs.

[0190] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0191] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A screen recording method, characterized in that: The method is applied to a target application process corresponding to a target application, the target application process includes a main rendering thread, a secondary rendering thread and an encoding thread, the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by the target application; the method includes: By means of the main rendering thread, the rendering data stored in the back buffer of the main rendering thread is copied to the bridge buffer; the bridge buffer is bound with a shared texture identifier; Obtaining the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; drawing the input rendering surface provided by the encoding thread based on the rendering data to obtain an output rendering surface, and transmitting the output rendering surface to the encoding thread; The encoding thread performs encoding processing based on the output rendering surface to obtain recorded video frame data.

2. The method according to claim 1, characterized in that The method further comprises: Creating, by the secondary rendering thread, a secondary graphics program context corresponding to the secondary rendering thread according to the primary graphics program context corresponding to the primary rendering thread, and determining the primary graphics program context as a parent graphics program context of the secondary graphics program context; The shared texture identifier created in the main rendering thread corresponding to the parent graphics program context supports use by the secondary rendering thread corresponding to the secondary graphics program context.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Detecting, by the main rendering thread, whether the shared texture data has been initialized; If not, creating the shared texture identifier and the bridge buffer through the main rendering thread, and binding the shared texture identifier and the bridge buffer; executing the process of copying the rendering data stored in the back buffer in the main rendering thread to the bridge buffer through the main rendering thread; If so, the main rendering thread is executed to copy the rendering data stored in the back buffer in the main rendering thread to the bridge buffer.

4. The method according to claim 1, characterized in that The step of copying the rendering data stored in the back buffer of the main rendering thread to the bridge buffer through the main rendering thread includes: By means of the main rendering thread, setting the bridge buffer as a current write buffer in the main rendering thread, and setting the back buffer as a current read buffer in the main rendering thread; The main rendering thread executes a buffer copy function to copy the rendering data from the current read buffer to the current write buffer.

5. The method according to claim 4, characterized in that The executing a buffer copy function by the main rendering thread to copy the rendering data from the current read buffer to the current write buffer includes: The buffer copy function is executed by the main rendering thread to copy the color data stored in the color buffer in the current read buffer to the current write buffer.

6. The method according to claim 4, characterized in that The method further comprises: In response to a screen recording stop operation triggered by the target application, restoring a read frame buffer target and a write frame buffer target in the main rendering thread through the main rendering thread, and restoring the read frame buffer target and the write frame buffer target to values ​​before responding to the screen recording operation; The read frame buffer target is used to indicate the current read buffer, and the write frame buffer target is used to indicate the current write buffer.

7. The method according to claim 1, characterized in that The step of drawing the input rendering surface provided by the encoding thread based on the rendering data to obtain the output rendering surface includes: Building a shader through the secondary rendering thread; Drawing the rendering result based on the mapping relationship between the vertex coordinates of the input rendering surface and the texture coordinates of the rendering data by the shader; The rendering result is output to the input rendering surface to obtain the output rendering surface.

8. The method according to claim 1, characterized in that The method further comprises: Creating a media encoder through the encoding thread; Creating the input rendering surface through the media encoder, and transmitting the input rendering surface to the secondary rendering thread; The encoding process is performed based on the output rendering surface through the encoding thread to obtain the recorded video frame data, including: The media encoder performs encoding processing based on the output rendering surface to obtain the video frame data.

9. The method according to claim 1, characterized in that: The method further comprises: Obtaining a recording video size set by the target application; the recording video size is used to indicate the size of a video frame obtained by screen recording; The input rendering surface corresponding to the size of the recorded video is created through the encoding thread, and the input rendering surface is provided to the secondary rendering thread, so as to obtain the video frame data corresponding to the size of the recorded video based on the input rendering surface recording.

10. The method according to claim 1, characterized in that The method further comprises: Obtaining a recording video frame rate set by the target application; the recording video frame rate is used to indicate a frame rate of a video obtained by screen recording; The step of drawing the input rendering surface provided by the encoding thread based on the rendering data to obtain the output rendering surface includes: The input rendering surface is rendered based on the rendering data according to the rendering cycle corresponding to the recorded video frame rate through the secondary rendering thread to obtain the output rendering surface.

11. A screen recording device, characterized in that: The device supports the running of a target application process corresponding to a target application, wherein the target application process includes a main rendering thread, a secondary rendering thread, and an encoding thread, wherein the secondary rendering thread and the encoding thread are started in response to a screen recording operation triggered by the target application; The device comprises: A data copy module, used for copying the rendering data stored in the back buffer of the main rendering thread to the bridge buffer through the main rendering thread; the bridge buffer is bound with a shared texture identifier; A rendering module, configured to obtain the rendering data from the bridge buffer based on the shared texture identifier through the secondary rendering thread; draw the input rendering surface provided by the encoding thread based on the rendering data to obtain an output rendering surface, and transmit the output rendering surface to the encoding thread; The encoding module is used to perform encoding processing based on the output rendering surface through the encoding thread to obtain the recorded video frame data.

12. A computer device, characterized in that: The device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the screen recording method according to any one of claims 1 to 10 according to the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by an electronic device, the screen recording method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the screen recording method according to any one of claims 1 to 10 is implemented.

Images