Screen recording method, device, equipment and storage medium

Abstract

The application belongs to the multimedia field and discloses a screen recording method, device, equipment and storage medium. The method comprises the following steps: obtaining screen data according to a created screen recording thread; constructing an image buffer according to the screen data; and analyzing image data in the image buffer according to a virtual address of the image buffer to obtain screen recording data. According to the application, the screen data is obtained according to the created screen recording thread, the image buffer is constructed according to the screen data, and the image data in the image buffer is analyzed according to the virtual address of the image buffer to obtain the screen recording data. Compared with the existing image processor which stores the synthesized screen data into the image buffer without the display synthesis system, the above-mentioned method can record video pictures under HDMI, analog television, digital television and other channels.

Description

Technical Field

[0001] This invention relates to the field of video recording technology, and in particular to a screen recording method, apparatus, device, and storage medium. Background Technology

[0002] In existing technologies, media codecs or media recorders acquire screen data through the Surface they create, and this screen data is produced by the graphics processing unit (GPU). Furthermore, the SurfaceFlinger system targets virtual displays, and its compositing method is uniformly COMPOSITION_GLES. The data composited by the GPU is directly placed into the image buffer corresponding to the surface created by the media codec or media recorder. However, HDMI channel video data is not provided to the SurfaceFlinger system. Therefore, the native screen recording logic cannot record video data from HDMI, analog TV, digital TV, and other channels. Thus, how to record video data from HDMI, analog TV, digital TV, and other channels has become a pressing technical problem to be solved.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a screen recording method, apparatus, device, and storage medium, aiming to solve the technical problem of how to record video data under channels such as HDMI, analog TV, and digital TV.

[0005] To achieve the above objectives, the present invention provides a screen recording method, the method comprising the following steps:

[0006] Obtain screen data based on the created screen recording thread;

[0007] Construct an image buffer based on the screen data;

[0008] The image data in the image buffer is parsed based on the virtual address of the image buffer to obtain the screen recording data.

[0009] Optionally, the step of constructing an image buffer based on the screen data includes:

[0010] Obtain the shared memory handle corresponding to the screen data using a preset interface description language;

[0011] An image buffer is constructed based on the shared memory handle.

[0012] Optionally, after the step of parsing the image data in the image buffer according to the virtual address of the image buffer to obtain the screen recording data, the method further includes:

[0013] Upon receiving a screenshot command, determine whether the screen is currently being recorded.

[0014] If not, call the preset screenshot interface to obtain screenshot data;

[0015] The screenshot result is determined based on the screenshot data and the screenshot parameters in the screenshot command.

[0016] Optionally, after determining whether the screen recording state is currently in progress upon receiving a screenshot command, the method further includes:

[0017] If so, generate a screenshot request based on the screenshot instruction and add the screenshot request to the screenshot queue;

[0018] When the screen recording ends, copy the screen recording data.

[0019] The screenshot result is determined based on the screen recording data and the screenshot requirement.

[0020] Optionally, the step of determining the screenshot result based on the screen recording data and the screenshot requirement includes:

[0021] Determine the screenshot time and format based on the screenshot requirements;

[0022] The target screen data is obtained from the screen recording screen data based on the screenshot time;

[0023] The target screen data is parsed and its format converted to obtain the screenshot result.

[0024] Optionally, before the step of obtaining screen data according to the created virtual display thread, the method further includes:

[0025] Upon receiving a screen recording command, determine whether the current state is in screenshot mode;

[0026] If not, create a screen recording thread in the virtual display module to obtain screen data.

[0027] Optionally, the step of obtaining screen data according to the created screen recording thread includes:

[0028] Screen data is acquired using a preset sampling method based on the created screen recording thread.

[0029] Furthermore, to achieve the above objectives, the present invention also provides a screen recording device, the device comprising:

[0030] The screen data acquisition module is used to acquire screen data based on the created screen recording thread.

[0031] An image cache construction module is used to construct an image buffer based on the screen data;

[0032] The parsing module is used to parse the image data in the image buffer according to the virtual address of the image buffer to obtain the screen recording data.

[0033] Optionally, the image cache construction module is further configured to obtain the shared memory handle corresponding to the screen data through a preset interface description language;

[0034] An image buffer is constructed based on the shared memory handle.

[0035] Optionally, the parsing module is further configured to determine whether the screen is currently being recorded when a screenshot command is received;

[0036] If not, call the preset screenshot interface to obtain screenshot data;

[0037] The screenshot result is determined based on the screenshot data and the screenshot parameters in the screenshot command.

[0038] Optionally, the parsing module is further configured to, if so, generate a screenshot request based on the screenshot instruction and add the screenshot request to the screenshot queue;

[0039] When the screen recording ends, copy the screen recording data.

[0040] The screenshot result is determined based on the screen recording data and the screenshot requirement.

[0041] Optionally, the parsing module is further configured to determine the screenshot time and format based on the screenshot requirements;

[0042] The target screen data is obtained from the screen recording screen data based on the screenshot time;

[0043] The target screen data is parsed and its format converted to obtain the screenshot result.

[0044] Optionally, the screen data acquisition module is further configured to determine whether the screen is currently in a screenshot state when a screen recording command is received;

[0045] If not, create a screen recording thread in the virtual display module to obtain screen data.

[0046] Optionally, the screen data acquisition module is further configured to acquire screen data according to a preset sampling method based on the created screen recording thread.

[0047] In addition, to achieve the above objectives, the present invention also proposes a screen recording device, the device comprising: a memory, a processor, and a screen recording program stored in the memory and executable on the processor, the screen recording program being configured to implement the steps of the screen recording method described above.

[0048] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a screen recording program, which, when executed by a processor, implements the steps of the screen recording method described above.

[0049] This invention acquires screen data based on a created screen recording thread; constructs an image buffer based on the screen data; and parses the image data in the image buffer according to the virtual address of the image buffer to obtain screen recording data. Because this invention acquires screen data based on a created screen recording thread; constructs an image buffer based on the screen data; and parses the image data in the image buffer according to the virtual address of the image buffer to obtain screen recording data, compared to existing image processors that store composited screen data in an image buffer without going through a display compositing system, the above method of this invention can record video images from HDMI, analog TV, digital TV, and other channels. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the structure of a screen recording device in the hardware operating environment involved in the embodiments of the present invention;

[0051] Figure 2 This is a flowchart illustrating the first embodiment of the screen recording method of the present invention;

[0052] Figure 3 This is a diagram illustrating the sampling method of the first embodiment of the screen recording method of the present invention;

[0053] Figure 4 This is a logical schematic diagram of the first embodiment of the screen recording method of the present invention;

[0054] Figure 5 This is a flowchart illustrating the second embodiment of the screen recording method of the present invention;

[0055] Figure 6 This is a structural block diagram of the first embodiment of the screen recording device of the present invention.

[0056] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0058] Reference Figure 1 , Figure 1 This is a schematic diagram of the screen recording device structure of the hardware operating environment involved in the embodiments of the present invention.

[0059] like Figure 1 As shown, the screen recording device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0060] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the screen recording device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0061] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a screen recording program.

[0062] exist Figure 1 In the screen recording device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the screen recording device of the present invention can be set in the screen recording device, and the screen recording device calls the screen recording program stored in the memory 1005 through the processor 1001 and executes the screen recording method provided in the embodiment of the present invention.

[0063] Based on the aforementioned screen recording device, this invention provides a screen recording method, referring to... Figure 2 , Figure 2This is a flowchart illustrating the first embodiment of the screen recording method of the present invention.

[0064] In this embodiment, the screen recording method includes the following steps:

[0065] Step S10: Obtain screen data based on the created screen recording thread.

[0066] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a mobile phone, tablet computer, or personal computer, or an electronic device or screen recording service capable of performing the above functions. The following description uses the screen recording service as an example to illustrate this embodiment and the subsequent embodiments.

[0067] It should be understood that, based on the native Android screen recording process, the entire screen recording process is essentially the GPU continuously compositing various layers into the buffer corresponding to the VirtualDisplaySurface. Therefore, referring to the principle of VirtualDisplaySurface, a new VirtualDisplaySurface can be created, which can be named VdpVirtualDisplaySurface. In the VdpVirtualDisplaySurface creation thread, screen data is continuously retrieved from the underlying layer and submitted to the buffer created by MediaCodec, replacing the GPU's layer compositing action. Here, the VirtualDisplaySurface is the screen recording thread created.

[0068] It should be understood that MediaCodec provides audio and video compression encoding and decoding functions, while MediaMuxer can mix audio and video to generate multimedia files, such as MP4 files. Similar to MediaRecorder, both require obtaining screen recording permissions through MediaProjectionManager and processing screen data in the callback. The above screen recording methods all follow the standard Android system call flow. These methods can only record the UI (Android 9.0 and below can record video data from SurfaceView), therefore, video cannot be recorded through channels such as HDMI, analog TV, and digital TV.

[0069] In practice, the screen recording service references the principle of VirtualDisplaySurface and creates a new screen recording thread VirtualDisplaySurface to obtain screen data from the underlying layer.

[0070] Furthermore, in order to improve the efficiency of acquiring screen data, step S10 includes: acquiring screen data through a preset sampling method according to the created screen recording thread.

[0071] It should be noted that the preset sampling method can be a YUV sampling method, which can be found in the following reference. Figure 3 , Figure 3 This diagram illustrates the sampling method of the first embodiment of the screen recording method of the present invention. The YUV sampling methods mainly include three types: YUV444, YUV422, and YUV420.

[0072] YUV4:4:4 sampling, each Y corresponds to a set of UV components, and on average, one pixel occupies 3 bytes.

[0073] YUV4:2:2 sampling, where every two Y values ​​share a set of UV components, with each pixel occupying an average of 2 bytes.

[0074] YUV4:2:0 sampling, where every four Y values ​​share a single UV component, with an average pixel occupying 1.5 bytes.

[0075] YUV storage method

[0076] YUV is divided into three components: "Y" represents luminance (or luma), which is the grayscale value; while "U" and "V" represent chroma (or chroma), which are used to describe the color and saturation of an image and to specify the color of a pixel.

[0077] There are two main types of YUV formats: planar and packed.

[0078] For the YUV format of planar, all pixels' Y values ​​are stored consecutively first, followed by all pixels' U values, and then all pixels' V values.

[0079] For packed The YUV format stores the Y, U, and V values ​​of each pixel in a continuous, interleaved manner.

[0080] I420:YYYYYYYYYUUVV=>YUV420P

[0081] YV12:YYYYYYYYYVVUU=>YUV420P

[0082] NV12:YYYYYYYYUVUV => YUV420SP

[0083] NV21:YYYYYYYYVUVU => YUV420SP

[0084] The corresponding definition in Android is:

[0085] HAL_PIXEL_FORMAT_YV12

[0086] HAL_PIXEL_FORMAT_YCrCb_420_SP

[0087] Furthermore, to avoid conflicts between screen recording and screenshotting, which could lead to interruption or abnormality in screenshotting, before step S10, the method further includes: upon receiving a screen recording command, determining whether the current state is a screenshot state; if not, creating a screen recording thread in the virtual display module to obtain screen data.

[0088] It should be noted that, to avoid terminal errors or malfunctions during screen recording, upon receiving a screen recording command, it first determines whether a screenshot state is currently in effect. If not, a screen recording thread is created in the virtual display module to acquire screen data. If a screenshot state is currently in effect, a screen recording request is generated according to the screen recording command and added to the screen recording queue. For details, please refer to the steps in the second embodiment for generating a screenshot request according to the screenshot command and adding the screenshot request to the screenshot queue; these steps will not be elaborated upon here.

[0089] Step S20: Construct an image buffer based on the screen data.

[0090] It should be noted that constructing the image buffer based on the screen data can be achieved by obtaining the shared memory handle of the original image buffer and constructing the image buffer based on the shared memory handle and the screen data.

[0091] Furthermore, in order to record video footage from channels such as HDMI, analog TV, and digital TV, step S20 may include: obtaining a shared memory handle corresponding to the screen data through a preset interface description language; and constructing an image buffer based on the shared memory handle.

[0092] It should be noted that the preset interface description language can be HIDL (Interface Description Language), which is an interface description language used to specify the interface between HAL and its users.

[0093] In practice, the screen recording service submits screen data to a buffer created by MediaCodec to replace the GPU's compositing layer action. IScreenRecord passes the handle of the shared memory corresponding to the Graphicbuffer to the screen recording service via HIDL. After receiving the shared memory handle, the ScreenRecord Service reconstructs the image buffer.

[0094] Step S30: Parse the image data in the image buffer according to the virtual address of the image buffer to obtain the screen recording data.

[0095] It should be noted that parsing the image data in the image buffer based on the virtual address of the image buffer may involve obtaining the virtual address of the image buffer, obtaining the image data in the image buffer based on the virtual address, and performing processing such as parsing the image data, creating a recording thread, and color format conversion.

[0096] In specific implementation, the screen recording service obtains the virtual address of the image buffer, retrieves the image data in the image buffer based on the virtual address, and performs processing on the image data, such as parsing image parameters, initializing the platform to create a recording thread, controlling recording, color format conversion, RingBuffer management, multi-process screenshotting, screen recording logic processing, obtaining screenshots from the underlying layer, and screen data required for screen recording.

[0097] In specific implementation, it can be referred to Figure 4 , Figure 4 This is a logical schematic diagram of the first embodiment of the screen recording method of the present invention, referring to... Figure 4It is known that the screen recording service creates a new thread, VdpVirtualDisplaySurface, to continuously retrieve screen data from the underlying layer and submit it to the buffer created by MediaCodec, replacing the GPU's compositing layer action. IScreenRecord passes the shared memory handle corresponding to the Graphicbuffer to ScreenRecordService via HIDL. Upon receiving the shared memory handle, the ScreenRecordService reconstructs the Graphicbuffer and passes its corresponding virtual address to the unified screen recording logic implementation. The unified screen recording logic implementation's main functions include: parsing screenshot parameters, initializing the platform to create a recording thread, controlling recording, color format conversion, RingBuffer management, multi-process screenshotting, and screen recording logic processing. Then, the processed data is converted according to the default data format of each platform, including MTK or RTK.

[0098] This embodiment acquires screen data based on a created screen recording thread; constructs an image buffer based on the screen data; and parses the image data in the image buffer according to the virtual address of the image buffer to obtain screen recording data. Because this embodiment acquires screen data based on a created screen recording thread; constructs an image buffer based on the screen data; and parses the image data in the image buffer according to the virtual address of the image buffer to obtain screen recording data, compared to existing image processors that store composited screen data in an image buffer without going through a display compositing system, this embodiment can record video footage from HDMI, analog TV, digital TV, and other channels.

[0099] refer to Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the screen recording method of the present invention.

[0100] Based on the first embodiment described above, in this embodiment, after step S30, the method further includes:

[0101] Step S40: Upon receiving a screenshot command, determine whether the screen is currently in recording mode.

[0102] In practice, when the screen recording service receives a screenshot command, it determines whether it is currently in screen recording mode.

[0103] Furthermore, to accommodate screen recording scenarios, if a screenshot is needed simultaneously with screen recording, directly calling the underlying screenshot interface would cause a conflict, leading to screen recording interruption or abnormality. Therefore, after step S40, the method further includes: if the screen is currently recording, generating a screenshot request based on the screenshot instruction and adding the screenshot request to the screenshot queue; copying the screen recording data at the end of screen recording; and determining the screenshot result based on the screen recording data and the screenshot request.

[0104] It should be noted that the screenshot command and the screenshot requirement may include information such as the screenshot time and the required screenshot format. Determining the screenshot result based on the screen recording data and the screenshot requirement may involve determining the screenshot time and format based on the screenshot requirement; obtaining the target screen data from the screen recording data based on the screenshot time; and performing data parsing and format conversion on the target screen data to obtain the screenshot result.

[0105] It should be noted that the target screen data can be screen data including screenshot data corresponding to the screenshot time. The step of parsing and format conversion of the target screen data to obtain the screenshot result can involve parsing the target screen data and formatting the screenshot result to the required screenshot format.

[0106] Step S50: If not, call the preset screenshot interface to obtain screenshot data.

[0107] It should be noted that if you are not currently in screen recording mode, you can directly call the existing screenshot interface to obtain screenshot data. The preset screenshot interface can be the normalized screenshot interface in SurfaceFlinger.

[0108] It should be understood that Android's native screenshot logic is to take a screenshot via the screencap command in the serial port, which is a standard method provided by Android.

[0109] The ScreenCap screenshot process involves iterating through the Layers in SurfaceFlinger, preparing the information needed by the GPU for compositing, and placing it into a LayerSettings container. This Vector is then given to the GPU for rendering, and the rendered data is placed in a GraphicBuffer. The application layer retrieves the RGB data from the GraphicBuffer and encodes it into the required format to generate the screenshot result.

[0110] In practice, if the screen is not currently being recorded, the normalized screenshot interface is called in SurfaceFlinger (SurfaceFlinger is a class and also a thread that continuously calculates and refreshes the content to be displayed on the current screen through a loop) to obtain screenshot data through the platform's private screenshot method.

[0111] Step S60: Determine the screenshot result based on the screenshot data and the screenshot parameters in the screenshot command.

[0112] It should be noted that the screenshot parameters may include the required screenshot format, for example, a screenshot requires RGBA format. Determining the screenshot result based on the screenshot data and the screenshot parameters in the screenshot command can be done by processing and converting the screenshot data according to the required format to generate the screenshot result. For example, the RTK2851D platform's underlying screenshot support is YV12, while screenshots require RGBA format; therefore, the following conversion is necessary:

[0113] libyuv::I420ToABGR(srcData,bufferStrideY,&srcData[bufferOffsetV],bufferStrideV,&srcData[bufferOffsetU],bufferStrideU,dstData,width*4,width,height);

[0114] It should be noted that: bufferStrideV = bufferStrideU = bufferStrideY /

[0115] For example, the MTK9638 platform's default screenshot data format is NV21, while screenshots require ARGB format. Therefore, the following conversion is necessary:

[0116] libyuv::int NV21ToARGB(const uint8*src_y,int src_stride_y,constuint8*src_uv,int src_stride_uv,uint8*dst_argb,int dst_stride_argb,int width,int height)

[0117] It should be noted that: src_stride_y = src_stride_uv

[0118] In practical implementation, to avoid conflicts caused by different screenshot requirements, each application needs to call a normalized screenshot interface when taking screenshots. Java layer calls can be made through the screenshot interface of SystemService or by reflecting on the `screenshot` method in SurfaceControl. As the scenarios requiring screenshot functionality increase, to avoid conflicts across multiple scenarios and to standardize related common logic, thus improving the development efficiency of this module, all screenshot requirements are unified in the ScreenRecord Service. Each platform only needs to connect to the corresponding screenshot interface. The requirements can be easily obtained by calling the interface according to the following instructions to obtain the RAW data after the screenshot;

[0119] An interface has been added to sky_hal_videoctrl.cpp.

[0120] / **

[0121] *This API is called when a screenshot is needed in Skyhal. After the call is complete, the screenshot data will be saved in screenDataBuf.

[0122] * By obtaining the virtual address using a lock, the data corresponding to that buffer can be read.

[0123] *Header file: #include "ScreenCapture.h"

[0124] *Statement: using::android::skw_screenrecord::ScreenCapture;

[0125] * using::android::GraphicBuffer;

[0126] * using::android::sp;

[0127] *Other: This interface supports multi-threaded calls.

[0128] *return:0 success, -1 failure

[0129] ** /

[0130] S32SkyHalVideoCtrl::getVideoScreenData(sp <graphicbuffer>screenDataBuf,SKY_BOOL bWithUI)

[0131] {

[0132] S32 ret=-1;

[0133] sp <screencapture>IScreenCap;

[0134] IScreenCap=ScreenCapture::allocScreenCapture();

[0135] if(IScreenCap)

[0136] ret=IScreenCap->getScreenShotData(screenDataBuf,bWithUI);

[0137] return ret;

[0138] }

[0139] The GraphicBuffer is created as follows:

[0140] / **

[0141] *inWidth is the pre-trimmed width

[0142] *inHeight pre-trimmed height

[0143] *inFormat HAL_PIXEL_FORMAT_RGBA_8888 For other formats, please refer to [the documentation / reference].

[0144] * / system / core / include / system / graphics-base-v1.0.h

[0145] *inUsage can refer to GraphicBuffer::USAGE_HW_TEXTURE|

[0146] *GraphicBuffer::USAGE_SW_WRITE_RARELY

[0147] ** /

[0148] GraphicBuffer(uint32_tinWidth,uint32_tinHeight,PixelFormatinFormat,uint32_tinUsage)

[0149] For example:

[0150] sp <graphicbuffer>graphicBuffer=new GraphicBuffer(1920,1080,

[0151] HAL_PIXEL_FORMAT_RGBA_8888,GraphicBuffer::USAGE_HW_TEXTURE|

[0152] GraphicBuffer::USAGE_SW_WRITE_RARELY);

[0153] Get the virtual address of the data corresponding to the GraphicBuffer.

[0154] uint8_t*capure_addr = NULL;

[0155] graphicBuffer->lock(GRALLOC_USAGE_SW_WRITE_OFTEN|GRALLOC_USAGE_SW_READ_OFTEN,(void**)(&capure_addr));

[0156] Dump screenshot data

[0157] mkdir -p / data / vendor / tmp; chmod777 / data / vendor / tmp

[0158] setpropthird.get.dumpRecievedScreenData1

[0159] After setting the prop, when a screenshot is triggered, the data will be automatically saved to / data / vendor / tmp / RecievedScreenData.bin. After saving, this prop will be reset to 0.

[0160] Add the relevant libraries to Android.mk

[0161] LOCAL_SHARED_LIBRARIES+=\

[0162] vendor.skyworth.hardware.screenrecord@1.0\

[0163] android.hidl.allocator@1.0\

[0164] android.hidl.memory@1.0\

[0165] libhidlmemory\

[0166] libCaptureScreenClientVendor\

[0167] libui

[0168] If this service is called from the system directory, the corresponding library is libCaptureScreenClient.

[0169] Location of screenshot-related code:

[0170] Screenshot client-side code implementation: SkyExecutables\ScreenCaptureService\client

[0171] Screenshot ServiceHIDLInterface: vendor\skyworth\hardware\interfaces\screenrecord

[0172] Screenshot of ServiceSeLinux related permissions:

[0173] vendor\skyworth\driverBase\android\sepolicy\Androidx\common

[0174] Common logic implementation for screenshot service: SkyExecutables\ScreenCaptureService\imp\common

[0175] Screenshot functionality is implemented across various platforms: SkyExecutables\ScreenCaptureService\imp\xxx(mtk)

[0176] In this embodiment, upon receiving a screenshot command, it determines whether the screen is currently recording. If not, it calls a preset screenshot interface to obtain screenshot data. The screenshot result is then determined based on the screenshot data and the screenshot parameters in the screenshot command. Currently, if a screenshot is needed simultaneously with screen recording, the existing method directly calls the underlying screenshot interface, which can cause conflicts, leading to recording interruptions or errors. Therefore, in this embodiment, if a screenshot is needed simultaneously with screen recording, the screenshot request is added to a queue. After capturing screen data during recording, a copy of the data is copied to the screenshot buffer to obtain the screenshot data. This avoids recording interruptions or errors caused by screenshots.

[0177] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the screen recording device of the present invention.

[0178] like Figure 6 As shown, the screen recording device proposed in this embodiment of the invention includes:

[0179] Screen data acquisition module 10 is used to acquire screen data according to the created screen recording thread;

[0180] Image cache construction module 20 is used to construct an image buffer based on the screen data;

[0181] The parsing module 30 is used to parse the image data in the image buffer according to the virtual address of the image buffer to obtain the screen recording data.

[0182] This embodiment acquires screen data based on a created screen recording thread; constructs an image buffer based on the screen data; and parses the image data in the image buffer according to the virtual address of the image buffer to obtain screen recording data. Because this embodiment acquires screen data based on a created screen recording thread; constructs an image buffer based on the screen data; and parses the image data in the image buffer according to the virtual address of the image buffer to obtain screen recording data, compared to existing image processors that store composited screen data in an image buffer without going through a display compositing system, this embodiment can record video footage from HDMI, analog TV, digital TV, and other channels.

[0183] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0184] In addition, for technical details not described in detail in this embodiment, please refer to the screen recording method provided in any embodiment of the present invention, which will not be repeated here.

[0185] Based on the first embodiment of the screen recording device of the present invention described above, a second embodiment of the screen recording device of the present invention is proposed.

[0186] In this embodiment, the image cache construction module 20 is further configured to obtain the shared memory handle corresponding to the screen data through a preset interface description language;

[0187] An image buffer is constructed based on the shared memory handle.

[0188] Furthermore, the parsing module 30 is also used to determine whether the screen is currently being recorded when a screenshot command is received;

[0189] If not, call the preset screenshot interface to obtain screenshot data;

[0190] The screenshot result is determined based on the screenshot data and the screenshot parameters in the screenshot command.

[0191] Furthermore, the parsing module 30 is also used to, if so, generate a screenshot request based on the screenshot instruction and add the screenshot request to the screenshot queue;

[0192] When the screen recording ends, copy the screen recording data.

[0193] The screenshot result is determined based on the screen recording data and the screenshot requirement.

[0194] Furthermore, the parsing module 30 is also used to determine the screenshot time and screenshot format according to the screenshot requirements;

[0195] The target screen data is obtained from the screen recording screen data based on the screenshot time;

[0196] The target screen data is parsed and its format converted to obtain the screenshot result.

[0197] Furthermore, the screen data acquisition module 10 is also used to determine whether the current state is a screenshot state when a screen recording command is received;

[0198] If not, create a screen recording thread in the virtual display module to obtain screen data.

[0199] Furthermore, the screen data acquisition module 10 is also used to acquire screen data according to the created screen recording thread through a preset sampling method.

[0200] Other embodiments or specific implementations of the screen recording device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0201] Furthermore, this embodiment of the invention also proposes a storage medium storing a screen recording program, which, when executed by a processor, implements the steps of the screen recording method described above.

[0202] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0203] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0205] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.< / graphicbuffer> < / screencapture> < / graphicbuffer>

Claims

1. A screen recording method, characterized in that, The screen recording method includes the following steps: Screen data is acquired using YUV sampling based on the created screen recording thread; Construct an image buffer based on the screen data; The image data in the image buffer is parsed according to the virtual address of the image buffer to obtain the screen recording data; The step of constructing an image buffer based on the screen data includes: obtaining a shared memory handle corresponding to the screen data through the HIDL interface description language; Construct an image buffer based on the shared memory handle; Before the step of acquiring screen data via YUV sampling based on the created screen recording thread, the method further includes: Upon receiving a screen recording command, determine whether the current state is in screenshot mode; If the current screen recording state is being captured, a screen recording request is generated according to the screen recording instruction and the screen recording request is added to the screen recording queue; The parsing of the image data in the image buffer includes: The image data in the image buffer is processed by at least one of the following: parsing image parameters, initializing the platform to create a recording thread, controlling recording, color format conversion, RingBuffer management, multi-process screenshotting, screen recording logic processing, obtaining screenshot data from the underlying layer, and screen data required for screen recording.

2. The screen recording method as described in claim 1, characterized in that, After the step of parsing the image data in the image buffer according to the virtual address of the image buffer to obtain the screen recording data, the method further includes: Upon receiving a screenshot command, determine whether the screen is currently being recorded. If not, call the preset screenshot interface to obtain screenshot data; The screenshot result is determined based on the screenshot data and the screenshot parameters in the screenshot command.

3. The screen recording method as described in claim 2, characterized in that, After determining whether the screen recording mode is currently active upon receiving a screenshot command, the process also includes: If so, generate a screenshot request based on the screenshot instruction and add the screenshot request to the screenshot queue; When the screen recording ends, copy the screen recording data. The screenshot result is determined based on the screen recording data and the screenshot requirement.

4. The screen recording method as described in claim 3, characterized in that, The steps for determining the screenshot result based on the screen recording data and the screenshot requirement include: Determine the screenshot time and format based on the screenshot requirements; The target screen data is obtained from the screen recording screen data based on the screenshot time; The target screen data is parsed and its format converted to obtain the screenshot result.

5. A screen recording device, characterized in that, The screen recording device includes: The screen data acquisition module is used to acquire screen data using YUV sampling based on the created screen recording thread. An image cache construction module is used to construct an image buffer based on the screen data; The parsing module is used to parse the image data in the image buffer according to the virtual address of the image buffer to obtain the screen recording data; The image cache construction module is also used to obtain the shared memory handle corresponding to the screen data through the HIDL interface description language; Construct an image buffer based on the shared memory handle; The screen data acquisition module is also used to determine whether the screen is currently in a screenshot state when a screen recording command is received; If the current screen recording state is being captured, a screen recording request is generated according to the screen recording instruction and the screen recording request is added to the screen recording queue; The parsing module is also used to perform at least one of the following processes on the image data in the image buffer: parsing image parameters, initializing the platform to create a recording thread, controlling recording, color format conversion, RingBuffer management, multi-process screenshot, screen recording logic processing, obtaining screenshot data from the underlying layer, and screen data required for screen recording.

6. A screen recording device, characterized in that, The device includes: a memory, a processor, and a screen recording program stored in the memory and executable on the processor, the screen recording program being configured to implement the steps of the screen recording method as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium stores a screen recording program, which, when executed by a processor, implements the steps of the screen recording method as described in any one of claims 1 to 5.

Images