Recording method, terminal device and medium

Abstract

This application provides a recording method, terminal device, and medium, relating to the field of terminal device technology. The recording method is applied to a first terminal device, which includes at least two types of processors. The recording method includes: acquiring first information reflecting the resource usage of a first application on the first terminal device, the resource including at least one of at least two types of processors; acquiring second information based on the first information, the second information indicating the processor used to perform the recording, the processor indicated by the second information being one of at least two types of processors; and performing recording using the processor indicated by the second information to record the screen and / or audio of the first application. This application combines the resource usage of the application being recorded with the selection of the processor from multiple processors for recording, thus helping to minimize the impact of recording on the application's resource usage and thereby reducing the impact of recording on application performance.

Description

Technical Field

[0001] This application relates to the field of terminal equipment technology, and in particular to a recording method, terminal equipment and medium. Background Technology

[0002] Terminal devices can record the application's visuals and audio in real time while the application is running, and save the recording as a video file. However, recording will affect the application's use of device resources, thus impacting application performance. Summary of the Invention

[0003] This application provides a recording method, terminal device, and medium. By considering the resource usage of the application being recorded, a processor is selected from a variety of processors to perform the recording. This helps to minimize the impact of recording on the application's use of device resources, thereby reducing the impact of recording on application performance.

[0004] In a first aspect, embodiments of this application provide a recording method applied to a first terminal device, the first terminal device including at least two types of processors; the recording method includes: acquiring first information, the first information reflecting the resource occupancy of the first terminal device by a first application, the resource including at least one of at least two types of processors; acquiring second information based on the first information, the second information indicating the processor used to perform the recording, the processor indicated by the second information being one of at least two types of processors; and performing recording using the processor indicated by the second information to record the screen and / or audio of the first application. That is, the first application is the application being recorded.

[0005] In this embodiment, the terminal device can dynamically adjust the recording strategy according to the resource usage of the recorded application. This allows the recording strategy to be adapted to the resource usage of the recorded application, thereby minimizing the impact of recording on the application's use of device resources.

[0006] Optionally, the first information includes: the identifier of the first application and the identifier of the first terminal device. In this way, the terminal device can obtain the recording strategy based on the identifier information, without needing to collect data on the application's resource usage.

[0007] Optionally, based on the first information, obtaining the second information includes: sending the first information to the cloud to request the processor used to perform the recording.

[0008] Compared to the implementation method where the terminal device determines the recording strategy itself, since the update of the determination rules used by the terminal device is delayed, when determining the recording strategy through the cloud, the recording strategy can be accurately determined based on the real-time updated determination rules.

[0009] Optionally, obtaining the second information based on the first information further includes: receiving the second information sent by the cloud based on the first information and the mapping information; wherein the mapping information is used to describe the mapping relationship between the identification information and the processor, and the identification information includes the application identifier and the terminal device identifier; the processor indicated by the second information is the processor mapped to the first information. This facilitates the cloud in quickly and accurately finding the recording strategy, and allows developers to optimize the mapping information as needed based on user feedback and other information.

[0010] Optionally, obtaining the second information based on the first information further includes: if no information from the cloud is received indicating the processor used to perform the recording, then obtaining the third information, which includes the resource utilization rate of the first application on the device; and generating the second information based on the third information.

[0011] Terminal devices can initiate local performance testing when not in use to determine their own recording strategy, thus enabling dynamic adjustment of the recording strategy during periods of inactivity.

[0012] Optionally, after generating the second information based on the third information, the recording method further includes sending the second information to the cloud. This helps address the issue of other terminal devices not using the same identifier information in the short term, thus eliminating the need for other terminal devices to initiate local performance testing.

[0013] Optionally, the first information includes: the resource utilization rate of the first application on the device. This helps to avoid situations where the application is not actively used.

[0014] Optionally, the first terminal device stores utilization thresholds for each of at least two types of processors; based on the first information, second information is obtained, including generating the second information based on the first information and the utilization thresholds. This helps to generate a recording strategy that adapts to the application's resource usage on the device.

[0015] Optionally, at least two types of processors include a central processing unit (CPU) and a graphics processing unit (GPU). If the first information indicates that the GPU utilization rate of the first application does not exceed a first threshold, or indicates that the GPU utilization rate of the first application exceeds the first threshold and the CPU utilization rate of the first application exceeds a second threshold, then the processor indicated by the second information is the GPU. If the first information indicates that the GPU utilization rate of the first application exceeds the first threshold and the CPU utilization rate of the first application does not exceed the second threshold, then the processor indicated by the second information is the CPU. This helps to achieve reasonable use of CPU and GPU resources and reduce the impact of recording on the application's use of CPU and GPU resources.

[0016] Optionally, before obtaining the first information, the recording method further includes: receiving a recording request while the first application is running; and performing the step of obtaining the first information according to the recording request. This allows users to issue recording requests as needed during application runtime.

[0017] Optionally, the user interface of the first application includes a recording control; receiving a recording request includes receiving a user's trigger operation on the recording control. This provides convenience for users to use the application's recording function, facilitating the issuance of recording requests on demand.

[0018] Optionally, the first application is a game application. This helps to mitigate the impact of recording exciting game footage and audio on game smoothness.

[0019] Secondly, embodiments of this application provide a recording device applied to a first terminal device, the first terminal device including at least two types of processors; the recording device includes: a first acquisition module, used to acquire first information, the first information being used to reflect the occupation of device resources of the first terminal device by a first application, the device resources including at least one of at least two types of processors; a second acquisition module, used to acquire second information based on the first information, the second information being used to indicate the processor used to perform recording, the processor indicated by the second information being one of at least two types of processors; and a recording module, used to perform recording using the processor indicated by the second information to record the screen and / or audio of the first application.

[0020] Thirdly, embodiments of this application provide an electronic chip, including: a processor for executing computer program instructions stored in a memory, wherein when the computer program instructions are executed by the processor, the electronic chip is triggered to perform the method as described in any of the first aspects.

[0021] Fourthly, embodiments of this application provide a terminal device, which includes a processor and a memory for storing computer program instructions, wherein when the computer program instructions are executed by the processor, the terminal device is triggered to execute the method as described in any of the first aspects.

[0022] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method as described in any of the first aspects.

[0023] In a sixth aspect, embodiments of this application provide a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method as described in any of the first aspects.

[0024] The technical effects of the aforementioned aspects can be referenced from each other, and will not be elaborated further here. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below.

[0026] Figure 1 A timing diagram illustrating the implementation process of recording performed by a terminal device according to an embodiment of this application;

[0027] Figure 2 A schematic diagram of the user interface of a game application provided in one embodiment of this application;

[0028] Figure 3 A schematic diagram illustrating the sharing of recorded video by a terminal device according to one embodiment of this application;

[0029] Figure 4a This is a schematic diagram illustrating a user's request to record via third-party recording software, provided as an embodiment of this application.

[0030] Figure 4b This is a schematic diagram illustrating a user's request to record via the operating system's recording function, as provided in one embodiment of this application.

[0031] Figure 5 A schematic diagram of a user interface supporting user adjustment of recording strategies, provided as an embodiment of this application;

[0032] Figure 6 A timing diagram illustrating the implementation process of recording by an application client, as provided in one embodiment of this application;

[0033] Figure 7a This is a timing diagram providing support for cloud-generated mapping information in one embodiment of this application;

[0034] Figure 7b A timing diagram for reporting game performance reports to the cloud, provided as an embodiment of this application;

[0035] Figure 8 A schematic diagram illustrating the rules for formulating a recording strategy according to one embodiment of this application;

[0036] Figure 9 This application provides a timing diagram for generating mapping information in the cloud based on application performance reports, as one embodiment of the present application.

[0037] Figure 10 A timing diagram of the recording strategy provided by the application client to synchronize with the cloud, as provided in one embodiment of this application;

[0038] Figure 11 A timing diagram showing the application client performing recording according to the recording strategy in the cloud, as provided in one embodiment of this application;

[0039] Figure 12 A timing diagram illustrating the implementation process of recording by another application client provided in one embodiment of this application;

[0040] Figure 13 This is a schematic diagram illustrating the process of an application client generating a recording strategy through application performance detection, as provided in one embodiment of this application.

[0041] Figure 14 A timing diagram for generating a recording strategy and executing recording for an application client provided in one embodiment of this application;

[0042] Figure 15 This is a schematic diagram of the structure of a client framework provided in one embodiment of this application;

[0043] Figure 16 This is a schematic diagram of a cloud architecture provided in one embodiment of this application;

[0044] Figure 17 A schematic flowchart illustrating a recording method provided in one embodiment of this application;

[0045] Figure 18 This is a schematic diagram of the structure of a terminal device provided in one embodiment of this application;

[0046] Figure 19 This is a software structure block diagram of a terminal device provided in one embodiment of this application. Detailed Implementation

[0047] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0048] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0049] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0050] It should be understood that the term "at least one" as used in this document refers to one or more, and "more than one" refers to two or more. The term "and / or" as used in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0051] It should be understood that although the terms "first," "second," etc., may be used to describe the set thresholds in the embodiments of this application, these set thresholds should not be limited to these terms. These terms are only used to distinguish the set thresholds from each other. For example, without departing from the scope of the embodiments of this application, the first set threshold may also be referred to as the second set threshold, and similarly, the second set threshold may also be referred to as the first set threshold.

[0052] Terminal devices (such as smartphones) can record the screen and audio of an application in real time while it is running (or an application client), and save the recording as a video file to support users in storing or sharing the recording.

[0053] Feasibly, the aforementioned application can be any application already installed on the terminal device, especially applications where the user has a recording need, such as game applications, video conferencing applications, drawing applications, etc. For example, a smartphone can record exciting game moments and audio in real time while running a game application and save it as a video file.

[0054] In one feasible implementation, the application may have a recording function, and the terminal device may use the application's recording function to perform recording (i.e., in-application recording).

[0055] In one example, such as Figure 1 As shown, the process of recording performed by the terminal device may include the following steps 101 to 107.

[0056] Step 101: The application interaction module in the application client receives the user's request to start recording.

[0057] Feasibly, while the application client is running on the terminal device, the user can request the application client to start recording when the user needs to use the application's recording function.

[0058] Conveniently, the application client's user interface may include a recording control. The recording control can have a closed state and an open state, allowing the user to switch the state of the recording control to request to start or stop recording. The closed state indicates that recording has not started, while the open state indicates that recording has started or is in progress. Thus, the user can request to start recording by triggering (e.g., clicking) the recording control in the closed state. Correspondingly, the application interaction module can receive the user's request to start recording.

[0059] Taking the application client as an example of a game application, see [link / reference]. Figure 2 The terminal device can display a recording control 201 while showing the game screen of the game application. In response to the user clicking the recording control 201, the terminal device can control the recording control 201 to switch between an on and off state.

[0060] Step 102: The application interaction module requests the recording screen from the application engine in the application client based on the start recording request.

[0061] In applications where the user interface includes a recording control, one possible implementation is that the application interaction module can request the recording screen from the application engine when it detects that the user has triggered the recording control, which is in a closed state.

[0062] For example, while using a game application, a user can click the closed recording control 201 when they believe a highlight moment is about to occur or has already begun. After the user clicks the closed recording control 201, the application's interaction module requests the game engine to record the footage. The highlight moment can refer to every instant in a brilliant segment of gameplay, showcasing skillful maneuvers, excellent teamwork, or other impressive actions by the player.

[0063] In other feasible implementations, the application interaction module can also detect whether the preset recording conditions are met in real time or periodically after detecting that the user has triggered the recording control which is in a closed state. If the conditions are met, the module requests the recording screen from the application engine; otherwise, it does not request the recording screen.

[0064] For example, after the user clicks the recording control 201, which is in the closed state, the application interaction module in the game application can detect whether a game highlight moment exists according to the preset game highlight moment judgment rules, and request the recording screen from the game engine when the game highlight moment is detected. In this way, compared to the start time of recording, the user can click the recording control 201 in advance, and the game application can automatically start recording the game highlight moment when it detects the game highlight moment.

[0065] Step 103: In response to the application interaction module's request to record the screen, the application engine returns the recording screen to the application interaction module.

[0066] For example, the application engine can render the application screen and return the rendered screen as a recorded screen to the application interaction module.

[0067] Feasibly, the application engine can render the application screen in real time while the application client is running, and send the rendered application screen to the display module of the terminal device for display. In response to a request from the application interaction module to record the screen, the application engine can also send the real-time rendered application screen to the application interaction module.

[0068] Feasibly, the application interaction module can trigger the application engine to stop providing recording footage when the user requests to end recording. Before this, the application engine can continuously return the recording footage at various times to the application interaction module in sequence.

[0069] Step 104: The application interaction module sends the received recording screen to the recording module in the application client.

[0070] Step 105: The recording module generates a recorded video based on the recording screen sent by the application interaction module.

[0071] If feasible, the recording module has video encoding capabilities, then the recording module can perform video encoding on the recorded screen to generate a recorded video.

[0072] Alternatively, the application client can also include an audio module for collecting audio data. In addition to recording the video footage, the recording module can also combine the audio data collected by the audio module during recording to generate the recorded video.

[0073] Step 106: The recording module saves the recorded video to the local storage of the terminal device.

[0074] In other embodiments, if a user requests to share the recorded video, the terminal device can also share the recorded video to other users' terminal devices.

[0075] Step 107: The recording module uploads the recorded video to cloud storage.

[0076] In practice, besides generating recorded videos, the application client can also provide some basic video editing techniques (such as stickers, beautification, compositing, special effects, etc.) for users to edit the recorded videos as needed to generate finished video products. After generating the finished video product, the terminal device can not only store the finished video product locally and / or in the cloud, but also share the finished video product.

[0077] In one example, such as Figure 3As shown, client A has an application SDK (Software Development Kit) installed, and the application SDK has recording functionality. Client A can record video and obtain video footage by running the application SDK. For example, Figure 3 The application SDK in the middle can provide Figure 1 The application client SDK shown is... Figure 3 The video material can be the recorded video generated in step 105 above.

[0078] See Figure 3 Client A, by running the application SDK, can obtain video footage, query the video template provided by the application server, and then replace the corresponding content in the video template with the video footage and other pre-prepared materials (such as text, images, music, etc.) to generate the final video. The generated video can be stored in client A's local storage. Later, if a user requests to share the final video to client B, client A can export the final video and share it to client B.

[0079] Among other feasible implementation methods, Figure 3 The order of the steps shown may vary, and this application does not impose any restrictions on this. For example, client A may prepare other materials after obtaining the video materials. Alternatively, client A may query video templates before obtaining the video materials.

[0080] In addition to the in-application recording mentioned above, the terminal device can also use third-party recording software or the recording function provided by the terminal device's operating system to perform recording (i.e., out-of-application recording).

[0081] Conveniently, after a user launches the third-party recording software, the terminal device can display the user interface of the third-party recording software, and this user interface can include recording controls. If the user triggers the recording controls, the third-party recording software can begin recording.

[0082] For example, see Figure 4a The terminal device can display the user interface of the third-party recording software via a pop-up window, and this user interface includes a recording control 401. If the user clicks the recording control 401, the third-party recording software starts recording. If the user clicks the recording control 401 again, the third-party recording software stops recording.

[0083] Taking game video recording as an example, if the terminal device can run a game application while the third-party recording software is recording, the third-party recording software can record both game footage and audio. The user can launch the game application before or after launching the third-party recording software.

[0084] Conveniently, the operating system's shortcut toolbar could include a recording switch. If the user triggers the recording switch, the terminal device can display recording controls via a floating window or similar method. If the user then triggers the recording controls, the terminal device can begin recording.

[0085] For example, see Figure 4b The terminal device can display a quick settings toolbar via a drop-down menu, which includes a recording switch 402. If the user clicks the recording switch 402, the terminal device can display a recording control 403 via a floating window. If the user clicks the recording control 403, the terminal device starts recording. If the user clicks the recording control 403 again, the terminal device stops recording.

[0086] Taking game video recording as an example, if the terminal device can run a game application during recording, it can record both game visuals and audio. The user can launch the game application before or after triggering the recording switch.

[0087] In one feasible implementation, the terminal device may consistently use a specific type of processor to perform recording. For example, the processor used may be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or the like.

[0088] Both application operation and recording consume device resources (such as CPU, GPU, memory, etc.). This is especially true for application operation scenarios with high resource consumption, poor device resource capabilities, and heavy device load, such as game live streaming and game video sharing scenarios. In such cases, there may be competition for device resources between application operation and video recording.

[0089] In this context, "game live streaming" refers to a scenario where a terminal device is simultaneously running a game application and broadcasting the game's visuals and audio. "Game video sharing" refers to a scenario where a terminal device is simultaneously running a game application and sharing recorded game videos.

[0090] As we can see, the recording process affects an application's use of device resources, thus impacting application performance. Taking gaming applications as an example, the video recording process can include steps such as video encoding. Using the CPU for video encoding generally performs better for high-quality video output, but it consumes more CPU resources, potentially leading to decreased game performance. For instance, under high CPU load, stuttering or frame drops may occur. Conversely, using the GPU for video encoding can provide a smooth recording experience with lower GPU utilization, but under high GPU load, it may affect the game's frame rate and smoothness.

[0091] Different applications typically have varying resource usage on devices, and different terminal devices can provide different amounts of device resources. Therefore, if a single processor is consistently used for recording, a mismatch between the recording strategy (or recording method, which reflects the processor used for recording) and the resource usage of the application being recorded can easily occur, significantly impacting application performance. For example, if a terminal device consistently uses the CPU for recording, and an application has a high CPU usage, the recording execution will affect the application's CPU resource consumption, thus impacting application performance.

[0092] In another feasible implementation, the terminal device supports users in selecting recording strategies on demand, that is, allowing users to choose the processor to use for recording and executing recording based on the user's selection. For example, the processor used can be a CPU, GPU, etc.

[0093] For example, see Figure 5 When displaying the game screen of a game application, the terminal device can display not only the recording control 201, but also the game settings control 501. If the user clicks the game settings control 501, the terminal device can display a pop-up window with the corresponding "CPU recording" strategy control 502 and the corresponding "GPU recording" strategy control 503, allowing the user to select one of the recording strategies as needed.

[0094] If the user clicks control 502, meaning the user selects a recording strategy of using the CPU to record video, the terminal device can control control 502 to be selected while control 503 is deselected, and use the CPU to perform recording. Conversely, if the user clicks control 503, meaning the user selects a recording strategy of using the GPU to record video, the terminal device can control control 503 to be selected while control 502 is deselected, and use the GPU to perform recording.

[0095] While the above implementation allows users to select recording strategies on demand, most users struggle to understand the rationale behind their choices. Therefore, they often rely on subjective feelings to determine the timing and target of their selections. This can easily lead to a mismatch between the chosen strategy and the device resource usage of the application being recorded. For example, a user might choose to use the CPU for recording when the application is using a high amount of CPU. Furthermore, this implementation requires users to manually configure the recording strategy, which also impacts the user experience.

[0096] To support the adaptation of the recording strategy used during the recording of an application's screen and / or audio to the device resource usage of the application being recorded, so as to minimize the impact of recording on the application's use of device resources, in one embodiment of this application, the processor to be used for recording can be selected from a variety of processors according to the device resource usage of the application being recorded, that is, the recording strategy can be dynamically adjusted according to the device resource usage of the application being recorded.

[0097] In one embodiment, the application described above can be an application that consumes a significant amount of device resources during runtime, such as some game applications. In other embodiments, the application described above can also be any application running on the terminal device.

[0098] In one feasible implementation, the cloud can provide a list of applications. The terminal device can dynamically adjust the recording strategy for each application in the list. For applications not in the list, the terminal device can, for example, use a fixed processor for recording or select any processor. In other feasible implementations, the terminal device can also support user-provided application lists or allow users to adjust the application list provided by the cloud.

[0099] For example, the device resources of the terminal device may include memory, processor, etc. The processor in the terminal device may include, for example, a CPU, GPU, digital signal processor (DSP), image signal processor (ISP), etc. In one embodiment, the processor used for recording may be either a CPU or a GPU.

[0100] By dynamically adjusting the recording strategy based on the device resource usage of the application being recorded, the consumption of device resources by video recording can be effectively managed. This helps ensure a balance between application smoothness and recording quality, minimizing the conflict between recording quality and application performance, and thus reducing the impact of recording on application performance and optimizing the recording experience. Furthermore, since the terminal device can automatically adjust the recording strategy, this adjustment can be done without user intervention, thereby improving the user experience.

[0101] In one embodiment, the terminal device can obtain the application's resource utilization rate to reflect the application's resource usage. Thus, the terminal device can determine a recording strategy based on the utilization rate data (i.e., the resource utilization rate of the recorded application).

[0102] When a terminal device uses occupancy data to obtain a recording strategy, one feasible implementation is that the terminal device can use... Figure 13 The occupancy data acquisition method described in the illustrated embodiment (see steps a1 to a4 below) is used to acquire occupancy data, which helps to improve the accuracy of occupancy data and thus helps to obtain an accurate recording strategy.

[0103] Different applications typically have varying resource consumption patterns, and different terminal devices can provide different amounts of device resources. When a specific application runs on a specific terminal device, its resource consumption rate can be determined through testing. Therefore, the combination of the application identifier and the terminal device identifier can determine the application's resource consumption rate. Thus, in another embodiment, the terminal device can obtain both the application identifier and the terminal device identifier to reflect the resource consumption of the recorded application. In this way, the terminal device can obtain a recording strategy based on the identifier information (i.e., the terminal device identifier and the application identifier of the recorded application) without needing to collect the application's resource consumption rate.

[0104] In one embodiment, the terminal device can determine its own recording strategy based on the resource usage of the recorded application (such as the aforementioned resource usage data or identification information) and pre-acquired rules. Since the terminal can dynamically adjust the recording strategy without interacting with the cloud, this ensures the timeliness of the dynamic adjustment and is applicable to scenarios where the terminal cannot connect to the cloud.

[0105] Practically, if the terminal device determines the recording strategy based on occupancy data, the determination rule mentioned above may include an occupancy threshold. In one feasible implementation, the terminal device can use... Figure 13 The recording strategy acquisition method described in the illustrated embodiment (see steps a1 to a5 below) is used to acquire the recording strategy based on the occupancy rate data.

[0106] If the terminal device determines the recording strategy based on the identification information, the determination rule mentioned above may include mapping information. The mapping information can be used to describe the mapping relationship between the identification information and the processor, indicating the recording strategy that can be obtained based on the identification information.

[0107] For example, the mapping information stored in the terminal device can be based on the following Figure 8Alternatively, the mapping information generated by the recording strategy formulation rules shown in Table 2. In other embodiments, other feasible recording strategy formulation rules may also exist. For example, in the recording strategy obtained based on other recording strategy formulation rules, the processor indicated by the recording strategy can be a processor other than the CPU and GPU. This application does not limit this.

[0108] Since mapping information typically does not cover all application identifiers, all terminal device identifiers, or various combinations of application identifiers and terminal device identifiers, when a terminal device uses identifier information to determine a recording strategy, there may be situations where no recording strategy is found after querying the mapping information (or, in short, an idle period). Feasibly, when a terminal device is idle, it can obtain occupancy rate data and determine the recording strategy based on that data.

[0109] In practice, the pre-acquired rules can be provided to each terminal device by the cloud. When the rules need to be optimized, the cloud can update the rules and then update the updated rules to each terminal device. In one example, an operating system update may include an update of the rules, in which case the rules can be updated to the terminal devices along with the operating system.

[0110] In another embodiment, the terminal device can obtain information reflecting the resource usage of the recorded application and send it to the cloud to obtain a recording strategy. The cloud can determine the recording strategy based on the resource usage of the recorded application and stored rules, and then send the determined result to the terminal device.

[0111] Feasibly, the terminal device can collect occupancy data and report it to the cloud. The cloud can then determine the recording strategy based on the occupancy data, and the determination rule can include an occupancy threshold.

[0112] Since any terminal device can obtain the recording strategy through the cloud, there is no need to store the occupancy threshold in each terminal device, and therefore there is no need to update the occupancy threshold.

[0113] Feasibly, the terminal device can collect identification information and report it to the cloud. The cloud can then determine the recording strategy based on the identification information, and the determination rule can include mapping information.

[0114] Since any terminal device can obtain the recording strategy through the cloud, there is no need to store a large amount of mapping information in each terminal device, and therefore there is no need to update the mapping information.

[0115] When the cloud uses mapping information to determine the recording strategy, there may be periods of inactivity. Feasibly, when such periods occur in the cloud, the terminal device can obtain occupancy rate data and combine it with an occupancy rate threshold to determine the recording strategy. Furthermore, the terminal device can also report the determined recording strategy to the cloud for updating the mapping information.

[0116] See one example. Figure 13 When the cloud is not in use, the terminal device can perform the following steps a1 to a5 to determine the recording strategy.

[0117] Feasibly, when changes to the determining rules are needed, the cloud can update the determining rules in a timely manner. Thus, compared to the implementation method where the terminal device determines the recording strategy itself, since the update of the determining rules used by the terminal device has a time delay, when determining the recording strategy through the cloud, it is possible to achieve accurate determination of the recording strategy based on the real-time updated determining rules.

[0118] It is feasible for the terminal device to perform recording in either in-application or out-of-application recording.

[0119] In the case of in-app recording, the terminal device can receive a recording start request from the user through the application's recording function while the application is running. Upon receiving the recording start request, the terminal device can obtain information reflecting the device resource usage of the application being recorded.

[0120] When recording outside of an application, the terminal device can first run the application and then receive a recording start request from the user via third-party recording software or the operating system's recording function. Upon receiving the recording start request, the terminal device can obtain information reflecting the device resource usage of the application being recorded.

[0121] In addition, the terminal device can also run the application only after receiving a record start request from the user through third-party recording software or the operating system's recording function. That is, the terminal device has started recording before running the application. In this case, the terminal device can obtain information reflecting the usage of device resources by the recorded application when it starts running the application.

[0122] See Figure 6 One embodiment of this application provides an implementation process for recording performed by an application client (i.e., in-application recording), which may include the following steps 601 to 606. The terminal device has an application client installed, and the application client has recording functionality, thus supporting the user's need for in-application recording. Figure 6 In the illustrated embodiment, the application client is not only the entity performing the recording operation, but also the object being recorded.

[0123] Figure 6 In the illustrated embodiment, the cloud determines a recording strategy based on the application identifier and terminal device identifier reported by the application client, combined with mapping information, and synchronizes the recording strategy to the terminal device. The application client then uses the processor indicated by the recording strategy to perform recording. Since the application identifier and terminal device identifier reflect the application's resource usage on the device, the processor used for recording is determined based on the resource usage of the application being recorded. This helps to reduce the impact of video recording on the device resource usage of the application being recorded.

[0124] Step 601: The application client receives a request to start recording.

[0125] Feasibly, while the application is running on the terminal device, the user can request to start recording from the application client when video recording is needed. For example, see... Figure 2 When a user feels the need to record gameplay highlights and audio during the use of a game application, they can click the closed recording control 201 to request to start recording.

[0126] Step 602: The application client reports its application identifier and terminal device identifier to the cloud.

[0127] In a feasible manner, in response to a start recording request, the application client can obtain the application identifier and terminal device identifier of the application client, and report the obtained identifier information to the cloud in order to request a recording strategy from the cloud.

[0128] In one feasible implementation, refer to Figure 10 When a user requests to start recording, the application client can obtain identification information and report it to the recording policy management module in the cloud via the interface " / record / perf / getPolicy" to request synchronization of the recording policy in the cloud. The cloud can determine the recording policy by finding the processor that has a mapping relationship with the identification information based on the mapping information, and then return the determined recording policy to the application client.

[0129] Step 603: The cloud determines the recording strategy based on the application identifier and terminal device identifier, combined with the mapping information, and returns the recording strategy.

[0130] Feasibly, the mapping information can reflect the mapping relationship between the processor and the application identifier and the terminal device identifier. Then, the cloud can use the mapping information to find the processor that has a mapping relationship with the application identifier and the terminal device identifier sent by the application client, so as to determine the recording strategy of the application client and enable the application client to use the found processor to perform recording.

[0131] In one feasible implementation, the application client can obtain and report not only identification information but also the resource occupancy rate of the recorded application on the device. Specifically, the application client can obtain both the identification information and the current resource occupancy rate data. The cloud can then perform operations such as recording strategy verification based on the resource occupancy rate reported by the application client. For example, the cloud can verify whether the difference between the resource occupancy rate reported by the application client and the rate provided by the application testing end meets the requirements. If yes, the cloud can determine the recording strategy based on the mapping information; otherwise, it can perform exception handling. For instance, this exception handling might include determining the recording strategy based on the mapping information and outputting exception information to remind developers to verify the accuracy of the mapping information.

[0132] Feasibly, the cloud can generate mapping information based on the application performance report provided by the application testing terminal. See [link to feasible implementation] for details. Figure 7a The process of supporting the generation of mapping information in the cloud can include the following steps 701 to 703.

[0133] Step 701: The application testing terminal performs performance testing on the application and generates an application performance report.

[0134] For example, application developers or testers can use performance testing tools on the testing device to perform performance tests on the application. For example, a performance testing tool could be smartPerf.

[0135] Feasibly, the application testing platform can test the resource utilization of the same application when running on different terminal devices, and it can also test the resource utilization of different applications when running on the same terminal device. The application testing platform can generate application performance reports based on the test results.

[0136] Practically, an application performance report may include the resource utilization of a specific application on a specific terminal device, the identifier of the application, and the identifier of the terminal device. In one example, assuming device resources include CPU and GPU, the generated application performance report may include the information in Table 1 below.

[0137] Table 1

[0138] Game Name Game package name Equipment type Game type CPU utilization GPU utilization game1 com.game1.x Mate60 MMO 40％ 70％ game2 com.game2.x Mate40 SLG 30％ 75％ game3 com.game3.x Nova Flip Leisure 68％ 45％

[0139] As shown in Table 1, in an application performance report, the application identifier may include "game1", the terminal device identifier may include "Mate60", and the application's utilization of device resources may include "CPU utilization 40%, GPU utilization 70%", that is, when the game application "game1" is running on the terminal device "Mate60", the game application uses 40% of the device's CPU resources and 70% of the device's GPU resources.

[0140] The content shown in Table 1 is only a few examples of application performance reports. The application performance reports generated by the application testing end may also include other application performance reports besides the three application performance reports shown in Table 1. For example, one application performance report includes the CPU and GPU utilization of the game application "game1" on the terminal device "Mate40", and another application performance report includes the CPU and GPU utilization of the game application "game1" on the terminal device "Nova Flip".

[0141] In practice, an application performance report may include not only the application's identifier, the terminal device's identifier, and the application's resource utilization, but also other information. This other information may be used for auxiliary identification of the application or terminal device. Referring to Table 1, taking a game application as an example, the application performance report may also include the game package name and game type.

[0142] By conducting detailed performance testing during the application development phase and assessing the application's resource consumption on different hardware configurations, the cloud can accurately generate mapping information to determine recording strategies, ensuring the scientific validity and effectiveness of these strategies. The generation of mapping information ensures that the terminal device can dynamically adjust its recording strategy based on the application's resource usage during recording, thereby reducing the impact of video recording on application performance.

[0143] Step 702: The application test client uploads the application performance report to the cloud.

[0144] It is feasible to connect one or more application test terminals in the cloud, and the cloud can aggregate the application performance reports reported by the connected application test terminals to generate mapping information.

[0145] In one feasible implementation, the specific implementation process of steps 701 to 702 may include: the developer controls the test terminal device to use the performance testing tool to test the application, and the performance testing tool reports the application performance report generated by the test application to the cloud database.

[0146] For example, see Figure 7bDevelopers can manipulate the testing device to input a game application with the package name com.game1.x into a performance testing tool to request game testing. They can then obtain a game performance report, perfReport1, returned by the tool after testing the application. This report indicates the CPU and GPU usage of the game application on the running device. Developers can then upload this perfReport1 report to a cloud database using an interface such as " / record / perf / uploadReport".

[0147] Step 703: The cloud generates mapping information based on the application performance report.

[0148] Feasibly, an application performance report can include the application identifier, the terminal device identifier, and the application's resource utilization on the device. The cloud can formulate rules based on the recording strategy, determining the recording strategy according to the recorded application's resource utilization, i.e., determining the processor used for recording when the application is running on the terminal device. Subsequently, the cloud can establish a mapping relationship between the determined processor and the application identifier and terminal device identifier, thereby generating mapping information.

[0149] Feasibly, the recording strategy formulation rules can allow the cloud to determine the recording strategy based on the device resource utilization of the application being recorded and based on pre-stored thresholds. The threshold values ​​can be set by cloud developers based on experience.

[0150] In one feasible implementation, the set threshold can be applied to all applications and all application usage scenarios. In other feasible implementations, the threshold can be set according to different specific scenarios and applications. For example, the threshold value for game applications can differ from the threshold value for other types of applications.

[0151] In one embodiment, the application's utilization of device resources may include CPU utilization and GPU utilization, and the pre-stored thresholds may include the corresponding CPU utilization threshold (e.g., denoted as α) and the corresponding GPU utilization threshold (e.g., denoted as β).

[0152] In one feasible implementation, the recording strategy formulation rules can be as follows: Figure 8 As shown. See also Figure 8The cloud can determine the recording strategy as using the GPU to perform recording when the application's GPU utilization does not exceed β, regardless of whether the application's CPU utilization exceeds α; and it can also determine the recording strategy as using the GPU to perform recording when the application's GPU utilization exceeds β and the application's CPU utilization exceeds α. In this way, GPU resources can be prioritized for recording, especially when GPU resources are relatively idle.

[0153] See Figure 8 The cloud can determine the recording strategy to use the CPU for recording when the application's GPU utilization exceeds β and the application's CPU utilization does not exceed α. In this way, more idle CPU resources can be used to perform recording.

[0154] Referring to Table 1 above, taking the game application "game1" and the terminal device "Mate60" as examples, the cloud can compare the CPU utilization rate "40%" with the threshold α mentioned above, and compare the GPU utilization rate "70%" with the threshold β mentioned above. Based on the comparison results, it can determine whether the processor that has a mapping relationship with "game1" and "Mate60" is the CPU or the GPU.

[0155] In another embodiment, the application's utilization of device resources may include GPU utilization, and the pre-stored threshold may include the corresponding GPU utilization threshold (e.g., including β1 and β2).

[0156] In one feasible implementation, the recording strategy formulation rules can be as shown in Table 2 below.

[0157] Table 2

[0158]

[0159] Referring to Table 2, the cloud can determine the recording strategy to use the GPU to perform recording when the application's GPU utilization does not exceed β1; and determine the recording strategy to use the CPU to perform recording when the application's GPU utilization exceeds β2.

[0160] Referring to Table 1 above, taking the game application "game1" and the terminal device "Mate60" with β2 = 60% as an example, the cloud can compare the GPU utilization rate "70%" with the above threshold β2, and then determine the processor that has a mapping relationship with "game1" and "Mate60" as the CPU based on the comparison results.

[0161] It is evident that for gaming applications with high GPU usage, using the CPU to record video can prevent video recording from further increasing the GPU load. This helps to balance device resources and reduce issues such as stuttering and low frame rates during recording, thus improving the user experience.

[0162] In one feasible implementation, step 703 may include: the cloud-based recording strategy management module synchronizing application performance reports from the cloud database, and then formulating rules (e.g., based on the recording strategy) Figure 8 (Or, according to the recording strategy formulation rules shown in Table 2 above), mapping information is generated based on the application performance report. The mapping information reflects the mapping relationship between the identification information and the processor, thus reflecting the recording strategy that can be obtained based on the identification information.

[0163] For example, based on the recording strategy shown in Table 2, a mapping relationship between the CPU and "game1" and "Mate60" can be constructed, that is, the recording is performed using the CPU according to the recording strategy available to "game1" and "Mate60".

[0164] For example, see Figure 9 The cloud-based recording strategy management module can synchronize application performance reports (such as the game performance report perfReport1 mentioned above) from the cloud database, and then generate mapping information (such as the recording strategy recordPolicy1 obtained from perfReport1) based on the application performance reports and the rules formulated according to the recording strategy.

[0165] The cloud can generate mapping information based on the application performance reports reported in real time by the application testing end, so as to support the terminal device to dynamically adjust the recording strategy. This can balance recording quality and application performance and reduce the impact of video recording on application performance.

[0166] By generating mapping information, the cloud can subsequently synchronize recording strategies to terminal devices based on the mapping information. The generation of mapping information makes it easier for the cloud to quickly and accurately find the recording strategy, and it also makes it easier for developers to optimize the mapping information as needed based on user feedback on the recording strategy provided by the cloud.

[0167] Step 604: The application client parses the recording strategy.

[0168] In one embodiment, the recording strategy distributed from the cloud can be parsed using the interface provided by the json library.

[0169] Step 605: The application client performs recording using the processor indicated by the recording policy. Practically, by performing recording, the application client can record its own screen and / or audio.

[0170] For example, if the recording policy instructs that video be recorded using the CPU, the application client can use the CPU to perform recording. This can specifically include using the CPU to perform image rendering operations, video encoding / decoding operations, post-processing operations, etc. For example, post-processing operations could include... Figure 3 The process shown illustrates replacing content in a video template with video footage and other materials.

[0171] For example, if the recording policy indicates that video should be recorded using the GPU, the application client can use the GPU to perform recording, which may include using the GPU to perform image rendering operations, video encoding / decoding operations, post-processing operations, etc.

[0172] In one feasible implementation, see [link to relevant documentation]. Figure 11 The specific implementation process of step 605 may include: after the application client parses the recording policy issued by the cloud (for example, the recording policy recordPolicy1 mentioned above), the application client performs recording using GPU or CPU based on the parsing result. When recording with GPU, the recording is loaded through the shader framework, and when recording with CPU, the recording is encoded through pure thread filter.

[0173] Feasibly, the recording process using the "load via shader framework" method can include: the video decoding stage can be implemented using software decoding, such as using the ffmpeg decoding library; the post-processing stage can be implemented using the filter function of the ffmpeg decoding library; and the video encoding stage can be completed using the ffmpeg encoding library to perform video encoding and video format encapsulation.

[0174] FFmpeg (Fast Forward MPEG) is software used for video encoding, decoding, and format conversion. MPEG (Moving Pictures Experts Group) is an international standard for motion picture compression algorithms.

[0175] Feasibly, the recording process using the "pure thread filter encoding" method can include: the video decoding stage can be implemented using GPU hardware decoding, and the post-processing stage can convert the decoded YUV image to an RGB image and then perform shader processing. For example, shader processing can achieve a post-processing effect of merging a texture onto the recorded video.

[0176] YUV is a color encoding method, where Y represents luminance and U and V represent the two chromaticity components. RGB is a color standard, where R represents red, G represents green, and B represents blue. Shader is a technique used for rendering graphics.

[0177] In one feasible implementation, during the recording process, the application client can adjust recording parameters (such as resolution, frame rate, etc.) according to the device resource usage of the application being recorded. This helps to effectively reduce the consumption of device resources and ensure the smooth operation of the application.

[0178] Figure 6 The illustrated embodiment, based on application performance testing and cloud policy management, can optimize the impact of in-application recording on application performance, thereby helping to improve the smoothness of application operation during recording.

[0179] Step 606: The application client receives the request to end recording and ends the recording.

[0180] It is feasible that during the recording process performed by the application client, the user can request to end the recording when the video recording needs to be stopped.

[0181] For example, see Figure 2 After the user clicks the recording control 201, which is in the closed state, the game application starts recording, and the recording control 201 is in the open state. During the recording process, if the user feels that they need to stop recording the game highlights and audio, they can click the recording control 201, which is in the open state, to request to stop recording.

[0182] In practice, the application client can respond to a record end request, stop recording, and store the recorded video on the terminal device. Afterward, the user can share the recorded video as needed, or store it in the cloud, etc.

[0183] In one feasible implementation, the application client can provide a user feedback mechanism, allowing users to report feedback during or after recording. For example, users can report issues encountered during recording, such as stuttering. The cloud can then perform optimizations based on user feedback, such as optimizing mapping information or occupancy thresholds. This not only improves the user experience but also makes the recording function more user-friendly and intelligent.

[0184] Figure 6 In the illustrated embodiment, the cloud determines the recording strategy based on the mapping information. However, when the cloud queries the mapping information, there may be a period of inactivity, preventing the determination of a recording strategy. Conveniently, when there is an inactivity period in the cloud, the application client can obtain the resource utilization rate of the application being recorded and determine the recording strategy based on the utilization rate threshold. Furthermore, the application client can report the determined recording strategy to the cloud for updating the mapping information. The specific implementation process under the inactivity period can be found below. Figure 12 The description of the illustrated embodiments will not be elaborated here.

[0185] By uploading application performance reports, application client recording strategies during off-peak hours, and user feedback information to the cloud to form a database, developers can easily use this data to conduct in-depth analysis and optimization of recording strategies. This makes the recording strategies more applicable, such as being suitable for different game scenarios and adapting to the device hardware configurations of different gamers.

[0186] Figure 6 In the illustrated embodiment, the cloud determines the recording strategy based on the application identifier and terminal device identifier reported by the application client. In this way, the application client does not need to collect the resource usage of the recorded application, but can dynamically adjust the recording strategy according to the resource usage of the recorded application.

[0187] Unlike the cloud-based approach of determining recording strategies based on identifier information, other feasible embodiments may also determine the recording strategy based on the device resource utilization rate of the application being recorded. In this way, the cloud does not need to generate the aforementioned mapping information, and the application testing end does not need to generate and report application performance reports; the cloud can determine the recording strategy simply by using a storage utilization threshold. Furthermore, determining the recording strategy based on a utilization threshold can avoid the aforementioned idle recording situation.

[0188] Unlike implementations that determine recording strategies via the cloud, other feasible embodiments allow the application client to determine its own recording strategy. When determining the recording strategy, the application client can pre-obtain a resource occupancy threshold and combine it with the resource occupancy rate of the application being recorded and the threshold to determine the recording strategy. Alternatively, the application client can pre-obtain mapping information and combine it with identification and mapping information to determine the recording strategy. In this way, the application client can determine the recording strategy without interacting with the cloud.

[0189] See Figure 12 One embodiment of this application provides an implementation process for recording performed by an application client (i.e., in-application recording), which may include the following steps 1201 to 1210. Figure 12 In the illustrated embodiment, the application client is not only the entity performing the recording operation, but also the object being recorded.

[0190] Steps 1201 to 1202 are the same as steps 601 to 602 above.

[0191] Step 1203: The cloud determines that no recording strategy was found based on the application identifier and terminal device identifier, combined with the mapping information, and returns an empty recording strategy.

[0192] Feasibly, the cloud can generate mapping information based on the application performance report provided by the application testing end. For details on how to support cloud-based generation of mapping information, please refer to [link / reference needed]. Figure 6 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0193] Feasibly, the cloud can return an empty recording strategy if it fails to find the mapping information. Besides returning an empty recording strategy, other embodiments may also return a notification message indicating that no recording strategy was found.

[0194] Step 1204: The application client parses the recording strategy.

[0195] Since no recording strategy was found in step 1203, the parsed recording strategy can be empty.

[0196] Step 1205: The application client obtains the application client's resource utilization rate on the device.

[0197] Feasibly, during periods of inactivity, the application client can initiate local performance monitoring according to a performance adaptive mechanism to detect performance data within an observation window and determine its recording strategy based on the detected data. The performance data can be used to obtain the application client's utilization of device resources.

[0198] In other feasible implementations, the application client can also send the detected performance data to the cloud, which will then determine the recording strategy and obtain the recording strategy determined by the cloud.

[0199] Besides situations where the application is not actively recording, other reasons (such as a disconnection in communication between the cloud and the terminal, or a timeout in the cloud response) can also prevent the application client from obtaining the recording strategy from the cloud. In this case, the application client can also start local performance monitoring.

[0200] Step 1206: The application client determines the recording strategy based on the obtained occupancy rate and the pre-obtained occupancy rate threshold.

[0201] Step 1207: The application client uses the processor indicated by the recording policy to perform recording.

[0202] Feasibly, the application client can record the application client's screen and / or audio by performing recording.

[0203] In one feasible implementation, see [link to relevant documentation]. Figure 13 The specific implementation process of steps 1205 to 1207 may include the following steps a1 to a7.

[0204] Step a1: Start application performance testing.

[0205] Feasibly, the application client can execute step a1 after step 1204.

[0206] Step a2: Define the observation window.

[0207] It is feasible to use a viewing window for a preset duration (e.g., 5 seconds) to collect enough performance data to accurately determine the application client's utilization of device resources.

[0208] Step a3: Collect performance data in the observation window in real time.

[0209] Feasibly, the application client can continuously monitor the performance data in the observation window, such as the application client's CPU performance metrics, the application's GPU performance metrics, the application's memory usage, and the application's frame rate.

[0210] Practically, CPU performance metrics can include CPU utilization, which can be calculated by measuring the ratio of CPU activity time over a period of time (e.g., 1 second) to the total time.

[0211] Practically, GPU performance metrics can include GPU utilization, which can be obtained through GPU-related APIs (such as NVIDIA's NVML and AMD's ADL).

[0212] Feasibly, for memory usage, the current available memory and the memory used can be detected to assess whether memory resources are sufficient.

[0213] Feasibly, for frame rate, taking game applications as an example, the smoothness of the game can be judged by recording the number of frames rendered in the observation window.

[0214] In one feasible implementation, the application client can obtain performance data in real time by calling APIs provided by the operating system (such as the Performance API of the Windows operating system, the proc file system of the Linux operating system, and the system interface provided by the mobile operating system).

[0215] Step a4: Determine the application's CPU and GPU utilization based on the collected performance data.

[0216] In one feasible implementation, the application client can calculate an average of the CPU utilization rate detected in real time within the observation window to represent the application's CPU utilization rate, and an average of the GPU utilization rate detected in real time within the observation window to represent the application's GPU utilization rate. This helps to obtain accurate utilization rate data.

[0217] Step a5: Determine the recording strategy based on the application's CPU and GPU usage, and then execute step a6 or step a7.

[0218] In one example, it can be combined Figure 8 The recording strategy shown defines rules that select the processor to record from the application's CPU and GPU usage.

[0219] Step a6: If the processor indicated by the recording strategy is the CPU, use the CPU to perform the recording.

[0220] Step a7: If the processor indicated by the recording policy is the GPU, use the GPU to perform the recording.

[0221] In one feasible implementation, if the application client obtains the recording strategy based on the device resource utilization rate of the recorded application (e.g., by obtaining it itself or through the cloud), the application client can also detect changes in utilization rate in real time during the recording process, and obtain the recording strategy again as needed based on the changed utilization rate, according to the degree of change in utilization rate.

[0222] For example, if the occupancy rate changes significantly over a prolonged period, it indicates that the current recording strategy may not be suitable for the changed occupancy rate, and a new recording strategy can be obtained. If the newly obtained recording strategy differs from the currently used one, the recording strategy can be switched to use the newly obtained one. If the occupancy rate does not change significantly over a prolonged period, the existing recording strategy can be maintained.

[0223] Step 1208: The application client receives the request to end recording and ends the recording.

[0224] Step 1208 is the same as step 606. For specific technical implementation, please refer to the above-mentioned explanation of step 606.

[0225] Step 1209: The application client reports the recording strategy to the cloud.

[0226] In one feasible implementation, see [link to relevant documentation]. Figure 14 The specific implementation process of steps 1207 to 1209 may include: if the recording policy issued by the cloud-based policy management module is empty, the application client can start local performance detection to obtain the application client's resource utilization rate on the device, and determine the recording policy based on the obtained utilization rate. When recording with GPU, the policy is loaded through the shader framework; when recording with CPU, the policy is encoded through pure thread filter. After the application client finishes recording, it reports the recording policy to the cloud database through the " / record / perf / uploadReport" interface.

[0227] In practice, in addition to reporting the recording strategy, the application client can also report the application identifier and the terminal device identifier to the cloud.

[0228] Step 1210: The cloud updates the mapping information based on the processor used for recording reported by the application client.

[0229] Referring to step 1203, since the existing mapping information does not include a processor mapped to the identification information reported by the application client (e.g., denoted as identification information 1), there is a situation where the processing is not completed. After the application client reports the recording policy to the cloud, the cloud can establish a mapping relationship between identification information 1 and the processor indicated by the recording policy (e.g., denoted as processor 1) to generate the corresponding mapping information 1, and add mapping information 1 to the existing mapping information set. After this, if the application client on other terminal devices reports identification information 1, there will be no situation where the processing is not completed. In this way, the problem of other terminal devices completing the processing for the same identification information in the short term can be solved, and other terminal devices do not need to start local performance detection.

[0230] Feasibly, after updating the existing mapping information set in the cloud based on mapping information 1, if the cloud receives an application performance report corresponding to identifier information 1 from the application testing terminal, the cloud can generate mapping information 2 based on the application performance report corresponding to identifier information 1. If the mapping relationship described by mapping information 2 is different from that described by mapping information 1, mapping information 1 can be updated and replaced with mapping information 2. In this way, the problem of inaccurate recording strategies provided by some terminal devices affecting the recording effect of other devices for a long time can be addressed.

[0231] One embodiment of this application provides a video recording system architecture, which includes a client architecture and a cloud architecture (or server architecture).

[0232] See Figure 15 The client architecture includes application client 1501, operating system (OS) 1502, and hardware 1503.

[0233] The application client 1501 includes application logic 1504, application engine 1505, and video recording module 1506.

[0234] The video recording module 1506 includes an adaptive interface layer 1507, a video encapsulation module 1508, a performance detection module 1509, a policy adaptive module 1510, a policy synchronization module 1511, a software decoding module 1512, a hardware decoding module 1513, a software encoding module 1514, a hardware encoding module 1515, and a post-processing module 1516.

[0235] For example, application client 1501 can be a game application or the like installed on a terminal device.

[0236] For example, the operating system 1502 can be the Android operating system, etc.

[0237] For example, hardware 1503 may include device resources such as CPU, GPU, and memory.

[0238] Application logic 1504 can be a logical architecture used in the development and design of application client 1501, responsible for handling the functions and process control in application client 1501.

[0239] Application Engine 1505 can be used to provide recording footage, such as rendering application footage to generate recording footage.

[0240] The adaptive interface layer 1507 can provide an interface for the application client 1501 to interact with the cloud, so as to support the application client 1501 to report the application client 1501's usage of device resources to the cloud through the interface, synchronize the recording strategy provided by the cloud, and report the recording strategy generated by the performance adaptive mechanism to the cloud.

[0241] The video encapsulation module 1508 can be used to encapsulate the recorded footage provided by the application engine 1505 into video material.

[0242] The performance detection module 1509 can be used to detect the performance of the application client 1501 according to the performance adaptive mechanism, such as detecting the CPU utilization and GPU utilization of the application client 1501.

[0243] In one example, the performance testing module 1509 can perform application performance testing during the aforementioned off-duty period.

[0244] The policy adaptation module 1510 can be used to generate a recording policy based on the performance detected by the performance detection module 1509.

[0245] The policy synchronization module 1511 can be used to obtain information reflecting the usage of device resources by the application client 1501. The obtained information is reported to the cloud via the adaptive interface layer 1507 to request the recording policy from the cloud and receive the recording policy synchronized from the cloud.

[0246] The software decoding module 1512 can be used to perform video decoding using software decoding, such as decoding video footage.

[0247] The hardware decoding module 1513 can be used to perform video decoding, such as decoding video footage.

[0248] The post-processing module 1516 can be used to post-process the decoded video material.

[0249] The software encoding module 1514 can be used to implement video encoding using software encoding, such as encoding post-processed video footage to generate a finished video product.

[0250] The hard-encoding module 1515 can be used to implement video encoding using hard encoding, such as encoding post-processed video footage to generate a finished video product.

[0251] Figure 15 The specific technical implementation of each module in the client architecture shown can be found in the relevant technical descriptions in other embodiments of this application, and will not be repeated here.

[0252] See Figure 16 The cloud architecture includes a recording policy management module 1601, an operating system (OS) 1602, a database 1603, and hardware 1604. The recording policy management module 1601 includes a recording policy gateway interface 1605, a performance report upload module 1606, a performance report parsing module 1607, a performance report storage module 1608, a policy configuration portal 1609, a policy matching module 1610, a policy synchronization module 1611, a policy rule set 1612, and a policy generation module 1613.

[0253] Database 1603 can be used to store application performance reports submitted by the application test client, recording strategies for application clients during idle periods, and user feedback information submitted by application clients.

[0254] The recording policy gateway interface 1605 can provide an interface for interaction between the cloud and application clients, enabling application clients to report the device resource usage of the recorded application to the cloud, synchronize the recording policy provided by the cloud, and report the recording policy generated by the performance adaptive mechanism to the cloud.

[0255] The performance report upload module 1606 can provide an interface for interaction between the cloud and the application testing end, so that the application testing end can report application performance reports to the cloud through the interface.

[0256] The performance report parsing module 1607 can be used to parse application performance reports submitted by the application test terminal.

[0257] The performance report storage module 1608 can be used to store application performance reports submitted by the application testing end.

[0258] The strategy configuration portal 1609 can be used to support developers in optimizing recording strategies, such as setting recording strategy rules, adjusting occupancy thresholds, and adjusting mapping information.

[0259] The policy generation module 1613 can be used to formulate rules and generate mapping information based on application performance reports and recording policies. The mapping information reflects the mapping relationship between identification information (application identifier and device identifier) ​​and recording policies. The identification information reflects the application's usage of device resources, while the recording policy indicates the processor used for recording.

[0260] Policy rule set 1612 may include various mapping information generated in the cloud.

[0261] The strategy matching module 1610 can be used to find recording strategies that have a mapping relationship with the identification information reported by the application client based on the various mapping information generated in the cloud.

[0262] The policy synchronization module 1611 can be used to synchronize the recorded policies found by the policy matching module 1610 to the application client.

[0263] Figure 16 The specific technical implementation of each module in the cloud architecture shown can be found in the relevant technical descriptions in other embodiments of this application, and will not be repeated here.

[0264] See Figure 17 One embodiment of this application provides a recording method applied to a first terminal device, the first terminal device including at least two processors; the recording method may include the following steps 1701 to 1703.

[0265] It is feasible that the first terminal device can be any terminal device, such as a smartphone, tablet computer or other user terminal.

[0266] Figure 17 In the illustrated embodiments, the at least two processors are processors that can be used for video recording. In one embodiment, the at least two processors include a CPU and a GPU. In other feasible implementations, the at least two processors may also include other types of processors.

[0267] Step 1701: Obtain first information. The first information is used to reflect the device resource usage of the first application on the first terminal device. The device resources include at least one of at least two types of processors.

[0268] Figure 17 In the illustrated embodiment, the first application is the application being recorded. The terminal device can record the screen and / or audio of the first application during its operation, either through in-application recording or out-of-application recording.

[0269] In one embodiment of this application, the first information may include an identifier of a first application and an identifier of a first terminal device. The identifier information (i.e., the application identifier and the terminal device identifier) ​​can reflect the resource usage of the first application, specifically reflecting the application's usage of each of the at least two processors mentioned above. Taking the at least two processors including a CPU and a GPU as an example, the identifier information can reflect the application's CPU usage and GPU usage.

[0270] Unlike the aforementioned first information which includes identification information, in another embodiment of this application, the first information may include the resource utilization rate of the first application on the first terminal device. The utilization rate data (i.e., the resource utilization rate of the first application on the terminal device) can reflect the resource utilization of the first application, specifically reflecting the utilization of the corresponding processor by the first application.

[0271] For example, utilization data could include an application's CPU utilization and GPU utilization, reflecting the application's usage of both the CPU and GPU processors. Thus, it could be based on, for example... Figure 8 The recording strategy is defined by the rules shown, and the recording strategy is obtained.

[0272] For example, utilization data can include the application's GPU utilization, which reflects the application's usage of the GPU, a type of processor. Thus, rules can be formulated based on recording strategies such as those shown in Table 2 to obtain a recording strategy.

[0273] In one embodiment of this application, the first application can be any application on the first terminal device. In other feasible implementations, the first application can be any application in a set of applications, where the applications in the set are all applications that consume a lot of device resources during runtime, such as some game applications.

[0274] In one embodiment of this application, before step 1701, the recording method may further include: receiving a recording request when running the first application; and performing step 1701 according to the recording request.

[0275] Conveniently, the first application may have a recording function, allowing the user to issue a recording request through the first application. Based on this, in one embodiment of this application, the user interface of the first application includes a recording control; the step of receiving a recording request may include: receiving a user's trigger operation on the recording control.

[0276] In one example, when a user needs to record, they can click the recording control. The first application can then respond to the user's click on the recording control and obtain the aforementioned first information.

[0277] For example, see Figure 2 The first application can be a game application, and the recording control can be control 201. The user can click control 201 to request the game application to start recording game footage.

[0278] In other embodiments, users can also issue recording requests through third-party recording software or the recording function of the operating system. In this case, the terminal device can receive the recording request before or after running the first application.

[0279] Step 1702: Based on the first information, obtain the second information, which is used to indicate the processor used to perform the recording. The processor indicated by the second information is one of at least two processors.

[0280] Alternatively, the terminal device can obtain the second information via the cloud. Thus, in one embodiment of this application, step 1702 may include: sending the first information to the cloud to request the processor used to perform the recording.

[0281] In one embodiment, the first information can be the aforementioned occupancy rate data. The terminal device can send the occupancy rate data to the cloud, so that the cloud can generate a recording strategy based on the occupancy rate data. Thus, the second information can be the recording strategy obtained based on the occupancy rate data. By obtaining the recording strategy based on the occupancy rate data, the idle period situation described in other embodiments of this application can be avoided.

[0282] In one embodiment, the first information can be the aforementioned identification information. The terminal device can send the identification information to the cloud so that the cloud can generate a recording strategy based on the identification information. Thus, the second information can be the recording strategy obtained based on the identification information. Since the terminal device can obtain the recording strategy through the identification information, it is not necessary to obtain occupancy data.

[0283] In one embodiment of this application, step 1702 may further include: after sending the first information to the cloud, receiving the second information sent by the cloud based on the first information and the mapping information; wherein the mapping information is used to describe the mapping relationship between the identification information and the processor, and the identification information includes the application identifier and the terminal device identifier; the processor indicated by the second information is the processor mapped to the first information.

[0284] For example, the mapping information can be generated by the cloud based on... Figure 8 Alternatively, it can be generated by the recording strategy rules shown in Table 2. The cloud can use the mapping information to find the processor that maps to the identification information sent by the terminal device.

[0285] In one embodiment of this application, the above-mentioned at least two processors include a CPU and a GPU; if the first information (such as the above-mentioned identification information or the above-mentioned utilization rate data) indicates that the first application's GPU utilization rate does not exceed the first threshold, or indicates that the first application's GPU utilization rate exceeds the first threshold and the first application's CPU utilization rate exceeds the second threshold, then the processor indicated by the second information is a GPU; if the first information indicates that the first application's GPU utilization rate exceeds the first threshold and the first application's CPU utilization rate does not exceed the second threshold, then the processor indicated by the second information is a CPU.

[0286] When a terminal device obtains a recording strategy based on identification information, there may be situations where a recording strategy cannot be obtained (i.e., a no-show situation). To address the need for dynamically adjusting the recording strategy during no-show situations, in one embodiment of this application, step 1702 may further include: if no information from the cloud indicating the processor used to execute recording is received, then obtaining third information, the third information including the resource utilization rate of the first application on the device; and generating second information based on the third information. Therefore, the terminal device can obtain resource utilization data and automatically generate a recording strategy based on the resource utilization data when a no-show situation exists.

[0287] In one embodiment, the terminal device obtains occupancy data in the manner described in steps a1 to a4 above.

[0288] To avoid situations where other terminal devices are left without recording when requesting recording strategies from the cloud based on the same identification information, in one embodiment of this application, after generating the second information based on the third information, the recording method may further include: sending the second information to the cloud.

[0289] In one feasible implementation, the first terminal device can upload the second information and the identification information together to the cloud. The cloud can then construct a mapping relationship between the processor indicated by the second information and the identification information. Thus, when other terminal devices request a recording strategy from the cloud based on the same identification information, the cloud can return the recording strategy to them.

[0290] In other feasible embodiments, the terminal device may also send the occupancy rate data to the cloud when there is no turn, and obtain the recording strategy generated by the cloud based on the occupancy rate data.

[0291] For example, see Figure 8 The first threshold can be Figure 8 The second threshold β shown can be Figure 8 α is shown in the diagram.

[0292] In a feasible manner, besides obtaining the second information through the cloud, the terminal device can also obtain the second information independently. Thus, in one embodiment of this application, the first information can be the aforementioned identification information or the aforementioned occupancy rate data, and the first terminal device generates the second information based on the first information. In this way, the terminal device does not need to interact with the cloud and can determine the recording strategy independently based on the identification information or occupancy rate data.

[0293] In one embodiment of this application, the first terminal device stores the utilization rate threshold corresponding to each of the above-mentioned at least two processors, and the first information is the utilization rate of the device resources of the first terminal device by the first application. Then step 1702 may include: generating second information based on the first information and the utilization rate threshold.

[0294] Step 1703: Using the processor indicated by the second information, perform recording to record the screen and / or audio of the first application.

[0295] For example, if the second information is a recording strategy instructing CPU recording, the terminal device can use the CPU to perform recording; if the second information is a recording strategy instructing GPU recording, the terminal device can use the GPU to perform recording.

[0296] visible, Figure 17 In the illustrated embodiment, the terminal device can dynamically adjust the recording strategy according to the resource usage of the recorded application. This allows the recording strategy to be adapted to the resource usage of the recorded application, thereby minimizing the impact of recording on the application's use of device resources. Figure 17 The specific technical implementation of the illustrated embodiment can be found in the relevant descriptions in other embodiments of this application, and will not be repeated here.

[0297] The method provided in any embodiment of this application can be applied to... Figure 18 The terminal device 100 shown. Figure 18A schematic diagram of the terminal device 100 is shown. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0298] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0299] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, GPUs, ISPs, controllers, video codecs, DSPs, baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0300] In some embodiments, the processor 110 may be a system-on-a-chip (SoC), which may include a CPU and may further include other types of processors. In some embodiments, the processor 110 may be a PWM control chip.

[0301] The processor 110 may also include necessary hardware accelerators or logic processing hardware circuitry, such as an ASIC, or one or more integrated circuits for controlling the execution of the program. Furthermore, the processor 110 may have the capability to operate one or more software programs, which may be stored in a storage medium.

[0302] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0303] In some embodiments, the memory of the terminal device 100 may be a read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), or other types of dynamic storage devices capable of storing information and instructions. It may also be an electrically erasable programmable read-only memory (EEPROM), or any computer-readable medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer.

[0304] In some embodiments, the processor 110 and the memory can be combined into a single processing device, or they can be separate components. The processor 110 can be used to execute program code stored in the memory. In specific implementations, the memory can be integrated into the processor 110, or it can be independent of the processor 110.

[0305] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0306] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0307] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0308] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0309] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0310] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0311] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0312] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0313] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0314] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0315] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0316] The ISP is used to process data fed back by camera 193. Camera 193 is used to capture still images or videos. An object passes through the lens to generate an optical image that is projected onto the photosensitive element.

[0317] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0318] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of terminal device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0319] Terminal device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0320] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 194. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. When a touch operation is applied to the display screen 194, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor. The terminal device 100 can also calculate the touch location based on the detection signal from the pressure sensor.

[0321] A touch sensor, also known as a "touch device," can be located on the display screen 194. The touch sensor and the display screen 194 together form a touchscreen, also called a "touchscreen." The touch sensor detects touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some embodiments, the touch sensor may also be located on the surface of the terminal device 100, in a different position than the display screen 194.

[0322] The software system of terminal device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture.

[0323] In addition, the terminal device runs an operating system on top of the aforementioned components. For example... operating system, operating system, Operating systems, etc. Applications can be installed and run on an operating system.

[0324] The software system of terminal device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of terminal device 100.

[0325] Figure 19 This is a software structure block diagram of the terminal device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom: application layer, application framework layer (Framework, FWK), system library and runtime layer, hardware abstraction layer (HAL), and kernel layer.

[0326] The application layer can include a series of application packages, such as camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, SMS and other applications.

[0327] The application framework layer provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 19 As shown, the application framework layer may include a view system, notification manager, resource manager, window manager, content provider, etc.

[0328] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0329] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating the device, and flashing indicator lights.

[0330] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0331] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0332] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0333] The system library and runtime layer includes the system library and the Android Runtime.

[0334] System libraries can include multiple functional modules. For example: 3D graphics processing libraries (e.g., OpenGL ES), surface managers, 2D graphics engines (e.g., SGL), media libraries, etc.

[0335] The 3D graphics processing library is used for 3D graphics drawing, image rendering, compositing, and layer processing. The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications, such as OpenGL ES. The 2D graphics engine is a drawing engine for 2D graphics, such as SGL. The media library supports playback and recording of various common audio and video formats, as well as still image files. The media library supports various audio and video encoding formats, such as: Moving Picture Experts Group (MPEG4), High-Compression Digital Video Codec Standard (H.264), Moving Picture Experts Compression Standard Audio Layer 3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate Codec (AMR), Joint Picture Experts Group (JPG), Portable Web Graphics (PNG), etc.

[0336] The Android runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core libraries comprise two parts: one part contains the functionalities that Java calls, and the other part is the core Android library itself. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0337] The Hardware Abstraction Layer (HAL) acts as a bridge between software and hardware. It encapsulates the underlying hardware driver, providing a generic interface for the application framework to call the driver. The HAL is an abstract interface for the device kernel driver, providing application programming interfaces (APIs) for accessing the underlying device to higher-level Java API frameworks. The HAL contains multiple library modules, each implementing an interface for a specific type of hardware component. The HAL provides APIs for higher-level Java API frameworks to access the underlying device; when an API framework requests access to the electronic device's hardware, the Android system loads the corresponding library module for that hardware component.

[0338] The Hardware Abstraction Layer (HAL) can communicate with the application framework layer through the Hardware Abstraction Layer Interface Definition Language (HIDL). HIDL is an interface description language that specifies the interface between HAL and FWK.

[0339] The kernel layer is the layer between hardware and software. At a minimum, it includes drivers for the camera, display, Bluetooth, audio, and sensors, as well as drivers for Ultra Wide Band (UWB), touchscreen, and IoT device management. The kernel layer is the foundation of the Android system, through which the system's final functionalities are implemented.

[0340] Understandable, Figure 19 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0341] This application embodiment also provides a recording device applied to a first terminal device, the first terminal device including at least two types of processors; the recording device includes: a first acquisition module, used to acquire first information, the first information being used to reflect the occupation of device resources of the first terminal device by a first application, the device resources including at least one of at least two types of processors; a second acquisition module, used to acquire second information based on the first information, the second information being used to indicate the processor used to perform recording, the processor indicated by the second information being one of at least two types of processors; and a recording module, used to perform recording using the processor indicated by the second information to record the screen and / or audio of the first application.

[0342] This application also provides an electronic chip, which is installed in a terminal device (UE). The electronic chip includes a processor for executing computer program instructions stored in a memory. When the computer program instructions are executed by the processor, the electronic chip is triggered to execute the method steps provided by the terminal device in any method embodiment of this application.

[0343] This application also proposes a terminal device, which includes a communication module, a memory for storing computer program instructions, and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the terminal device is triggered to execute the method steps provided by any method embodiment of this application.

[0344] This application also proposes a server device, which includes a communication module, a memory for storing computer program instructions, and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the server device is triggered to execute the method steps provided by any method embodiment of this application that are executed by the cloud.

[0345] This application also provides a terminal device, which includes multiple antennas, a memory for storing computer program instructions, a processor for executing the computer program instructions, and a communication device (such as a communication module that can implement 5G communication based on the NR protocol). When the computer program instructions are executed by the processor, the terminal device is triggered to execute the method steps provided by any method embodiment of this application.

[0346] Specifically, in the embodiments of this application, one or more computer programs are stored in the aforementioned memory. The one or more computer programs include instructions that, when executed by the aforementioned device, cause the aforementioned device to perform the method steps provided in the embodiments of this application that are executed by the terminal device.

[0347] Furthermore, the devices, apparatuses, and modules described in the embodiments of this application may be implemented by computer chips or physical entities, or by products with certain functions.

[0348] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0349] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0350] Specifically, this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the method steps provided by the terminal device in this application.

[0351] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method steps provided in this application embodiment that are executed by a terminal device.

[0352] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0353] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0354] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0355] An integrated unit implemented as a software functional unit can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0356] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0357] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0358] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments of this application can be implemented using electronic hardware, computer software, or a combination of electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0359] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the same or similar parts between the various embodiments of this application can be referred to mutually. For example, the specific working processes of the systems, devices, and units described in the embodiments of this application can be referred to the corresponding processes in the method embodiments of this application, and will not be repeated here.

[0360] The above description is merely a specific embodiment of this application and is not intended to limit this application. The scope of protection of this application shall be determined by the claims.

Claims

1. A recording method, characterized in that, Applied to a first terminal device, the first terminal device including at least two processors; the method includes: Obtain first information, which is used to reflect the device resource usage of the first application on the first terminal device, and the device resources include at least one of the at least two processors; Based on the first information, second information is obtained, the second information being used to indicate the processor used to perform the recording, and the processor indicated by the second information is one of the at least two processors; Using the processor indicated by the second information, perform recording to record the screen and / or audio of the first application.

2. The method according to claim 1, characterized in that, The first information includes: the identifier of the first application and the identifier of the first terminal device.

3. The method according to claim 2, characterized in that, Based on the first information, obtain the second information, including: The first information is sent to the cloud to request the processor used to perform the recording.

4. The method according to claim 3, characterized in that, Obtaining the second information based on the first information further includes: Receive the second information sent by the cloud based on the first information and the mapping information; The mapping information is used to describe the mapping relationship between the identification information and the processor, and the identification information includes the application identifier and the terminal device identifier. The processor indicated by the second information is the processor that maps to the first information.

5. The method according to claim 3, characterized in that, Obtaining the second information based on the first information further includes: If no information is received from the cloud indicating the processor used to perform the recording, then third information is obtained, including the resource utilization rate of the first application on the device. The second information is generated based on the third information.

6. The method according to claim 5, characterized in that, After generating the second information based on the third information, the method further includes: The second information is sent to the cloud.

7. The method according to claim 1, characterized in that, The first information includes: the resource utilization rate of the first application on the device.

8. The method according to claim 7, characterized in that, The first terminal device stores the utilization rate threshold corresponding to each of the at least two types of processors; Based on the first information, obtain the second information, including: The second information is generated based on the first information and the occupancy threshold.

9. The method according to any one of claims 1-8, characterized in that, The at least two processors include a central processing unit and a graphics processing unit; If the first information indicates that the first application's utilization rate of the image processor does not exceed the first threshold, or indicates that the first application's utilization rate of the image processor exceeds the first threshold and the first application's utilization rate of the central processing unit exceeds the second threshold, then the processor indicated by the second information is the image processor. If the first information indicates that the first application's utilization rate of the image processor exceeds the first threshold, and the first application's utilization rate of the central processing unit does not exceed the second threshold, then the processor indicated by the second information is the central processing unit.

10. The method according to any one of claims 1-8, characterized in that, Before obtaining the first information, the method further includes: A recording request was received while the first application was running; According to the recording request, the step of obtaining the first information is executed.

11. The method according to claim 10, characterized in that, The user interface of the first application includes recording controls; The receipt of a recording request includes: receiving a user's trigger operation on the recording control.

12. The method according to any one of claims 1-8, characterized in that, The first application is a game application.

13. A terminal device, characterized in that, The terminal device includes a processor and a memory for storing computer program instructions, wherein when the computer program instructions are executed by the processor, the terminal device is triggered to perform the method as described in any one of claims 1-12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-12.

Images