Cross-device screen recording system, method, and related apparatus

Abstract

Embodiments of the present application provide a cross-device screen recording system, method and related device, the screen recording system comprises a first electronic device and a second electronic device, and a communication connection is established between the first electronic device and the second electronic device, wherein: the first electronic device is configured to: display a first interface of a first application, the first interface is related to the second electronic device; in response to a first operation acting on the first interface, generate a screen recording instruction; and send the screen recording instruction to the second electronic device, wherein the screen recording instruction indicates to start a screen recording function, the screen recording function is a highlight function and / or a mirror control function; the second electronic device is configured to: according to the screen recording instruction, acquire video data and audio data corresponding to the screen recording function through a highlight module; and send the video data and the audio data to the first electronic device. Through the present application, the highlight function and the mirror control function can be started at the same time, conflict detection is avoided, and user experience is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, in particular to a cross-device screen recording system and method and related device. BACKGROUND

[0002] With the development of electronic technology and mobile Internet, a user can have multiple electronic devices, such as a mobile phone, a tablet computer, a smart TV, a smart screen, etc. The user's multiple electronic devices can be interconnected for use. For example, the user's mobile phone and smart TV or smart screen can be connected.

[0003] When the user watches a video through the smart TV or smart screen, for the wonderful segments in the video (such as the wonderful segments of a football goal or the wonderful plot segments of a TV series, etc.), the user can store these wonderful segments or share them with other users through the "highlight" function. When the user wants to watch the screen content of the mobile phone through the smart TV or smart screen, the user can project the screen content of the smart TV or smart screen on the smart phone through the "mirror control" function.

[0004] However, the "highlight" function and the "mirror control" function cannot be started at the same time. When they are started at the same time, the two functions need to be conflict detected, that is, when the "highlight" is started, the "mirror control" needs to be paused; when the "mirror control" is started, the "highlight" needs to be paused. In this way, not only the computing cost is increased, but also the user experience is affected. SUMMARY

[0005] The cross-device screen recording system, method and related device provided by the embodiments of the present application can start the highlight function and the mirror control function at the same time, thereby improving the user experience.

[0006] In a first aspect, the present application provides a cross-device screen recording system, the screen recording system comprising a first electronic device and a second electronic device, and a communication connection is established between the first electronic device and the second electronic device, wherein:

[0007] The first electronic device is configured to:

[0008] display a first interface of a first application, the first interface being related to the second electronic device;

[0009] generate a screen recording instruction in response to a first operation acting on the first interface;

[0010] send the screen recording instruction to the second electronic device, wherein the screen recording instruction indicates to start a screen recording function, and the screen recording function is a highlight function and / or a mirror control function;

[0011] The second electronic device is configured to:

[0012] According to the screen recording instruction, the highlight moment module is configured to acquire video data and audio data corresponding to the screen recording function.

[0013] The video data and the audio data are sent to the first electronic device.

[0014] In the embodiments of the present application, whether the screen recording instruction is of the highlight moment function or the screen recording instruction is of the mirror control function, the second electronic device acquires the video file corresponding to the screen recording function through the highlight moment module. Therefore, the highlight moment function and the mirror control function can be started simultaneously, and no conflict detection is required when starting the highlight moment function and the mirror control function, thereby reducing internal processing and improving user experience.

[0015] In a possible implementation of the first aspect, the second electronic device is specifically configured to:

[0016] When it is determined that the screen recording function is started, the interface of the recording module is called through the highlight moment module to acquire the audio data and the video data;

[0017] The audio data and the video data are sent to the screen projection module of the first electronic device;

[0018] The first electronic device is configured to display a recording screen of the audio data and the video data corresponding to the mirror control function.

[0019] As can be seen, the interface of the recording module is called by the highlight moment module, so in the case that the interface of the recording module can be called by only one module at a time, since the recording content corresponding to the highlight moment function and the screen recording content corresponding to the mirror control function are both acquired through the highlight moment module, the highlight moment function and the mirror control function can be started simultaneously.

[0020] In a possible implementation of the first aspect, the second electronic device is specifically configured to:

[0021] The acquired audio data and video data are saved to the tail of the first cache queue;

[0022] When there is data in the first cache queue, the audio data and the video data located at the head of the first cache queue are taken out through a sending thread;

[0023] The audio data and the video data located at the head of the first cache queue are sent to the screen projection module of the first electronic device;

[0024] In a case where the receiving success information from the first electronic device is received, the audio data and the video data located at the head of the first cache queue are deleted.

[0025] It can be seen that the second electronic device, in order to ensure the data consistency of the recording content recorded by the recording module, can first cache the data into the first cache queue when the recording content is acquired. In a case where the first electronic device successfully receives the data, the cached data is deleted, which can ensure the correctness of data transmission.

[0026] In a possible implementation of the first aspect, the first electronic device is further configured to:

[0027] write the audio data and the video data located at the head of the first cache queue into the tail of a second cache queue;

[0028] send the receiving success information to the second electronic device.

[0029] It can be seen that, in order to ensure the consistency and correctness of the data, the first electronic device, after receiving the data from the second electronic device, can first cache the data into the second cache queue, and then send the information of successful caching to the second electronic device.

[0030] In a possible implementation of the first aspect, the first electronic device is specifically configured to:

[0031] in a case where there is data in the second cache queue, take out the audio data and the video data located at the head of the second cache queue;

[0032] synchronously process the audio data and the video data at the head of the second cache queue to obtain a recording picture, and display the recording picture;

[0033] in a case where the video data and the audio data need to be saved, save the audio data and the video data into a synthesis module.

[0034] It can be seen that the data in the second cache queue is cached in order, so that the data obtained from the second cache queue can ensure the consistency and correctness of the data, and a correct video picture can be provided.

[0035] In a possible implementation of the first aspect, the first electronic device is further configured to:

[0036] in response to a second operation acting on the first interface, generate a stop recording screen instruction;

[0037] send the stop recording screen instruction to the second electronic device, where the stop recording screen instruction indicates to stop the recording screen function;

[0038] The second electronic device is configured to:

[0039] stop the screen recording function according to the stop screen recording instruction.

[0040] It can be seen that the first electronic device can stop the screen recording function of the second electronic device by sending a notification screen recording instruction to the second electronic device.

[0041] In a possible implementation of the first aspect, the second electronic device is specifically configured to:

[0042] In a case where it is determined according to the stop screen recording instruction that the mirror control function and the highlight function are both stopped, the video data and the audio data corresponding to the screen recording function are stopped from being recorded. It can be seen that in a case where the highlight function and the mirror control function are both stopped, the second electronic device does not need to record, that is, does not need to save the video data and the audio data, and does not need to synchronize the audio data and the video data to the first electronic device.

[0043] In a possible implementation of the first aspect, the second electronic device is specifically configured to:

[0044] In a case where it is determined according to the stop screen recording instruction that the mirror control function is not stopped and the highlight function is stopped, the video data and the audio data corresponding to the screen recording function are stopped from being saved according to the stop screen recording instruction.

[0045] The second electronic device sends the saved video file information corresponding to the screen recording function to the first electronic device.

[0046] It can be seen that because the highlight function is stopped, only the saving of the video data and the audio data is stopped, but the synchronization of the audio data and the video data to the first electronic device is not stopped.

[0047] In a possible implementation of the first aspect, the second electronic device is specifically configured to: in a case where it is determined according to the video file information that the recording is successful, display the video file corresponding to the highlight function. It can be seen that after the highlight function is stopped, the received data is verified, and the recorded video file can be displayed.

[0048] In a second aspect, the present application provides a cross-device screen recording method applied to a first electronic device, which can include:

[0049] The first electronic device displays a first interface of a first application, and generates a screen recording instruction in response to a first operation acting on the first interface.

[0050] sending the screen recording instruction to the second electronic device, wherein the screen recording instruction indicates to start a screen recording function, the screen recording function is a highlight function and / or a mirror control function, and the second electronic device is configured to record video data corresponding to the screen recording function and the audio data;

[0051] receiving the video data and the audio data from the second electronic device;

[0052] displaying a recording picture corresponding to the mirror control function based on the video data and the audio data.

[0053] In a possible implementation of the second aspect, after the receiving the video data and the audio data from the second electronic device, and before the displaying the recording picture corresponding to the mirror control function based on the video data and the audio data, the method further includes:

[0054] writing the audio data and the video data into a tail of a second cache queue;

[0055] sending receiving success information to the second electronic device, wherein the second electronic device is configured to delete the audio data and the video data in the first cache queue according to the receiving success information.

[0056] In a possible implementation of the second aspect, the displaying the recording picture corresponding to the mirror control function based on the video data and the audio data includes:

[0057] when there is data in the second cache queue, taking out audio data and video data at a head of the second cache queue;

[0058] synchronously processing the audio data and the video data at the head of the second cache queue to obtain a recording picture, and displaying the recording picture.

[0059] In a possible implementation of the second aspect, the method further includes: when it is necessary to save the video data and the audio data, saving the audio data and the video data into a synthesis module.

[0060] In a possible implementation of the second aspect, the method further includes:

[0061] generating a stop screen recording instruction in response to a second operation acting on the first interface;

[0062] sending the stop screen recording instruction to the second electronic device, wherein the second electronic device is configured to return saved video file information to the first electronic device according to the stop screen recording instruction;

[0063] In a case where it is determined according to the video file information that recording is successful, a video file corresponding to the highlight function is displayed.

[0064] In a third aspect, the present application provides a cross-device screen recording method, applied to a second electronic device, which can include:

[0065] Receiving a screen recording instruction from a first electronic device, wherein the screen recording instruction indicates to start a screen recording function, and the screen recording function is a highlight function and / or a mirror control function;

[0066] Obtaining video data and audio data according to the screen recording instruction;

[0067] Sending the video data and the audio data to the first electronic device.

[0068] In a possible implementation of the third aspect, the obtaining of the video data and the audio data according to the screen recording instruction includes:

[0069] In a case where it is determined that the screen recording function is started, obtaining the audio data and the video data by calling an interface of a recording module through a highlight module.

[0070] In a possible implementation of the third aspect, the sending of the video data and the audio data to the first electronic device includes:

[0071] Saving the obtained audio data and video data to a tail of a first cache queue;

[0072] In a case where there is data in the first cache queue, taking out audio data and video data located at a head of the first cache queue through a sending thread;

[0073] Sending the audio data and the video data located at the head of the first cache queue to a screen projection module of the first electronic device.

[0074] In a possible implementation of the third aspect, after the sending of the video data and the audio data to the first electronic device, the method further includes:

[0075] In a case where receiving success information from the first electronic device is received, deleting the audio data and the video data located at the head of the first cache queue.

[0076] In a possible implementation of the third aspect, the method further includes:

[0077] Receiving a stop screen recording instruction from the first electronic device, wherein the stop screen recording instruction indicates to stop the screen recording function;

[0078] stop the screen recording function according to the stop screen recording instruction.

[0079] In a possible implementation of the third aspect, the stopping the screen recording function according to the stop screen recording instruction comprises:

[0080] In a case where it is determined according to the stop screen recording instruction that the mirror control function and the highlight function are to be stopped, stop recording video data and audio data corresponding to the screen recording function.

[0081] In a possible implementation of the third aspect, the stopping the screen recording function according to the stop screen recording instruction comprises:

[0082] In a case where it is determined according to the stop screen recording instruction that the mirror control function is not to be stopped and the highlight function is to be stopped, stop saving video data and audio data corresponding to the screen recording function according to the stop screen recording instruction.

[0083] send, to the first electronic device, saved video file information corresponding to the screen recording function.

[0084] In a fourth aspect, an electronic device is provided, which comprises one or more processors, a memory, wherein the memory is coupled to the one or more processors, the memory is configured to store computer program codes, the computer program codes comprise computer instructions, and the one or more processors invoke the computer instructions to enable the electronic device to perform the cross-device screen recording method described in the second aspect or any possible implementation of the second aspect, the third aspect or any possible implementation of the third aspect.

[0085] In a fifth aspect, a chip or chip system is provided, which comprises at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is configured to run computer programs or instructions to perform the cross-device screen recording method described in the second aspect or any possible implementation of the second aspect, the third aspect or any possible implementation of the third aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0086] In a possible implementation, the chip or chip system described above in the embodiments of the present application further comprises at least one memory, and the at least one memory stores instructions. The memory can be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0087] In a sixth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program. The computer program is executed by a processor to cause a computer to perform the cross-device screen recording method described in the second aspect or any possible implementation manner of the second aspect, the third aspect or any possible implementation manner of the third aspect.

[0088] In a seventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a communication device, causes the communication device to perform the cross-device screen recording method described in the second aspect or any possible implementation manner of the second aspect, the third aspect or any possible implementation manner of the third aspect.

[0089] It should be understood that the description of technical features, technical solutions, advantages or similar language in the present application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of a feature or advantage means that the specific technical feature, technical solution or advantage is included in at least one embodiment. Therefore, the description of technical features, technical solutions or advantages in the specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and advantages described in the embodiments can be combined in any appropriate manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions or advantages of a specific embodiment. In other embodiments, additional technical features and advantages can be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0090] The following describes the drawings used in the embodiments of the present application.

[0091] Figure 1 is an architecture schematic diagram of a cross-device screen recording system provided by an embodiment of the present application;

[0092] Figure 2A is a schematic diagram of a user interface 1121 of a simulated remote controller provided by an embodiment of the present application;

[0093] Figure 2B is a conflict scenario schematic diagram provided by an embodiment of the present application;

[0094] Figure 3 is a flowchart of a call recording state module provided by an embodiment of the present application;

[0095] Figure 4 is a structural schematic diagram of an electronic device 100 provided by an embodiment of the present application;

[0096] Figure 5is an interaction schematic diagram provided by an embodiment of the present application, in which a first electronic device calls a recording interface in a second electronic device;

[0097] Figure 6 is a cross-device screen recording method provided by an embodiment of the present application;

[0098] Figures 7A-7C is a timing schematic diagram provided by an embodiment of the present application, which is about “highlight moments” and “mirror control”. DETAILED DESCRIPTION

[0099] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and the appended claims of the present application, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0100] Hereinafter, the terms “first” and “second” are only for the purpose of description, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with “first” and “second” can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of “a plurality of” is two or more, unless otherwise specified.

[0101] Figure 1 is an architecture schematic diagram of a cross-device screen recording system provided by an embodiment of the present application. As shown in the figure, Figure 1 the system architecture can include a first electronic device 101 and a second electronic device 102. The first electronic device 101 can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle terminal, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the second electronic device 102 can be a smart screen device, a smart television (TV) or other electronic devices with video playback. The present application does not make specific limitations thereto.

[0102] In the embodiments of the present application, the first electronic device 101 and the second electronic device 102 establish a connection through wired or wireless means. Based on the established connection, the first electronic device 101 and the second electronic device 102 can be used together. In specific implementation, the wireless communication network accessed by the first electronic device 101 and the second electronic device 102 can be a network established based on wireless local area networks (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), or infrared (IR) technology, etc. For example, the first electronic device 101 and the second electronic device 102 access the same Wi-Fi local area network. For another example, the first electronic device 101 and the second electronic device 102 access the wireless communication network based on Wi-Fi peer to peer (P2P). For yet another example, the first electronic device 101 and the second electronic device 102 access the wireless communication network based on Bluetooth communication. The wired communication network accessed by the first electronic device 101 and the second electronic device 102 can be a network established based on Universal Serial Bus (USB) connection and high definition multimedia interface (HDMI) and other wired communication technologies.

[0103] As shown in Figure 1 In the embodiments of the present application, the first electronic device 101 is taken as a mobile phone and the second electronic device 102 is taken as a smart screen as an example. The smart space application is installed in the first electronic device 101, and the connected smart screen (i.e. the second electronic device 102) is displayed on the first interface 112 of the smart space application, and the today's use time length (4 hours and 48 minutes) of the smart screen and the on-off control (used to turn on / off the first electronic device 101) are also displayed. The mirror control, remote controller, smooth connection call, search video, message board, remote home watching, etc. are also displayed on the first interface 112 of the smart space application, and the control service interface of the corresponding function can be triggered by any one of the function controls.

[0104] In a possible implementation, the first electronic device 101 receives the user operation acting on the "remote controller" control, and can display a remote control interface as shown in Figure 2AThe diagram shows the user interface 1121 of the analog remote control. The user interface 1121 displays directional controls (up, down, left, right), an OK control, volume up / down controls, an exit control, and a menu control. These controls function similarly to the physical buttons on a physical remote control, allowing the user to operate the second electronic device 102 using the analog remote control on the second electronic device 102, just like using a physical remote control.

[0105] like Figure 2A As shown, during the video playback on the second electronic device 102, if the user sees something interesting, they can operate the "OK control" on the user interface 1121 of the first electronic device 101. The first electronic device 101 can respond to the user's operation, or the physical remote control of the second electronic device 102 can respond to the user's operation on the "OK control". Figure 2B (Not shown) A screen recording command can be sent to the second electronic device 102 to trigger the second electronic device 102 to record the exciting moments. The second electronic device 102 can record the screen according to the screen recording command, and obtain a video clip of preset length.

[0106] In one implementation, during the playback of a video on the second electronic device 102, in addition to triggering screen recording via a simulated remote control (such as the user interface 1121 of the first electronic device 101) or a physical remote control, screen recording can also be triggered in other ways. For example, during the playback of a video on the second electronic device 102, a screen recording command issued by the user's voice can be received, and the second electronic device 102 can record the screen according to the voice command to obtain a video clip of a preset duration.

[0107] from Figure 2A It can also be seen that during the recording of highlights by the second electronic device 102, a "Recording in progress" indicator can be displayed on the user interface 1011 of the second electronic device 102. In one implementation, the second electronic device 102 records a video file of a preset duration via a playback service. In another implementation, the second electronic device 102 can extract video and audio data of a preset duration from the already recorded video and audio data, and synthesize the extracted audio and video data into a video file of the highlights, with the playback duration of the video file being the preset duration.

[0108] For example, the maximum recording time can be 5 minutes, that is, the maximum preset duration can be 5 minutes. Within 5 minutes, the user can pause the recording by using a simulated remote control, a physical remote control or other means.

[0109] from Figure 2AIt can also be seen that after recording ends, the second electronic device 102 can display a user interface 1012. The user interface 1012 displays a thumbnail of the video clip and the recording time, and prompts "Video has been saved to the gallery." In one implementation, in response to a user operation on the "cancel control" in the user interface 1012, the second electronic device 102 can cancel sharing the recorded highlight video file to the first electronic device 101. In another implementation, in response to a user operation on the "share control" in the user interface 1012, if the second electronic device 102 has not enabled the sharing function, it displays a user interface 1013. The user interface 1013 can display a prompt to enable "sharing" and operation controls. In response to a user operation on the "enable control," the second electronic device 102 can synchronize the highlight video file to the gallery of the first electronic device 101.

[0110] During the recording of highlights by the second electronic device 102, if the user wishes to use the mirror control function, they can operate... Figure 1 The mirror control control on the first interface 112 of the first electronic device 101, in response to the user operation, allows the first electronic device 101 to send a mirror control command to the second electronic device 102 to trigger the second electronic device 102 to activate the mirror control function. However, since the screen recording of highlights on the smart screen requires the user to turn it on, and because the underlying capabilities of the smart screen are limited (i.e., the underlying chip only supports single-channel decoding), triggering the mirror control function during the recording of highlights on the smart screen requires conflict detection. Therefore, it needs to interact with other applications to detect conflicts and display the result as shown below. Figure 2B The diagram illustrates a conflict scenario. Figure 2B As shown, if "Highlights" is active and the second electronic device 102 is not on the input source interface, the user can control the second electronic device 102 using the mirror control on the first electronic device 101. When the second electronic device 102 is on the input source interface, if the user continues to click the mirror control on the first electronic device 101, a pop-up window will appear on the first electronic device 101 to prompt the user of a scene conflict. The prompt may include, but is not limited to, "Highlights on the smart screen are in use. To use mirror control, please turn off 'Highlights' on the smart screen. Do you want to turn it off?".

[0111] from Figure 2BIt can also be seen that, in an implementation, after the user turns off the highlight moment of the second electronic device 102 through the first electronic device 101, a user interface 1014 is displayed on the second electronic device 102, and a pop-up window in the user interface 1014 displays that the highlight moment has been turned off. The user can issue a pop-up window for turning on the highlight moment again by pressing the OK key on the simulated remote controller, and a user interface 1015 is displayed on the second electronic device 102. In the user interface 1015, the pop-up window content includes but is not limited to “After turning on, when watching the live broadcast in the input terminal application, the OK key can be pressed for a long time to record a highlight segment of about 15 seconds, and share it with nearby terminal devices. Also, in the smart space app-remote controller, the OK key can be pressed for a long time to experience this function”. The second electronic device 102 can trigger the highlight moment again in response to the user operation on the “turn on control” in the pop-up window.

[0112] From Figure 2B It can also be seen that, because the mirror control has entered the video interface (that is, the first electronic device 101 displays the display content of the second electronic device 102), if the user triggers the highlight moment, a user interface 1016 is displayed on the second electronic device 102, and a pop-up window in the user interface 1016 can prompt the user of a scene conflict, and the pop-up window content includes but is not limited to: “If you want to use the highlight moment, you need to exit the mirror control of the mobile phone”. Do you want to exit?

[0113] It should be noted that, in the process of “mirror control”, the first electronic device 101 can be referred to as a peer, a screen mirroring peer, or a screen mirroring peer device, and the second electronic device 102 can be referred to as a source, a screen mirroring source, or a screen mirroring source device. For example, in the process of mirror projection, the second electronic device 102 plays a video in response to the user's operation on the video player application, and displays the picture content in the video on a large screen and plays the audio content in the video. The second electronic device 102 transmits the picture content and the audio content in the video as projection content to the first electronic device 101 through a wireless communication network. After receiving the projection content, the first electronic device 101 displays the picture content in the video on the screen and plays the audio content in the video, thereby playing the video, so that the same resource content is presented on the first electronic device 101 and the second electronic device 102 in the process of mirror projection. The mirror projection scene has been exemplarily described, and it can be understood that the mirror projection can also be applied to other various scenes, and the application embodiments do not limit the application scenarios.

[0114] It can be known from the above description that the highlight moment and the mirror control cannot be started at the same time. That is, in the process of using the highlight moment, if the mirror control needs to be started, the highlight moment needs to be paused. In the process of using the mirror control, if the highlight moment needs to be started, the mirror control needs to be paused.

[0115] Next, the reason why the "highlight moment" and the "mirror control" cannot be started at the same time is introduced.

[0116] Please refer to Figure 3 , Figure 3 is a flowchart of a process of calling a recording state module provided by an embodiment of the present application. As shown in Figure 3 , after the first electronic device 101 receives the user operation of starting the "highlight moment" through the UI module, the recording state module and the webSocket communication module are called to send the screen recording instruction of the "highlight moment" to the command processing module of the second electronic device 102. After the command processing module in the second electronic device 102 processes the screen recording instruction, the screen recording instruction is sent to the highlight moment module, and the interface of the recording state module is called to obtain the audio data and the video data from the video recording state module and the audio recording state module. In the process of starting the "highlight moment", after the first electronic device 101 receives the user operation of starting the "mirror control" through the UI module, the screen projection module is called to send the mirror control instruction to the screen control processing module of the second electronic device 102. The screen control processing module of the second electronic device 102 also needs to call the interface of the recording state module to obtain data from the video recording state module and the audio recording state module.

[0117] However, since the interface of the recording state module can only be called by one of the modules, it cannot be called at the same time. That is, if the highlight moment module calls the interface of the recording state module, the screen control processing module cannot call the interface of the recording state module, and the "mirror control" needs to be paused. If the screen control processing module calls the interface of the recording module, the highlight moment module cannot call the interface of the recording module, and the "highlight moment" needs to be paused.

[0118] The hardware structure of the first electronic device 101 and the second electronic device 102 in the system shown in Figure 1 is described below.

[0119] For example, Figure 4 is a structural diagram of an electronic device 100 provided by an embodiment of the present application. The electronic device can be the first electronic device 101 or the second electronic device 102 in Figures 1-3 .

[0120] As shown in Figure 4As shown, the electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0121] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0122] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0123] The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0124] The processor 110 can also include a memory that stores instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The cache memory can hold instructions or data that the processor 110 has recently used or is likely to use again. If the processor 110 needs to use the instructions or data again, it can be retrieved directly from the cache memory. This avoids repeated accesses to the main memory, reducing the latency of the processor 110 and thus improving the efficiency of the system.

[0125] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0126] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 can include multiple sets of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 can be coupled to the touch sensor 180K through an I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface and implement the touch function of the electronic device 100.

[0127] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple sets of I2S buses. The processor 110 can be coupled to the audio module 170 through the I2S bus, enabling communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can deliver audio signals to the wireless communication module 160 through the I2S interface, enabling the function of answering a phone call through a Bluetooth headset.

[0128] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 can be coupled with the wireless communication module 160 through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, realizing the function of answering a phone call through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0129] The UART interface is a general-purpose serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is usually used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface, realizing the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, realizing the function of playing music through a Bluetooth headset.

[0130] The MIPI interface can be used to connect the processor 110 and peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes the camera serial interface (CSI), the display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface, realizing the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface, realizing the display function of the electronic device 100.

[0131] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 and the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0132] The USB interface 130 is an interface that meets the USB standard specification, which can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect a headset to play audio through the headset. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0133] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a limitation on the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 can also use different interface connection modes or a combination of multiple interface connection modes in the above embodiments.

[0134] The charging management module 140 is configured to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the electronic device 100. The charging management module 140 can charge the battery 142 and also supply power to the electronic device through the power management module 141.

[0135] The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160, etc. The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0136] The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0137] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device 100. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be arranged in the same device as at least part of the modules of the processor 110.

[0138] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a microphone 170B, etc.), or displays an image or a video through the display 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110 and disposed in the same device as the mobile communication module 150 or other functional modules.

[0139] The wireless communication module 160 can provide a wireless communication solution including a wireless local area network (WLAN) (e.g., a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like, which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 can also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification thereon, and radiate the signal as an electromagnetic wave via the antenna 2.

[0140] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidu navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0141] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0142] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), or the like. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0143] The electronic device 100 can implement the photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.

[0144] The ISP is configured to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing to convert it into an image visible to the naked eye. The ISP can also algorithmically optimize the noise and brightness of the image. The ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0145] The camera 193 is configured to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, or the like format. In some embodiments, the electronic device 100 can include one or N cameras 193, where N is a positive integer greater than 1.

[0146] The digital signal processor is used to process digital signals, in addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 is in frequency point selection, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0147] The video codec is used to compress or decompress digital video. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0148] The NPU is a neural-network (NN) computing processor, which processes input information quickly by drawing on the structure of a biological neural network, such as drawing on the transmission mode between human brain neurons, and can also constantly self-learn. Through the NPU, the electronic device 100 can realize intelligent cognitive applications, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0149] The internal memory 121 can include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs).

[0150] The random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, such as the fifth generation DDR SDRAM commonly referred to as DDR5 SDRAM), etc.; the non-volatile memory can include a magnetic disk storage device, a flash memory.

[0151] The flash memory can include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operation principle, and can include single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential order, and can include universal flash storage (UFS) and embedded multi media Card (eMMC), etc. according to the storage specification.

[0152] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of an operating system or other programs running, and can also be used to store data of users and application programs, etc.

[0153] The non-volatile memory can also store executable programs and store data of users and application programs, etc., and can be loaded into the random access memory in advance for direct reading and writing by the processor 110.

[0154] The external memory interface 120 can be used to connect an external non-volatile memory to realize the expansion of the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, files such as music and video are saved in the external non-volatile memory.

[0155] The electronic device 100 can realize the audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, etc. For example, music playing, recording, etc.

[0156] The audio module 170 is used to convert digital audio information into analog audio signals, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or part of the function modules of the audio module 170 can be arranged in the processor 110.

[0157] The speaker 170A, also known as a "loudspeaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0158] The receiver 170B, also called the "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a phone call or a voice message, the receiver 170B can be held close to the ear to answer the voice.

[0159] The microphone 170C, also called the "microphone", "sound collector", is used to convert sound signals into electrical signals. In some other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals, but also achieve noise reduction functions. In some other embodiments, the electronic device 100 can be provided with three, four or more microphones 170C, which can not only collect sound signals and reduce noise, but also identify the source of the sound and achieve directional recording functions.

[0160] The earphone interface 170D is used to connect a wired earphone. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0161] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0162] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shaking of the electronic device 100, and calculates the distance that the lens module needs to compensate according to the angle, so that the lens can offset the shaking of the electronic device 100 by reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B can also be used for navigation and motion sensing game scenarios.

[0163] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude, assists positioning and navigation by measuring the air pressure value by the barometric pressure sensor 180C.

[0164] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of a flip cover by using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the cover or the opening and closing state of the flip cover, the electronic device 100 can set a flip cover automatic unlocking feature.

[0165] The acceleration sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions, typically, three axes. The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the electronic device posture, applied to the landscape / portrait screen switching, pedometer, etc.

[0166] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance by infrared or laser. In some embodiments, the scene is shot, and the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0167] The proximity light sensor 180G can include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode can be an infrared light emitting diode.

[0168] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the display screen 194 brightness according to the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when shooting. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket to prevent false touch.

[0169] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application lock, fingerprint shooting, fingerprint answering, etc.

[0170] The temperature sensor 180J is used to detect temperature.

[0171] The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called "touch screen".

[0172] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals of the human body sound part vibration bone block.

[0173] The key 190 includes the power-on key, the volume key, etc. The key 190 can be a mechanical key. It can also be a touch key. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0174] The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompt, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The touch operation acting on different regions of the display screen 194 can also correspond to different vibration feedback effects of the motor 191. Different application scenarios (such as time reminders, received messages, alarms, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0175] The indicator 192 can be an indicator light, which can be used to indicate the charging state, the power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0176] Please refer to Figure 5 , Figure 5 is an interaction schematic diagram provided by an embodiment of the present application for calling a recording interface in a second electronic device. As shown in Figure 5 , the first electronic device 101 includes one or more of a user interface UI module 501, a recording state module 502, a cast module 503, an audio-video synchronization module 504, and a websocket communication module 505. The second electronic device 102 includes one or more of a command processing module 506, a highlight module 507, and a recording module 508 (including a video recording module 5081 and an audio recording module 5082).

[0177] Exemplarily, the user interface UI module 501 on the first electronic device 101 is used to control the UI interface of the smart space application. The first electronic device 101 can start the “highlight” and “mirror control” of the second electronic device 102 through the user interface UI module 501.

[0178] In one implementation, in the process of starting the “highlight” of the second electronic device 102 through the user interface UI module 501 of the first electronic device 101, the recording state module 502 is called through the state synchronization channel. The recording state module 502 calls the audio-video synchronization module 504 through the video receiving state synchronization channel. The audio-video synchronization module 504 calls the websocket communication module 505 to send a screen recording instruction of the “highlight” to the second electronic device 102.

[0179] In one implementation, in the process of starting the “mirror control” of the second electronic device 102 through the user interface UI module 501 of the first electronic device 101, the cast module 503 is called through the state synchronization channel. The cast module 503 calls the audio-video synchronization module 504. The audio-video synchronization module 504 calls the websocket communication module 505 to send a cast instruction of the “mirror control” to the second electronic device 102.

[0180] It can be seen that, whether the "mirror control" function is started or the "highlight moment" function is started, the first electronic device 101 calls the audio-video synchronization module 504, and then sends the instruction of the corresponding function to the second electronic device 102 through the websocket communication module 505.

[0181] After the second electronic device 102 receives the "highlight moment" recording instruction and / or the "mirror control" screen projection instruction from the first electronic device 101, the received instruction can be processed by the command processing module, and then the interface of the recording module 508 is called by the highlight moment module 507 to obtain the audio data and video data of the "highlight moment" corresponding to the recording instruction and / or the audio data and video data of the "mirror control" corresponding to the screen projection instruction from the video recording module 5081 and the audio recording module 5082.

[0182] It can be seen that, whether the audio data and video data corresponding to the screen projection instruction are obtained or the audio data and video data corresponding to the recording instruction are obtained, the interface of the recording module 508 can be called by the highlight moment module 507 to obtain. Therefore, in the case that the interface of the recording module 508 can only be referenced by one module, because the highlight moment module 507 only needs to call the interface of the recording module 508 in the above process, there will be no conflict between the "highlight moment" and the "mirror control". Therefore, in the case of using the "highlight moment", if the "mirror control" is started, there will be no pop-up window as shown in Figure 2B to prompt the user of the scene conflict. Or, in the case of using the "mirror control", if the "highlight moment" is started, there will also be no pop-up window to prompt the user of the scene conflict.

[0183] As shown in Figure 6 , a cross-device recording method provided by an embodiment of the present application is applied to the system as shown in Figure 1 , which includes a first electronic device and a second electronic device, and the method includes but is not limited to the following steps:

[0184] Step S601, the first electronic device generates a recording instruction in response to a first operation acting on a first interface.

[0185] Among them, the first electronic device is installed with a first application, the first electronic device displays a first interface of the first application, and the first application can be a smart space application. The first interface displays information related to the second electronic device which has a communication connection with the first electronic device. The first electronic device can control the second electronic device through the control service provided by the first application, such as controlling the second electronic device to record.

[0186] In one implementation, when the user watches the exciting content during the playing of the video by the second electronic device, the first electronic device can be used to control the second electronic device to record the exciting content. For example, the first electronic device can receive a first operation (e.g., a click) of the user on the "OK control" of the user interface 1121 shown in FIG. 11B. In response to the first operation of the user, the first electronic device can generate a recording instruction for the "highlight moment". The recording instruction for the "highlight moment" carries a first identifier, which indicates that the recording instruction corresponds to the instruction for the "highlight moment" function. For example, a value of "1" of the first identifier indicates that the "highlight moment" is started, and a value of "0" of the first identifier indicates that the "highlight moment" is paused. Figure 2A For example, the first electronic device can receive a first operation (e.g., a click) of the user on the "mirror control" control of the first interface 112 shown in FIG. 11A. In response to the first operation of the user, the first electronic device can generate a recording instruction for the "mirror control". The recording instruction for the "mirror control" carries a second identifier, which indicates that the recording instruction corresponds to the instruction for the "mirror control" function. For example, a value of "1" of the second identifier indicates that the "mirror control" is started, and a value of "0" of the second identifier indicates that the "mirror control" is paused.

[0187] In another implementation, during the operation of the second electronic device (e.g., the second electronic device is in a powered-on state and displays content), if the user wants to mirror the content displayed on the second electronic device to the first electronic device, the first electronic device can be used to control the second electronic device to perform the mirroring. For example, the first electronic device can receive a first operation (e.g., a click) of the user on the "mirror control" control of the first interface 112 shown in FIG. 11A. In response to the first operation of the user, the first electronic device can generate a recording instruction for the "mirror control". The recording instruction for the "mirror control" carries a second identifier, which indicates that the recording instruction corresponds to the instruction for the "mirror control" function. For example, a value of "1" of the second identifier indicates that the "mirror control" is started, and a value of "0" of the second identifier indicates that the "mirror control" is paused. Figure 1

[0188] Step S602: The first electronic device sends a recording instruction to the second electronic device.

[0189] The first electronic device and the second electronic device establish a connection through a wired or wireless manner, and based on the established connection, the first electronic device and the second electronic device can be used together. Therefore, the first electronic device can send a recording instruction to the second electronic device through a wired or wireless manner. The recording instruction carries a recording identifier, which includes a first identifier and a second identifier. The first identifier indicates that the recording instruction corresponds to the instruction for the "highlight moment" function, and the second identifier indicates that the recording instruction corresponds to the instruction for the "mirror control" function.

[0190] Step S603: The second electronic device acquires video data and audio data corresponding to the recording function through the highlight moment module according to the recording instruction.

[0191] ​Specifically, after receiving the screen recording instruction, the second electronic device can determine whether to start the screen recording function according to a first value of a screen recording identifier carried in the screen recording instruction. For example, the first value is 1, indicating that the screen recording function is started.

[0192] In one implementation, when the value of the screen recording identifier is 1, it indicates that the screen recording instruction is an instruction for starting the “highlight moment” function. Therefore, the recorded content includes the content corresponding to the “highlight moment”. For example, the recorded content can be a video segment in a video played by the second electronic device.

[0193] In another implementation, when the value of the screen recording identifier is 1, it indicates that the screen recording instruction is an instruction for starting the “mirror control” function. Therefore, the recorded content includes the content corresponding to the “mirror control”. For example, the recorded content can be the content displayed by the second electronic device.

[0194] Therefore, after the second electronic device determines to start the screen recording function according to the first value carried in the screen recording instruction, the second electronic device can call the interface of the recording module through the highlight moment module, so as to obtain the recorded video file. The recorded video file includes the video file corresponding to the screen recording function.

[0195] It can be understood that in the case where the “highlight moment” function is started and the “mirror control” function is started, the screen recording identifier can be used to distinguish the starting of the “highlight moment” function and the starting of the “mirror control” function. Further, the video file corresponding to the recording function in the recorded video file can also be distinguished, that is, the video file of the “highlight moment” and the video file of the “mirror control” can be distinguished.

[0196] In one possible implementation, in the running state, the second electronic device can first cache a 15-second video file in the recording module. In the case where no operation of triggering the screen recording by the user is received, the second electronic device can delete the recorded 15 seconds, so as to release the memory space. Then, the second electronic device continues to cache the next 15-second video file in the recording state module. By analogy, the second electronic device can continuously / periodically cache 15-second video files in the recording state module, and delete the recorded 15 seconds in the case where no operation of triggering the screen recording by the user is received, so as not to occupy too much memory, and to ensure that the latest recorded 15-second video file can be quickly provided when the user triggers the screen recording.

[0197] In one implementation, after the second electronic device obtains the corresponding audio data and video data from the recording state module according to the recording content, before the second electronic device sends the audio data and video data to the first electronic device, the second electronic device can sequentially input the audio data and video data carrying the screen recording identifier into the first cache queue (occupying a fixed memory space), and when the first cache queue is full, discard the audio data and video data that are input into the first cache queue first, so that new audio data and video data can be continuously input into the first cache queue, so that the first cache queue can continuously store the latest played audio data and video data.

[0198] Exemplarily, the data packets (including audio data and video data) in the recording module are written into the first cache queue for storage, and the order of the data packets is unchanged when writing, and the currently written data packet is located after the previously written data packet, that is, the order is sorted according to the order of obtaining, so as to ensure that the data packets in the first cache queue are sorted according to the order of obtaining, that is, the order of the data packets in the first cache queue is the same as the order of the video data packets in the recording module, so that the data packets can be extracted according to the order in the first cache queue when extracting data from the first cache queue, and the accuracy of the extracted data packets is ensured.

[0199] Step S604, the second electronic device sends the video data and audio data to the first electronic device.

[0200] In one implementation, when there are data packets (including audio data and video data) in the first cache queue of the second electronic device, a sending thread is started, and the second electronic device can take out the head data packet in the first cache queue through the sending thread, and then send a video file containing the above data packet to the first electronic device. That is, after starting the screen recording function, the second electronic device synchronizes the video data and audio data corresponding to the screen recording function to the first electronic device in real time.

[0201] In one implementation, during the process in which the second electronic device sends the video data and audio data to the first electronic device, a reply from the first electronic device side is needed, and after receiving the receiving success information from the first electronic device, the second electronic device can remove the above data packet from the head of the first cache queue. In one implementation, if no reply from the first electronic device is received within a preset time (for example, 1 second), the second electronic device needs to send the above data packet again until a reply from the first electronic device is received.

[0202] Step S605, the first electronic device displays the recording screen corresponding to the mirror control function.

[0203] Specifically, the first electronic device can synchronize and merge the corresponding video data and audio data according to the screen recording instruction to obtain a screen recording segment including the video data and the audio data, and display the recording content.

[0204] It can be understood that the "mirror control" function is that the second electronic device sends the displayed content to the first electronic device, so that the first electronic device can display the content displayed by the second electronic device in real time. The "highlight" function is that the second electronic device records a video file of a preset time length, and after triggering the stop of the "highlight" function, the second electronic device sends the saved video file information to the first electronic device, and in the case of determining that the recording is successful according to the video file information, the video file corresponding to the "highlight" function is displayed.

[0205] In a possible implementation, after the first electronic device receives the video data and the audio data synchronized by the second electronic device, the first electronic device obtains and displays the recording picture of the "mirror control".

[0206] In an implementation, after the first electronic device receives the data packet (including the audio data and the video data) from the second electronic device, the first electronic device inputs the data packet to the tail of the second cache queue, and then replies to the second electronic device that the data packet is received successfully. Therefore, when there is a data packet in the second cache queue, the first electronic device takes out the head data packet of the second cache queue. On one hand, the head data packet is processed synchronously and the "mirror control" content processed is displayed, and on the other hand, the head data is stored in the synthesis module (MediaMuxer). After receiving the instruction for triggering the end of the "highlight", the first electronic device can obtain the data packet of the preset time length from the synthesis module, synchronize and merge the data packet to obtain and display the video file of the "highlight".

[0207] In step S606, the first electronic device generates a stop screen recording instruction in response to the second operation acting on the first interface.

[0208] For example, after the first electronic device starts the "highlight" function and the "mirror control" function of the second electronic device, the first electronic device can generate a stop screen recording instruction in response to the second operation acting on the first interface, such as a function for closing the "highlight" function or closing the "mirror control" function, so that the first electronic device can close the "highlight" function and the "mirror control" function of the second electronic device.

[0209] In step S607, the first electronic device sends the stop screen recording instruction to the second electronic device.

[0210] The stop screen recording instruction indicates to stop a screen recording function corresponding to the highlight moment function or a screen recording function corresponding to the mirror control function. The second electronic device stops the screen recording function according to the stop screen recording instruction.

[0211] The first electronic device sends video file information to the first electronic device.

[0212] In step S608, the second electronic device sends the video file information to the first electronic device.

[0213] In one implementation, when the second electronic device determines to stop the mirror control function and the highlight moment function according to the stop screen recording instruction, the second electronic device stops recording video data and audio data corresponding to the screen recording function, that is, the second electronic device stops synchronizing the audio data and the video data to the first electronic device and also stops saving the audio data and the video data in the first cache queue.

[0214] In one implementation, when the second electronic device determines not to stop the mirror control function and to stop the highlight moment function according to the stop screen recording instruction, the second electronic device stops saving video data and audio data corresponding to the screen recording function, but can synchronize the audio data and the video data to the first electronic device.

[0215] In step S609, the first electronic device displays a recording picture corresponding to the highlight moment function when it is determined that the recording is successful according to the video file information.

[0216] It can be understood that after the first electronic device receives the video file information saved from the second electronic device, the first electronic device can verify the video file synthesized by the synthesis module according to the video file information. When the verification is successful, it indicates that the recording of the "highlight moment" is successful, and the first electronic device can display the recording picture of the "highlight moment", such as a recording cover.

[0217] Next, a timing diagram corresponding to the scenario of the embodiments of the present application is introduced.

[0218] Scenario one: The first electronic device 101 starts the "highlight moment" of the second electronic device 102 and obtains a video file corresponding to the "highlight moment".

[0219] Scenario two: The first electronic device 101 starts the "mirror control" of the second electronic device 102 and obtains a video file corresponding to the "mirror control".

[0220] Scenario three: The first electronic device 101 stops the recording of the "highlight moment" of the second electronic device 102.

[0221] Scenario four: The first electronic device 101 stops the recording of the "mirror control" of the second electronic device 102.

[0222] The first electronic device 101 is installed with a first application, which can be a smart space application. Through the first application, the second electronic device 102 can be interacted with and controlled. For example, the second electronic device 102 is controlled to start a “highlight moment” function and record a video file corresponding to the “highlight moment”. The second electronic device 102 is controlled to start a “mirror control” function and record a video file corresponding to the “mirror control”. The second electronic device 102 is controlled to stop the “highlight moment” function. The second electronic device 102 is controlled to stop the “mirror control” function.

[0223] Figures 7A-7C is a timing diagram provided by an embodiment of the present application about the “highlight moment” and the “mirror control”.

[0224] As shown in Figures 7A-7C , the first electronic device 101 includes a UI module, a screen projection module, a recording state module, and an audio-video synchronization module. The second electronic device 102 includes a command processing module, a highlight moment module, and a recording module.

[0225] Figure 7A is a timing diagram provided by an embodiment of the present application about the first electronic device sending a screen recording instruction to the second electronic device, including but not limited to the following execution steps:

[0226] S701, the UI module receives a first operation.

[0227] During the process of the second electronic device 102 playing a video, when the user watches a highlight content, the user can start the recording of the “highlight moment” of the second electronic device 102 through the first electronic device 101 or other ways. If the user wants to project the content displayed in the second electronic device 102 to the first electronic device 101 for display, the user can start the recording of the “mirror control” of the second electronic device through the first electronic device. The present application does not make any limitation on the order of starting the “highlight moment” and the “mirror control”. The “highlight moment” can be started first and then the “mirror control” is started, or the “mirror control” can be started first and then the “highlight moment” is started.

[0228] Exemplarily, the UI module includes a user-visual control corresponding to the first application (for example, a smart control application), such as Figure 2A the “remote controller” control shown in and Figure 1 the “mirror control” control shown in. The UI module receives a first operation (for example, a click) acting on the “OK” control in the “remote controller” control, and the first operation is used to start the recording of the “highlight moment”.

[0229] Exemplarily, the UI module receives a first operation (e.g., a click) acting on the "mirror control" control, and the first operation is used to start recording of the "mirror control". It should be noted that the manner of starting the "highlight" and the "mirror control" is not limited to the first operation, and can also be voice control, etc.

[0230] S702, the UI module sends a recording instruction of the "highlight" to the recording state module.

[0231] The recording state module is used to manage recording of the "highlight", and the recording instruction of the "highlight" is used to indicate that the "highlight" function is started, and to manage the "highlight" content that the user wants to save in the second electronic device.

[0232] S703, the UI module sends a recording instruction of the "mirror control" to the cast module.

[0233] The cast module is used to manage recording of the "mirror control", and the recording instruction of the "mirror control" is used to indicate that the "mirror control" function is started, and to manage the content displayed by the second electronic device.

[0234] S704, the recording state module starts recording of the audio and video module.

[0235] The audio and video module is used to save the recording content returned by the second electronic device. When the "highlight" function is started, the audio and video module receives the recording instruction of the "highlight", and thus starts recording, and sets a first identifier (e.g., HighlightStatus) in a recording identifier in the audio and video module to 1. The first identifier is a flag bit of the "highlight", and the value is 1, indicating that the "highlight" function is started.

[0236] S705, the cast module starts recording of the audio and video module.

[0237] The audio and video module is used to save the recording content returned by the second electronic device. When the "mirror control" function is started, the audio and video module receives the recording instruction of the "mirror control", and thus starts recording, and sets a second identifier (e.g., CastStatus) in a recording identifier in the audio and video module to 1. The second identifier is a flag bit of the "mirror control", and the value is 1, indicating that the "mirror control" function is started.

[0238] S706, the audio and video module sends a start message of the recording instruction to the command processing module.

[0239] In one implementation, when the audio / video module receives the start, the communication module (e.g., WebSocket Server) in the first electronic device can connect the second electronic device according to the IP address. The communication module (e.g., WebSocket Client) in the second electronic device will respond to the connection. After the connection channel is successfully established, the audio / video module sends a screen recording instruction to the command processing module, and the screen recording instruction carries a screen recording identifier. Exemplarily, the message format of the screen recording instruction is as follows:

[0240] {

[0241] "taskId":"5C0E***FE6FA",

[0242] "acticon":"syncVideoAndAudio",

[0243] "HighlightStatus":"0\1",

[0244] "CastStatus":"0\1"

[0245] }

[0246] S707, the command processing module sends a data capture message related to the screen recording instruction to the highlight moment module.

[0247] In one implementation, before determining the recording content, when the command processing module (e.g., CmdServer) of the second electronic device receives a message related to the screen recording instruction sent by the WebSocket Client of the first electronic device through the WebSocket Server, the process communication RecordManager between services can be established through the highlight moment module. The switch state is read through the getSwitchState() method of the process communication RecordManager established through the highlight moment module.

[0248] Exemplarily, in the case where the value of the switch state is 1, it indicates that the second electronic device starts the highlight moment function. In the case where the value of the switch state is 0, it indicates that the second electronic device does not start the highlight moment function. When the highlight moment function is not started, an error code of start failure is returned to the first electronic device, and a pop-up window is displayed on the second electronic device. The content in the pop-up window is used to prompt the user to turn on the highlight moment function. Exemplarily, the content in the pop-up window includes an OK button. The second electronic device starts the highlight moment function in response to the input operation of the OK button, and then needs to go through S701-S706 again.

[0249] Exemplarily, when the screen recording capability has been started, the command processing module calls doRecord(flag, callback) of the RecordManager. Wherein, flag is true, indicating to save the recording result, and flag is false, indicating not to save the recording result, and only return the audio data and the video data through the callback.

[0250] S708, the highlight moment module requests the video data and the audio data from the recording module.

[0251] When the highlight moment module receives the screen recording instruction of starting recording, the highlight moment module requests the audio data and the video data from the recording module. That is, the highlight moment module calls the interface of the recording module to acquire the audio data and the video data from the MTK video recording module and the MTK audio recording module.

[0252] Exemplarily, the highlight moment module acquires the video data through the acquireCastStrem() method of the MtkCast, and acquires the audio data through the acquireAudioCastStrem() method of the MtkCast.

[0253] S709, the recording module returns the video data and the audio data to the highlight moment module.

[0254] Exemplarily, when the recording result needs to be saved, the returned video data VencStreamInf and the audio data AudioEncoderStreamInfo can be written through the writeSampleData() method of the MediaMuxer in the Android Framework.

[0255] S710, the highlight moment module returns the video data and the audio data to the command processing module.

[0256] During the recording process, the command processing module (for example, the CmdServer) synchronizes the data through the callback method of doRecord(true, callback) in the RecordManager, wherein the video data VencStreamInf and the audio data AudioEncoderStreamInfo are returned, and the callback returned data contains a flag bit flag, flag is true, indicating that it has been saved, and flag is false, indicating that it has not been saved.

[0257] S711, the command processing module sends a start result message to the audio and video module.

[0258] When the command processing module receives the returned audio data and / or video data, it sends a start result message to the audio / video module. Exemplarily, the message format is as follows:

[0259] {

[0260] "taskId":"5C0E***FE6FA",

[0261] "acticonCode":"**"

[0262] }

[0263] Further, the audio / video module sends a start result message to the recording state module.

[0264] The start result message is used to inform the recording state module of the start result of the second electronic device for the "highlight moment".

[0265] Further, the audio / video module sends a start result message to the cast module.

[0266] The start result message is used to inform the cast module of the start result of the second electronic device for the "mirror control".

[0267] Exemplarily, when the actionCode in the start result message is 202, it indicates that the screen recording capability switch of the second electronic device is not turned on; when the actionCode in the start result message is 203, it indicates that the screen recording capability of the second electronic device has been turned on.

[0268] Figure 7B is a timing diagram provided by an embodiment of the present application for a second electronic device to send a data packet to a first electronic device, including but not limited to the following steps:

[0269] S712, the command processing module sends a data packet to the audio / video module.

[0270] The data packet includes audio data and video data.

[0271] Exemplarily, when the command processing module returns the video data VencStreamInf by using the callback method in RecordManager, it first calculates the data sequence number, size, and check value (such as sha256 value) of the video data in the video file, and then saves the video data at the tail of the video data queue of the first cache queue for sending.

[0272] Exemplarily, when the command processing module returns the audio data AudioEncoderStreamInfo by using the callback method in the RecordManager, it first calculates the data sequence number, size, and check value (such as sha256 value) of the audio data in the video file, and then saves the audio data at the tail of the audio data queue of the first cache queue for sending.

[0273] Therefore, when there is a data packet in the first cache queue, a sending thread is started, which takes out the head data packet in the first cache queue and sends the data packet to the audio / video module on the first electronic device side. At the same time, after receiving a reply from the first electronic device that the data packet has been successfully received, the data packet is removed from the head of the first cache queue. If no reply is received from the first electronic device within a preset time (for example, within 1 second) after sending the data packet, the data packet can be resent.

[0274] Exemplarily, the message format of the sent data packet is as follows:

[0275] {

[0276] "taskId":"5C0E***FE6FA",

[0277] "fileName":"Highlight2023-08-30.mp4",

[0278] "tag":"audio" / "video",

[0279] "currentIndex":1,

[0280] "saveTag":true,

[0281] "dataSize":"10240",

[0282] "data":

[0283] "D5052951A89E32********************************7B6110F7796859DFE6FA",

[0284] "dataSha256":"50529*************************7B6F77DFE6F"

[0285] }

[0286] S713, the audio / video module stores the data packet into the second cache queue.

[0287] When the audio and video module of the first electronic device receives the data packet, the audio data in the data packet is inserted into the tail of the audio data queue of the second cache queue, and the video data in the data packet is inserted into the tail of the video data queue of the second cache queue. Finally, the audio and video module returns information about receiving the successful data packet to the command processing module.

[0288] Exemplarily, the returned message format is as follows:

[0289] {

[0290] "taskId":"5C0E***FE6FA",

[0291] "tag":"audio" / "video",

[0292] "currentIndex":1,

[0293] "result":"success"

[0294] }

[0295] S714, the audio and video module sends the data packet to the screen projection module.

[0296] Exemplarily, when there is video data in the video data queue of the first cache queue, the audio and video module can dequeue the head data in the video data queue and first send the data to the cast module.

[0297] In one implementation, when saveTag is true, it indicates that the audio and video module needs to save the data. The audio and video module can write the data packet into the cache of the MediaMuxer through the wrieSampleData() method of the MediaMuxer in the Android Framework.

[0298] Exemplarily, when there is audio data in the audio data queue of the first cache queue, the audio and video module can dequeue the head data in the audio data queue and first send the data to the cast module.

[0299] In one implementation, when saveTag is true, it indicates that the audio and video module needs to save the data. The audio and video module can write the data packet into the cache of the synthesis module (for example, MediaMuxer) through the wrieSampleData() method of the MediaMuxer in the Android Framework.

[0300] S715, the screen projection module processes the data packet to obtain a recording picture of "mirror control".

[0301] S716, the screen projection module sends the recording picture of the "mirror operation" to the UI module.

[0302] S717, the UI module displays the recording picture of the "mirror operation".

[0303] After the UI module receives the video information, the interface is refreshed to display the recording picture of the "mirror operation".

[0304] Figure 7C is a timing diagram provided by an embodiment of the present application for a first electronic device to stop a "highlight" and / or "mirror operation" of a second electronic device, including but not limited to the following steps:

[0305] S718, the UI module receives a second operation.

[0306] For example, during the recording of the "highlight", the UI module receives a second operation (for example, a click) on the "OK" control in the "remote control" control shown in FIG. 8, and the second operation is used to stop the recording of the "highlight". It should be noted that when the recording of the "highlight" reaches a preset time length (for example, 5 minutes), the recording of the "highlight" can be automatically triggered to stop. Or receiving a voice command to stop the "highlight" can also trigger the recording of the "highlight" to stop, and the present application does not make any limitation on this. Figure 2A For example, during the recording of the "mirror operation", the UI module receives a second operation (for example, a click) on the "mirror operation" control shown in FIG. 9, and the second operation is used to stop the recording of the "mirror operation".

[0307] Figure 1

[0308] S719, the UI module sends a stop recording instruction of the "highlight" to the recording state module.

[0309] S720, the UI module sends a stop recording instruction of the "mirror operation" to the cast module.

[0310] S721, the recording state module stops the recording of the audio and video module.

[0311] When the "highlight" is stopped, the audio and video module receives the stop recording instruction of the "highlight", sets the value of the first identifier (for example, HighlightStatus) to 0, and simultaneously executes step S723.

[0312] S722, the screen projection module stops the recording of the audio and video module.

[0313] ​​When the "mirror control" is paused, the audio and video module determines whether the value of the first identifier (e.g., HighlightStatus) is 1. If the value is 1, it indicates that the highlight moment is in use, and the stop recording instruction is not sent. If the value is 0, it indicates that the highlight moment is not in use. Then, step S723 is executed, otherwise, the process is not executed further, and the value of the second identifier (e.g., CastStatus) is set to 0.

[0314] S723, the audio and video module sends a stop recording instruction pause message to the command processing module.

[0315] The stop recording instruction carries a recording identifier. For example, the message format of the stop recording instruction is as follows:

[0316] {

[0317] "taskId":"5C0E***FE6FA",

[0318] "acticon":"stopSyncVideoAndAudio",

[0319] "HighlightStatus":"0\1",

[0320] "CastStatus":"0\1"

[0321] }

[0322] S724, the command processing module sends a stop recording instruction to the highlight moment module.

[0323] In one implementation, when the command processing module receives the message of the stop recording instruction, if the first identifier (e.g., HighlightStatus) in the recording instruction is 0 and the second identifier (e.g., CastStatus) is 1, the command processing module pauses the file saving by using the stopSaveFile method in RecordManager.

[0324] In another implementation, when the command processing module receives the message of the stop recording instruction, if the first identifier (e.g., HighlightStatus) in the recording instruction is 0 and the second identifier (e.g., CastStatus) is 0, the command processing module pauses the recording synchronization by using the stopRecord method in RecordManager.

[0325] S725, the highlight moment module returns the saved video file information to the command processing module. In one implementation, when the highlight moment module receives the stopSaveFile call, the saved video file information can be returned through the onResult method of the callback.

[0326] S726, the command processing module returns a save message of the video file to the audio and video module.

[0327] In one implementation, when the command processing module returns the saved video file information through the onResult method, the returned video file information is sent to the audio and video module on the first electronic device side. Exemplarily, the sent message format is as follows:

[0328] {

[0329] "taskId":"5C0E***FE6FA",

[0330] "acticonCode":"**"

[0331] "fileName":"Highlight2023-08-30.mp4",

[0332] "partCount":"100",

[0333] "fileSize":"101238912",

[0334] "sha256":"50529*************************7B6F77DFE6F"

[0335] }

[0336] S727, the audio and video module acquires the video file.

[0337] In one implementation, when the returned message format is actionCode=200, it indicates that the video recording is successful. When the recording is successful, the synthesis module (for example, MediaMuxer) is notified to generate the video file. The generated video file includes a check value (such as sha25). The check value is compared with the check value (such as sha25) in S726, and the file size is compared. If they are the same, it indicates that the recording of the "highlight moment" is successful, and the video file of the "highlight moment" is acquired.

[0338] S728, the audio and video module returns the video file to the recording state module.

[0339] Exemplarily, the audio and video module returns the video file of the "highlight moment" to the recording state module.

[0340] S729, the recording state module sends the video file to the UI module.

[0341] Exemplarily, the recording state module sends the video file of the "highlight moment" to the UI module.

[0342] S730, the UI module displays the video file.

[0343] Exemplarily, the UI module displays the page content corresponding to the video file of the "highlight moment".

[0344] The term "user interface (UI)" in the specification and claims of the present application and the drawings is a medium interface for interaction and information exchange between an application or an operating system and a user, which realizes conversion between an internal form of information and a form acceptable by the user. The user interface of an application is source code written in a specific computer language such as Java, extensible markup language (XML), and the like. The interface source code is parsed, rendered, and finally presented as content recognizable by the user such as pictures, texts, buttons, and the like controls on a terminal device. A control (widget) is a basic element of the user interface, and typical controls include a toolbar, a menu bar, a text box, a button, a scrollbar, a picture, and a text. The properties and content of the controls in the interface are defined by tags or nodes, such as XML. <textview> 、 <imgview> 、 <videoview>The interface is defined by nodes that specify the controls contained in the interface. One node corresponds to one control or property in the interface, and the nodes are parsed and rendered to present the content visible to the user. In addition, many applications, such as hybrid applications, also contain web pages in the interface. A web page, also referred to as a page, can be understood as a special control embedded in the interface of an application. The web page is a source code written in a specific computer language, such as hyper text markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed by a browser or a web page display component similar to the function of a browser to present content recognizable to the user. The specific content contained in the web page is also defined by tags or nodes in the web page source code, such as HTML defines the content of the web page by tags such as 、 、 <video> 、 <canvas>to define the elements and attributes of a web page.

[0345] A common form of user interface is a graphic user interface (GUI), which refers to a user interface that displays in a graphical manner. It can be an icon, window, control, etc. interface element displayed in the display screen of an electronic device, wherein the control can include an icon, button, menu, tab, text box, dialog box, status bar, navigation bar, Widget, etc. visual interface element.

[0346] It should be understood that each step in the above method embodiments provided by the present application can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software. The method steps disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor.

[0347] The present application also provides an electronic device, which can include a memory and a processor. The memory can be used to store a computer program, and the processor can be used to call the computer program in the memory to enable the electronic device to execute the method in any one of the above embodiments.

[0348] The present application also provides a chip system, which includes at least one processor for realizing the functions involved in the method executed by the electronic device in any one of the above embodiments.

[0349] In a possible design, the chip system further includes a memory for saving program instructions and data, and the memory is located in the processor or outside the processor.

[0350] The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0351] Optionally, the processor in the chip system can be one or more. The processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which realizes by reading software codes stored in the memory.

[0352] Optionally, the memory in the chip system can also be one or more. The memory can be integrated with the processor, or can be separately arranged from the processor, and the embodiments of the present application are not limited in this regard. Exemplarily, the memory can be a non-transient processor, for example, a read-only memory (ROM), which can be integrated on the same chip as the processor, or can be separately arranged on different chips. The embodiments of the present application do not make specific limitations on the type of the memory and the arrangement manner of the memory and the processor.

[0353] Exemplarily, the chip system can be a field programmable gate array (FPGA), can be an application specific integrated circuit (ASIC), can also be a system on chip (SoC), can also be a central processor unit (CPU), can also be a network processor (NP), can also be a digital signal processor (DSP), can also be a micro controller unit (MCU), can also be a programmable logic device (PLD) or other integrated chip.

[0354] The present application also provides a computer program product, which comprises a computer program (also referred to as code or instruction), which, when executed, causes a computer to perform the method executed by the electronic device in any one of the above embodiments.

[0355] The present application also provides a computer readable storage medium, which stores a computer program (also referred to as code or instruction). When the computer program is executed, it causes a computer to perform the method executed by the electronic device in any one of the above embodiments.

[0356] The embodiments of the present application can be combined in any manner to achieve different technical effects.

[0357] In the above embodiments, all or part of the processes can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the processes can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes described in the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk), etc.

[0358] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiments can be instructed by a computer program to complete the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes ROM or random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

[0359] In summary, the above only describes the embodiments of the technical solutions of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present application shall be included in the protection scope of the present application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A cross-device screen recording system, the system comprising: The screen recording system comprises a first electronic device and a second electronic device, and a communication connection is established between the first electronic device and the second electronic device, wherein: The first electronic device is configured to: display a first interface of a first application, the first interface being related to the second electronic device; generate a screen recording instruction in response to a first operation acting on the first interface; send the screen recording instruction to the second electronic device, wherein the screen recording instruction instructs to start a screen recording function, the screen recording function comprising a highlight function and a mirror control function, the mirror control function referring to the second electronic device sending picture content and audio content in a played video as screen projection content to the first electronic device; The second electronic device is configured to: obtain video data and audio data corresponding to the screen recording function through a highlight module according to the screen recording instruction, the video data comprising the picture content, and the audio data comprising the audio content; send the video data and the audio data to the first electronic device; The first electronic device is configured to: display a recording picture corresponding to the mirror control function based on the audio data and the video data; display a video file corresponding to the highlight function in a case where it is determined that recording is successful according to video file information corresponding to the screen recording function.

2. The system of claim 1, wherein, The second electronic device is specifically configured to: obtain the audio data and the video data through an interface of a recording module by calling the highlight module in a case where it is determined that the screen recording function is started; send the audio data and the video data to a screen projection module of the first electronic device.

3. The system of claim 2, wherein, The second electronic device is specifically configured to: save the obtained audio data and video data to a tail of a first cache queue; in a case where there is data in the first cache queue, take out audio data and video data located at a head of the first cache queue through a sending thread; send the audio data and the video data located at the head of the first cache queue to the screen projection module of the first electronic device; in a case where receiving success information from the first electronic device is received, delete the audio data and the video data located at the head of the first cache queue.

4. The system of claim 3, wherein, The first electronic device is further configured to: write the audio data and the video data located at the head of the first cache queue to a tail of a second cache queue; send the receiving success information to the second electronic device.

5. The system of claim 4, wherein, The first electronic device is specifically configured to: in a case where there is data in the second cache queue, take out audio data and video data located at a head of the second cache queue; synchronously process the audio data and the video data at the head of the second cache queue to obtain a recording picture, and display the recording picture; in a case where the video data and the audio data need to be saved, save the audio data and the video data into a synthesis module.

6. The system of any one of claims 1-5, wherein: The first electronic device is further configured to: generate a stop screen recording instruction in response to a second operation acting on the first interface; sending the stop screen recording instruction to the second electronic device, wherein the stop screen recording instruction indicates to stop the screen recording function; the second electronic device is configured to: stop the screen recording function according to the stop screen recording instruction.

7. The system of claim 6, wherein, the second electronic device is specifically configured to: in a case where it is determined according to the stop screen recording instruction that the highlight function is to be stopped and the mirror control function is not to be stopped, stop saving the video data and the audio data corresponding to the screen recording function.

8. The system of claim 7, wherein, the second electronic device is specifically configured to: in a case where it is determined according to the stop screen recording instruction that the mirror control function is to be stopped and the highlight function is to be stopped, stop recording the video data and the audio data corresponding to the screen recording function; send the saved video file information corresponding to the screen recording function to the first electronic device.

9. A cross-device screen recording method, characterized in that, The method applied to a first electronic device, the method comprises: the first electronic device displays a first interface of a first application, and generates a screen recording instruction in response to a first operation acting on the first interface; send the screen recording instruction to a second electronic device, wherein the screen recording instruction indicates to start a screen recording function, the screen recording function includes a highlight function and a mirror control function, the mirror control function means that the second electronic device sends the picture content and audio content in the video played as screen projection content to the first electronic device, the second electronic device is configured to record video data and audio data corresponding to the screen recording function, the video data includes the picture content, and the audio data includes the audio content; receive the video data and the audio data from the second electronic device; display a recording picture corresponding to the mirror control function based on the video data and the audio data; in a case where it is determined according to video file information corresponding to the screen recording function that recording is successful, display a video file corresponding to the highlight function.

10. The method of claim 9, wherein, After receiving the video data and the audio data from the second electronic device, before displaying the recording picture corresponding to the mirror control function based on the video data and the audio data, the method further comprises: write the audio data and the video data into the tail of a second cache queue; send a receiving success information to the second electronic device, wherein the second electronic device is configured to delete the audio data and the video data located in a first cache queue according to the receiving success information.

11. The method of claim 10, wherein, The method of displaying the recording picture corresponding to the mirror control function based on the video data and the audio data comprises: in a case where there is data in the second cache queue, take out the audio data and the video data located at the head of the second cache queue; synchronously process the audio data and the video data at the head of the second cache queue to obtain a recording picture, and display the recording picture.

12. The method of claim 11, wherein, The method further comprises: in a case where it is necessary to save the video data and the audio data, save the audio data and the video data into a synthesis module.

13. The method according to any one of claims 9-12, characterized in that, The method further comprises: generate a stop screen recording instruction in response to a second operation acting on the first interface; The second electronic device is configured to return saved video file information to the first electronic device according to the stop screen recording instruction.

14. A cross-device screen recording method, comprising: The method applied to the second electronic device comprises: receiving a screen recording instruction from the first electronic device, wherein the screen recording instruction indicates to start a screen recording function, and the screen recording function comprises a highlight function and a mirror control function; obtaining video data and audio data according to the screen recording instruction, wherein the mirror control function refers to that the second electronic device sends picture content and audio content of a video played as screen projection content to the first electronic device; sending the video data and the audio data to the first electronic device, wherein the video data comprises the picture content, and the audio data comprises the audio content, and the first electronic device is configured to display a recording picture corresponding to the mirror control function based on the audio data and the video data, and the first electronic device is further configured to display a video file corresponding to the highlight function when it is determined that recording is successful according to video file information corresponding to the screen recording function.

15. The method of claim 14, wherein, The method further comprises: in a case where it is determined that the screen recording function is started, obtaining the audio data and the video data by calling an interface of a recording module through a highlight module.

16. The method of claim 15, wherein, The method further comprises: saving the obtained audio data and video data to a tail of a first cache queue; in a case where there is data in the first cache queue, taking out audio data and video data located at a head of the first cache queue through a sending thread; and sending the audio data and the video data located at the head of the first cache queue to a screen projection module of the first electronic device.

17. The method of claim 16, wherein, The method further comprises: in a case where receiving success information is received from the first electronic device, deleting the audio data and the video data located at the head of the first cache queue.

18. The method according to any one of claims 14-17, characterized by, The method further comprises: receiving a stop screen recording instruction from the first electronic device, wherein the stop screen recording instruction indicates to stop the screen recording function; stopping the screen recording function according to the stop screen recording instruction.

19. The method of claim 18, wherein, The method further comprises: in a case where it is determined to stop the mirror control function and the highlight function according to the stop screen recording instruction, stopping recording of the video data and the audio data corresponding to the screen recording function.

20. The method of claim 19, wherein, The method further comprises: in a case where it is determined not to stop the mirror control function and to stop the highlight function according to the stop screen recording instruction, stopping saving of the video data and the audio data corresponding to the screen recording function; and sending saved video file information corresponding to the screen recording function to the first electronic device.

21. An electronic device, comprising: The electronic device comprises one or more processors; a memory; wherein the memory is coupled with the one or more processors, the memory is configured to store computer program codes, the computer program codes comprise computer instructions, and the one or more processors invoke the computer instructions to enable the electronic device to perform the method in any one of claims 9-20.

22. A chip system, characterized by The chip system is applied to an electronic device, and the chip system comprises one or more processors, and the processor is configured to invoke computer instructions to enable the electronic device to perform the method in any one of claims 9-20.

23. A computer program product comprising instructions, wherein: When the computer program product runs on the electronic device, the electronic device is enabled to perform the method in any one of claims 9-20.

24. A computer-readable storage medium for storing instructions, the computer-readable storage medium comprising: When the instructions run on the electronic device, the electronic device is enabled to perform the method in any one of claims 9-20.

Images