Screen recording method and electronic equipment

Abstract

The invention provides a screen recording method and electronic equipment, and the method comprises the steps: displaying a first interface of a first application, the first interface comprises a first region and a second region, the first region is a video playing region, and the second region is a non-video playing region; in response to a first click operation of the recording start button, recording display content corresponding to the first area through the video note application; in response to a first click operation of a recording ending button, ending recording to obtain a first video; in response to the first user operation, a second interface of the first application is displayed, and the second interface does not contain the video playing area; in response to a second click operation of the recording start button, recording the display content of the whole interface of the second interface through the video note application; and in response to a second click operation of the recording ending button, ending recording to obtain a second video. Screen recording is carried out based on the currently displayed content of the display screen, and the user operation cost problem of the regional recording function is improved.

Description

Screen recording method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on November 17, 2022, with application number 202211442149.6 and application name “A Screen Recording Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computers, and in particular to a screen recording method and electronic device. Background Art

[0003] With the development of digital media, screen recording function has emerged to record the content or operations displayed on electronic devices for sharing and preservation.

[0004] The screen recording feature includes region recording, specifically recording a user-selected area by dragging. However, this method relies on the user dragging to select the recording area, which is costly. For example, the time required to drag and define a recording area is high.

[0005] Therefore, how to provide a screen recording method that improves the user operation cost of the regional recording function has become an important research topic in the technical field.

[0006] Summary of the Invention

[0007] The present invention provides a screen recording method, which records the screen based on the content currently displayed on the display screen, thereby improving the user operation cost problem of the regional recording function.

[0008] In a first aspect, the present application provides a screen recording method, the method comprising: starting a first application, displaying a first interface of the first application, the first interface comprising a first area and a second area, the first area being a video playback area, and the second area being a non-video playback area; in response to a user operation of calling out a function bar of a video note application, displaying the function bar, the function bar comprising a start recording button, the video note application comprising a screen recording function; in response to a first click operation on the start recording button, recording the display content corresponding to the first area through the video note application, and displaying an end recording button in the function bar, while not displaying the start recording button; in response to a first click operation on the end recording button, ending the recording, obtaining a first video, and displaying the end recording button in the function bar The start recording button is displayed, and the end recording button is not displayed. The first video includes the display content of the first area and does not include the display content of the second area; in response to the first user operation, the second interface of the first application is displayed, and the second interface does not include the video playback area; in response to the second click operation of the start recording button, the display content of the entire interface of the second interface is recorded through the video note application, and the end recording button is displayed in the function bar, and the start recording button is not displayed; in response to the second click operation of the end recording button, the recording is ended to obtain the second video, and the start recording button is displayed in the function bar, and the end recording button is not displayed. The second video includes all the display content of the second interface.

[0009] In this application, the first interface and the second interface are both the entire interface corresponding to the first application. Exemplarily, the first application supports video playback, and the first area in the first interface is one of the video playback areas for playing videos through the first application.

[0010] For example, the first application is an application that the preset video note application supports recording, and the view corresponding to the first area is an area corresponding to a view belonging to the first application that the preset video note application supports recording. For example, a user can customize a desired list of applications and views that the video note application supports and / or does not support recording.

[0011] Exemplarily, the display content of the first area includes video content and may also include barrage content, etc.

[0012] In some other screen recording methods, when recording the screen, the default is to record the display content of the entire display area of ​​the display screen, and regional recording is not supported. The recorded screen data is likely to contain images that the user does not want to record, resulting in a large amount of screen recording data, which brings certain storage pressure to electronic devices and other problems.

[0013] However, through the method provided by this application, when a screen recording instruction is received and the interface currently displayed on the display screen includes the above-mentioned first area (video playback area), the display content corresponding to the first area is recorded instead of full-screen recording. On the one hand, it meets the user's video area recording needs and improves the problem of large data volume and high storage pressure of screen recording data. On the other hand, if the interface currently displayed on the display screen is a second interface that does not include a video playback area, the entire interface of the second interface is recorded. That is, when the user's recording requirement is not for video area recording, the entire interface of the currently displayed first application (second interface) is recorded to meet the user's different recording needs in different scenarios.

[0014] In some other screen recording methods, in the area recording scenario, the user is required to drag and select the display area by himself, and obtain the screen recording data in the form of screenshots during recording. The operation cost of recording area selection is high, and a large amount of area screenshot data will be generated during the recording process, which will take up a large amount of memory space.

[0015] However, through the method provided by this application, when a screen recording instruction is received and the display screen currently displays the first interface containing the above-mentioned first area, the display content corresponding to the first area will be automatically recorded. If the display screen currently displays the second interface that does not contain the video playback area, the display content of the entire second interface will be automatically recorded. The user does not need to drag and select the display area, which improves the problem of high user operation costs. In addition, the method provided by this application calls the screen recording application in the electronic device to record the display content of the first area during screen recording, rather than obtaining the screen recording data by screenshot, which improves the problem of memory space occupied by the large amount of regional screenshot data generated during the screen recording process.

[0016] In this application, the descriptions of the click operations of the start recording button include the first click operation, the second click operation, the third click operation, the fourth click operation, etc., which are for the convenience of distinguishing this click operation in the current scenario from another click operation in another scenario, rather than indicating that there are different click methods or trigger methods for the start recording button.

[0017] Exemplarily, the user operation of calling up the function bar of the video note application is a preset shortcut key operation. For example, the shortcut key operation can be a combination of physical keys or a virtual key (such as a three-finger swipe down).

[0018] In some descriptions of this application, the start recording button is also called the start recording icon, and the end recording button is also called the end recording icon.

[0019] In one possible implementation, after the recording is ended, the method further includes: in response to a second user operation, displaying the second interface and the interface of the second application in a split-screen display, the interface of the second application does not include a video playback area; in response to a third click operation of the start recording button, recording the display content of the entire interface of the second interface through the video note application, and displaying the end recording button in the function bar, and not displaying the start recording button; in response to the third click operation of the end recording button, ending the recording, obtaining a third video, and displaying the start recording button in the function bar, and not displaying the end recording button, the third video includes all the display content of the second interface, and does not include the display content of the interface of the second application.

[0020] It is understandable that the end of recording described here can be after the end of recording in the above-mentioned 'end recording in response to the first click operation of the end recording button', or it can be after the end of recording in the above-mentioned 'end recording in response to the second click operation of the end recording button', and this article does not limit this.

[0021] Exemplarily, the second application may be an application that does not support video playback, and any interface of the second application does not include a video playback area. Exemplarily, the second application is not included in a preset list of applications that support recording of the video note application.

[0022] Through the method provided by this application, when a screen recording instruction is received and the display screen currently displays the second interface of the first application and the interface of the second application, the second interface of the first application is recorded, and the interface of the second application is not recorded. That is, full-screen recording is not blindly performed, further meeting the user's video area recording needs in different scenarios (such as the regional screen recording needs in the split-screen scenario), improving the problems of large data volume and high storage pressure of the screen recording data. In addition, the user does not need to drag and select the recording area, further improving the operating cost problem of the user's regional screen recording needs in different scenarios.

[0023] In one possible implementation, after the recording is ended, the method further includes: in response to a third user operation, displaying the interface of the second application and the interface of the third application in a split-screen display, and the interface of the third application does not include a video playback area; in response to a fourth click operation of the start recording button, recording the display content of the entire display area of ​​the display screen through the video note application, and displaying the end recording button in the function bar, and not displaying the start recording button; in response to the fourth click operation of the end recording button, ending the recording and obtaining a fourth video, and the fourth video includes the display content of the interface of the second application and the interface of the third application.

[0024] It is understandable that the end of recording described here can be after the end of recording in the above-mentioned 'end recording in response to the first click operation of the end recording button', or after the end of recording in the above-mentioned 'end recording in response to the second click operation of the end recording button', or after the end of recording in the above-mentioned 'end recording in response to the third click operation of the end recording button', and this document does not limit this.

[0025] Exemplarily, the third application is an application that does not support video playback, and any interface of the third application does not include a video playback area. Exemplarily, the third application is not included in the preset list of applications that support recording of the video note application.

[0026] Through the method provided in this application, when a screen recording instruction is received and the display screen currently displays the interface of the second application and the interface of the third application, the entire display area of ​​the display screen is recorded. That is, when the interfaces displayed on the display screen do not include the interfaces of applications supported by the video note application for recording, full-screen recording is performed, thereby further meeting the user's full-screen recording needs in the corresponding scenarios, and, without the user having to drag and select the recording area, further improving the operational cost problem of the user's full-screen recording needs in the corresponding scenarios.

[0027] In one possible implementation, the method further includes: in response to the first click operation of the start recording button, displaying the interface of the video note application in a floating window, and displaying a preview image in the preview area of ​​the interface of the video note application; in response to the first click operation of the end recording button, displaying a preview image and a play icon in the preview area, and playing the first video in response to the click operation of the play icon.

[0028] In one possible implementation, the method further includes: in response to the second click operation of the start recording button, displaying the interface of the video note application in a floating window, and displaying a preview image in the preview area of ​​the interface of the video note application; in response to the second click operation of the end recording button, displaying a preview image and a play icon in the preview area, and playing the first video in response to the click operation of the play icon.

[0029] In one possible implementation, the identifier of the first application is included in the application whitelist, the identifier information of the view corresponding to the first area is included in the view whitelist, the identifier information of the view corresponding to the second area is not included in the view whitelist, the identifier of the second application and the identifier of the third application are not included in the application whitelist, the application whitelist is used to record a list of identifier information of applications that the video note application supports recording, the view whitelist is used to record a list of identifier information of views that the video note application supports recording, and each view in the view whitelist comes from the corresponding application in the application whitelist.

[0030] Exemplarily, each application in the application whitelist is an application including a video playback function, and each view in the view whitelist is a view for playing video data.

[0031] For example, one or more applications in the application whitelist may be applications that the user customizes and selects as desired video note application support recording, and one or more views in the view whitelist may be views that the user customizes and selects as desired video note application support recording.

[0032] Therefore, the application identification information and view identification information of the desired recording area are pre-set based on the application whitelist and view whitelist, and the target recording area is determined based on the application whitelist, view whitelist, and the window information currently displayed on the display screen, rather than blindly recording in full screen, and the user does not need to manually drag and divide the recording area, thereby improving the problem of excessive screen recording data volume caused by blindly performing full-screen recording, and improving the user operation cost problem of the regional screen recording function, and can also meet the user's different screen recording needs in various different scenarios. For example, when the displayed page contains a video playback area, the user's screen recording need of recording the video playback area is met; when the displayed page does not contain a video playback area but contains the interface of the application desired to be recorded, the entire interface of the application is recorded to meet the regional screen recording need in the split-screen scenario; and when the displayed page does not contain a video playback area and does not contain the interface of the application desired to be recorded, the entire display content of the display area corresponding to the display screen is recorded to meet the full-screen recording need.

[0033] In the present application, the above-mentioned display of the first interface of the first application can be understood as the display screen currently only displays the first interface of the first application, or it can also be understood as the display screen currently only displays the first interface and the interface of the video note application. The above-mentioned display of the second interface of the first application can be understood as the display screen currently only displays the second interface and the interface of the above-mentioned video note application (including the above-mentioned function bar, video note document and other contents). In the present application, the above-mentioned display of the second interface and the interface of the second application in a split-screen display manner can be understood as the display screen currently displays but is not limited to the second interface and the interface of the second application. For example, it can also display the interface of one or more other applications that are not included in the application whitelist. The above-mentioned display of the interface of the second application and the interface of the third application in a split-screen display manner can be understood as the display screen currently displays but is not limited to the interface of the second application and the interface of the third application. For example, it can also display the interface of one or more other applications that are not included in the application whitelist.

[0034] In one possible implementation, if before the electronic device receives the above-mentioned first click operation through the video note application, the first interface of the above-mentioned first application displayed on the display screen also includes a third area, and the view identifier corresponding to the third area is included in the view whitelist, then the above-mentioned recording of the display content corresponding to the first area through the video note application may specifically include: recording the display content of the smallest rectangular area on the display screen that includes the first area and the third area.

[0035] In a possible implementation, after displaying the function bar and before recording the display content corresponding to the first area through the video note application, the method also includes: obtaining a first window information set, the first window information set is a set of window information of at least one window displayed on the display screen, and the first window information set includes window information of the first interface; obtaining a first set based on the first window information set and the application whitelist, each application in the first set is included in the first window information set and included in the application whitelist, and the first set includes the first application; obtaining the size information of the target view based on the first set, the view whitelist and the first window information set, determining that the size information of the target recording area is the size information of the target view, the target view is the view corresponding to the first application in the view whitelist, and the target view is the view corresponding to the first area; recording the display content corresponding to the first area through the video note application includes: recording the display content corresponding to the first area based on the size information of the target view recorded in the target recording area.

[0036] Specifically, in response to a user operation of calling out a function bar of the video note application, the video note application enables an accessibility service and continuously detects the window state based on the accessibility service. After the accessibility service detects a window state change triggered by the user operation of calling out the function bar of the video note application, the first window information set is obtained.

[0037] Exemplarily, the identifier corresponding to each element in the first set is the package name of the application. The view identifiers recorded in the view whitelist correspond one-to-one to the application identifiers recorded in the application whitelist.

[0038] Optionally, the above-mentioned acquisition of the size information of the target view based on the first set, the view whitelist and the first window information set, and determination of the size information of the target recording area as the size information of the target view can be specifically as follows: according to the address of the element in the first set, the size information of the reference view is acquired in sequence based on the first window information set and it is determined whether the acquired size information of the reference view is a valid value, the reference view is the view corresponding to the reference application in the view whitelist, and the reference application is the application corresponding to the address of the element in the first set; if the size information of the target view is acquired and the element address of the first application in the first set is greater than the element address of any target application, determination is made that the size information of the target recording area is the size information of the target view; the target application is included in the first set, the size information of the view corresponding to the target application is a valid value, and the target application is not the first application.

[0039] In other words, the size information of the target recording area is the size information corresponding to the last view ID among the M view IDs for which size information can be obtained (or the size information is a valid value). The M view IDs are, in order, the views in the view whitelist that correspond to the M application package names in the application package name set. For details, please refer to the detailed description of the relevant method steps in Figure 8B and will not be repeated here.

[0040] Exemplarily, if the size information of the reference view is obtained, which can also be understood as the size information of the reference view being a valid value, the size information of the target recording area is determined to be the size information of the reference view, and the size information of the reference view of the reference application corresponding to the next element address in the first set is sequentially traversed to obtain. If the size information of the reference view is determined to be a valid value, the size information of the target recording area is determined to be the size information of the current reference view. If the size information of the reference view cannot be obtained, the size information of the reference view of the reference application corresponding to the next element address is continuously traversed to obtain until the element address reaches the last address in the first set.

[0041] Optionally, the above-mentioned acquisition of the size information of the target view based on the first set, the view whitelist, and the first window information set, and determination of the size information of the target recording area as the size information of the target view can be specifically as follows: determining a first view set based on the first set and the view whitelist, wherein each view in the first view set is included in the view whitelist, and the application corresponding to each view in the first view set is included in the first set, and the first view set includes the first view corresponding to the first area; determining a second view set, wherein the second view set is a set of target views, and the target view is an element in the first view set whose size information is a valid value; when the second view set only includes the first view, determining the size information of the target recording area as the size information corresponding to the first view; or, when the second view set includes the first view, and the first view is the first or last element in the second view set, determining the size information of the target recording area as the size information of the first view. For details, please refer to the detailed description of the relevant method steps in Figure 8A, which will not be described in detail here.

[0042] In one possible implementation, after displaying the second interface of the first application and before recording the display content of the entire interface of the second interface through the video note application, the method also includes: obtaining a second window information set, the second window information set is a set of window information of at least one window displayed on the display screen, and the second window information set includes window information of the second interface; obtaining a second set based on the second window information set and the application whitelist, each application in the second set is included in the second window information set and included in the application whitelist, and the second set includes the first application; if the size information corresponding to the identifier of the target view is not obtained based on the second set, the view whitelist and the second window information set, then determining that the size information of the target recording area is the size information of the entire display area of ​​the second interface, and the target view is the view corresponding to the first application in the view whitelist; recording the display content of the entire interface of the second interface through the video note application includes: recording the display content of the entire interface of the second interface based on the size information of the entire display area of ​​the second interface recorded in the target recording area.

[0043] Exemplarily, the second user operation may specifically be a series of operation instructions corresponding to displaying the second interface and the interface of the second application in a split-screen display manner.

[0044] Specifically, after the video note application detects the window state change triggered by displaying the second interface and the interface of the second application in a split-screen display manner based on the accessibility service, it obtains the above-mentioned second window information set.

[0045] Optionally, if the size information corresponding to the identifier of the target view is not obtained based on the second set, the view whitelist, and the second window information set, then determining the size information of the target recording area to be the size information of the entire display area of ​​the second interface, and the target view is the view corresponding to the first application in the view whitelist, includes: according to the address of the element in the second set, obtaining the size information of the reference view based on the second window information set, the reference view is the view corresponding to the reference application in the view whitelist, and the reference application is the application corresponding to the address of the element in the first set; if the size information of the reference view is not obtained, determining the size information of the target recording area to be the size information of the entire display area of ​​the second interface. For details, please refer to the detailed description of the relevant method steps in Figure 8B, which will not be described in detail here.

[0046] That is to say, when traversing to obtain the size information of the view corresponding to each application in the second set, no valid size information can be obtained. Then, based on the inclusion of the first application and the application whitelist, the size information of the target recording area is determined to be the size information of the entire display area of ​​the second interface.

[0047] Optionally, a third view set is determined based on the second set and the view whitelist, each view in the first view set is included in the view whitelist, and the application corresponding to each view in the third view set is included in the second set; based on the fact that the second set only includes the first application, the third view set does not include views corresponding to any area in the second interface, and does not include views corresponding to any area of ​​the interface of the second application, the size information of the target recording area is determined to be the second size information corresponding to the entire display area of ​​the second interface.

[0048] In a possible implementation, each element in the application whitelist is identification information of an application including a video playback function, and each element in the view whitelist is identification information of a view used to play video data.

[0049] In the second aspect, the present application provides a screen recording method, which includes: starting a video note application, and in response to a user operation of calling out a function bar of the video note application, displaying the above-mentioned function bar including a start recording button; based on the user operation, displaying a target interface; when the target interface includes a video playback area, in response to a first click operation of the start recording button, recording the display content corresponding to the video playback area in the target interface; when the target interface does not include a video playback area and the application corresponding to the target interface is an application that supports video playback, in response to a second click operation of the start recording button, recording the display content of the entire interface of the target interface; when the target interface does not include a video playback area and the application corresponding to the target interface is not an application that supports video playback, in response to a click operation of the start recording button, recording the display content of the entire display area of ​​the display screen.

[0050] Specifically, in response to a click operation of the start recording button, the display content of the area corresponding to the size information of the target recording area is recorded. The target recording area is determined based on the following method: when the window state of the display screen changes, the window information set of the current window is obtained. After obtaining the window information set of the current window, the target recording area is determined based on the window information set, specifically including: obtaining a target set based on the window information set and the application whitelist, each application in the target set is included in the window information set and included in the application whitelist; obtaining a target view corresponding to a reference application (the reference application is any element in the target set) in the view whitelist based on the target set; when it is determined that the size information corresponding to the target view is obtained, the size information of the target recording area is determined to be the size information corresponding to the target view. When it is determined that the size information of the target view corresponding to each reference application in the target set cannot be obtained and the target set contains the target application, the size information of the target recording area is determined to be the size information corresponding to the interface of the target application. Alternatively, when the number of elements in the target set is 0, the size information of the target recording area is determined to be the size information of the entire display area of ​​the display screen. Exemplarily, when the function bar including the start recording button is displayed, the window state of the display screen changes, triggering acquisition of a window information set of the current window, and determining the target recording area based on the window information set.

[0051] Exemplarily, in the case of the aforementioned display target interface, the window state of the display screen changes, triggering the acquisition of the window information set of the current window, and determining the aforementioned target recording area based on the window information set. Exemplarily, the aforementioned display target interface can be the 'display of the first interface of the first application' described in the above-mentioned first aspect, or the 'display of the second interface of the first application' described in the above-mentioned first aspect, or the 'display of the second interface and the interface of the second application' described in the above-mentioned first aspect, or the 'display of the interface of the second application and the interface of the third application' described in the above-mentioned first aspect.

[0052] For explanations of names such as application whitelist, view whitelist, window information set, and target recording area, please refer to the relevant description of the first aspect and will not be elaborated here.

[0053] By adopting the method provided in the embodiment of the present application, the recording area is determined based on the specific content of the current display interface of the display screen, rather than blindly recording the full screen, and the user does not need to manually drag and divide the recording area, thereby improving the problem of excessive screen recording data volume caused by blindly performing full-screen recording, and improving the user operation cost problem of the regional screen recording function.

[0054] In addition, it can also meet users' different screen recording needs in a variety of different scenarios. For example, when the displayed target interface includes a video playback area, the display content of the video playback area is recorded to meet the regional screen recording needs in the video playback scenario; when the displayed target interface does not include a video playback area, but the displayed target interface is the interface of the application you want to record, the entire interface of the target interface is recorded to meet the regional screen recording needs in the split-screen scenario; and when the displayed target interface does not include a video playback area and does not include the interface of the application you want to record, the entire display content of the display area corresponding to the display screen is recorded to meet the full-screen recording needs.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors, a memory, and a display screen; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, the one or more processors calling the computer instructions to enable the electronic device to execute the method shown in the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect.

[0056] In a fourth aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions to enable the electronic device to execute the method shown in the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect.

[0057] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when run on an electronic device, enables the electronic device to execute the method shown in the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect.

[0058] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions, characterized in that when the instructions are run on an electronic device, the electronic device executes the method shown in the first aspect or any possible implementation of the first aspect or the second aspect or any possible implementation of the second aspect.

[0059] It is understood that the electronic device provided in the third aspect, the chip provided in the fourth aspect, the computer program product provided in the fifth aspect, and the computer storage medium provided in the sixth aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] FIG1 is a schematic diagram of a user interface for calling out a function bar related to screen recording in a video note application according to an embodiment of the present application;

[0061] FIG2 is a schematic diagram of a user interface related to starting screen recording in response to a click operation on a start screen recording icon of an electronic device provided by an embodiment of the present application;

[0062] FIG3 is a schematic diagram of a user interface related to ending screen recording in response to a click operation on an end screen recording icon of an electronic device provided by an embodiment of the present application;

[0063] FIG4 is a schematic diagram of a user interface for recording a screen using the screen recording method provided by the present application in a floating window scenario provided by an embodiment of the present application;

[0064] FIG5 is a schematic diagram of a user interface for recording a screen using the screen recording method provided by the present application in a split-screen scenario provided by an embodiment of the present application;

[0065] FIG6 is a schematic diagram of a user interface for recording a screen using the screen recording method provided by the present application in a split-screen scenario provided by an embodiment of the present application;

[0066] FIG7 is a timing flow chart of a screen recording method provided in an embodiment of the present application;

[0067] FIG8A is a flowchart of a method for implementing step S706 in FIG7 according to an embodiment of the present application;

[0068] FIG8B is another method flow chart of a specific implementation of step S706 in FIG7 provided in an embodiment of the present application;

[0069] FIG9A is a representation of size information of a recording area provided by an embodiment of the present application;

[0070] FIG9B is another representation of the size information of the recording area provided in an embodiment of the present application;

[0071] FIG10 is a schematic diagram of a user interface in a scenario where the number N of view IDs included in a view set is 2, provided by an embodiment of the present application;

[0072] FIG11 is a timing flow chart of another screen recording method provided in an embodiment of the present application;

[0073] FIG12 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application;

[0074] FIG13 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of this application clearer, this application will be further described below with reference to the accompanying drawings.

[0076] The terms "first" and "second" in the specification, claims, and drawings of this application are used only to distinguish different objects and are not used to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to the process, method, product, or device.

[0077] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It will be understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0078] In this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three and more than three, and "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0079] To facilitate understanding, some concepts related to the embodiments of the present application are described below by way of example for reference.

[0080] (1) Video note application

[0081] In this application, the video note application supports screen recording, and the video file obtained by the screen recording can be recorded in the corresponding note document. In addition to the screen recording function, the video note application can also support recording text and / or pictures in the note document, which is not limited in this article. In this application, the video note application can be a separate application (application, app) or a function integrated into the note application (such as a memo application), which is not limited in this article.

[0082] (2) View

[0083] A view is a class (or object) in interface programming design, representing the basic building block of the user interface. A view corresponds to a rectangular area on the display screen, which can be used for interface drawing and processing events such as touch events. It is the base class for building user interface components (such as buttons, text areas, etc.).

[0084] For example, text, controls, images, or media streams (e.g., video streams) can each correspond to an independent view, such as a text view (textview), a button view (button), or an image view (imageview). For example, as shown in user interface 1a in FIG1 , the video playback area 101 can correspond to an independent view, the video full-screen zoom icon 102 can correspond to an independent view, and the display area corresponding to the "comment area" in the comment area 103 can correspond to an independent view.

[0085] It is understandable that different areas on the display screen are demarcated by different views to display corresponding content, which is determined during the user interface design. For example, during the project development process, corresponding view areas are designed to display elements such as text, controls, or pictures. Each view will have a corresponding view identifier (view id) for identifying the view. In the user interface design, the content that you want to display in the view can be edited based on the view id.

[0086] Generally, a view can correspond to a single text, control, or image. A view can also correspond to a group of views consisting of multiple texts, and / or controls, and / or images. This document does not limit this.

[0087] Generally, the display area corresponding to a view is rectangular. The dimensions of a view (and therefore a rectangular display area) include the coordinates of its upper-left corner, width, and height. As you can see, the shape of a view's display area may vary based on platform services, and may also be other shapes such as a circle. This document does not limit this.

[0088] (3) Accessibility

[0089] Accessibility services are functions provided to help specific groups of people with accessibility needs. For example, accessibility services can include one or more of screen reading, on-demand reading, and Braille keyboard.

[0090] The accessibility service can receive some events (AccessibilityEvent) issued by the window management service (WindowManageService). These events are state changes related to the display content in the window, such as button clicks, input box content changes, focus changes, etc. Specifically, after the accessibility service in the electronic device is enabled, when the window management service detects a window state change (window state change), the window management service can send an AccessibilityEvent event to the accessibility service through the AccessibilityEvent callback function. In this application, sending messages between modules (such as the window management service and the accessibility service) refers to each module obtaining data through function calls, or transmitting data through inter-process communication.

[0091] The window state change mentioned above may refer to operations such as clicking, dragging, or updating any view displayed on the display screen. Generally, after receiving an AccessibilityEvent, the accessibility service can obtain the window information of the current display screen to provide accessibility services to the user based on the window information.

[0092] The above introduction to the concepts can be used in the following embodiments.

[0093] The advantages of the screen recording method provided in the embodiment of the present application are described in detail below in combination with other screen recording methods:

[0094] In some other screen recording methods, electronic devices default to recording the entire display area. This creates a high probability that the recorded data will contain images that the user did not intend to record, resulting in a large amount of recorded data and placing a certain amount of storage pressure on the electronic device. Alternatively, some video editing software can be used to edit the video data within a certain range, preserving the content of specific areas of the video. However, video editing software requires high mobile phone performance and is difficult to operate, making it unsuitable for large-scale use.

[0095] In view of this, an embodiment of the present application provides a method for selecting regional screen recording or full-screen recording based on the specific content currently displayed on the display screen, thereby improving the problem that the recorded screen data is likely to contain images that the user does not want to record and the amount of data obtained is large.

[0096] Exemplarily, using the screen recording method provided in the embodiment of the present application, an application whitelist and a view whitelist can be pre-set in the electronic device, wherein the application whitelist is used to record the preset applications that support video note application recording, the application whitelist can include the identifiers of the applications that support video note application recording, the view whitelist includes the views corresponding to the preset recording areas of each application in the application whitelist, and the view whitelist includes the identifiers (view ids) of the views corresponding to the preset recording areas. Optionally, each application in the application whitelist can set a separate view whitelist, such as the view whitelist of the target application, which can record the views corresponding to the preset recording areas in each interface of the target application. An application can include multiple interfaces. After the video note application is enabled, the window status is continuously detected based on the accessibility service. After detecting that the window status has changed, the window information set is obtained based on the accessibility service. If it is determined based on the window information set, the application whitelist and the view whitelist that there is a target view in the view currently displayed on the display screen, and the target view comes from an application in the application whitelist and the identifier of the target view is included in the view whitelist, then the electronic device determines to record the area corresponding to the target view through the video note application. If it is determined that there is no target view but there is a target window, and the target window is from an application in the application whitelist, the electronic device records the target window through the video note application. If it is determined that neither the above target window nor the above target view exists, the electronic device performs full-screen recording through the video note application.

[0097] Generally, when a user uses third-party screen recording software to record the screen, if the user's recording requirement is for a currently playing video, the user's desired recording area is the video's playback area, and the user's undesired recording area is other areas outside of the video's playback area. For example, as shown in user interface 1c in FIG2 , if the user's recording requirement is for a currently playing video, the user's desired recording area is video playback area 201, and the user's undesired recording area is other areas outside of video playback area 201, such as the detailed introduction, comment area, etc. If the user's recording requirement is not for the playing video screen, the user's desired recording area is the entire window corresponding to the application, or the user's desired recording area is the display area of ​​the entire display screen.

[0098] For example, in an embodiment of the present application, the application whitelist and view whitelist stored by the electronic device through the video note application can be determined according to specific needs. For example, each application identifier in the application whitelist is an identifier of an application that includes a video playback function, and each view identifier in the view whitelist is an identifier of a view corresponding to the video screen playback area in the application that includes a video playback function.

[0099] Based on this, in some scenarios, when the electronic device's display screen currently displays a video playback area (target view) from an application whitelist and whose view identifier is included in the view whitelist, and a screen recording start instruction is received, the electronic device will record the video playback area through the video note application. This effectively improves the problem of a high probability that the recorded screen data contains scenes that the user does not want to record, and the resulting large amount of recorded screen data.

[0100] In some scenarios, when the display screen of the electronic device currently displays an application interface (target window) that is not a video playback interface and whose view identifier is not included in the view whitelist, and a start screen recording instruction is received, the electronic device records the target window through the video note application. Also, in some scenarios, when the application interface currently displayed on the display screen of the electronic device is a non-video playback interface and the application interface is not from the application whitelist, and a start screen recording instruction is received, the electronic device records the display area of ​​the entire display screen through the video note application. Thus, while effectively improving the problem that the recorded screen data has a high probability of containing images that the user does not want to record and the amount of data of the recorded screen data is large, the user's screen recording needs in different scenarios are further met.

[0101] Some other screen recording methods support regional recording, specifically recording a user-selected display area. This method, on the one hand, has high user operation costs, such as the time required to drag and define a recording area. On the other hand, this regional recording method mostly uses regional screenshots and screenshot splicing to generate screen recording data. During the screen recording process, a large amount of regional screenshot data (also known as regional screenshot data) is generated, which requires a large amount of memory space and places high performance requirements on the electronic device.

[0102] By adopting the method provided in the embodiment of the present application, on the one hand, the electronic device determines the recording area through the video note application based on the current window information set, the pre-stored application whitelist, the view whitelist and the accessibility service, rather than performing regional screen recording according to the area defined by the user's dragging, thereby reducing the user's operating costs while satisfying the user's regional screen recording function.

[0103] On the other hand, after determining the recording area, the electronic device obtains the screen recording data through the video note application by screen recording, rather than generating the screen recording data by taking regional screenshots and splicing the screenshots, thereby improving the problem that the screen recording process occupies a large memory space and has high performance requirements for the electronic device.

[0104] The screen recording method provided in the embodiment of the present application is described in detail below in conjunction with the user interface provided in the embodiment of the present application.

[0105] In an embodiment of the present application, a video note-taking application is installed on an electronic device, wherein the electronic device provides a user with a preset shortcut key to access a function bar related to screen recording corresponding to the video note-taking application. The shortcut key can be a combination of one or more virtual keys and / or physical keys, which is not limited herein. The following describes the application of the screen recording method provided by the present application in conjunction with several scenarios.

[0106] Scenario 1:

[0107] Please refer to Figure 1. The electronic device is a mobile phone terminal. The electronic device runs a first application. For example, the first application is a video application. The electronic device displays the first interface of the first application in full-screen display mode through the display screen. For example, the first interface is the page for the first application to play the video as shown in the user interface 1a in Figure 1. In response to the shortcut key operation of calling up the function bar (in this application, the function bar is the function bar corresponding to the video note application), the electronic device displays the function bar 10 as shown in the user interface 1b in Figure 1. The function bar 10 includes a start screen recording icon 104. In some other possible implementations, the function bar 10 may also include a screenshot icon 105 and / or an edit icon 106. The function bar 10 can also be understood as a toolbar or a floating bar. It is understandable that in the background implementation of the electronic device, displaying the function bar 10 can be achieved by calling a control or component.

[0108] Optionally, in the above-mentioned user interface 1b, the electronic device displays the function bar 10 in response to the shortcut key operation for calling out the function bar. Specifically, the electronic device may start the video note application and display the above-mentioned function bar 10 through the display screen in response to the operation of the above-mentioned shortcut key when the video note application has not been started (it can also be understood that no page corresponding to the video note application is displayed on any layer in the display screen). Alternatively, it may be that the electronic device has already run the video note application in the background. For example, the electronic device starts running the video note application based on a click operation on the application icon of the video note application, but after starting running the video note application, it starts running the first application and displays the page of the first application on the top layer. In the process of running the video note application and the first application, the electronic device calls out the function bar 10 in response to the operation of the above-mentioned shortcut key.

[0109] Exemplarily, the electronic device displays the function bar 10 in response to the shortcut key operation for calling out the function bar, specifically including executing steps S701 to S705 as shown in FIG. 7 .

[0110] S701, start the video note application.

[0111] It is understandable that after receiving the shortcut key operation on the function bar corresponding to the video note application, the electronic device starts running the video note application and displays the above-mentioned function bar 10 on the display screen.

[0112] In an embodiment of the present application, corresponding to the software framework diagram shown in FIG13 , the video note application belongs to the application layer.

[0113] S702: The video note application enables accessibility services.

[0114] For example, the Google native service provides an enabling function enableAccessibilityServices() for accessibility services. The video note application can enable the system function of the accessibility service module in the electronic device by calling the enabling function enableAccessibilityServices(). The accessibility service module in the electronic device actively starts the accessibility service after receiving the call of the enabling function enableAccessibilityServices().

[0115] Optionally, after the video note application is started and enabled, the video note application calls the enable function to enable the accessibility service regardless of whether the accessibility service module is currently enabled.

[0116] Or, optionally, after the video note application is enabled and it is determined that the accessibility service module is not currently enabled, the enable function is called to enable the accessibility service. Exemplarily, the above-mentioned video note application enabling the accessibility service specifically includes: after starting the video note application, determining whether the accessibility service module is currently enabled, for example, an object of the accessibility service management class (AccessibilityManager) can be instantiated, and the isEnabled() method is called based on the instantiated object of AccessibilityManager to determine whether the accessibility service module is currently enabled. If not, the enable function is called to enable the accessibility service. If so, the video note application does not call the enable function to enable the accessibility service, that is, step S702 is not executed, and step S703 is directly executed.

[0117] It is understandable that after receiving the call request for the above-mentioned enable function, if the accessibility service module determines that the current accessibility service module is not enabled, the accessibility service module is enabled; if it is determined that the current accessibility service module is already enabled, the enabled state can be kept unchanged. The specific enabling method of the accessibility service module depends on the specific design of Google's native service, and this document does not limit this.

[0118] Exemplarily, enabling the accessibility service module specifically refers to enabling the system capabilities of the accessibility service class (AccessibilityService class) in the accessibility service module. The system capabilities of the AccessibilityService class include: receiving the AccessibilityEvent event issued by the window management service to indicate a change in window state (i.e., window state change); after receiving the AccessibilityEvent, the window information AccessibilityWindowInfo related to the AccessibilityEvent can be obtained based on the getwindows() method, and the view information AccessibilityNoteInfo in the window related to the AccessibilityEvent can be obtained based on the AccessibilityWindowInfo.getRoot() method.

[0119] In some other possible implementations, the electronic device may also execute the above step S702 (enabling accessibility services) after enabling the video note application and receiving the above shortcut key operation, that is, enabling the accessibility service after receiving the instruction of the above shortcut key operation, and not enabling the accessibility service before starting the video note application but receiving the above shortcut key operation.

[0120] S703, the video note application continuously detects the window status based on the accessibility service.

[0121] It should be noted that the video note application detects the window state through the accessibility service. Optionally, if the window state changes, the system will feedback to the video note application through the accessibility service, thereby detecting the window state. In some implementations, it can also be understood as continuously detecting the window state. Exemplarily, the video note application inherits the AccessibilityService class, that is, with AccessibilityService as the parent class, creates an AccessibilityService subclass, and the AccessibilityService subclass has the same system capabilities as the AccessibilityService parent class. Exemplarily, the AccessibilityService subclass receives an AccessibilityEvent sent by the window management service to indicate a change in the window state (that is, a window state change). Alternatively, the AccessibilityService subclass can detect that the callback function of the AccessibilityEvent is called to indicate a window state change, which is not limited in this document. In this application, after enabling the accessibility service, the accessibility service subclass performs the task of continuously detecting the window state.

[0122] In the embodiment of the present application, corresponding to the software framework diagram shown in FIG13 , the window management service belongs to the window management module in the application framework layer.

[0123] S704: After the video note application detects that the window state has changed, it obtains a window information set based on the accessibility service.

[0124] For example, after step S701, when the electronic device receives a click operation on the comment area 103, it displays the interface corresponding to the comment area 103. Since the click operation on the comment area 103 is a view click operation, that is, a window state change, the accessibility service subclass in the video note application can detect the window state change corresponding to the click operation on the comment area 103, thereby triggering the execution of step S704 to obtain the window information set of the current window.

[0125] For example, after the video note application is started, the accessibility service is enabled, and the accessibility service subclass can detect the window state. Because the video note application displays the function bar 10 in the user interface 1b shown in Figure 1 in response to the shortcut key operation, which is a type of window state change (the specific change is the view area corresponding to the newly added function bar 10 in the interface), the accessibility service subclass can detect the window state change related to the display of the function bar 10, thereby triggering the execution of the above step S704 to obtain the window information set of the current window.

[0126] In an embodiment of the present application, after the window state changes, the video note application can obtain the window information AccessibilityWindowInfo related to AccessibilityEvent based on the getWindows() method, and can obtain the view information AccessibilityNoteInfo contained in the AccessibilityWindowInfo based on the .getRoot() method.

[0127] In an embodiment of the present application, a window information set is a set of window information for one or more windows displayed on a display screen. Each element in the window information set is window information corresponding to a window displayed on the display screen. For example, the window information set can be an array of a list type. The window information for a window includes an application package name and information about one or more views. The views displayed by the window originate from the application corresponding to the application package name. The information about each view includes at least size information corresponding to the view ID of the view.

[0128] S705 , the video note module determines the size information of the target recording area based on the window information set, the application whitelist, and the view whitelist.

[0129] In the embodiment of the present application, the application whitelist is pre-set in the video note application and is used to record the package names of applications that the video note application supports recording. The view whitelist is pre-set in the video note application and is used to record the view IDs of views that the video note application supports recording, corresponding to the applications in the application whitelist.

[0130] For example, each application package name in the application whitelist is the identifier of an application that includes video playback functionality, and each view identifier in the view whitelist is the view ID of a view corresponding to the video playback area within the application that includes video playback functionality. Optionally, the video note application may also provide users with the ability to customize the data items included in the application whitelist and view whitelist, though this is not limited herein.

[0131] It is understandable that the package name of the application is public information and can be obtained on the official website of the software. The view ID of the view of the relevant interface in the application may be public or private. If the view ID is private, the view whitelist can be obtained based on communication, authorization, etc.

[0132] Exemplarily, the first association relationship between the application whitelist and the view whitelist is shown in Table 1 below. The identification information of the application supported by the video note application for recording includes application package name A, application package name B, and application package name C, the identification information of the view corresponding to application package name A supported by the video note application for recording includes view id a1 and view id a2, the identification information of the view corresponding to application package name B supported by the video note application for recording includes view id b1, and the identification information of the view corresponding to application package name C supported by the video note application for recording includes view id c1 and view id c2.

[0133] Table 1

[0134] For ease of description, any application package name in the application whitelist is collectively referred to as a reference application package name. In the embodiment of the present application, there may be one or more view ids corresponding to the reference application package name in the view whitelist, which is not limited herein.

[0135] The specific implementation of step S706 is described in detail below with reference to FIG8A . As shown in FIG8A , the video note module determines the size information of the target recording area based on the window information set, the application whitelist, and the view whitelist, including:

[0136] S801: Obtain a set of recordable application package names based on a window information set and an application whitelist.

[0137] In an embodiment of the present application, each element in the above application package name set is an application package name included in both the window information set and the application whitelist.

[0138] Exemplarily, S801 specifically includes: obtaining the view information AccessibilityNoteInfo of each window information in the window information set one by one based on the .getRoot() method. For example, for ease of description below, the AccessibilityNoteInfo corresponding to each window information in the window information set is collectively referred to as Root. Furthermore, calling the .getPackageName() method through the Root to obtain the application package name 1 contained in the Root, where the application package name 1 is the application package name contained in the window information corresponding to the Root, and then determining whether the application package name 1 is included in the application whitelist. If so, the application package name 1 is included as an element in the above-mentioned application package name set. Otherwise, the application package name 1 is not included as an element in the application package name set, thereby obtaining the above-mentioned application package name set.

[0139] S802: Determine whether the number M of elements included in the application package name set is greater than 0.

[0140] If M is greater than 0, the video note module executes step S803; if M is equal to 0, the video note module executes step S811.

[0141] In the embodiment of the present application, if M is greater than 0, step S803 is executed; if M is equal to 0, step S811 is executed.

[0142] S803: Acquire a first view set based on the window information set, the application package name set, and a first association relationship between the application whitelist and the view whitelist.

[0143] Each element in the first view set is a view identifier, wherein each view identifier in the first view set is included in the view whitelist, and the application package name corresponding to the view identifier is included in the above application package name set.

[0144] For ease of description, each element in the application package name set is referred to as a reference application package name below.

[0145] Exemplarily, S803 specifically includes: based on the first association relationship, sequentially obtaining one or more view IDs corresponding to each reference application package name in the view whitelist, and using the obtained one or more view IDs as the first view set. For example, if the application package name set includes application package name A and application package name B, then the view IDs corresponding to application package name A in the view whitelist are sequentially obtained, specifically including view ID a1 and view ID a2, and the view ID corresponding to application package name B in the view whitelist is sequentially obtained, specifically including view ID b1, so that the first view set includes view ID a1, view ID a2, and view ID b1.

[0146] S804: Determine whether the first view set is not empty.

[0147] Exemplarily, determining whether the first view set is not empty specifically includes: determining whether the number of elements corresponding to the first view set is greater than 0; if so, the first view set is not empty, otherwise.

[0148] Exemplarily, determining whether the first view set is not empty may further include determining whether the first view set contains an element with a valid value, that is, an element with a value that is not an invalid value (e.g., an invalid value such as null or undefined), but rather contains size information representing a region of a view. If so, the first view set is not empty; otherwise,

[0149] In the embodiment of the present application, if it is determined that the first view set is empty, step S810 is executed.

[0150] If it is determined that the first view set is not empty, the video note application may have the following three optional execution schemes:

[0151] Optionally, if it is determined that the first view set is not empty, step S805 is executed.

[0152] Or, optionally, if it is determined that the first view set is not empty, step S806 is executed.

[0153] Alternatively, if it is determined that the first view set is not empty, step S807 is executed.

[0154] S805, taking each view in the first view set as the current view in turn, and determining whether the size information corresponding to the current view is a valid value; until the size information corresponding to the current view is determined to be a valid value, then determining the size information of the target recording area to be the size information corresponding to the current view; if the size information corresponding to each current view is an invalid value, executing step S810.

[0155] It is understandable that in the above step S805, if there are one or more view identifiers whose size information is not an invalid value in the first view set, then the size information of the above target recording area is the first view identifier among the one or more view identifiers whose size information is not an invalid value.

[0156] Exemplarily, the data type of the first view set is a List queue type, and each element (view element) from the head to the tail of the queue is traversed in sequence as the current view, and it is determined whether the size information of the current view is a valid value. If so, it is determined that the size information of the target recording area is the size information corresponding to the current view, and the traversal of the next element of the current view in the first view set is no longer continued.

[0157] For example, the current view is recorded as view A. Step S805 may specifically include the following: 1) Passing view A's identifier (i.e., view id) as an input parameter, and calling the .findAccessibilityNotesByViewId(view id) method via the target Root to determine whether the window information set contains size information corresponding to view A. The target Root is the AccessibilityNoteInfo of the window information of the application corresponding to view A. 2) If the size information obtained for view A is valid (i.e., not null), use this size information as the size information of the target recording area, and stop traversing the next view element in the first view set. 3) If the size information obtained for view A is invalid (e.g., null or undefined), use the next view element of view A in the first view set as the current view, and determine whether the size information of the current view is valid. This process continues until the size information corresponding to the current view is valid, then determine the size information of the target recording area as the size information corresponding to the current view, and so on. 4) If the size information corresponding to each current view is invalid, execute step S810.

[0158] In an embodiment of the present application, if the first view set is an ordered set, such as a list set, the order of traversing the elements in the first view set is the same as the address order of the elements in the set. For example, the first element, the second element, and so on in the first view set are traversed in sequence.

[0159] S806, taking each view in the first view set as the current view in turn, and determining whether the size information corresponding to the current view is a valid value. If so, determining that the size information of the target recording area is the size information corresponding to the current view; if the size information corresponding to each current view is an invalid value, executing step S810.

[0160] It is understandable that in the above step S806, if there are one or more view identifiers whose size information is not an invalid value in the first view set, then the size information of the above target recording area is the last view identifier among the one or more view identifiers whose size information is not an invalid value.

[0161] Exemplarily, the current view is recorded as view A, and the above step S806 may specifically include: 1) taking the identifier of view A as an input parameter, and calling the .findAccessibilityNotesByViewId(view id) method through the target Root to determine whether the size information corresponding to view A is recorded in the above window information set. 2) If the size information of view A obtained is a valid value (that is, not null), then the size information is used as the size information of the target recording area, and the next view element of view A in the first view set is used as the current view to determine whether the size information of the current view is a valid value. 3) If the size information of view A obtained is an invalid value (for example, null or undefined), then the next view element of view A in the first view set is used as the current view to determine whether the size information of the current view is a valid value, and so on. 4) If the size information corresponding to each current view is an invalid value, execute step S810.

[0162] S807 , taking views in the first view set as current views in sequence, and determining whether size information corresponding to the current views is a valid value; if so, determining that the current view is an element in the second view set.

[0163] It is understandable that in the above step S807, if there are one or more view identifiers whose size information is not an invalid value in the first view set, then the above second view set is a set consisting of the one or more view identifiers whose size information is not an invalid value.

[0164] Exemplarily, the current view is recorded as view A. The above S807 may specifically include: 1) taking the identifier of view A as an input parameter, and calling the .findAccessibilityNotesByViewId(view id) method through the target Root to determine whether the size information corresponding to view A is recorded in the above window information set. 2) If the size information of view A obtained is a valid value (that is, not null), view A is taken as an element in the second view set, and the next view element of view A in the first view set is taken as the current view to determine whether the size information corresponding to the current view is a valid value, and so on.

[0165] In an embodiment of the present application, each element in the second view set is a view identifier view id. In some other possible implementations, each element in the second view set may also be size information corresponding to the view identifier, or each element in the second view set may also be an element that records the association between the view identifier and the size information corresponding to the view identifier. This document does not limit this.

[0166] Exemplarily, in response to the above-mentioned shortcut key operation, the video note application displays the function bar 10 of the user interface 1b (first interface) shown in Figure 1, and detects a window state change based on the function bar 10 displayed in the user interface 1b, and obtains the window information set corresponding to the user interface 1b. Based on the window information set, the application whitelist, and the view whitelist, it is determined that the package name identifier of the above-mentioned first application (video application) is included in the application whitelist, that is, the above-mentioned application package name set contains the package name identifier of the first application, and the view where the playback area 201 shown in the first interface (user interface 1b) is located is included in the view whitelist, that is, the above-mentioned second view set contains the view identifier of the playback area 201. Then, the electronic device determines that the above-mentioned target recording area is the playback area 201, and stores the size information of the target recording area. Specifically, the size information of the playback area 201 is obtained based on the above-mentioned .findAccessibilityNotesByViewId(view id) method.

[0167] S808: Determine whether the number N of elements included in the second view set is greater than 0.

[0168] In the embodiment of the present application, when N is greater than 0, step S809 is executed; when N is equal to 0, step S810 is executed.

[0169] S809: Determine size information of the target recording area based on the second view set.

[0170] In the embodiment of the present application, depending on the number N of elements included in the view set, there are the following scenarios and implementation methods for determining the size information of the target recording area based on the second view set in step S809.

[0171] Method 1:

[0172] N is 1. That is, the second view set records only one view A, and the size information corresponding to the view A (eg, view id A) is a valid value, and the size information of the target recording area is determined to be the size information of the view id A.

[0173] Method 2:

[0174] If N is greater than or equal to 2, that is, the second view set contains at least two view identifiers, the target recording area is the area corresponding to the last element in the second view set. For example, if the view set is a List type of data and the last element in the view set is view identifier A and its size information, the target recording area is the area corresponding to the size information of view identifier A.

[0175] In another possible implementation, N is greater than or equal to 2, and the target recording area may also be the area corresponding to the first element in the view set, which is not limited in this document.

[0176] Method 3:

[0177] Optionally, the representation of the size information of the view identifier can be in the form of representation A and representation B, where, as shown in FIG9A , in representation A, the size information of the view identifier is: the coordinates of the upper left corner of the rectangle (including horizontal and vertical coordinates) 901, the width of the rectangle 902, and the height of the rectangle 903. It is understandable that the origin of the horizontal and vertical coordinates in representation A or representation B can be a point on any display area on the display screen, and this document does not limit this. As shown in FIG9B , in representation B, the size information of the view identifier is: the vertical coordinate 904 of the upper edge of the rectangle, the vertical coordinate 905 of the lower edge of the rectangle, the horizontal coordinate 906 of the left edge of the rectangle, and the horizontal coordinate 907 of the right edge of the rectangle.

[0178] In method 3, N is greater than or equal to 2, and step S805 determines the target recording area based on the second view set, specifically including: based on at least two views included in the second view set, determining a minimum rectangular area as the size information of the above-mentioned target recording area, and the minimum rectangular area is the minimum rectangular area including each view of the at least two views.

[0179] Illustratively, if the representation format of the size information of the view identifiers recorded in the view set is the representation format A, the representation format of the size information of each view identifier in the view set is converted to representation format B. Then, based on the converted ordinates of at least two upper rectangular edges, the ordinates of at least two lower rectangular edges, the abscissas of at least two left rectangular edges, and the abscissas of at least two right rectangular edges, the ordinate of the upper largest rectangular edge, the ordinate of the lower largest rectangular edge, the abscissa of the left largest rectangular edge, and the abscissa of the right largest rectangular edge are determined as the size information of the target recording area.

[0180] Exemplarily, if the representation form of the size information of the view identifier recorded in the above-mentioned view set is the above-mentioned representation form B, it is not necessary to convert the representation form of the size information of each view identifier in the view set into the representation form B, but directly based on the vertical coordinates of at least two rectangular upper edges, the vertical coordinates of at least two rectangular lower edges, the horizontal coordinates of at least two rectangular left edges, and the horizontal coordinate information of at least two rectangular right edges recorded in the view set, determine that the vertical coordinate of the largest rectangular upper edge, the vertical coordinate of the largest rectangular lower edge, the horizontal coordinate of the largest rectangular left edge, and the horizontal coordinate of the largest rectangular right edge are the size information of the above-mentioned target recording area.

[0181] Method 4:

[0182] N is greater than or equal to 2, and the video note module can determine a target recording area in at least two recordable view size information based on the user's click operation.

[0183] Exemplarily, N is equal to 2, as shown in Figure 10, the view set includes view id1 and its size information, view id2 and its size information, which correspond to area 1001 and area 1002 in Figure 10 respectively. The electronic device can output prompt information instructing the user to select the area to be recorded, such as highlighting area 1001 and area 1002, and popping up a window to inform the user that "there are two recordable areas, please click on one of the areas to record", so that the above-mentioned target recording area can be determined based on user input. For example, after receiving the click coordinates input by the user, it is determined whether the click coordinates are in area 1001 or area 1002. If the click coordinates are in area 1001 but not in area 1002, the target recording area is determined to be area 1001. If the click coordinates are in area 1002 but not in area 1001, the target recording area is determined to be area 1002. It is understood that when pre-storing the view whitelist, view identifiers for each view ID that do not overlap can be pre-stored, thereby avoiding the problem of the click coordinates being included in both area 1001 and area 1002. In another possible implementation, areas corresponding to two or more views can be selected as target recording areas based on the user's click operation, which is not limited herein.

[0184] S810: Determine size information of a target recording area based on the application package name set.

[0185] Depending on the number M of elements included in the application package name set, the above-mentioned determination of the target recording area based on the application package name set may have the following different scenarios and processing methods.

[0186] Method 5:

[0187] M is equal to 1, that is, the application package name set includes only one application package name, for example, application package name A.

[0188] Specifically, determining the size of the target recording area based on the application package name set is as follows: determining the display area occupied by the application corresponding to application package name A on the display screen as the target recording area. Exemplarily, the window size of the entire interface corresponding to the application is obtained based on the application package name and the GetClientRect() method.

[0189] For example, in some scenarios, the electronic device is running a video note application, a first application, and a second application. The application package name of the first application is included in the above-mentioned application whitelist, the application package name of the second application is not included in the above-mentioned application whitelist, and the second user interface displayed by the first application through the display screen of the electronic device does not contain any view included in the view whitelist. Then, the video note application determines that the display area occupied by the first application on the display screen is the above-mentioned target recording area.

[0190] Method 6:

[0191] M is greater than or equal to 2, that is, the application package name set includes at least two application package names.

[0192] Optionally, the determining of the size information of the target recording area based on the application package name set may be: determining the target recording area to be the entire display area of ​​the display screen of the electronic device.

[0193] Optionally, the above-mentioned determination of the target recording area based on the application package name set can also be specifically as follows: determining that the target recording area includes the display area occupied by the application corresponding to each application package name in the application package name set on the display screen, and the target recording area is a continuous and uninterrupted area (it can also be understood that there is no truncation area).

[0194] Exemplarily, in the display area occupied on the display screen by the applications corresponding to at least two application package names included in the application package name set, the value with the largest vertical coordinate is used as the vertical coordinate of the upper edge of the target recording area, the value with the smallest vertical coordinate is used as the vertical coordinate of the lower edge of the target recording area, the value with the smallest horizontal coordinate is used as the vertical coordinate of the left edge of the target recording area, and the value with the largest horizontal coordinate is used as the vertical coordinate of the right edge of the target recording area.

[0195] Optionally, the above-mentioned determination of the target recording area based on the application package name set can also be specifically as follows: receiving a user click operation, determining the target application package name in the application package name set based on the user click operation, determining the display area occupied on the display screen and containing the coordinate information corresponding to the user click operation, and using the display area occupied on the display screen by the application corresponding to the target application package name as the above-mentioned target recording area.

[0196] S811: Determine that the target recording area is the entire display area of ​​the display screen of the electronic device.

[0197] In the embodiment of the present application, if M is equal to 0, the target recording area is determined to be the entire display area of ​​the display screen.

[0198] Exemplarily, when it is determined that M is equal to 0, the electronic device may further output a first prompt message to prompt that there is no view that can be recorded separately and the recording area has been set to full screen.

[0199] For example, in some scenarios, the electronic device is running a video note application, a second application, and a third application, and the application package name of the second application and the application package name of the third application are not included in the above application whitelist. The video note application determines that the above target recording area is the entire display area of ​​the display screen.

[0200] It should be noted that the size information corresponding to the current view is the size information of the current view.

[0201] In another possible implementation, after determining that the first application is an application in the application whitelist, all views in the view whitelist can be traversed (optionally, all views in the view whitelist of the first application can be traversed), that is, the size information of each view in the view whitelist is obtained. If the view in the view whitelist is currently being displayed, the size information of the view can be obtained, that is, the size information of the view is a valid value (optionally, if the size information is not 0, it is a valid value); if the view in the view whitelist is not currently being displayed, the size information of the view cannot be obtained, that is, the size information of the view is an invalid value (optionally, if the size information is 0, it is an invalid value).

[0202] If the size information of the current view is obtained, that is, the size information corresponding to the current view is a valid value, the size information of the target recording area is determined to be the size information corresponding to the current view; if the size information of the current view is not obtained, that is, the size information corresponding to the current view is an invalid value, the size information of the next view is obtained. It should also be noted that if the size information of multiple views is obtained, the size information of the target recording area is the size information of the most recently obtained view. That is, with respect to the size information of the target recording area, the size information of the view obtained later will overwrite the size information of the view obtained earlier.

[0203] It should also be noted that an application can contain multiple interfaces, and an interface can contain multiple views. For an interface of an application, only one view can be added to the view whitelist. In this way, when displaying a certain interface of the application, only the size information of the view contained in the currently displayed interface can be obtained. If there is only one view in the view whitelist for this interface, the area corresponding to the view in the view whitelist in the interface can be used as the target recording area.

[0204] Optionally, after step S802, the size information of the target recording area may be determined based on the step method flow shown in FIG8B . Specifically, as shown in FIG8B , after step S802, if it is determined that M is greater than 0, step S812 is executed; if it is determined that M is equal to 0, step S811 is executed.

[0205] Among them, S812 specifically includes: taking the application package names in the application package name set as the current application in turn, and determining whether the size information of the reference view corresponding to the current application is a valid value; if so, determining the size information of the target recording area as the size information of the reference view; if the size information of the reference view corresponding to each application package name in the application package name set is an invalid value, executing step S810.

[0206] For example, the view identifiers recorded in the view whitelist correspond one-to-one with the application identifiers recorded in the application whitelist. That is, each application in the application whitelist corresponds to only one view supported for recording by the video note application. For example, if the application whitelist includes application package A, the number of view IDs corresponding to application package A in the view whitelist is 1. For example, the view ID corresponding to application package A in the view whitelist is only view ID a1.

[0207] Based on the one-to-one correspondence between the view identifiers recorded in the view whitelist and the application identifiers recorded in the application whitelist, the video note application does not need to obtain the above-mentioned first video set, but directly traverses the address order of the elements in the application package name set to obtain the size information of the reference view corresponding to the corresponding application package name.

[0208] Exemplarily, the above M is equal to 2, the first element in the first application package name set is application package name A, the second element is application package name B, the view id corresponding to application package name A in the view whitelist is only view id a1, and the number of view ids corresponding to application package name B is only view id b1. According to the address of the element in the application package name collection, based on the .findAccessibilityNotesByViewId(view id) method, first obtain the size information of view id a1 corresponding to application package name A. If, when obtaining the size information of view id a1 corresponding to application package name A, the size information of the view corresponding to view id a1 can be obtained (that is, the obtained size information of view id a1 is a valid value), then assign the size information of view id a1 to the above-mentioned target recording area, and continue to traverse according to the address of the element in the application package name collection to obtain the size information of view id b1 corresponding to application package name B. If the size information of view id b1 can be obtained (that is, the size information of view id b1 is a valid value), then update the target recording area again, specifically by assigning the size information of view id b1 to the target recording area. If the size information for view ID a1 corresponding to application package name A cannot be obtained, the size information for view ID b1 corresponding to application package name B is further traversed according to the address of the element in the application package name set. The target recording area is determined based on whether the size information for view ID b1 is valid. If valid size information cannot be obtained by traversing according to the address of the element in the application package name set and obtaining the size information for the view corresponding to each application package name in the application package name set, step S811 above is executed.

[0209] That is to say, the size information of the above-mentioned target recording area is the size information corresponding to the last view id whose size information can be obtained (or the size information is a valid value) among the M view ids, and the M view ids are respectively the views in the view whitelist that correspond to the M application package names in the application package name set.

[0210] The following describes how to execute the screen recording method provided in an embodiment of the present application when an electronic device receives a screen recording start instruction based on a video note application.

[0211] For example, please refer to the user interface 1b in Figure 2. In response to the click operation of the start screen recording icon 104, and in the corresponding window information in the user interface 1b, the package name of the video application is included in the application whitelist, and the view where the playback area 201 shown in the first interface (user interface 1b or user interface 1c) is located is included in the view whitelist, the electronic device determines that the above-mentioned target recording area is the playback area 201.

[0212] After determining that the target recording area is the playback area 201, the electronic device starts recording the display content corresponding to the playback area 201, and displays the user interface 1c in Figure 2, wherein the end screen recording icon 202 is displayed in the function bar 10, and the start screen recording icon 104 is not displayed; and a new note document 203 for recording the screen recording data is created, and the note document 203 is displayed as a floating window in the display area in the lower right corner of the display screen, and the preview area 204 in the editing area of ​​the note document 203 displays a first preview image 2041. It can be understood that in the background implementation of the electronic device, the start screen recording icon 104 and the end screen recording icon 202 are implemented by the same control, but the corresponding display icons in the control will change accordingly at different recording times.

[0213] Among them, the first preview image 2041 can be an image frame of the above-mentioned target recording area corresponding to the starting moment when the electronic device starts recording, or the first preview image 2041 can also be an image frame corresponding to the current recording moment and the target recording area, which is not limited in this article.

[0214] In some possible implementations, during screen recording, the recording duration may also be displayed in the above-mentioned function bar 10; or, if a maximum recording duration (for example, 5 minutes) is set for the screen recording duration, and the recording duration of the current screen recording data is limited to not be greater than the maximum recording duration, then the above-mentioned function bar 10 may also display a recording countdown corresponding to the maximum recording duration.

[0215] It is understood that in other scenarios, the electronic device may also launch the video note application before receiving a click operation on the start screen recording icon 104. For example, before the user clicks the start screen recording icon 104 or executes the aforementioned shortcut key operation, the user opens a note document A created in the video note application. The electronic device may display the interface of the note document A on the display screen, for example, as a floating window in the lower right corner of the display screen, or may display the interface of the note document A on a layer other than the top layer of the display screen. In this scenario, in response to the click operation on the start screen recording icon 104, the electronic device determines that the target recording area is the playback area 201 and begins recording the display content corresponding to the playback area 201. Note document A is used as the note document for recording the screen recording data, eliminating the need to create a new note document 203 for recording the screen recording data. Note document A is displayed as a floating window, and a first preview image 2041 is displayed in the editing area of ​​note document A.

[0216] In one possible implementation, during the screen recording process, the display brightness of the target screen recording area on the display screen is higher than the brightness of other display areas. For example, the display brightness of the play area 201 in the user interface 1c is higher than the brightness of other display areas.

[0217] As shown in the user interface 1c of FIG2 , the preview area 204 is located in the line above the cursor in the text editing area. This is only an example. The preview area 204 may also be located in other positions, such as after the cursor. This document does not limit this.

[0218] Exemplarily, in response to the click operation on the start screen recording icon 104 , the electronic device specifically executes the following steps S706 to S708 .

[0219] S706: The video note application receives an instruction to start screen recording.

[0220] That is, the video note application receives a click operation on the start screen recording icon 104 .

[0221] In one possible implementation, since the click operation on the start screen recording icon 104 is a click operation on the view, which is a type of window state change, the video note application can also detect the window state change event related to the start screen recording instruction based on the accessibility service subclass when receiving the start screen recording instruction, and then execute the above steps S704 to S705.

[0222] Alternatively, in another possible implementation, after the video note application receives the start screen recording instruction and before the next end screen recording instruction is received, if the accessibility service subclass detects a window state change again, the video note application is preset to no longer execute the acquisition of the window information set in step S704 and the above step S705. That is, after receiving the click operation on the start screen recording icon 104, the video note application no longer executes the above steps S704 to S705.

[0223] S707, the video note application sends the size information of the target recording area to the screen recording program, and accordingly, the screen recording program receives the size information of the target recording area.

[0224] Exemplarily, before receiving the instruction to start screen recording, the size information of the target recording area determined and stored by the video note application based on steps S703 to S705 is the size information corresponding to the playback area 201 in the user interface 1c shown in FIG2 . Specifically, in the user interface 1c shown in FIG2 , the application package name set includes the application package name of the first application, the first view set includes the view ID of the view corresponding to the playback area 201, and the size information of the target recording area is the size information corresponding to the playback area 201.

[0225] In the embodiment of the present application, the screen recording program is a system screen recording program that comes with the electronic device. Corresponding to the software architecture diagram shown in Figure 13, the screen recording program is a screen recording application module in the application layer.

[0226] S708, the screen recording program records the display content corresponding to the target recording area to obtain screen recording data.

[0227] Exemplarily, in response to the operation of ending screen recording, the video note application sends an end screen recording instruction to the screen recording program. After receiving the end screen recording instruction, the screen recording program ends the screen recording and generates a video file (screen recording data). It is understandable that the video note application can determine the end of screen recording based on the user's click operation on the end screen recording icon and send an end screen recording instruction to the screen recording program. In some other possible implementations, the video note application can also send an end screen recording instruction to the screen recording program based on the preset first duration and the start time of screen recording after determining that the screen recording duration is equal to the first duration. This document does not limit this.

[0228] Generally, the size information of the screen recording area recorded in the screen recording program of the electronic device is consistent with the size information of the display screen by default. For example, the class used to store the size information of the screen recording area is the screen recording model (ScreenRecordModel). In the embodiment of the present application, after receiving the size information of the target recording area, the screen recording program can set the size information of the recording area to be consistent with the size information of the target recording area, overwriting the default size information of the display screen.

[0229] For example, in an embodiment of the present application, based on the fact that the size information of the recording area recorded by the screen recording model (ScreenRecordModel) of the electronic device defaults to the size information of the display screen, a 'set screen recording parameters (setVideoArea) method' is added based on the screen recording service (ScreenRecordService) provided by the system. In the setVideoArea method, the size information of the screen recording area recorded in the ScreenRecordModel can be reset.

[0230] For example, after the video note application sends the size information of the above-mentioned target recording area to the system screen recording service (ScreenRecordService), the ScreenRecordService notifies the setVideoArea method of the size information of the target recording area, and based on the setVideoArea method, sets the size information of the screen recording area recorded in the ScreenRecordModel to be consistent with the size information of the target recording area.

[0231] Optionally, after the video note application detects a change in the window state, it obtains a window information set based on the accessibility service. Specifically, the window information set may be obtained based on the accessibility service when the window state change is detected based on the accessibility service and the video note application is not currently in the screen recording state. That is, during screen recording, if the video note application detects a change in the window state based on the accessibility service, the video note application does not obtain the above-mentioned window information set. Instead, during non-screen recording, when the video note application detects a change in the window state based on the accessibility service, the video note application obtains the above-mentioned window information set.

[0232] Optionally, during screen recording, if the video note application detects a change in the window status based on the accessibility service, the video note application can also obtain the above-mentioned window information set and update the size information of the target recording area when the size information of the target recording area needs to be updated. However, it is understandable that the updated size information of the target recording area will not affect the recording area in the current recording process.

[0233] For example, referring to user interface 1c and user interface 1d in FIG3 , after the electronic device receives a click operation on the end screen recording icon 202 in the interface of user interface 1c, user interface 1d as shown in FIG3 is displayed. Specifically, in response to the click operation on the end screen recording icon 202, the electronic device ends recording, generates a first video file (corresponding to the above-mentioned screen recording data), displays the start screen recording icon 104 in the function bar 10, does not display the end screen recording icon 107, and displays a second preview image 2042 and a start play icon 2043 in the preview area 204 of the note document 203. In response to the click operation on the start play icon 2043, the electronic device can play the first video file. Unlike the above-mentioned user interface 1c, during screen recording, the content displayed in the preview area 204 shown in the above-mentioned user interface 1c does not include the start play icon 2043.

[0234] The second preview image 2042 is identical to the image displayed in the first preview image 2041. For example, the first preview image 2041 is the image frame corresponding to the start time of recording the display content of the playback area 201, while the second preview image 2042 is the image frame corresponding to the start time of the first video file. It is understood that, depending on the specific design, the first preview image 2041 and the second preview image 2042 may also be different, and this is not limited herein.

[0235] For example, the first video file may be stored in a storage directory corresponding to the video note application, and its data format may be mp4. Alternatively, in one possible implementation, in response to a click operation anywhere in the preview area 204, the first video file begins to play, and in response to another click operation anywhere in the preview area 204, the first video file ends to play.

[0236] In some other possible implementations, after the electronic device responds to the click operation of the end screen recording icon 202 and ends the recording, it can also display the recording time of the video file, the playback progress bar, and the full-screen magnification icon in the preview area 204, etc. This document does not limit this.

[0237] In some other scenarios, if in addition to the above-mentioned playback area 201, there are one or more other views included in the view whitelist in the above-mentioned user interface 1c, then the above-mentioned target recording area can be the minimum rectangular area that includes the playback area 201 and the display area corresponding to each of the one or more other views.

[0238] Scenario 2:

[0239] As shown in user interface 4a in FIG4 , the electronic device is a mobile phone terminal. The electronic device is running a first application (such as the above-mentioned video application), a second application (such as a chat application), and a video note application. The electronic device displays the chat application page in full screen on the display screen, displays the page of the video application playing the video in a floating window on the display screen, and displays the note document 402 with the theme of "Conference Notes A" in the video note application in a floating window on the display screen. For example, the user manually shrinks the interface corresponding to the video note application into a floating window and moves it to the display area corresponding to the note document 402 in FIG4 . In this scenario, the electronic device displays the function bar 10 shown in user interface 4a in response to the above-mentioned shortcut key operation. Specifically, the electronic device responds to the above-mentioned shortcut key operation, executes the above-mentioned steps S701 to S705, and in step S705, based on the fact that the application package name of the first application (video application) is included in the application whitelist, the application package name of the second application (chat application) is not included in the application whitelist, and the window corresponding to the first application, that is, the view corresponding to the playback area 401 in the user interface 4a is included in the view whitelist, determines and stores the size information of the above-mentioned target recording area as the size information corresponding to the view id of the view corresponding to the above-mentioned playback area 401.

[0240] In response to the click operation on the start screen recording icon 104 in the interface of the user interface 4a, the electronic device reads the stored size information of the target recording area (specifically, the size information corresponding to the view id of the view corresponding to the playback area 401), starts recording the display content corresponding to the playback area 401, and displays the user interface 4b as shown in Figure 4. Specifically, the end screen recording icon 202 is displayed in the function bar 10, and the start screen recording icon 104 is not displayed; and a preview image 403 is displayed in the preview area in the editing area of ​​the note document 402. For the description of the preview image, please refer to the relevant description of other embodiments in this article and will not be described in detail here. Specifically, the playback area 401 is recorded through the above steps S707 to S708, and in response to the click operation of the end screen recording icon 202, the recording is ended and a video file A is generated. The video file A contains the display content of the playback area 401.

[0241] In an embodiment of the present application, during screen recording, the electronic device records other display content in the display content of the area corresponding to the size information of the target recording area except for the display content of the video note application.

[0242] For example, before starting to record the display content of the playback area 401, there is a first overlapping area between the display area of ​​the note document 402 in the user interface 4b and the playback area 401. Optionally, in the first overlapping area, the display priority of the interface corresponding to the note document 402 is higher than the display priority of the interface of the first application (that is, the interface corresponding to the playback area 401). However, during the screen recording process of starting to record the display content of the playback area 401, the electronic device can shield the display content of the note document 402, not record the display content corresponding to the video note application, but record other display content in the target recording area except the display content of the note document interface. The video file A finally recorded contains the video content, barrage content, etc. played by the first application through the playback area 401, but does not contain the interface content of the note document 402.

[0243] For example, before starting to record the display content of the playback area 401, if there is a second overlapping area between the interface of other applications and the display area of ​​the interface of the first application in the user interface 4b, and the other applications refer to applications other than the video note application and the first application, for example, the interface of the other application is the interface corresponding to the chat application, then optionally, the display priority of the interface of the other application in the second overlapping area is higher than the display priority of the interface of the first application (that is, the interface corresponding to the playback area 401). And during the screen recording of the display content of the playback area 401, the display content of the second overlapping area in the playback area 401 recorded by the electronic device is specifically the display content of the interface of the other application in the second overlapping area, rather than the display content of the playback interface of the first application in the second overlapping area, and the display content of other areas of the playback area 401 except before the second overlapping area recorded by the electronic device is specifically the display content of the playback interface of the first application in these other areas. The video file A finally recorded includes the display content of the interface of the other application in the second overlapping area, and the display content of the playback interface of the first application in the other area, but does not include the display content of the playback interface of the first application in the second overlapping area.

[0244] In an embodiment of the present application, recording the target recording area is to record the display content of the display area corresponding to the size of the target recording area in the display screen, which may include video content and barrage content corresponding to the current video content, rather than just recording the video stream. For example, in some possible scenarios, the playback area 401 in the user interface 4a also displays barrage content 405. The video file A obtained by recording the display content corresponding to the playback area 401 using the screen recording method provided by the present application includes the adapted content played at the playback area 401 and the barrage content 405.

[0245] Scenario 3:

[0246] As shown in the user interface 5a in Figure 5, the electronic device is a foldable screen mobile phone terminal or a tablet terminal. The electronic device displays the second interface of the first application (video application) through the display area on the left side of the display screen, and displays the page of the second application (such as the above-mentioned chat application) through the display area on the right side of the display screen. In this scenario, the electronic device responds to the above-mentioned shortcut key operation and displays the function bar 10 shown in the user interface 5a. Specifically, the electronic device responds to the above-mentioned shortcut key operation and executes the above-mentioned steps S701 to S705. In step S705, based on the fact that the application package name of the first application is included in the application whitelist, the application package name of the second application is not included in the application whitelist, and the second interface corresponding to the first application, that is, the view corresponding to each display area in the user interface 5a is not included in the view whitelist, the size information of the above-mentioned target recording area is determined and stored as the entire interface 501 of the above-mentioned second interface. This meets the regional screen recording requirements in the split-screen scenario.

[0247] In response to the click operation of the start screen recording icon 104 in the interface of the user interface 5a, the electronic device starts recording the display content corresponding to the entire interface 501 of the video application, and displays the user interface 5b as shown in Figure 5. Specifically, the end screen recording icon 202 is displayed in the function bar 10, and the start screen recording icon 104 is not displayed; and a newly created note document 502 for recording screen recording data is displayed in the display area in the lower right corner of the display screen as a floating window, and a preview image 503 is displayed in the editing area of ​​the note document 502. Specifically, the display content of the entire interface 501 of the second interface is recorded through the above steps S707 to S708, and in response to the click operation of the end screen recording icon 202, the recording is ended and a second video file is generated. The second video file contains the display content of the entire interface 501 of the second interface, and does not contain the display content of the interface of the second application.

[0248] It is understandable that in the user interface provided in the embodiment of the present application, the interface for displaying the preview image through the video note document in a floating window during screen recording is mostly displayed in the lower right corner of the screen, but this article does not limit the specific display position of the interface corresponding to the video note document. For example, in some other scenarios, in the user interface 5b, if the interface of the above-mentioned second application (chat application) is displayed through the left half of the display screen, and the second interface of the first application (video application) is displayed through the right half of the display screen, then after the electronic device determines that the above-mentioned target recording area is the entire interface 501 of the second interface, it can also display the above-mentioned video note document 502 in the lower left corner area of ​​the display screen, instead of displaying the above-mentioned video note document 502 in the lower right corner area of ​​the display screen.

[0249] Scenario 4:

[0250] As shown in user interface 6a in Figure 6, the electronic device is a foldable screen mobile phone terminal or tablet terminal. The electronic device displays the page of a third application (e.g., a travel application) in the left display area of ​​the display screen, and displays the page of a second application (e.g., the chat application) in the right display area of ​​the display screen. In this scenario, the electronic device responds to the operation instruction of the shortcut key and displays the function bar 10 shown in user interface 6a. Specifically, in response to the shortcut key operation, the electronic device executes steps S701 to S705 above, and in step S705, based on the fact that the application package name of the second application and the third application are not included in the application whitelist, determines and stores the size information of the target recording area as the size information corresponding to the entire display area of ​​the display screen.

[0251] In response to clicking the start screen recording icon 104, the electronic device begins recording the display content corresponding to the display area of ​​the entire display screen of the electronic device and displays the user interface 6b shown in Figure 6. Specifically, the end screen recording icon 202 is displayed in the function bar 10, and the start screen recording icon 104 is not displayed; and a newly created note document 601 for recording the screen recording data is displayed in a floating window on the display screen, with a preview image 602 displayed in the editing area of ​​the note document 601.

[0252] In some other scenarios, the above-mentioned user interface 6b (or user interface 6a) may also display the interfaces of one or more other applications whose application package names are not included in the application whitelist. This document does not limit this. Correspondingly, the size information corresponding to the above-mentioned target recording area is still the size information of the display area of ​​the display screen.

[0253] Optionally, the function bar 10 may be located in the middle of the display screen, or at the left or right edge of the display area of ​​the display screen, which is not limited herein.

[0254] Optionally, the size information of the target recording area is recorded in the video note application, and the position of the function bar 10 can be in the middle of the target recording area, or at the left edge or right edge of the target recording area. The target recording area is used to indicate the size information of the area that the electronic device can currently record. When the electronic device starts recording, the recorded area is consistent with the size information of the target recording area. The default size of the recordable area can be the size of the display area corresponding to the display screen. The electronic device can determine the size information of the target recording area based on the above step S706. Specifically, when the electronic device is running the video note application, in response to the above shortcut key operation, the above function bar 10 is called out, and the size information of the target recording area is determined based on the above steps S704 to S706, and the position of the function bar 10 is determined to be in the middle of the target recording area.

[0255] It should be noted that in the above scenarios 1 to 4, the method for calling up the function bar 10 of the video note application is the corresponding shortcut key. However, this does not mean that the function bar 10 can only be called up by shortcut keys. Other operation methods are also possible, and this document does not limit this. For example, a virtual control A for calling up the function bar 10 can be provided in the video note application. When the electronic device receives a click operation on the virtual control A, the function bar 10 is called up.

[0256] Optionally, the electronic device can fold (or hide) the function bar 10 in response to a click operation on the start screen recording icon 104 after starting screen recording, that is, the function bar 10 is not displayed, and, in response to a click operation in the recording area, the function bar 10 will be displayed again.

[0257] The following describes in detail the above steps S701 to S708 in conjunction with the timing diagram of the program code class shown in Figure 11. Corresponding to Figure 13, the code classes shown in Figure 11 all belong to the application layer, wherein the application startup class, whitelist constant management class, accessibility service subclass, and screen recording parameter management class are included in the video note application of the application layer, and the screen recording service class and screen recording model class are included in the screen recording application module of the application layer.

[0258] Specifically, as shown in FIG11 , the screen recording method provided in the embodiment of the present application includes:

[0259] S1101, the application startup class (HnNotePadAppliction) starts the video note application.

[0260] In the embodiment of the present application, starting the video note application can also be understood as starting the video note function.

[0261] In an embodiment of the present application, the HnNotePadAppliction class is a functional module in the video note application that is responsible for detecting a startup request for the video note application, starting a thread running the video note application after detecting the startup request, and displaying a related user interface through an electronic device.

[0262] Exemplarily, HnNotePadAppliction starts running the video note application based on a click operation on the application icon of the video note application.

[0263] Exemplarily, when the video note application is not started, HnNotePadAppliction starts the video note application after detecting the operation instruction of the above-mentioned shortcut key for calling up the function bar of the video note application.

[0264] HnNotePadAppliction can also start the video note application based on other possible triggering methods, which are not limited in this article.

[0265] In the embodiment of the present application, the video note application can be system software of the electronic device, or it can be third-party software, which is not limited in this document.

[0266] S1102: Enable accessibility services.

[0267] In the embodiment of the present application, after HnNotePadAppliction starts the video note application, it requests the electronic device to enable the system capability of the accessibility service parent class (AccessibilityService parent class) in the accessibility service module, and calls the accessibility service subclass to start running the accessibility service subclass. The accessibility service subclass is shown as VideoAccessibilityService in Figure 11. The accessibility service subclass (VideoAccessibilityService) is a functional module in the video note application for monitoring window status changes.

[0268] S1103, the accessibility service subclass (VideoAccessibilityService) continues to detect the window status.

[0269] For details, please refer to the relevant instructions in the above step S703, which will not be described in detail here.

[0270] S1104: VideoAccessibilityService detects that the window state has changed, and obtains a window information set based on the accessibility service.

[0271] For the description of the window information collection, please refer to the relevant description in the above step S704, which will not be described in detail here.

[0272] S1105 , VideoAccessibilityService reads the application whitelist and view whitelist from the whitelist constant management class (ScreenRecordConstant).

[0273] In the embodiment of the present application, the whitelist constant management class (ScreenRecordConstant) is a functional module for recording the data of the application whitelist and view whitelist in the video note application. VideoAccessibilityService can obtain the data of the application whitelist and view whitelist by accessing the ScreenRecordConstant class.

[0274] For detailed descriptions of the application whitelist and view whitelist, please refer to the relevant description in step S706 above, which will not be detailed here.

[0275] S1106 , the VideoAccessibilityService determines the size information of the target recording area based on the window information set, the application whitelist, and the view whitelist.

[0276] Regarding the specific execution method of determining the target recording area, please refer to the relevant description in step S706 above and will not be described in detail here.

[0277] S1107, VideoAccessbilityService sends the size information of the target recording area to the screen recording parameter management class (ScreeRecordManager), and accordingly, ScreenRecordManager receives the size information of the target recording area.

[0278] In an embodiment of the present application, the ScreenRecordManager class can be understood as a functional module in the video note application for managing screen recording parameters and responding to start recording instructions and end recording instructions.

[0279] S1108: ScreenRecordManager stores the size information of the target recording area.

[0280] S1109, ScreenRecordManager receives a command to start screen recording.

[0281] For detailed instructions on the start screen recording instruction, please refer to the relevant description in step S706 above, which will not be detailed here.

[0282] In one possible implementation, when ScreenRecordManager receives an instruction to start screen recording, VideoAccessbilityService also receives an event regarding a click operation on the instruction to start screen recording, and after receiving the event regarding a click operation on the instruction to start screen recording, executes the above steps S1104 and S1106 to determine whether the size information of the target recording area needs to be updated, and if so, updates it; otherwise, keeps the size information of the target recording area unchanged.

[0283] It is understandable that after receiving the above-mentioned start screen recording instruction, the changes in the interface displayed on the display screen generally only include the start recording icon becoming the end recording icon, and the video note document for recording the screen recording data being displayed in a floating window in the display area in the lower right corner of the display screen. However, the changes in window information corresponding to these changes will not cause the size information of the target recording area to change, because the application whitelist provided by this application does not include the identification information of the video note application, that is, even if a timing problem occurs between step S1110 and step S1106 or S1107, it will not cause the size information of the target recording area to be inconsistent with the current window. In an optional manner, in order to avoid the problem that the size information of the target recording area used by the ScreenRecordManager class is not the latest data because the VideoAccessbilityService and the ScreenRecordManager update the target recording area based on the start screen recording instruction while the ScreenRecordManager class uses the size information of the target recording area (for example, executing step S1110 to send the size information of the target recording area to the screen recording service class), that is, the size information of the target recording area is sent to the screen recording service class before the target recording area is updated, resulting in the timing problem that the target recording area used is not the latest data, a timer corresponding to the second duration can also be set, the timer is started after receiving the recording instruction, and step S1110 is executed after the timer expires. For example, the first duration can be the total average duration of executing steps S1104 and S1106 in the sample data.

[0284] In another possible implementation, after ScreenRecordManager receives the start screen recording instruction, VideoAccessbilityService sends a first notification. After receiving the end screen recording instruction in step S1113, ScreenRecordManager sends a second notification to VideoAccessbilityService. After receiving the first notification, VideoAccessbilityService does not process the detected window state change, that is, does not execute steps S1104 to S1106. Only after receiving the second notification does VideoAccessbilityService respond to the detected window state change, that is, execute steps S1104 to S1106.

[0285] S1110, ScreenRecordManager sends the size information of the target recording area to the screen recording service class (ScreenRecordService), and accordingly, ScreenRecordService receives the size information of the target recording area.

[0286] In an embodiment of the present application, the screen recording service class (ScreenRecordService) is a screen recording service interface in a screen recording application of an electronic device, and the video note application can perform screen recording tasks by calling the screen recording application.

[0287] S1111, ScreenRecordService starts the screen recording service and calls the setVideoArea method to set the screen recording parameters in the screen recording model class (ScreenRecordModel) to be consistent with the size information of the above-mentioned target recording area.

[0288] For the description of the setVideoArea method and the ScreenRecordModel class, please refer to the relevant description in step S709 above.

[0289] Exemplarily, ScreenRecordService starts the screen recording service through the onStarCommand method.

[0290] Exemplarily, the size information of the target recording area and the screen recording parameters indicated by the ScreenRecordModel include: the horizontal coordinate of the upper left corner vertex of the rectangular area, the vertical coordinate of the upper left corner vertex, the width, and the height. The above-mentioned call to set the screen recording parameters (setVideoArea) method sets the screen recording parameters in the screen recording model class (ScreenRecordModel) to be consistent with the size information of the target recording area, specifically including: setting the horizontal coordinate of the upper left corner vertex of the screen recording parameter to the horizontal coordinate of the upper left corner vertex of the target recording area (shown as setVideoAreaLeft in Figure 11), setting the vertical coordinate of the upper left corner vertex of the screen recording parameter to the vertical coordinate of the upper left corner vertex of the target recording area (shown as setVideoAreaTop in Figure 11), setting the width of the screen recording parameter to the width of the target recording area (shown as setVideoAreaWidth in Figure 11), and setting the height of the screen recording parameter to the height of the target recording area (shown as setVideoAreaHeight in Figure 11).

[0291] S1112, ScreenRecordModel starts recording the screen based on the screen recording parameters.

[0292] S1113, ScreenRecordManager receives the instruction to end recording.

[0293] S1114, ScreenRecordManager sends an end-recording instruction to ScreenRecordModel, and accordingly, ScreenRecordModel receives the end-recording instruction.

[0294] S1115, ScreenRecordModel generates screen recording data.

[0295] S1116, ScreenRecordModel sends the screen recording data to ScreenRecordManager, and accordingly, ScreenRecordManager receives and stores the screen recording data.

[0296] For the description of the screen recording data, please refer to the relevant instructions on the screen recording data in step S708 above and / or the relevant instructions on the above-mentioned first video file, video file A, second video file, and third video file, which will not be detailed here.

[0297] Exemplarily, after receiving the screen recording data, the ScreenRecordManager can store the screen recording data. For example, the screen recording data can be stored in a directory for storing data of the video note application in the electronic device, and an association relationship can be established with the corresponding note document in the screen recording data. In addition, after receiving the screen recording data, the ScreenRecordManager can also insert and display the screen recording data in the note document corresponding to the video note application. For example, as shown in the user interface 1d in Figure 3, a first preview image 2041 of the screen recording data and a start play icon 2043 are displayed in the preview area 204 of the note document 203, and the screen recording data is played in response to a click operation of the start play icon 2043.

[0298] It is understandable that the accessibility service provided by the Google native platform detects the window status, and after detecting the change in the window status, the application identifier corresponding to the current window and the size corresponding to the view identifier are obtained based on some function methods for obtaining window information provided by the Google native platform. Since the accessibility service provided by the Google native platform has already taken corresponding measures to protect user privacy, the information is obtained by using some open function interfaces provided by the Google native platform for obtaining window information, which does not involve privacy information related to the user's personal information. To further prevent the risk of user information leakage, in a possible implementation method, the electronic device can also use the screen recording method provided by this application to record the screen after obtaining user authorization. For example, when the electronic device uses the video note application for the first time, the overview of the screen recording method provided by this application is displayed to the user through the display screen, and the user authorization is applied for. After obtaining the authorization, the screen recording method provided by this application is used to record the screen. Alternatively, a switch button function for enabling the screen recording method provided by this application to record the screen can be provided to the user in the video note application, and an overview of the screen recording method provided by this application can be displayed to the user through the display screen in response to the user operation. The switch button is in the off state by default, and in the off state, the video note application records the screen in full screen mode by default. In response to the user's turning on operation of the switch button, the electronic device uses the screen recording method provided in this application to record the screen.

[0299] It is understandable that the screen recording method provided in the embodiment of the present application can be executed by any electronic device that records the screen through a target application and is configured with an accessibility service function. The target application is an application with a screen recording function, for example, the target application can be the above-mentioned video note application, or the target application can also be a video note function in the screen recording application that comes with the electronic device, or the target application can also be other third-party screen recording applications, which is not limited in this document. Exemplarily, the form of the electronic device can be a non-folding screen mobile phone terminal, a folding screen mobile phone terminal, a tablet terminal, a laptop computer, a desktop computer, a laptop computer, or a handheld computer, etc., which is not limited in this document.

[0300] For example, please refer to FIG12 , which takes the electronic device 100 shown in FIG12 as an example to describe in detail the structural diagram of the electronic device provided in this application.

[0301] The electronic device 100 may include a processor 110, an external memory 120, an internal storage area 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor module 170, a button 180, a camera 190, and a display screen 191. The sensor module 170 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0302] It is understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in Figure 12, or combine certain components, or split certain components, or arrange the components differently. The components shown in Figure 12 can be implemented in hardware, software, or a combination of software and hardware. For example, the electronic device 100 may also include an audio module (speaker, receiver, microphone, headphone jack, etc.), a subscriber identification module (SIM) card interface, and a motor, etc.

[0303] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0304] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0305] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0306] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface. It will be understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0307] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0308] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0309] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0310] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs the sound signal through the audio device or displays an image or video through the display screen 191. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0311] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0312] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0313] Electronic device 100 implements display functionality through a GPU, display screen 191, and an application processor. The GPU is a microprocessor for image processing that connects display screen 191 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0314] For example, in an embodiment of the present application, a combination of one or more physical keys in the keypad 180 serves as a shortcut key for calling the function bar 10 shown in FIG1 . After the electronic device receives a shortcut key instruction regarding the function bar 10 via the keypad 180 , the electronic device displays the function bar 10 via the display screen 191 . Furthermore, after the display screen 191 receives a touch operation on the start screen recording icon 104 in the function bar 10 , the display screen 191 records the display content of the target recording area.

[0315] The electronic device 100 can implement a shooting function through an ISP, a camera 190 , a video codec, a GPU, a display screen 191 , and an application processor.

[0316] The external memory 120 can be used to connect to an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory 120 to implement data storage functions. For example, files such as music and videos can be stored in the external memory card.

[0317] The internal storage area 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal storage area 121. The internal storage area 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal storage area 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0318] FIG13 is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0319] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the system is divided into four layers: application layer, application framework layer, runtime and system libraries, and kernel layer.

[0320] The application layer can include a series of application packages.

[0321] As shown in FIG13 , the application package in the application layer may include but is not limited to a screen recording application module, a camera, a call, a WLAN, music, a video and other applications (also referred to as an application (App)).

[0322] In an embodiment of the present application, the application layer may further include a video note application module, which is used to execute the screen recording method in an embodiment of the present application.

[0323] In some other possible implementations, the video note application module may also be located in other layers of the software architecture, such as the application framework layer, system library, kernel layer, etc., which are not limited here.

[0324] The application framework layer provides an application programming interface (API) and programming framework for the application layer's applications. The application framework layer includes some predefined functions.

[0325] As shown in Figure 13, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. In this application, the application framework layer may also include an accessibility service module.

[0326] The window manager is used to manage window programs. The content provider is used to store and obtain data and make this data accessible to applications. The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. The phone manager is used to provide communication functions for the electronic device 100. The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, etc. The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction.

[0327] Exemplarily, in the present application, the video note application module in the application layer implements an accessibility service subclass based on the accessibility service module, and detects the window status based on the accessibility service subclass. Specifically, the window manager (window management service) in the application framework layer can send a corresponding AccessibilityEvent to the accessibility service subclass based on the event callback function after detecting that the window status has changed. For example, when the video note application receives a click operation on the start screen recording icon, the window manager can detect an event related to the click operation on the start screen recording icon and send the corresponding AccessibilityEvent to the accessibility service subclass. After the accessibility service subclass receives the AccessibilityEvent, the video note application determines that the window status has changed, and obtains a window information set based on the accessibility service subclass, determines the target recording area based on the window information set, the application whitelist, and the view whitelist, and records the target recording area. Specifically, please refer to the detailed description of the relevant method embodiments in this article, which will not be described in detail here.

[0328] The runtime includes the core library and the virtual machine. The runtime is responsible for system scheduling and management.

[0329] The core library consists of two parts: one part is the function that the programming language (for example, Java language) needs to call, and the other part is the core library of the system.

[0330] The application layer and application framework layer can run in a virtual machine. The virtual machine can execute the application layer and application framework layer programming files (for example, Java files) as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0331] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0332] The kernel layer is the layer between hardware and software. The kernel layer may include display drivers, camera drivers, audio drivers, sensor drivers, etc.

[0333] As used in the above embodiments, the term “when…” may be interpreted to mean “if…” or “after…” or “in response to determining…” or “in response to detecting…”, depending on the context. Similarly, the phrases “upon determining…” or “if (stated condition or event) is detected” may be interpreted to mean “if determining…” or “in response to determining…” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0334] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0335] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0336] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A screen recording method, characterized in that: The method comprises: Launching a first application and displaying a first interface of the first application, wherein the first interface includes a first area and a second area, wherein the first area is a video playback area and the second area is a non-video playback area; In response to a user operation of calling out a function bar of a video note application, displaying the function bar, the function bar including a start recording button, the video note application including a screen recording function; In response to a first click operation on the start recording button, the display content corresponding to the first area is recorded by the video note application, and an end recording button is displayed in the function bar, while the start recording button is not displayed; In response to a first click operation on the end recording button, the recording is ended, a first video is obtained, and the start recording button is displayed in the function bar, while the end recording button is not displayed. The first video includes the display content of the first area and does not include the display content of the second area. In response to a first user operation, displaying a second interface of the first application, where the second interface does not include a video playback area; In response to a second click operation on the start recording button, the display content of the entire second interface is recorded by the video note application, and the end recording button is displayed in the function bar, and the start recording button is not displayed; In response to the second click operation of the end recording button, the recording is ended, a second video is obtained, and the start recording button is displayed in the function bar, while the end recording button is not displayed. The second video includes all display contents of the second interface.

2. The method according to claim 1, characterized in that After the recording is finished, the method further includes: In response to a second user operation, displaying the second interface and the interface of the second application in a split-screen display manner, where the interface of the second application does not include a video playback area; In response to a third click operation on the start recording button, the display content of the entire second interface is recorded by the video note application, and the end recording button is displayed in the function bar, and the start recording button is not displayed; In response to the third click operation of the end recording button, the recording is ended, a third video is obtained, and the start recording button is displayed in the function bar, while the end recording button is not displayed. The third video includes all the display content of the second interface, but does not include the display content of the interface of the second application.

3. The method according to claim 1 or 2, characterized in that After the recording is finished, the method further includes: In response to a third user operation, displaying the interface of the second application and the interface of the third application in a split-screen display manner, wherein the interface of the third application does not include a video playback area; In response to a fourth click operation on the start recording button, recording the display content of the entire display area of ​​the display screen through the video note application, and displaying the end recording button in the function bar without displaying the start recording button; In response to the fourth click operation of the end recording button, the recording is ended and a fourth video is obtained. Contains the display content of the interface of the second application and the interface of the third application.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In response to the first click operation of the start recording button, displaying the interface of the video note application in a floating window display mode, and displaying a preview image in a preview area of ​​the interface of the video note application; In response to a first click operation on the end recording button, a preview image and a start play icon are displayed in the preview area, and in response to a click operation on the start play icon, the first video is played.

5. The method according to claim 3, characterized in that The first application is included in the application whitelist, the view corresponding to the first area is included in the view whitelist, the view corresponding to the second area is not included in the view whitelist, and the second application and the third application are not included in the application whitelist. The application whitelist is used to record the preset list of applications that the video note application supports recording, and the view whitelist is used to record the preset list of views that the video note application supports recording. Each view in the view whitelist comes from the corresponding application in the application whitelist.

6. The method according to any one of claims 1 to 5, characterized in that After displaying the function bar and before recording the display content corresponding to the first area using the video note application, the method further includes: Acquire a first window information set, where the first window information set is a set of window information of at least one window displayed on a display screen, and the first window information set includes window information of the first interface; Acquire a first set based on the first window information set and the application whitelist, where each application in the first set is included in the first window information set and in the application whitelist, and the first set includes the first application; Determining, based on the first set, the view whitelist, and the size information of the target view obtained from the first window information set, the size information of the target recording area is the size information of the target view, the target view is the view corresponding to the first application in the view whitelist, and the target view is the view corresponding to the first area; The recording of the display content corresponding to the first area by the video note application includes: The display content corresponding to the first area is recorded based on the size information of the target view recorded in the target recording area.

7. The method according to claim 6, characterized in that The acquiring the size information of the target view based on the first set, the view whitelist, and the first window information set, and determining the size information of the target recording area as the size information of the target view includes: Acquire, based on the first window information set and according to the address of the element in the first set, size information of a first reference view, where the first reference view is a view corresponding to a first reference application in the view whitelist, and the first reference application is an application corresponding to the address of the element in the first set; If the size information of the target view is obtained, and the element address of the first application in the first set is greater than the element address of any target application, the size information of the target recording area is determined to be the size information of the target view; the target application is included in the first set, and the size information of the view corresponding to the target application is The effective value, and the target application is not the first application.

8. The method according to any one of claims 1 to 7, characterized in that After displaying the second interface of the first application and before recording the entire display content of the second interface by the video note application, the method further includes: Acquire a second window information set, where the second window information set is a set of window information of at least one window displayed on the display screen, and the second window information set includes window information of the second interface; Acquire a second set based on the second window information set and the application whitelist, wherein each application in the second set is included in the second window information set and in the application whitelist, and the second set includes the first application; If size information corresponding to the identifier of the target view is not obtained based on the second set, the view whitelist, and the second window information set, determining that the size information of the target recording area is the size information of the entire display area of ​​the second interface, and the target view is the view corresponding to the first application in the view whitelist; The recording of the display content of the entire second interface by the video note application includes: The display content of the entire interface of the second interface is recorded based on the size information of the entire display area of ​​the second interface recorded in the target recording area.

9. The method according to claim 8, characterized in that If size information corresponding to the identifier of the target view is not obtained based on the second set, the view whitelist, and the second window information set, determining that the size information of the target recording area is size information of the entire display area of ​​the second interface, and the target view is the view corresponding to the first application in the view whitelist, including: Acquire, based on the second window information set and according to the address of the element in the second set, size information of a second reference view, where the second reference view is a view corresponding to a second reference application in the view whitelist, and the second reference application is an application corresponding to the address of the element in the second set; If the size information of the second reference view is not obtained, the size information of the target recording area is determined to be the size information of the entire display area of ​​the second interface.

10. The method according to any one of claims 5 to 9, characterized in that Each element in the application whitelist is identification information of an application including a video playback function, and each element in the view whitelist is identification information of a view used to play video data.

11. An electronic device, characterized in that: The electronic device includes: one or more processors, a memory and a display screen; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 10.

12. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises instructions, which, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 10.

Images