Method, apparatus, electronic device, and storage medium for operating a user interface

Abstract

The present disclosure relates to a method, apparatus, terminal device and computer readable storage medium for operating an electronic device. The method for operating an electronic device comprises determining whether a screenshot start condition is satisfied; in response to determining that the screenshot start condition is satisfied, automatically screenshotting a user interface of the electronic device to obtain a plurality of screenshots associated with the user interface of the electronic device; and in response to determining that a screenshot stop condition is satisfied, stopping automatically screenshotting the user interface of the electronic device.

Description

Technical Field

[0001] This disclosure relates to the field of user interface processing technology, and in particular to a method, apparatus, electronic device, and storage medium for operating an electronic device. Background Technology

[0002] In related technologies, while using an electronic device (e.g., while watching a video playing on the device or wanting to share their actions on the device with others), users may wish to take screenshots of the device's user interface. Although the steps for taking screenshots of the user interface vary across different electronic devices, doing so can generally disrupt the user's current activity, causing inconvenience.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] According to a first aspect of this disclosure, a method for operating an electronic device is provided, comprising: determining whether a screenshot start condition is met; automatically taking screenshots of the user interface of the electronic device in response to determining that the screenshot start condition is met, to obtain a plurality of screenshots associated with the user interface of the electronic device; and stopping the automatic taking of screenshots of the user interface of the electronic device in response to determining that a screenshot stop condition is met.

[0005] According to a second aspect of this disclosure, an apparatus for operating an electronic device is provided, comprising: a determining unit configured to determine whether a screenshot start condition is met; a screenshot unit configured to automatically take screenshots of the user interface of the electronic device to obtain a plurality of screenshots associated with the user interface of the electronic device; and a stopping unit configured to stop taking screenshots of the user interface of the electronic device in response to determining that a screenshot stop condition is met.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the method for operating the electronic device as described in this disclosure.

[0007] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that stores a program, the program including instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the methods for operating the electronic device described in this disclosure. Attached Figure Description

[0008] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0009] Figure 1 This is a flowchart illustrating a method for operating an electronic device according to an exemplary embodiment of the present disclosure;

[0010] Figure 2 This is a schematic diagram of a user interface illustrating an example of determining whether a screenshot start condition is met according to an exemplary embodiment of the present disclosure;

[0011] Figure 3 This is a schematic diagram illustrating the user interface of a screen recording window according to an exemplary embodiment of the present disclosure;

[0012] Figure 4A This is a schematic diagram illustrating a user interface for displaying video and multiple screenshots in portrait mode on an electronic device according to an exemplary embodiment of the present disclosure;

[0013] Figure 4B This is a schematic diagram illustrating a user interface for displaying video and multiple screenshots in landscape mode on an electronic device according to an exemplary embodiment of the present disclosure;

[0014] Figure 5 This is a schematic diagram illustrating a user interface for editing displayed video and multiple screenshots according to exemplary embodiments of the present disclosure;

[0015] Figure 6 This is a structural block diagram illustrating an apparatus for operating an electronic device according to exemplary embodiments of the present disclosure;

[0016] Figure 7 This is a structural block diagram illustrating an exemplary electronic device that can be applied to exemplary embodiments of the present disclosure. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0019] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0020] It should be noted that, in this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0021] Furthermore, the terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Additionally, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0022] Furthermore, it should be noted that the names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0023] In this disclosure, "electronic device" can be any type of electronic device, such as, but not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc.

[0024] In this disclosure, a "window" is a medium through which code or a program is displayed on the user interface of an electronic device. In particular, an "application" is displayed on the user interface of an electronic device through an "application window".

[0025] In related technologies, while using an electronic device (e.g., while watching a video playing on the device or wanting to share their actions on the device with others), users may wish to take screenshots of the device's user interface. Although the steps for taking screenshots of the user interface vary across different electronic devices, doing so can generally disrupt the user's current activity, causing inconvenience.

[0026] Therefore, this disclosure provides a method for operating an electronic device. The method for operating an electronic device according to this disclosure can automatically take screenshots of the user interface of the electronic device to obtain multiple screenshots associated with the user interface for subsequent user operations. This allows the automatic screenshot function to be executed when the screenshot start conditions are met, simplifying the user's operation. Furthermore, the method for operating an electronic device according to this disclosure can automatically take screenshots of the user interface of the electronic device without requiring the user to manually take screenshots according to the operating specifications of different electronic devices, ensuring that the screenshot process does not affect the user's current operation.

[0027] Exemplary embodiments of the method for operating an electronic device according to the present disclosure will be further described below with reference to the accompanying drawings.

[0028] Reference Figure 1 A first aspect of this disclosure may include a method for operating an electronic device, the method comprising: determining whether a screenshot start condition is met (S101); in response to determining that the screenshot start condition is met, automatically taking screenshots of the user interface of the electronic device to obtain a plurality of screenshots associated with the user interface of the electronic device (S102); and in response to determining that a screenshot stop condition is met, stopping the automatic taking of screenshots of the user interface of the electronic device (S103).

[0029] In step S101, according to some embodiments, the electronic device may determine whether a screenshot start condition is met. According to some embodiments, the screenshot start condition may be a user's input to start a screenshot of the electronic device's user interface. According to some embodiments, the screenshot start condition may be a user's input to start a screenshot of a physical button on the electronic device. According to some embodiments, the screenshot start condition may also be independent of user input. How to determine whether a screenshot start condition is met will be described in detail below.

[0030] In step S102, according to some embodiments, taking an automatic screenshot of the user interface of the electronic device (also known as a screenshot) is performed automatically by the electronic device without requiring manual operation by the user.

[0031] According to some embodiments, the entire user interface of an electronic device can be automatically captured. According to other embodiments, a portion of the user interface of an electronic device can also be automatically captured, for example, automatically filtering out black borders in a video during the capture process.

[0032] In related technologies, screenshots of the user interface of electronic devices are usually displayed as screenshot animations.

[0033] According to some embodiments, when automatically taking a screenshot of the user interface of the electronic device, the screenshot animation may not be displayed on the user interface of the electronic device.

[0034] In this way, the screenshot animation will not interfere with the current display content of the electronic device's user interface, thus affecting the user's current operation of the electronic device, which improves the user's visual experience.

[0035] In related technologies, taking a screenshot of the user interface of an electronic device usually produces a screenshot sound effect.

[0036] According to some embodiments, when automatically taking a screenshot of the user interface of the electronic device, the electronic device may not produce a screenshot sound effect.

[0037] In this way, the audio effects of the screenshot will not interfere with the current playback content of the electronic device's audio output device, thus improving the user's auditory experience.

[0038] In step S103, according to some embodiments, the electronic device may determine whether a screenshot stop condition is met. According to some embodiments, the screenshot stop condition may be a user's input to stop inputting a screenshot of the electronic device's user interface. According to some embodiments, the screenshot stop condition may be a user's input to stop inputting a screenshot of a physical button on the electronic device. According to some embodiments, the screenshot stop condition may also be independent of user input. How to determine whether a screenshot stop condition is met will be described in detail below.

[0039] According to some embodiments, in response to determining that the screenshot stopping condition is met, thumbnails of the plurality of screenshots and / or the video can be directly displayed on the user interface of the electronic device. According to other embodiments, in response to determining that the recording stopping condition is met, the thumbnails of the plurality of screenshots and / or the video can be saved in a temporary folder instead of being directly displayed for subsequent user operations.

[0040] According to some embodiments, the user interface of the electronic device may be a touch-sensitive user interface.

[0041] The following describes how to determine whether the screenshot start condition is met in step S101.

[0042] According to some embodiments, determining whether the screenshot start condition is met may include: determining that the screenshot start condition is met in response to receiving an automatic screenshot instruction input for the user interface of the electronic device.

[0043] According to some embodiments, the automatic screenshot instruction input may include a click input, a press input, a swipe input, a release input, or a combination thereof. According to some embodiments, the automatic screenshot instruction input may include a single input, such as a click input to a screen recording / screenshot button, or a press input to a specific location on the user interface. According to other embodiments, the automatic screenshot instruction input may also include multiple inputs. Automatic screenshot instruction inputs including multiple inputs will be described in detail below.

[0044] In some cases, it is possible to determine the state of the electronic device's user interface when it receives the automatic screenshot instruction input.

[0045] According to some embodiments, the method may further include: receiving an automatic screenshot instruction input for the user interface of the electronic device, wherein determining whether the screenshot start condition is met includes: determining whether the user interface of the electronic device displays an application window when the automatic screenshot instruction input is received; in response to determining that the user interface of the electronic device displays an application window when the automatic screenshot instruction input is received, determining whether the application window is allowed to be screenshotted; and in response to determining that the application window is allowed to be screenshotted, determining that the screenshot start condition is met.

[0046] According to some embodiments, the user interface of the electronic device may display an application window, system desktop, task manager window, etc. when it receives the automatic screenshot instruction input.

[0047] If the user interface of the electronic device displays an application window when it receives the automatic screenshot instruction input, it can be further determined whether the application window allows being screenshotted.

[0048] In some cases, the application window is allowed to be captured. In these cases, the user interface displays the application window when it receives an automatic screenshot instruction, which confirms that the screenshot start conditions are met.

[0049] In other cases, application windows are not allowed to be screenshotted. According to some embodiments, the application window developer may not allow screenshots of the application window (e.g., for copyright or privacy reasons). According to some embodiments, the user sets the application window to not be screenshotted. The user may set the application window to not be screenshotted for reasons such as not wanting to be disturbed while operating the application window. For example, an application window that can be set by the user to not be screenshotted could be a game application window. Therefore, if the user interface displays an application window that is not allowed to be screenshotted when it receives an automatic screenshot instruction input, it can be determined that the screenshot initiation condition is not met.

[0050] In this way, it's possible to determine whether an application window is allowed to be screenshotted based on whether the user interface displays an application window when it receives an automatic screenshot instruction. For application windows that are not allowed to be screenshotted, if an automatic screenshot instruction is received at the application window (e.g., due to user error or the input method of the automatic screenshot instruction being the same as the input method of an operation in the displayed application window), the electronic device will not respond to the automatic screenshot instruction and begin taking a screenshot. This makes the response to the automatic screenshot instruction more accurate and improves the user experience.

[0051] According to some embodiments, determining whether the screenshot start condition is met further includes: in response to determining that the user interface of the electronic device is not displaying an application window when the automatic screenshot instruction input is received, determining that the screenshot start condition is met.

[0052] According to some embodiments, the user interface of the electronic device may not be an application window when it receives the automatic screenshot instruction input; for example, it may be a desktop window, a task manager window, etc. In these cases, since there may be no issue of being disturbed, the electronic device can directly determine that the screenshot start condition is met and begin automatic screenshotting, simplifying the steps of determining whether the screenshot start condition is met.

[0053] As mentioned above, the automatic screenshot instruction input may include a single input or multiple inputs.

[0054] According to some embodiments, receiving an automatic screenshot instruction input for the user interface of the electronic device may include: receiving a function menu call input for the user interface of the electronic device; in response to receiving the function menu call input, displaying a function selection menu on the user interface of the electronic device, wherein the function selection menu includes a screen recording screenshot function option; and receiving a selection for the screen recording screenshot function option.

[0055] Figure 2 This is a schematic diagram illustrating an example of a user interface 200 for determining whether a screenshot start condition is met according to an exemplary embodiment of the present disclosure. The following will be combined with... Figure 2 Describe how to determine the start condition for taking a screenshot by using the function selection menu and selecting function options on the function selection menu.

[0056] According to some embodiments, the automatic screenshot instruction input may include two inputs. The first input to the automatic screenshot instruction input includes a function menu call input for the user interface of the electronic device. (See also...) Figure 2 In response to receiving the function menu call input (e.g., a click input to a function menu call button, a press input to a user interface, a swipe input from the bottom edge of the user interface of the electronic device, etc.), a function selection menu 201 is displayed on the user interface of the electronic device. According to some embodiments, the function selection menu 201 may include multiple function options. Figure 2 In the example shown, the function selection menu 201 may include three function options: audio word segmentation function option 202-1, screen recording screenshot function option 202-2, and image word segmentation function option 202-3. Of course, this disclosure is not limited to this; the function selection menu may include any number of function options. The user can select one function option from the function menu 201. According to some embodiments, the automatic screenshot instruction input further includes the selection of the screen recording screenshot function option 202-2 as a second input to the automatic screenshot instruction input.

[0057] In this way, two inputs can be used to determine whether the screenshot start condition is met. This gives users more opportunities to confirm and more time to consider, and can prevent users from accidentally starting automatic screenshots instead of intentionally.

[0058] According to some embodiments, the method may further include: receiving an automatic screenshot instruction input for the user interface of the electronic device, wherein determining whether the screenshot start condition is met may include: obtaining the position of the automatic screenshot instruction input on the user interface of the electronic device; determining whether the user interface of the electronic device is within a text extractable area at the position; and determining that the screenshot start condition is met in response to determining that the user interface of the electronic device is not within a text extractable area at the position.

[0059] According to some embodiments, the user interface of an electronic device may include a text-extractable area and a text-non-extractable area. The text-extractable area refers to the area where text can be selected in response to text extraction input, such as the area containing text conversations in a chat application, or the area containing text body text in an email application. The text-non-extractable area refers to the area where text cannot be selected in response to text extraction input, such as the system desktop, or the area containing images in a photo album application. The text extraction input may include click input, press input, swipe input, release input, or a combination thereof targeting the text-extractable area. Therefore, text extraction input may overlap with automatic screenshot instruction input.

[0060] According to some embodiments, when the location of the automatic screenshot instruction input on the user interface of the electronic device is not within the text extraction area, the screenshot start condition is determined to be met, and a screenshot is taken. In this way, the screenshot start condition can be determined more accurately, without interfering with text extraction input, thus improving the user experience.

[0061] According to some embodiments, when the automatic screenshot instruction input includes multiple inputs (i.e., function menu call-up input and input for the screen recording screenshot function option), obtaining the position of the automatic screenshot instruction input on the user interface of the electronic device may include: obtaining the position of the function menu call-up input on the user interface of the electronic device. According to some embodiments, in response to determining that the function menu call-up input, which is the first input of the automatic screenshot instruction input, does not belong to the text extractable area, it is determined that a function selection menu can be displayed. Then, the selection of the screen recording screenshot function option in the function selection menu, which is the second input of the automatic screenshot instruction input, is performed.

[0062] In this way, to meet the conditions for starting a screenshot, the function menu, which serves as the first input for automatic screenshot, can be used to eliminate accidental operations on the text-extractable area. At the same time, the screen recording screenshot function option, which serves as the second input for automatic screenshot, confirms that the user wants to start automatic screenshot, further increasing the accuracy of determining whether the conditions for starting a screenshot are met and improving the user experience.

[0063] According to some embodiments, the method may further include: in response to determining that the user interface of the electronic device is a text-extractable area at the location, displaying a function selection menu on the user interface of the electronic device, wherein the function selection menu includes a screen recording screenshot function option; receiving a selection for the screen recording screenshot function option; and in response to receiving a selection for the screen recording screenshot function option, determining that the screenshot start condition is met.

[0064] According to some embodiments, when it is determined that the location of the automatic screenshot instruction input (e.g., press input) belongs to a text-extractable area, it is impossible to determine whether the user intends to perform text extraction or automatic screenshot as described in this disclosure. Therefore, according to some embodiments, when a user presses on part or all of the text-extractable area, a function selection menu can be displayed (e.g., pop-up) on the user interface to allow the user to further confirm the operation expected by the electronic device. According to some embodiments, the function selection menu may include, for example, screen recording screenshot function options, audio word segmentation function options, image word segmentation function options, text word segmentation function options, text extraction function options, etc. According to some embodiments, the user can select the screen recording screenshot function option (e.g., click the screen recording screenshot function option, press the screen recording screenshot function option, slide on the screen recording screenshot function option, etc.), and after the user selects the screen recording screenshot function option, it is determined that the screenshot start condition is met.

[0065] In this way, even if the user's automatic screenshot input is located within the text extraction area, the possibility of the user wanting to take an automatic screenshot is not completely ruled out. Instead, a function selection menu is displayed to give the user further options. When the user selects the screen recording screenshot function option associated with the automatic screenshot function, the electronic device can still perform the automatic screenshot function.

[0066] The exemplary manner of automatic screenshotting in step S102 will be described in detail below with reference to exemplary embodiments of the present disclosure.

[0067] According to some embodiments, automatically taking screenshots of the user interface of the electronic device may include automatically taking screenshots of the user interface of the electronic device at preset time intervals. According to some examples, the preset time interval may be once every 0.1 seconds, once every 0.2 seconds, once every 0.3 seconds, once every 0.4 seconds, once every 0.8 seconds, or once every 1 second, etc.

[0068] In related technologies, due to limitations in hand speed, manual screenshots of the user interface of electronic devices can only be taken at a low frequency. However, the method for operating electronic devices according to this disclosure enables high-speed automatic screenshots.

[0069] Furthermore, in related technologies, the frequency of manually taking screenshots of the user interface of an electronic device is unstable due to limitations in controllability. However, the method for operating an electronic device according to this disclosure can achieve uniform screenshots and can adjust the frequency of automatic screenshots as needed (which will be described in detail below).

[0070] In some cases, automatically taking screenshots of the user interface of an electronic device can take a long time and result in a large number of screenshots.

[0071] According to some embodiments, automatically taking screenshots of the user interface of the electronic device may include: obtaining in real time the elapsed time of automatically taking screenshots of the user interface of the electronic device; and dynamically adjusting the time interval for automatically taking screenshots of the user interface of the electronic device based on the real-time obtained elapsed time.

[0072] According to some embodiments, the elapsed time for automatically taking screenshots of the electronic device's user interface can be obtained in real time using a timer on the electronic device. The elapsed time can be the time that has passed since the start of automatically taking screenshots of the electronic device's user interface until the current moment.

[0073] In this way, the time interval for automatic screenshots (i.e., the rate of automatic screenshots) can be adjusted based on the elapsed time. Compared to manual screenshots in related technologies, the automatic screenshotting of the user interface of an electronic device according to exemplary embodiments of this disclosure is controllable.

[0074] According to some embodiments, when a short period of time has elapsed, the user interface of the electronic device can be automatically captured at short intervals; while when a longer period of time has elapsed, the user interface of the electronic device can be automatically captured at longer intervals.

[0075] According to some embodiments, the step of dynamically adjusting the time interval for automatically taking screenshots of the user interface of the electronic device based on the real-time acquired elapsed time may include: taking screenshots of the user interface of the electronic device at a first time interval when the real-time acquired elapsed time is within a first range; and taking screenshots of the user interface of the electronic device at a second time interval when the real-time acquired elapsed time is within a second range, wherein the values ​​in the first range are all less than the values ​​in the second range, and the first time interval is less than the second time interval.

[0076] Based on some examples, the first range can be (0, 15 seconds) [i.e., the elapsed time is greater than 0 seconds and less than or equal to 15 seconds], and the second range can be (15 seconds, 30 seconds) [i.e., the elapsed time is greater than 15 seconds and less than or equal to 30 seconds]. Based on some examples, the first time interval can be once every 0.2 seconds, and the second time interval can be once every 0.4 seconds. In other words, when the elapsed time for automatically taking a screenshot of the electronic device's user interface is no more than 15 seconds, a screenshot of the electronic device's user interface can be taken once every 0.2 seconds; when the elapsed time for automatically taking a screenshot of the electronic device's user interface is more than 15 seconds but no more than 30 seconds, a screenshot of the electronic device's user interface can be taken once every 0.4 seconds.

[0077] According to some embodiments, the step of dynamically adjusting the time interval for automatically taking screenshots of the user interface of the electronic device based on the real-time acquired elapsed time may further include: when the real-time acquired elapsed time is within a third range, automatically taking screenshots of the user interface of the electronic device at a third time interval, wherein the values ​​in the second range are all less than the values ​​in the third range, and the second time interval is less than the third time interval.

[0078] Based on some examples, the third range can be (30 seconds, +∞) (i.e., more than 30 seconds have elapsed), and the third time interval can be once every 0.8 seconds. In other words, when more than 30 seconds have elapsed since the automatic screenshot of the electronic device's user interface was taken, a screenshot of the electronic device's user interface can be taken once every 0.2 seconds.

[0079] The foregoing has described some examples of exemplary embodiments according to the present disclosure. Those skilled in the art should understand that the first range, second range, third range, first time interval, second time interval, and third time interval can be changed according to actual circumstances, and exemplary embodiments according to the present disclosure may also include a fourth range, a fifth range, and a fourth time interval, a fifth time interval, etc.

[0080] In this way, the longer the time elapsed since the automatic screenshot of the electronic device's user interface was taken, the longer the time interval between automatic screenshots becomes. This avoids taking too many screenshots, making subsequent user actions on these screenshots (such as filtering them) more convenient.

[0081] In some cases, multiple screenshots associated with the user interface of an electronic device, obtained using the method described above for dynamically adjusting the time interval of automatic screenshots, may be uneven in time.

[0082] Based on the example described above, the first range can be (0, 15 seconds) (i.e., the elapsed time is greater than 0 seconds and less than or equal to 15 seconds), and the second range can be (15 seconds, 30 seconds) (i.e., the elapsed time is greater than 15 seconds and less than or equal to 30 seconds). The first time interval can be once every 0.2 seconds, and the second time interval can be once every 0.4 seconds. The elapsed time acquired in real time is 30 seconds. During the 30 seconds of taking screenshots of the user interface of the electronic device, 75 screenshots (at intervals of 15 seconds / 0.2 seconds) associated with the user interface of the electronic device are obtained when the elapsed time was within the first range (i.e., the first 15 seconds). 37 screenshots (at intervals of 15 seconds / 0.4 seconds and rounded down) associated with the user interface of the electronic device are obtained when the elapsed time was within the first range (i.e., the first 15 seconds).

[0083] According to some embodiments, the second time interval is N times the first time interval, where N is a positive integer greater than 1, and the method may further include: when the real-time acquired elapsed time is within a second range, retaining one screenshot and deleting the others from every N screenshots associated with the user interface of the electronic device acquired when the elapsed time was in a first range.

[0084] Based on the examples described above, it can be seen that the second time interval is twice the first time interval. Therefore, one out of every two screenshots associated with the user interface of the electronic device acquired during the first time interval (e.g., the first 15 seconds) can be retained while the others are deleted. In other words, among the 75 screenshots associated with the user interface of the electronic device acquired in the first 15 seconds, one out of every two screenshots can be retained while the others are deleted. According to some examples, these 75 screenshots can be numbered chronologically, retaining those with even numbers and deleting those with odd numbers. No deletion is performed on the 37 screenshots associated with the user interface of the electronic device acquired in the latter 15 seconds. Thus, the final retained screenshots are evenly distributed over time (i.e., one every 0.4 seconds). According to other examples, it is also possible to retain the screenshots with odd numbers and delete the screenshots with even numbers.

[0085] In this way, even without knowing the duration of automatically taking screenshots of the user interface of the electronic device, the method according to the exemplary embodiments of this disclosure can not only adaptively adjust the screenshot time interval according to the duration of automatic screenshots, but also maintain the time uniformity of multiple screenshots finally obtained that are associated with the user interface of the electronic device, greatly enhancing the adaptive capability of automatic screenshots and improving the user experience.

[0086] According to some embodiments, the method may further include: recording the user interface of the electronic device in response to determining that the screenshot start condition is met, to obtain a video associated with the user interface of the electronic device; and stopping recording the user interface of the electronic device in response to determining that the screenshot stop condition is met.

[0087] According to some embodiments, recording the user interface of an electronic device (also known as screen recording) refers to recording the content displayed by the user interface of an electronic device within a certain period of time in a video format, and the resulting video is called the video associated with the user interface of the electronic device.

[0088] According to some embodiments, images displayed on the user interface can be recorded frame by frame and these images can be combined into a video.

[0089] While the above examples only illustrate some embodiments of screen recording, it should be noted that all technologies in the related art capable of recording user interfaces (or screens) can be used in this disclosure.

[0090] In this disclosure, the recording of the user interface of the electronic device is achieved by the electronic device itself, rather than by filming the user interface of the electronic device using an external device (e.g., a camera).

[0091] In this disclosure, recording time refers to the time elapsed from when the screenshot start condition is determined to be met until the screenshot stop condition is determined to be met.

[0092] According to some embodiments, recording the user interface of an electronic device can typically be used to record videos being played on the electronic device, record instructional demonstrations of applications, etc., but this disclosure is not limited to these embodiments.

[0093] According to some embodiments, the entire user interface of an electronic device can be recorded. According to other embodiments, a portion of the user interface of an electronic device can also be recorded, for example, automatically filtering out black borders in a video during recording.

[0094] Figure 3 This is a schematic diagram illustrating the user interface of a screen recording window according to an exemplary embodiment of the present disclosure. The following will be combined with... Figure 3 Describes the screen recording window that can be displayed when recording the user interface of an electronic device.

[0095] According to some embodiments, the method may further include: in response to determining that the automatic screenshot indication input satisfies the screenshot start condition, displaying a screen recording window 310 on the user interface 300 of the electronic device, wherein the screen recording window indicates that the user interface of the electronic device is being recorded.

[0096] According to some embodiments, no screen recording animation is displayed during the recording of the electronic device's user interface. Specifically, when automatically taking a screenshot of the electronic device's user interface, neither a screenshot animation nor a screenshot sound effect is displayed. In this case, although the electronic device is indeed recording and automatically taking screenshots after the screenshot start conditions are confirmed to be met, the user may doubt whether the electronic device is actually recording and automatically taking screenshots because they do not receive any feedback from the electronic device. Therefore, according to some embodiments, a screen recording window 310 is displayed to provide the user with visual feedback that the electronic device's user interface is being recorded.

[0097] According to some embodiments, the text prompt "Recording..." can be displayed in the screen recording window 310, thereby providing customers with more intuitive visual feedback. For example... Figure 3As shown in the example, the text prompt is located at the bottom of the screen recording window. Of course, this disclosure is not limited to this, and the text prompt can also be located at the top or middle of the screen recording window.

[0098] According to some embodiments, the elapsed time of the recording electronic device's user interface can be displayed in real time in the user interface 300 or the screen recording window 310. According to some embodiments, the real-time displayed elapsed time can have a certain degree of transparency, such as 60%. This allows the user to be reminded of the elapsed recording time in real time on the user interface of the recording electronic device.

[0099] In some cases, although a screen recording window is displayed on the user interface, the user may not want the screen recording window to be recorded into the video associated with the user interface of the electronic device, nor do they want the screen recording window to be automatically captured in at least one screenshot associated with the user interface of the electronic device.

[0100] According to some embodiments, automatically taking a screenshot of the user interface of the electronic device may include: not automatically taking a screenshot of the screen recording window during the automatic screenshot of the user interface of the electronic device; and recording the user interface of the electronic device may include: not recording the screen recording window during the recording of the user interface of the electronic device.

[0101] According to some embodiments, not recording the screen recording window means that during the recording of the user interface of the electronic device, the area of ​​the user interface of the electronic device other than the screen recording window and the area obscured by the screen recording window can be recorded, so that the screen recording window does not appear in the video.

[0102] According to some embodiments, not automatically taking screenshots of the screen recording window means that during the automatic screenshotting of the user interface of the electronic device, the area of ​​the user interface of the electronic device other than the screen recording window and the area obscured by the screen recording window can be automatically screenshotted, so that the screen recording window is not present in the multiple screenshots.

[0103] According to some embodiments, during the recording of an electronic device's user interface, screen recording windows displayed on the user interface are automatically filtered. The following describes in detail the technique for automatically filtering screen recording windows during recording.

[0104] According to some embodiments, the step of not automatically taking screenshots of the screen recording window and / or not recording the screen recording window may include: adjusting the capture attribute of the screen recording window to be uncaptureable; detecting and receiving the capture attribute of the screen recording window when starting to automatically take screenshots of the user interface of the electronic device and / or starting to record the user interface of the electronic device; in response to receiving that the capture attribute of the screen recording window is uncaptureable, not automatically taking screenshots of the screen recording window during the automatic screenshotting of the user interface of the electronic device and / or not recording the screen recording window during the recording of the user interface of the electronic device.

[0105] According to some embodiments, the user interface and screen recording window of an electronic device can be displayed on different layers. For example, the user interface of the electronic device is displayed on a first layer, and the screen recording window is displayed on a second layer, with the second layer displayed above the first layer. According to some embodiments, during display, since the screen recording window is displayed above the user interface, areas of the user interface obscured by the screen recording window will not be displayed.

[0106] In some embodiments, the capture attribute of the screen recording window is adjusted to be non-capturing. For example, the screen recording window is assigned the NOT_CAPTURE attribute value.

[0107] According to some embodiments, the capture attribute of the screen recording window displayed on the second layer is detected. If the capture attribute is not captureable, then during the recording of the user interface of the electronic device and / or the automatic screenshotting of the user interface of the electronic device, the screen recording window displayed on the second layer is automatically filtered, and only the user interface of the electronic device displayed on the first layer is recorded and / or the automatic screenshotting of the user interface of the electronic device displayed on the first layer is automatically captured.

[0108] In other words, the user interface of the electronic device is recorded as if the screen recording window does not exist, and screenshots of the user interface are automatically taken as if the screen recording window does not exist.

[0109] According to some embodiments, displaying a screen recording window on the user interface of the electronic device includes: rendering the user interface of the electronic device on a user interface view control in real time; and setting the user interface view control in the screen recording window to display a thumbnail of the user interface of the electronic device in real time in the screen recording window.

[0110] According to some embodiments, the user interface of an electronic device can be rendered in real time onto a user interface view control using virtual display technology. The user interface view control can be, for example, a Surface control. Virtual display technology can include, for example, the Virtual Display technology in the Android system.

[0111] In this way, the content displayed in the screen recording window can be synchronized with the content displayed in the user interface of the electronic device.

[0112] like Figure 3 As shown in the example, the content 301 displayed in the user interface 300 is a tree, while the content 311 displayed in real time in the screen recording window is a thumbnail of the tree, to visually reassure the user that the electronic device is indeed recording the currently displayed user interface.

[0113] In some embodiments, the aspect ratio of the screen recording window is the same as that of the user interface of the electronic device. In some embodiments, the four corners of the screen recording window are rounded for a more aesthetically pleasing appearance.

[0114] As mentioned above, it is possible to record a portion of the user interface of an electronic device, or to automatically take a screenshot of a portion of the user interface. It should be noted that although the following description uses recording as an example to detail the technique for recording a portion of the user interface of an electronic device, the same technique can certainly be used to take screenshots of a portion of the user interface.

[0115] According to some embodiments, when playing video, a portion of the user interface of an electronic device may be, for example, the video playback area.

[0116] According to some embodiments, the method may further include: during automatic screenshotting of the user interface of the electronic device, determining whether the user interface of the electronic device is displaying a video playback application window corresponding to a video playback application, wherein the video playback application window includes a video playback area, wherein the automatic screenshotting of the user interface of the electronic device includes: in response to determining that the user interface of the electronic device is displaying the video playback application window, only the video playback area is screenshotted, and the recording of the user interface of the electronic device includes: in response to determining that the user interface of the electronic device is displaying the video playback application window, only the video playback area is recorded.

[0117] According to some embodiments, it can be determined whether the currently displayed application window is a video playback application window based on information read from the electronic device's system about the application window currently being displayed on the user interface. According to some examples, the video playback application corresponding to the video playback application window could be TikTok, Huoshan Video, etc.

[0118] According to some embodiments, the video playback application window may display a video playback area, an advertisement area, a user comment area, a director or actor or voice actor display area, etc. However, when automatically taking screenshots of the video played in the video playback application window and / or recording the video played in the video playback application window, the user may only want to record or automatically take screenshots of the video playback area that is part of the user interface of the electronic device.

[0119] According to some embodiments, the automatically taking screenshots of the user interface of the electronic device and / or only recording the video playback area may include: determining the position of the video playback area on the user interface of the electronic device; and based on the position of the video playback area on the user interface of the electronic device, taking a screenshot of the video playback area and / or recording the video playback area.

[0120] According to some embodiments, the video playback area may be represented using coordinates. According to some examples, a coordinate system may be established with the vertex at the lower left corner of the user interface as the origin, the horizontal rightward direction of the user interface as the positive x-axis direction, and the vertical upward direction of the user interface as the positive y-axis direction. According to some embodiments, determining the position of the video playback area on the user interface of the electronic device includes determining the coordinates of the edges of the video playback area in the coordinate system, and taking all the coordinates within the edges as the video playback area.

[0121] According to some examples, in the case where the video playback area is a rectangular area, the linear functions of the four sides of the video playback area on the coordinate system may be determined. For example, the two horizontal sides may be y = a and y = b (where a < b) respectively, and the two vertical sides may be x = c and x = d (where c < d) respectively. Then, the area within all four sides is taken as the video playback area, for example, {c ≤ x ≤ d, and a ≤ y ≤ b}. According to some embodiments, corresponding the left side of the obtained video playback area to the pixels constituting the edges of the video playback area, the content displayed by these pixels may be recorded, thereby realizing recording a part of the user interface of the electronic device. Similarly, it is also possible to only intercept the content displayed by these pixels, thereby realizing automatically taking screenshots of a part of the user interface of the electronic device.

[0122] In this way, the interference of non-video playback areas is excluded, enabling the user to subsequently enjoy the video and / or screenshots of only the video playback area more purely, enhancing the user's viewing experience.

[0123] When the video being played is a movie, the video playback area can be further narrowed to determine the movie playback area. For example, the aspect ratio of a rectangular video playback area is typically 1.85:1, while the aspect ratio of a movie screen is typically 2.35:1. Therefore, the movie playback area can be further defined as, for example, {c≤x≤d, and a+(ba) / 9.4≤y≤b-(ba) / 9.4}.

[0124] According to some embodiments, determining the position of the video playback area on the user interface of the electronic device includes the following three methods.

[0125] In some cases, within a video playback application window, edge detection can be used to determine the video playback area because there is a significant change in grayscale at the boundary between the video playback area and its surrounding area.

[0126] According to some embodiments, determining the position of the video playback area on the user interface of the electronic device may include: detecting the border of the video playback area; and determining the area inside the border as the video playback area based on the border of the video playback area.

[0127] According to some embodiments, a method for detecting the border of the video playback area may include: acquiring an image of the current user interface of an electronic device; performing Gaussian smoothing denoising on the acquired image; performing convolution operation on the denoised image using an operator (e.g., Canny operator, Lapacian operator, etc.) to obtain the gradient of each pixel of the acquired image; determining pixels whose gradients are greater than a gradient threshold (e.g., the maximum value of the gradient); and connecting these pixels to obtain the border of the video playback area.

[0128] In some cases, video playback application developers often create multiple versions of the video playback application window to adapt to different electronic device models.

[0129] According to some embodiments, determining the position of the video playback area on the user interface of the electronic device may include: obtaining the resolution of the electronic device; determining the video playback application corresponding to the video playback application window currently displayed on the user interface of the electronic device; and determining the position of the video playback area on the user interface of the electronic device based on the obtained resolution of the electronic device and the determined video playback application corresponding to the video playback application window currently displayed on the user interface of the electronic device.

[0130] According to some embodiments, the resolution of the electronic device may include the resolution of the user interface and the resolution of the display screen. According to some embodiments, when playing video, the video playback area may depend on the resolution of the user interface and / or the display screen. Therefore, by determining the resolution of the user interface and / or the display screen, the video playback area can be accurately obtained.

[0131] According to some embodiments, determining the position of the video playback area on the user interface of the electronic device may include: obtaining the model of the electronic device; determining the video playback application corresponding to the video playback application window currently displayed on the user interface of the electronic device; and determining the position of the video playback area on the user interface of the electronic device based on the obtained model of the electronic device and the determined video playback application corresponding to the video playback application window currently displayed on the user interface of the electronic device.

[0132] According to some embodiments, the model number of an electronic device includes the manufacturer and the device's model. For example, the manufacturer of the obtained electronic device (phone) is Smartisan, the model is M1, and the currently displayed video playback application window corresponds to the video playback application TikTok. Based on the above information, the video playback area of ​​TikTok on the Smartisan M1 phone can be accurately obtained.

[0133] In some cases, the video playback area can be obtained by directly extracting the video stream from the electronic device's system.

[0134] According to some embodiments, the method may further include: acquiring the elapsed time of recording the user interface of the electronic device in real time; and determining that the screenshot stopping condition described in step S103 is met when the acquired elapsed time equals a stop threshold time. According to some embodiments, the stop threshold time may be preset by the user or set by the system default of the electronic device. According to some examples, the stop threshold time may be 20 seconds, 30 seconds, 1 minute, 2 minutes, etc.

[0135] According to some embodiments, the stop threshold time can be set to infinity.

[0136] According to some embodiments, the stop threshold time can be set to a sufficiently large value. In other words, the stop threshold time is set to a value that is almost impossible to reach when using the screen recording screenshot function normally. For example, the stop threshold time can be set to 0x7fffffff. The value 0x7fffffff is actually the maximum value in a 32-bit int. In seconds, 0x7fffffff is approximately 68 years, which can be approximated as setting the stop threshold time to infinity.

[0137] According to some embodiments, when the multiple screenshots are stored in memory, the available memory of the electronic device can be determined in real time. According to some embodiments, reserved memory can be allocated to the electronic device. This reserved memory may be memory required to ensure the normal operation of the electronic device's system. According to some examples, if the available memory determined in real time is reserved memory, then the elapsed time is determined as a stop threshold time.

[0138] According to other embodiments, when the plurality of screenshots are stored in the cache's storage area, the remaining space of the cache can also be determined in real time. According to some embodiments, reserved space can be allocated for the cache of the electronic device. According to some examples, if the available remaining space determined in real time is reserved space, then the elapsed time is determined as a stop threshold time.

[0139] According to some embodiments, the reserved memory and / or reserved space may vary depending on the model of the electronic device. According to some embodiments, the reserved memory and / or reserved space may also be associated with the application running on the electronic device. However, those skilled in the art will understand that this disclosure is not limited thereto, and the reserved memory and / or reserved space may be determined in any manner. Typically, the reserved memory and / or reserved space are not zero. However, in some cases, the reserved memory and / or reserved space may also be zero.

[0140] In some cases, users want to be able to determine when automatic screenshots will stop.

[0141] According to some embodiments, the method may further include: receiving a screenshot stop input for the user interface of the electronic device; and determining, in response to the screenshot stop input, that the screenshot stop condition is met.

[0142] In this way, users can stop the automatic screenshotting of the electronic device's user interface by taking a screenshot (e.g., clicking the screen recording stop button, clicking the screen recording window, pressing the user interface (e.g., user interface 300), etc.), which further increases the user's freedom of operation and enhances the user experience.

[0143] According to some embodiments, the method may further include: after stopping taking screenshots of the user interface of the electronic device and stopping recording the user interface of the electronic device, displaying thumbnails of the plurality of screenshots and thumbnails of the video on the user interface of the electronic device.

[0144] The following will combine Figure 4A and Figure 4B This describes in detail how to display recorded videos associated with the user interface of an electronic device, as well as multiple automatically captured screenshots associated with the user interface of the electronic device. Figure 4A This is a schematic diagram illustrating a user interface 400A displaying video and multiple screenshots in portrait mode on an electronic device according to an exemplary embodiment of the present disclosure. Figure 4B This is a schematic diagram illustrating a user interface 400B displaying video and multiple screenshots in landscape mode on an electronic device according to an exemplary embodiment of the present disclosure.

[0145] According to some embodiments, after stopping the recording of the user interface of the electronic device and stopping the taking of screenshots of the user interface of the electronic device, a thumbnail of the video and thumbnails of the multiple screenshots can be directly displayed on the user interface of the electronic device.

[0146] According to other embodiments, such as Figure 4A and Figure 4B As shown, after the recording of the user interface of the electronic device is stopped and the screenshotting of the user interface of the electronic device is stopped, video screenshot cards 410A and 410B can be displayed on the user interface 400A and 400B of the electronic device, and thumbnails of the video 420A and 420B and thumbnails of the plurality of screenshots 430A and 430B can be displayed on the video screenshot cards 410A and 410B.

[0147] In this disclosure, the aspect ratio is compared with 1 to determine whether the display orientation of the user interface and the thumbnails of the video and the thumbnails of the plurality of screenshots is portrait or landscape.

[0148] Specifically, in this disclosure, if the aspect ratio of the user interface is less than 1, the user interface is displayed in portrait orientation. If the aspect ratio of the user interface is greater than 1, the user interface is displayed in landscape orientation.

[0149] Specifically, in this disclosure, if the aspect ratio of the video thumbnail and the thumbnails of the plurality of screenshots is less than 1, then the display orientation of the video thumbnail and the thumbnails of the plurality of screenshots is portrait orientation. If the aspect ratio of the video thumbnail and the thumbnails of the plurality of screenshots is greater than 1, then the display orientation of the video thumbnail and the thumbnails of the plurality of screenshots is landscape orientation.

[0150] In some cases, during the recording of an electronic device's user interface and the automatic capture of screenshots of that user interface, the user may rotate the electronic device (e.g., rotating it from portrait to landscape orientation, or vice versa). In these cases, the thumbnails of the video and the thumbnails of the multiple screenshots may be displayed in an angle that is neither portrait nor landscape.

[0151] According to some embodiments, in response to the screenshot stop condition, thumbnails of screenshots whose display orientation is portrait or landscape can be retained and displayed, and thumbnails of screenshots whose display orientation is neither portrait nor landscape can be deleted.

[0152] This method ensures the neatness of the screenshots that are retained and displayed, resulting in a better visual experience.

[0153] Furthermore, in some of the aforementioned situations, the orientation of the electronic device may change during the recording of the device's user interface and the automatic screenshotting of that user interface, resulting in inconsistent orientations for multiple screenshots.

[0154] According to some embodiments, the display orientation of the video thumbnail and the thumbnails of the plurality of screenshots can be determined based on the orientation of the electronic device at the moment when screen recording stops.

[0155] According to some embodiments, the method may further include: determining a screenshot stopping direction, the screenshot stopping direction being the orientation of the electronic device at the moment when screenshotting the user interface of the electronic device is stopped and recording of the user interface of the electronic device is stopped; and displaying thumbnails of the plurality of screenshots and a thumbnail of the video on the user interface of the electronic device based on the screenshot stopping direction.

[0156] According to some embodiments, the screenshot stopping direction includes the portrait direction and the landscape direction as defined above.

[0157] According to some embodiments, the display orientation of the video thumbnail and the thumbnails of the plurality of screenshots is the same as the screenshot stopping orientation. According to some examples, when the screenshot stopping orientation is determined to be portrait mode, the display orientation of the video thumbnail and the thumbnails of the plurality of screenshots is also portrait mode. According to other examples, when the screenshot stopping orientation is determined to be landscape mode, the display orientation of the video thumbnail and the thumbnails of the plurality of screenshots is also landscape mode.

[0158] For example, the screenshot stops in landscape mode. Figure 4AAs shown, although the electronic device is currently oriented in portrait mode and the user interface 400A is also in portrait mode, the thumbnail 420A of the video and the thumbnails 430A of the multiple screenshots are displayed in landscape mode because the screenshot stopped in landscape mode. (Continue referring to...) Figure 4A Because the display orientation (landscape orientation) of the video thumbnail 420A and the thumbnails 430A of the multiple screenshots is inconsistent with the display orientation (portrait orientation) of the user interface 400A, a black border is displayed above and below these thumbnails. Figure 4B As shown, the electronic device's real-time orientation is landscape, the user interface 400A is displayed in landscape mode, and since the screenshot stops in landscape mode, the video thumbnail 420A and the thumbnails 430A of the multiple screenshots are also displayed in landscape mode. (Continue referring to...) Figure 4B Since the display orientation (landscape orientation) of the video thumbnail 420B and the thumbnails 430B of the multiple screenshots is consistent with the display orientation (portrait orientation) of the user interface 400B, no black border areas are displayed above and below these thumbnails.

[0159] In this way, even if the electronic device rotates during the recording of the user interface and the automatic screenshotting of the user interface (e.g., the electronic device rotates from portrait to landscape or vice versa), the thumbnails of the video and the thumbnails of the multiple screenshots can still be displayed in the same direction, so that the user's visual perception is consistent and very neat when watching.

[0160] According to some embodiments, the method may further include: receiving a display input for a thumbnail of a first screenshot among the plurality of screenshots; and, in response to the display input, displaying the first screenshot in full screen on the user interface of the electronic device; receiving a playback input for a thumbnail of the video; and, in response to the playback input, playing the video in full screen on the user interface of the electronic device.

[0161] According to some embodiments, playback input for the thumbnail of the video may include: play buttons 422A, 422B on the thumbnail of the video (see reference). Figure 4A and Figure 4B The input can include clicking on the thumbnail of the video, pressing on the thumbnail of the video, etc. According to some embodiments, the input for displaying the thumbnail of the first screenshot among the plurality of screenshots may include clicking on the zoom-in button on the thumbnail of the first screenshot, pressing on the thumbnail of the first screenshot, etc.

[0162] According to some embodiments, the video played in full screen is the original recorded video, with a resolution higher than the thumbnail of the video. The first screenshot displayed in full screen is the original image obtained by automatic capture, with a resolution higher than the thumbnail of the first screenshot.

[0163] According to some embodiments, black borders in multiple screenshots can be removed before displaying multiple thumbnails associated with a user interface of an electronic device. This allows the image after the black borders have been removed to be saved and shared during operations such as saving and sharing, as described below.

[0164] According to some embodiments, since there is a significant change in grayscale at the boundary between the black border area and other areas in the screenshot, edge detection methods can also be used to determine the black border area and delete the detected black border area in the screenshot.

[0165] Based on some examples, Gaussian smoothing can be applied to each of the plurality of screenshots for denoising; convolution operations can be performed on the denoised image using operators (e.g., Canny operator, Lapacian operator, etc.) to obtain the gradient of each pixel in each screenshot; pixels whose gradients are greater than a gradient threshold (e.g., the maximum value of the gradient) can be identified; these pixels can be concatenated to identify black border regions in each screenshot (if black border regions exist); and black border regions can be removed from screenshots containing black border regions.

[0166] In this way, if the black border area is not removed during the automatic screenshot of the electronic device's user interface, it can still be removed from multiple screenshots associated with the electronic device's user interface, thus reducing the memory usage of the electronic device during automatic screenshotting.

[0167] According to other embodiments, it is also possible to remove the black border area both during the automatic screenshotting of the user interface of the electronic device and in the multiple screenshots associated with the user interface of the electronic device.

[0168] According to some embodiments, the method may further include: receiving a first save input for a first save button 411A, 411B displayed on the user interface of the electronic device; and in response to the first save input, storing the plurality of screenshots to a screenshot folder and storing the video to a video folder.

[0169] According to some embodiments, the first save input may be a click input, a press input, or the like, of a first save button displayed on the user interface of the electronic device.

[0170] In this way, the video and multiple screenshots can be saved simultaneously through the first save input. Compared with related technologies, users do not need to save the video and multiple screenshots separately, simplifying the user's operation.

[0171] According to some embodiments, the method may further include: receiving a first share input for a first share button and a first application option displayed on a user interface of the electronic device, wherein the first application option corresponds to a first application; and in response to the first share input, sharing the plurality of screenshots and the video to the first application.

[0172] According to some embodiments, the first application option can be directly displayed on the user interface of the electronic device. The first sharing input can be a click input to a first share button or a click input to the first application option.

[0173] According to other embodiments, the first application option may not be directly displayed on the user interface of the electronic device, and the first sharing input includes a first input and a second input. The first input of the first sharing input includes a click, press, or swipe input on the first share button. In response to the first input of the first sharing input, a list of target sharing applications is displayed on the user interface, the list including the first application option. The second input of the first sharing input includes a click, press, or swipe input on the first application option. In response to the second input of the first sharing input, the video and the at least one screenshot are shared to the first application corresponding to the first application option.

[0174] In this way, there's no need to display the first application option on the electronic device's user interface before receiving the first share input, which makes the user interface cleaner, especially in cases with a large number of application options. Furthermore, dividing the first share input into a first input and a second input gives users more options when deciding which application to share videos and multiple screenshots to.

[0175] The following will combine Figure 5 Describe the editing operations for the video and multiple screenshots. Figure 5 This is a schematic diagram illustrating a user interface 500 for editing displayed videos and multiple screenshots according to an exemplary embodiment of the present disclosure.

[0176] According to some embodiments, the method may further include: receiving selection input for one or more thumbnails of the thumbnails of the video and the plurality of screenshots; selecting the one or more thumbnails in response to the selection input; and deselecting the one or more thumbnails again in response to the selection input.

[0177] According to some embodiments, the selection input for the thumbnail of the video may include clicking, pressing, etc., on the thumbnail of the video. According to some embodiments, the selection input for one or more thumbnails among the thumbnails of the plurality of screenshots may include clicking, pressing, swiping, etc., on the one or more thumbnails. For example, when the selection input is a swipe input, responding to the selection input to select the one or more thumbnails may include determining the swipe path of the swipe input on the user interface, determining the thumbnails traversed by the swipe path, and selecting the one or more thumbnails traversed by the swipe path as the selected thumbnails.

[0178] In this way, users can select thumbnails corresponding to screenshots in various ways according to their preferences and the number of screenshots they wish to select, providing users with greater freedom and improving the user experience.

[0179] According to some embodiments, in response to the selection input, one or more selected thumbnails can be highlighted. According to some examples, a selection marker can be displayed on the selected one or more thumbnails. For example, as... Figure 5 As shown, cut Figure 5 thumbnails and screenshots Figure 7 If the thumbnail is selected, then in the screenshot Figure 5 thumbnails and screenshots Figure 7 A checkmark symbol is displayed on the thumbnail as a selection marker (532). According to some examples, a bold border can be displayed around one or more selected thumbnails. According to some examples, the selected one or more thumbnails can be slightly enlarged in size. According to some examples, the brightness of one or more selected thumbnails can be increased.

[0180] According to some embodiments, in response to the selected input again, the selected one or more thumbnails may no longer be highlighted. According to some examples, the selection marker may no longer be displayed. For example, the checkmark symbol as selection marker 532 may no longer be displayed. According to some examples, the bold border surrounding the selected one or more thumbnails may no longer be displayed. According to some examples, one or more thumbnails that have been slightly enlarged may be reduced to their original size. According to some examples, the brightness of one or more thumbnails that have been brightened may be reduced to their original brightness.

[0181] This method allows for clearer feedback to users on which screenshots have been selected and which screenshots have been deselected.

[0182] According to some embodiments, the method may further include: receiving a second save input for a second save button displayed on the user interface of the electronic device when one or more thumbnails are selected; and in response to the second save input, storing the screenshots and / or videos corresponding to the selected one or more thumbnails in the plurality of screenshots and / or videos to a screenshot folder and / or a video folder.

[0183] According to some embodiments, a second save button may be displayed when the thumbnails of the video and the multiple screenshots are shown. According to some examples, when the thumbnails of the video and the multiple screenshots are displayed, the second save button cannot receive a second save input. According to other examples, when the thumbnails of the video and the multiple screenshots are displayed, the second save button cannot receive a second save input, and when one or more thumbnails are selected, the second save button is configured to receive a second save input. For example, when the second save button is in a state where it can receive a second save input, it can be displayed normally; while when the second save button is in a state where it cannot receive a second save input, it can be grayed out.

[0184] According to some embodiments and references Figure 5 A second save button 548 can be displayed when one or more of the thumbnails are selected. According to... Figure 5 As shown in some examples, when one or more thumbnails are selected, an edit border 540 is displayed on the video screenshot card 510. The upper edge of the edit border 540 may be above the topmost thumbnail of the selected one or more thumbnails, and the lower edge of the edit border 540 may be below the bottommost thumbnail of the selected one or more thumbnails. This provides a better spatial representation of the selected one or more thumbnails. According to some examples, the second save button 548 is displayed on the upper edge of the edit border 540, but this disclosure is not limited thereto.

[0185] According to some embodiments, the second save input for the second save button displayed on the user interface of the electronic device may include a click input, a press input, etc., for the second save button.

[0186] In this way, through the second save input, users can select their favorite videos and / or screenshots from the thumbnails of the videos and the thumbnails of the multiple screenshots, and save these videos and / or screenshots to the video folder and / or screenshot folder respectively.

[0187] According to some embodiments, the method may further include: when one or more thumbnails are selected, receiving a second sharing input for a second share button 546 and a second application option displayed on the user interface of the electronic device, wherein the second application option corresponds to a second application; and in response to the second sharing input, sharing the screenshots and / or videos corresponding to the selected one or more thumbnails from the plurality of screenshots and / or videos to the second application.

[0188] According to some embodiments, the various display times (e.g., when displaying thumbnails of the video and the thumbnails of the plurality of screenshots, when selecting one or more thumbnails, etc.), display positions (e.g., displayed on the user interface, displayed on the video screenshot card, displayed on the editing border, etc.), and display methods (e.g., able to receive second sharing input (normal display), unable to receive second sharing input (grayed out), etc.) of the second share button may be the same as or similar to the second save button described in detail above, and will not be repeated here.

[0189] According to some embodiments, the second sharing input for the second share button and the second application option displayed on the user interface of the electronic device may include a click input, a press input, etc., for the second share button.

[0190] According to some embodiments, various input methods for the second sharing input (e.g., directly displaying the second application option, or displaying a list of sharing target applications including the second application option in response to the first input of the second sharing input and sharing screenshots corresponding to one or more selected thumbnails to the second application corresponding to the second application option in response to the second input of the second sharing input) may be the same as or similar to the first sharing input described in detail above, and will not be repeated here.

[0191] In this way, through a second sharing input, users can select their favorite videos and / or screenshots from the thumbnails of the videos and the thumbnails of the multiple screenshots, and share these videos and / or screenshots. Figure 1 Share it to a second application.

[0192] According to some embodiments, the method may further include: receiving a puzzle input 542 for a puzzle button displayed on the user interface of the electronic device when one or more thumbnails are selected; and generating a stitched image of one or more screenshots corresponding to the selected one or more thumbnails in response to the puzzle input, wherein the stitched image spatially includes one or more screenshots corresponding to the selected one or more thumbnails.

[0193] According to some embodiments, various display timings of the puzzle button (e.g., when displaying the thumbnail of the video and the thumbnails of the multiple screenshots, when one or more thumbnails are selected, etc.), display positions (e.g., displayed on the user interface, displayed on the video screenshot card, displayed on the editing border, etc.), and display manners (e.g., can receive puzzle input (normal display), cannot receive puzzle input (grayed out), etc.) are similar to the second save button described in detail above, and will not be elaborated here.

[0194] According to some embodiments, when one or more thumbnails are selected, it is determined whether the one or more selected thumbnails include the thumbnail of the video. According to some examples, if the one or more selected thumbnails include the thumbnail of the video, the puzzle button cannot receive puzzle input. According to other examples, regardless of whether the one or more selected thumbnails include the thumbnail of the video, the puzzle button can receive puzzle input, and in response to the puzzle input, the thumbnail of the selected video is ignored, and only a montage of one or more screenshots corresponding to the one or more selected thumbnails is generated.

[0195] According to some embodiments, the puzzle input for the puzzle button displayed on the user interface of the electronic device may include click input, press input, etc. for the puzzle button.

[0196] According to some embodiments, the montage is a picture that spatially includes one or more screenshots corresponding to all the one or more selected thumbnails. The one or more screenshots can be arranged in the montage in any manner.

[0197] According to some embodiments, generating a montage of one or more screenshots corresponding to the one or more selected thumbnails among the multiple screenshots may include: determining the number of selected thumbnails; and if the number is equal to one, using the one screenshot corresponding to the selected one thumbnail as the montage; if the number is an odd number greater than one, generating the montage in a manner that one screenshot corresponding to the multiple selected thumbnails is arranged in any row and two screenshots corresponding to the multiple selected thumbnails are arranged in each of the remaining rows; if the number is an even number, generating the montage in a manner that two screenshots corresponding to the multiple selected thumbnails are arranged in each row. According to some examples, if the number of selected thumbnails is an odd number greater than 1, one screenshot can be arranged in the first row and two screenshots can be arranged in each of the remaining rows. For example, when the number of selected thumbnails is three, the three screenshots can be arranged in a "pin" shape. Of course, the present disclosure is not limited thereto. According to other examples, the screenshots corresponding to the multiple selected thumbnails can also be arranged in various shapes such as a circle, a heart shape, etc.

[0198] For example, when a user records a tutorial demonstration of an application, they can select screenshots representing several key steps in the demonstration and stitch these screenshots together into a composite image. This composite image allows the user to showcase the essence of their tutorial demonstration without having to watch the entire recorded video.

[0199] According to some embodiments, the method may further include: receiving animated input for an animated button 544 displayed on the user interface of the electronic device when one or more thumbnails are selected; and generating an animated GIF that displays one or more screenshots corresponding to the selected one or more thumbnails at a certain frequency in response to the animated input, wherein the animated GIF includes one or more screenshots corresponding to the selected one or more thumbnails in time.

[0200] According to some embodiments, the various display timings (e.g., when displaying thumbnails of the video and the thumbnails of the plurality of screenshots, when selecting one or more thumbnails, etc.), display positions (e.g., displayed on the user interface, displayed on the video screenshot card, displayed on the editing border, etc.), and display methods (e.g., able to receive animated input (normal display), unable to receive animated input (grayed out), etc.) of the animated button are similar to those of the second save button described in detail above, and will not be repeated here.

[0201] According to some embodiments, when one or more thumbnails are selected, it is determined whether the selected thumbnails include video thumbnails. According to some examples, if the selected thumbnails include video thumbnails, the GIF button cannot receive GIF input. According to other examples, regardless of whether the selected thumbnails include video thumbnails, the GIF button can receive GIF input, and in response to the GIF input, the selected video thumbnails are ignored, and only GIFs of one or more screenshots corresponding to the selected thumbnails are generated.

[0202] According to some embodiments, animated input for animated buttons displayed on the user interface of the electronic device may include click input, press input, etc. for the animated button.

[0203] According to some embodiments, an animated GIF is generated by continuously playing one or more screenshots corresponding to all selected thumbnails at a certain frame rate. The animated GIF is an image, which may be in GIF format, and includes, in time, one or more screenshots corresponding to all selected thumbnails.

[0204] For example, when a user is recording a movie, they can select multiple screenshots representing key scenes and plot points, and use these screenshots to generate an animated GIF. This GIF, which is much shorter than the movie's runtime, allows the user to showcase key scenes and plot points without having to watch the entire recorded film.

[0205] In some cases, the recording time may be relatively long, resulting in a relatively large number of screenshots. In such situations, one or more screenshots can be automatically recommended to the user based on certain criteria.

[0206] According to some embodiments, the method may further include: evaluating the plurality of screenshots based on preset rules to obtain a score associated with each of the plurality of screenshots; comparing the score associated with each screenshot with a score threshold; and highlighting one or more screenshots among the plurality of screenshots whose scores are greater than or equal to the score threshold on the user interface of the electronic device.

[0207] In some embodiments, the scoring threshold can be fixed. For example, the scoring threshold can be preset by the user or set by system default.

[0208] According to other embodiments, the scoring threshold can be varied. As some examples, the maximum score associated with each screenshot is taken, and a certain percentage (less than 1) of this maximum score is used as the scoring threshold, such as 90%, 95%, 97%, etc. Another example is that the scores associated with each screenshot are sorted, and the Nth highest score is used as the scoring threshold, where N is, for example, 1, 2, 3, 5, etc.

[0209] According to some embodiments, one or more screenshots with scores greater than or equal to the score threshold can be highlighted in various ways. For example, a star icon can be displayed on one or more thumbnails of the screenshots with scores greater than or equal to the score threshold. As another example, one or more thumbnails of the screenshots with scores greater than or equal to the score threshold can be displayed in a blinking manner.

[0210] According to some embodiments, the preset rules may include at least one of the following: whether a face exists in the screenshot, the completeness of the face in the screenshot, the resolution of the screenshot, and the sharpness of the screenshot.

[0211] According to some embodiments, screenshots containing faces score higher than screenshots without faces. Screenshots with high face completeness score higher than screenshots with low face completeness. Screenshots with high resolution score higher than screenshots with low resolution. Screenshots with high sharpness score higher than screenshots with low sharpness.

[0212] According to some embodiments, each item in the preset rules can be assigned a specific associated weight, and the item score of each item can be multiplied by the specific associated weight and summed to obtain the score of the screenshot.

[0213] The following will combine Figure 4A , Figure 4B and Figure 5 Describe other buttons on user interfaces 400A, 400B, and 500 and their corresponding functions. According to some embodiments, in response to input to the select all buttons 412A, 412B, and 512 (e.g., click input, press input, etc.), thumbnails of the plurality of screenshots can be selected. Furthermore, in response to further input to the select all buttons 412A, 412B, and 512 (e.g., click input, press input, etc.), the thumbnail of the video and the thumbnails of the plurality of screenshots can no longer be selected. According to some embodiments, in response to input to… Figure 5 The input to the exit selection button 550 shown (e.g., click input, press input, etc.) can deselect one or more thumbnails. According to some embodiments, in response to... Figure 4A , Figure 4B and Figure 5 The input of the close buttons 413A, 413B, and 513 shown (e.g., click input, press input, etc.) can close and exit the screen recording and screenshot function.

[0214] According to some embodiments, audio information associated with the user interface of an electronic device can be recorded synchronously during the recording of the user interface of the electronic device.

[0215] According to some embodiments, the method may further include: in response to a recording start condition being met, during recording of the user interface of the electronic device, acquiring audio information associated with the electronic device, wherein the audio information corresponds to the video on a timeline.

[0216] According to some embodiments, the audio information associated with the electronic device may include audio information played aloud through an audio output device of the electronic device. According to some embodiments, the audio information associated with the electronic device may include audio information acquired through an audio input device (e.g., a microphone). According to some embodiments, the audio information associated with the electronic device may also include audio information acquired directly from the underlying hardware of the electronic device. According to some embodiments, the audio information associated with the electronic device may also include a combination of two or more of the various types of audio information associated with the electronic device described above.

[0217] According to some embodiments, audio information can be extracted from the video associated with the user interface of the electronic device after recording of the user interface of the electronic device has stopped.

[0218] According to some embodiments, the method may further include: real-time identification and obtaining text information corresponding to the audio information on a timeline; and adding the text information to the video and / or the plurality of screenshots according to the timeline, so that the text information can be displayed simultaneously with the video and / or the plurality of screenshots.

[0219] According to some embodiments, a timeline-based approach allows audio and video information to correspond one-to-one at the same time. For example, if the current user interface is displaying a movie, and audio information can be acquired simultaneously during screen recording, then during playback, the video and audio information can be combined according to the timeline. This allows the user to both see the movie's visuals displayed on the user interface and hear the corresponding audio.

[0220] According to some embodiments, text information corresponding to video and audio information on the timeline can be identified in real time. For example, if the current user interface is displaying a movie, and audio information is simultaneously acquired and identified in real time during screen recording to obtain text information, then during playback, the video and text information can be combined according to the timeline. This allows users who are unable to speak, or who are hearing impaired, to both view the movie screen displayed on the user interface and see real-time subtitles corresponding to the movie.

[0221] According to some embodiments, a timeline-based approach allows audio, text, and video information to correspond one-to-one at the same time. For example, if the current user interface is displaying a movie, and audio information is simultaneously acquired and recognized in real time during screen recording to obtain text information, then during playback, video, audio, and text information can be combined according to the timeline, allowing the user to simultaneously view the movie visuals, the corresponding audio, and the real-time subtitles.

[0222] It should be noted that although the above description only uses movies as an example, this disclosure is not limited to any particular scene. Audio information can be acquired and text information corresponding to the audio information can be identified in real time during the recording of any scene.

[0223] According to some embodiments, a thumbnail of the audio information corresponding to the audio information can be displayed in the video screenshot card as described above. According to some embodiments, the audio information can be played in response to a playback input to the thumbnail of the audio information (e.g., a click input to a play button displayed on the thumbnail of the audio information, a press input to the thumbnail of the audio information, etc.). According to some embodiments, in response to the first save input described above, in addition to saving the video to a video folder and the plurality of screenshots to a screenshot folder, the audio information can also be saved to an audio folder. According to some embodiments, in response to the first share input described above, in addition to sharing the video and the plurality of screenshots to a first application, the audio information can also be shared to the first application.

[0224] Exemplary methods for operating electronic devices according to the present disclosure have been described above with reference to the accompanying drawings. Exemplary apparatuses for operating electronic devices according to the present disclosure, and exemplary embodiments of the electronic devices, will now be further described with reference to the accompanying drawings.

[0225] A second aspect of this disclosure may include a means for operating electronic equipment. Figure 6 This is a structural block diagram illustrating an apparatus for operating an electronic device according to exemplary embodiments of the present disclosure. Figure 6 As shown, the device 600 for operating an electronic device may include: a determining unit 601 configured to determine whether a screenshot start condition is met; a screenshot unit 602 configured to automatically take screenshots of the user interface of the electronic device to obtain multiple screenshots associated with the user interface of the electronic device; and a stopping unit 603 configured to stop taking screenshots of the user interface of the electronic device in response to determining that a screenshot stop condition is met.

[0226] According to some embodiments, the apparatus 600 for operating an electronic device may further include units configured to perform other steps of any of the foregoing methods.

[0227] The exemplary apparatus for operating an electronic device disclosed herein has been briefly described above. It should be noted that specific embodiments of each unit or feature of the exemplary apparatus for operating an electronic device disclosed herein can be found in the specific embodiments of the corresponding features in the foregoing exemplary methods for operating an electronic device disclosed herein, and therefore will not be repeated here. Furthermore, the units involved in describing the exemplary apparatus for operating an electronic device disclosed herein can be implemented in software or hardware. The names of the units do not, in some cases, constitute a limitation on the unit itself.

[0228] Furthermore, the functions of each unit of the exemplary apparatus for operating electronic devices disclosed herein can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0229] A third aspect of this disclosure may include an electronic device that may include a processor; and a memory storing a program that includes instructions that, when executed by the processor, cause the processor to perform any of the foregoing methods.

[0230] A fourth aspect of this disclosure may include a storage medium storing a program, the program including instructions that, when executed by a processor of an electronic device, cause the electronic device to perform any of the foregoing methods.

[0231] The following is for reference. Figure 7 It shows a schematic diagram of the structure of an electronic device 700 (e.g., the electronic device in this disclosure) suitable for implementing embodiments of the present disclosure. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0232] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 706 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0233] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 706 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0234] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 706, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0235] It should be noted that the storage medium described above in this disclosure can be a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0236] In some implementations, examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any networks currently known or developed in the future.

[0237] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0238] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine whether a screenshot start condition is met; in response to determining that the screenshot start condition is met, record the user interface of the electronic device to obtain a video associated with the user interface of the electronic device; automatically take screenshots of the user interface of the electronic device during the recording of the user interface of the electronic device to obtain multiple screenshots associated with the user interface of the electronic device; and in response to determining that a screenshot stop condition is met, stop recording the user interface of the electronic device and stop taking screenshots of the user interface of the electronic device.

[0239] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine whether a screenshot start condition is met; in response to determining that the screenshot start condition is met, record the user interface of the electronic device to obtain video associated with the user interface of the electronic device; automatically take screenshots of the user interface of the electronic device during the recording of the user interface of the electronic device to obtain multiple screenshots associated with the user interface of the electronic device; and in response to determining that a screenshot stop condition is met, stop recording the user interface of the electronic device and stop taking screenshots of the user interface of the electronic device.

[0240] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0241] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0242] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0243] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0244] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0245] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for operating an electronic device, comprising: Determine if the conditions for starting a screenshot are met; In response to determining that the screenshot start condition is met, the user interface of the electronic device is automatically captured at dynamic time intervals to obtain multiple screenshots associated with the user interface of the electronic device, and the automatic screenshot operation does not affect the user's current use of the electronic device, wherein the dynamic time interval is determined based on the elapsed time of the automatic screenshot, and the automatic screenshot of the user interface of the electronic device includes: acquiring the elapsed time of the automatic screenshot of the user interface of the electronic device in real time; when the acquired elapsed time is within a first range, automatically capturing the user interface of the electronic device at a first time interval; and when the acquired elapsed time is within a second range, automatically capturing the user interface of the electronic device at a second time interval, wherein the values ​​in the first range are all less than the values ​​in the second range, and the first time interval is less than the second time interval; and During the automatic screenshotting of the user interface of the electronic device, it is determined whether the user interface of the electronic device is displaying a video playback application window corresponding to the video playback application, wherein the video playback application window includes a video playback area; In response to determining that the screenshot start condition is met, the user interface of the electronic device is recorded to obtain a video associated with the user interface of the electronic device; wherein, in response to determining that the user interface of the electronic device is displaying the video playback application window, only the video playback area is recorded; In response to determining that the screenshot stop condition is met, the automatic screenshotting of the user interface of the electronic device is stopped.

2. The method according to claim 1, wherein, The step of determining whether the screenshot start condition is met includes: in response to receiving an automatic screenshot instruction input for the user interface of the electronic device, determining that the screenshot start condition is met.

3. The method according to claim 1, further comprising: Receives an automatic screenshot instruction input for the user interface of the electronic device. The determination of whether the screenshot start condition is met includes: Determine whether the user interface of the electronic device displays an application window when the automatic screenshot instruction is received; In response to determining that the user interface of the electronic device displays an application window when the automatic screenshot instruction input is received, it is determined whether the application window is allowed to be screenshotted; and In response to determining that the application window is allowed to be screenshotted, it is determined that the screenshot start condition is met.

4. The method according to claim 3, wherein, The determination of whether the screenshot start condition is met also includes: In response to determining that the user interface of the electronic device does not display an application window when the automatic screenshot instruction input is received, it is determined that the screenshot start condition is met.

5. The method according to any one of claims 2 to 4, wherein, The receiving of an automatic screenshot instruction input for the user interface of the electronic device includes: Receive function menu call input for the user interface of the electronic device; In response to receiving the function menu call input, a function selection menu is displayed on the user interface of the electronic device, wherein the function selection menu includes a screen recording / screenshot function option; and Receive the selection of the screen recording screenshot function options.

6. The method according to claim 1, further comprising: Receives an automatic screenshot instruction input for the user interface of the electronic device. The determination of whether the screenshot start condition is met includes: The location of the automatically captured screenshot instruction on the user interface of the electronic device is obtained; Determine whether the user interface of the electronic device at the location belongs to a text-extractable area; and In response to determining that the user interface of the electronic device is not located in a text-extractable area at the specified location, the screenshot start condition is determined to be met.

7. The method according to claim 6, wherein, The receiving of an automatic screenshot instruction input for the user interface of the electronic device includes: Receive function menu call input for the user interface of the electronic device; In response to receiving the function menu call input, a function selection menu is displayed on the user interface of the electronic device, wherein the function selection menu includes a screen recording / screenshot function option; and Receive the selection of the screen recording screenshot function options.

8. The method according to claim 7, wherein, The step of obtaining the location of the automatic screenshot instruction input on the user interface of the electronic device includes: The location of the function menu call-up input on the user interface of the electronic device is obtained.

9. The method according to claim 6, further comprising: In response to determining that the user interface of the electronic device is a text-extractable area at the location, a function selection menu is displayed on the user interface of the electronic device, wherein the function selection menu includes a screen recording / screenshot function option; Receive the selection of options for the screen recording screenshot function; In response to receiving a selection for the screen recording screenshot function option, it is determined that the screenshot start condition is met.

10. The method according to claim 1, wherein, The second time interval is N times the first time interval, where N is a positive integer greater than 1, and the method further includes: When the real-time acquired time is within the second range, one screenshot is retained and the others are deleted from every N screenshots associated with the user interface of the electronic device that were acquired when the time was previously within the first range.

11. The method according to claim 1, further comprising: In response to determining that the screenshot stop condition is met, recording of the user interface of the electronic device is stopped.

12. The method of claim 11, further comprising: In response to determining that the automatic screenshot instruction input meets the screenshot start condition, a screen recording window is displayed on the user interface of the electronic device. The screen recording window indicates that the user interface of the electronic device is being recorded.

13. The method of claim 12, wherein, The automatic screenshotting of the user interface of the electronic device includes: During the automatic screenshotting of the user interface of the electronic device, the screen recording window is not automatically screenshotted, and The recording of the user interface of the electronic device includes: The screen recording window is not recorded during the recording of the user interface of the electronic device.

14. The method according to claim 13, wherein, The phrase "not automatically taking screenshots of the screen recording window" and / or "not recording the screen recording window" includes: Adjust the capture attribute of the screen recording window to be uncaptureable; When automatically taking a screenshot of the user interface of the electronic device and / or when recording the user interface of the electronic device begins, the capture attributes of the screen recording window are read and received. In response to receiving that the screen recording window's capture attribute is not captureable, the screen recording window is not automatically captured during the automatic capture of the electronic device's user interface and / or the screen recording window is not recorded during the recording of the electronic device's user interface.

15. The method of claim 14, wherein, Displaying a screen recording window on the user interface of the electronic device includes: Render the user interface of an electronic device onto a user interface view control in real time; and The user interface view control is set in the screen recording window to display a thumbnail of the user interface of the electronic device in real time in the screen recording window.

16. The method of claim 11, further comprising: in, The automatic screenshotting of the user interface of the electronic device includes: in response to determining that the user interface of the electronic device is displaying the video playback application window, taking a screenshot only of the video playback area.

17. The method of claim 16, wherein, The automatic screenshotting of the user interface of the electronic device and / or the recording of only the video playback area includes: Determine the position of the video playback area on the user interface of the electronic device; and Based on the position of the video playback area on the user interface of the electronic device, a screenshot and / or recording of the video playback area is performed.

18. The method of claim 17, wherein, Determining the position of the video playback area on the user interface of the electronic device includes: Detect the border of the video playback area; and Based on the border of the video playback area, the area inside the border is defined as the video playback area.

19. The method of claim 17, wherein, Determining the position of the video playback area on the user interface of the electronic device includes: Obtain the resolution of the electronic device; Determine the video playback application corresponding to the video playback application window currently displayed on the user interface of the electronic device; and Based on the obtained resolution of the electronic device and the determined video playback application corresponding to the video playback application window currently displayed on the user interface of the electronic device, the position of the video playback area on the user interface of the electronic device is determined.

20. The method according to claim 1, further comprising: When the elapsed time obtained in real time equals the stop threshold time, it is determined that the screenshot stop condition is met.

21. The method according to claim 20, wherein, The stop threshold time is preset by the user or set by the system default of the electronic device.

22. The method according to claim 1, further comprising: Receive a screenshot of the user interface of the electronic device and stop input; as well as In response to the screenshot input stopping, it is determined that the screenshot stopping condition is met.

23. The method of claim 11, further comprising: After stopping the taking of screenshots of the user interface of the electronic device and stopping the recording of the user interface of the electronic device, thumbnails of the multiple screenshots and thumbnails of the video are displayed on the user interface of the electronic device.

24. The method of claim 23, further comprising: Determine the screenshot stopping direction, which is the orientation of the electronic device at the moment when the screenshot of the user interface of the electronic device is stopped and the recording of the user interface of the electronic device is stopped; as well as Based on the stopping direction of the screenshots, thumbnails of the multiple screenshots and a thumbnail of the video are displayed on the user interface of the electronic device.

25. The method of claim 23, further comprising: Receive display input for a thumbnail of the first screenshot among the plurality of screenshots; as well as In response to the display input, the first screenshot is displayed in full screen on the user interface of the electronic device; Receive playback input for a thumbnail of the video; as well as In response to the playback input, the video is played in full screen on the user interface of the electronic device.

26. The method of claim 23, further comprising: Receive a first save input in response to a first save button displayed on the user interface of the electronic device; as well as In response to the first save input, the plurality of screenshots are stored in the screenshot folder, and the video is stored in the video folder.

27. The method of claim 23, further comprising: Receive a first share input for a first share button and a first application option displayed on the user interface of the electronic device, wherein the first application option corresponds to a first application; as well as In response to the first sharing input, the plurality of screenshots and the video are shared to the first application.

28. The method of claim 23, further comprising: Receive selection input for one or more thumbnails of the video and the plurality of screenshots; In response to the selected input, select one or more thumbnails; as well as In response to the selected input again, the one or more thumbnails are no longer selected.

29. The method of claim 28, further comprising: When one or more of the thumbnails are selected, a second save input is received for a second save button displayed on the user interface of the electronic device; as well as In response to the second save input, the screenshots and / or videos corresponding to the selected one or more thumbnails in the plurality of screenshots and / or videos are stored in the screenshot folder and / or video folder.

30. The method of claim 28, further comprising: When one or more thumbnails are selected, a second share input is received for a second share button and a second application option displayed on the user interface of the electronic device, wherein the second application option corresponds to a second application; In response to the second sharing input, the screenshots and / or videos corresponding to the selected one or more thumbnails are shared to the second application.

31. The method of claim 28, further comprising: When one or more thumbnails are selected, puzzle input is received for the puzzle button displayed on the user interface of the electronic device; as well as In response to the image stitching input, a stitched image is generated that combines one or more screenshots from the plurality of screenshots that correspond to the selected one or more thumbnails. The stitched image spatially includes one or more screenshots corresponding to the selected one or more thumbnails.

32. The method according to claim 31, wherein, The step of generating a stitched image of one or more screenshots corresponding to the selected one or more thumbnails from the plurality of screenshots includes: Determine the number of thumbnails selected; and If the quantity is equal to one, then a screenshot corresponding to one of the selected thumbnails will be used as the stitched image; If the number is an odd number greater than one, the stitched image is generated by arranging one screenshot corresponding to the multiple screenshots of the selected multiple thumbnails in any row and two screenshots corresponding to the multiple screenshots of the selected multiple thumbnails in each of the remaining rows. If the number is even, the stitched image is generated by arranging two screenshots from each row corresponding to the selected thumbnails.

33. The method of claim 28, further comprising: When one or more of the thumbnails are selected, animated input is received for the animated button displayed on the user interface of the electronic device; as well as In response to the animated GIF input, an animated GIF is generated that displays one or more screenshots corresponding to the selected one or more thumbnails at a certain frequency. The animated image includes, in time, one or more screenshots corresponding to the selected one or more thumbnails.

34. The method of claim 23, further comprising: Based on preset rules, the multiple screenshots are evaluated to obtain a score associated with each of the multiple screenshots; The score associated with each screenshot is compared with the score threshold. as well as One or more screenshots with scores greater than or equal to the score threshold are highlighted on the user interface of the electronic device.

35. The method according to claim 34, wherein, The preset rules include at least one of the following: whether a face exists in the screenshot, the completeness of the face in the screenshot, the resolution of the screenshot, and the sharpness of the screenshot.

36. The method of claim 11, further comprising: In response to the fulfillment of the recording start condition, during the recording of the user interface of the electronic device, audio information associated with the electronic device is acquired, wherein the audio information corresponds to the video on the timeline.

37. The method of claim 36, further comprising: Real-time identification and acquisition of text information corresponding to the audio information on the timeline; as well as The text information is added to the video and / or the multiple screenshots according to the timeline, so that the text information can be played along with the video and / or displayed along with the multiple screenshots according to the timeline.

38. The method according to claim 1, wherein, When automatically taking a screenshot of the user interface of the electronic device, no screenshot animation is displayed on the user interface of the electronic device.

39. The method according to claim 1, wherein, When automatically taking a screenshot of the user interface of the electronic device, the electronic device does not produce a screenshot sound effect.

40. The method according to claim 1, wherein, The user interface of the electronic device is a touch-sensitive user interface.

41. A means for operating an electronic device, comprising: The determination unit is configured to determine whether the screenshot start condition is met. A screenshot unit is configured to automatically take screenshots of the user interface of the electronic device at dynamic time intervals to obtain multiple screenshots associated with the user interface of the electronic device, and to ensure that the automatic screenshot operation does not affect the user's current use of the electronic device. The dynamic time interval is determined based on the elapsed time of the automatic screenshot. The automatic screenshot of the user interface includes: acquiring the elapsed time of the automatic screenshot in real time; taking an automatic screenshot of the user interface at a first time interval when the acquired elapsed time is within a first range; and taking an automatic screenshot of the user interface at a second time interval when the acquired elapsed time is within a second range. The values ​​in the first range are all less than the values ​​in the second range, and the first time interval is less than the second time interval. The screenshot unit is further configured to, during automatic screenshotting of the user interface of the electronic device, determine whether the user interface of the electronic device is displaying a video playback application window corresponding to a video playback application, wherein the video playback application window includes a video playback area; in response to determining that the screenshot start condition is met, record the user interface of the electronic device to obtain a video associated with the user interface of the electronic device; wherein, in response to determining that the user interface of the electronic device is displaying the video playback application window, only the video playback area is recorded; The stop unit is configured to stop taking screenshots of the user interface of the electronic device in response to determining that the screenshot stop condition is met.

42. The apparatus of claim 41, further comprising: A unit configured to perform the steps of the method according to any one of claims 2 to 40.

43. An electronic device comprising: processor; as well as A memory storing a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 40.

44. A storage medium for storing a program, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 40.

Images