Method for acquiring materials and electronic equipment

Abstract

The embodiment of the invention provides a material obtaining method and electronic equipment, the method is executed by the electronic equipment, the electronic equipment is in communication connection with a handwriting pen, the method comprises the steps that under the condition that a first operation input by a user is received, a global lasso function is started, and the global lasso function is used for obtaining a material in the process that the user uses the electronic equipment; acquiring materials from a current interface of the electronic equipment; identifying and displaying the content circled by the user on the current interface by using the handwriting pen; and receiving a second operation of the user on the circled content, and obtaining the material according to the second operation. According to the method, the user can quickly obtain the materials in the process of using the electronic equipment, and the interaction efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a method for acquiring material and an electronic device. Background Art

[0002] When users use electronic devices for work, reading, and entertainment, when they see content that interests them or important content, they usually want to save the corresponding content (or material) so that they can view the corresponding content later when needed.

[0003] However, current users are usually unable to quickly obtain content materials when acquiring corresponding content. They need to go through tedious operations such as screenshots, photo editing, text extraction, and new document creation. As a result, users are unable to obtain materials anytime and anywhere, and the interaction efficiency is low. Summary of the invention

[0004] The present application provides a method for acquiring materials and an electronic device, which can enable a user to quickly acquire materials while using an electronic device and improve interaction efficiency.

[0005] In a first aspect, the present application provides a method for acquiring material, which is executed by an electronic device, and a communication connection is established between the electronic device and a stylus. The method includes: upon receiving a first operation input by a user, starting a global lasso function, the global lasso function is used for the user to acquire material from a current interface of the electronic device during the process of using the electronic device; identifying and displaying content circled by the user on the current interface using the stylus; receiving a second operation of the user on the circled content, and acquiring material according to the second operation.

[0006] The first operation is used to trigger the global lasso function. After the global lasso function is turned on, the user can use the stylus to circle the current interface of the electronic device to obtain materials. When the user uses the stylus to circle the current interface, the electronic device can recognize and display the circled content, and then the user can operate the circled content, such as copying, saving, etc., so that the material can be quickly obtained.

[0007] In one implementation, before executing the above method, the electronic device needs to establish a communication connection with the stylus, such as establishing a Bluetooth connection. After the electronic device establishes a communication connection with the stylus, a floating ball of the stylus may be displayed on the display screen of the electronic device. If the user clicks on the floating ball, the electronic device may pop up a function list of the stylus. The user may input the above first operation through the function list. For example, if the function list includes a global lasso function control, and the user selects the global lasso function control (i.e., inputs the first operation), the electronic device turns on the global lasso function.

[0008] In the above implementation, the user can use the stylus to perform global lasso (circle selection) at any location on the electronic device interface at any time and anywhere. After circling, the electronic device can automatically identify the content in the circled range and allow the user to perform quick save and other operations, which is convenient for the user to use in subsequent creation and improves the interaction efficiency.

[0009] In combination with the first aspect, in some implementations of the first aspect, the above-mentioned identification and display of content circled by the user on the current interface using a stylus pen includes: receiving an input event of the stylus pen on the current interface; identifying the input event, determining and displaying the content circled by the stylus pen on the current interface.

[0010] When the user circles on the current interface of the electronic device, it is done through the stylus connected to the electronic device. Therefore, the electronic device will receive the input event of the stylus on the current interface, and identify the input event to determine the content circled by the stylus on the current interface.

[0011] In one implementation, after the global lasso function is turned on, the method further includes: covering a transparent layer on the layer of the current interface, that is, the process of the stylus pen circling the content on the current interface is achieved by covering the transparent layer on the layer of the current interface of the electronic device, and the transparent layer is superimposed on the layer of the current interface of the electronic device, and does not affect the display of the content on the current interface of the electronic device. At the same time, the transparent layer can obtain the input event of the user using the stylus in real time, identify and draw the circling track, and the drawn circling track and the circling content can be displayed on the transparent layer to be presented to the user.

[0012] Optionally, the input event includes a pen-down event, a pen-movement event, and a pen-lifting event of the stylus.

[0013] In combination with the first aspect, in some implementations of the first aspect, the above-mentioned recognition of input events, determination and display of the content circled by the stylus on the current interface, includes: when a pen-down event of the stylus is recognized, taking a screenshot of the current interface to obtain a screenshot corresponding to the current interface, and recording the starting position of the stylus; when a stylus movement event is recognized, drawing a circle selection track on a transparent layer; when a stylus lift event is recognized, determining and displaying the content circled by the stylus on the current interface based on the starting position of the stylus, the screenshot corresponding to the current interface, and the circle selection track.

[0014] Among them, when the stylus pen drops a pen event, that is, the stylus pen starts to touch the display screen of the electronic device, the electronic device can take a screenshot of the current interface, and cache the screenshot for subsequent determination of the circled content. After the stylus pen drops a pen event, the user can use the stylus pen to move the circle on the transparent layer, and the electronic device can draw along the circle trajectory of the stylus pen, and draw a circle trajectory on the transparent layer. After the stylus pen is drawn and lifted, the electronic device can determine the content circled by the stylus pen based on the starting position of the stylus pen, the screenshot of the current interface, and the circle trajectory.

[0015] Optionally, when the user inputs a pen-down event with the stylus, the electronic device takes a screenshot of the current interface, and the subsequent movement events are all performed based on the screenshot, that is, the circled content is circled based on the screenshot. Therefore, the electronic device can correspond the corresponding position on the screenshot according to the starting position of the stylus and the circled track, and obtain a partial screenshot of the corresponding area in the screenshot, and this partial screenshot is the circled content.

[0016] In the above implementation, the user can obtain the corresponding material by arbitrarily circling the current interface of the electronic device using a stylus, thereby improving the efficiency of obtaining the material.

[0017] In combination with the first aspect, in some implementations of the first aspect, after determining and displaying the content circled by the stylus on the current interface, the method further includes: performing text recognition and / or intelligent cutout recognition on the circled content; when it is recognized that the circled content includes text, displaying a text recognition control; when it is recognized that the circled content includes an object that can be cutout, automatically cutting out the object.

[0018] Among them, after the electronic device displays the content circled by the stylus, in order to improve the intelligence of processing the circled content, the electronic device can also perform text recognition and / or intelligent cutout recognition on the circled content. In the case where it is recognized that the circled content includes text, the electronic device can display the corresponding text recognition control, and the user can subsequently choose to perform text recognition to obtain the corresponding text. In the case where it is recognized that the circled content includes an object that can be cut out (such as a portrait), the electronic device can automatically recognize the portrait and adsorb the edge line of the circled range on the outline of the portrait to complete the cutout processing.

[0019] In combination with the first aspect, in some implementations of the first aspect, the transparent layer is configured not to process input events from the user's finger and to intercept input events from the stylus.

[0020] It can be seen from the above description that in this application the user uses a stylus to circle and lasso. In order to improve the accuracy of identifying stylus input events, when the above electronic device overlays a transparent layer on the layer of the current interface, the transparent layer can be configured to not process the input events of the user's fingers, and only process the input events of the stylus (i.e., intercept the input events of the stylus) to reduce the interference of finger input events.

[0021] In combination with the first aspect, in some implementations of the first aspect, after identifying the content circled by the user using the stylus on the current interface, the above method also includes: if it is identified that the number of drawing points circled by the stylus on the transparent layer is less than a preset threshold, encapsulating the drawing points of the stylus as a click event through the transparent layer, and transmitting the click event to the layer of the current interface, and not displaying the content circled by the stylus on the current interface; responding to the click event through the layer of the current interface.

[0022] In some scenarios, the user may not use the stylus to draw continuous lines, but to perform dot (or click) operations, and these dot operations cannot constitute a circle selection result. Therefore, if the number of drawing points input by the stylus is relatively small, for example, less than the preset threshold, the electronic device can process it as a click operation. The electronic device can encapsulate the click operation as a click event through a transparent layer, and transmit it to the layer of the current interface of the electronic device. The layer of the current interface can respond to the click event, that is, it can perform corresponding actions, such as clicking to open an application; at this time, because this operation does not constitute a circle selection result, the electronic device no longer displays the circled content.

[0023] In combination with the first aspect, in some implementations of the first aspect, before receiving the second operation of the user on the circled content, the method further includes: modifying the configuration of the transparent layer to process input events of the user's finger.

[0024] From the above description, it can be seen that when the electronic device initially overlays a transparent layer on the layer of the current interface, the transparent layer is configured to not process finger input events. However, after the user completes the selection with the stylus, in order to facilitate the user to operate on the circled content, the transparent layer can be configured to process finger input events.

[0025] In combination with the first aspect, in some implementations of the first aspect, after identifying and displaying the content circled by the user on the current interface using a stylus, the method further includes: if a click operation of the user at a first position is received, canceling the display of the circled content, the first position being a position outside the range of the second position on the current interface, and the second position being the position of the circled content on the current interface.

[0026] In this implementation, if the user feels that the circled content is incorrect or inappropriate, the user can re-circle it. Optionally, the user can click outside the range of the circled content. After the electronic device receives the click operation, it can exit the current circle selection mode and cancel the display of the previously circled content, so that the user can re-circle it. In this way, it is convenient for the user to perform any circle selection operation anytime and anywhere, which greatly facilitates the process of obtaining materials.

[0027] In combination with the first aspect, in some implementations of the first aspect, the second operation includes copying, saving, sharing, text recognition, and dragging the circled content.

[0028] After the user has circled the content with the stylus pen, the electronic device can display the copy, save, and share controls, and the user can copy, save, or share the circled content in the form of a picture. If the circled content includes text, the electronic device can also display a text recognition control. The user can select the corresponding control according to their needs, that is, enter the second operation.

[0029] In combination with the first aspect, in some implementation methods of the first aspect, when the second operation is an operation of copying, saving or sharing the circled content, the above-mentioned obtaining material according to the second operation includes: copying, saving or sharing the picture corresponding to the circled content to obtain the picture corresponding to the circled content as the material.

[0030] That is, if the user selects the copy, save, or share operation, the electronic device obtains the material in the form of a picture. In this implementation, the user can directly copy, save, or share the circled content in the form of a picture, or when the electronic device recognizes that the circled content includes a cutout object, the electronic device can copy, save, or share the picture corresponding to the cutout object.

[0031] In combination with the first aspect, in some implementations of the first aspect, when the second operation is an operation of dragging the circled content, the above-mentioned obtaining material according to the second operation includes: dragging the picture corresponding to the circled content to the icon or display interface of the first application, and saving the picture corresponding to the circled content as material through the first application.

[0032] In this scenario, the user can also drag the picture corresponding to the circled content to other applications. Optionally, the user can drag the picture corresponding to the circled content by long pressing. Optionally, the user can drag the corresponding picture to the collection application (or global collection application), or, when the electronic device is in split-screen display mode, the user can drag the corresponding picture to other applications displayed in the split screen (such as a note application), which greatly simplifies the process of users obtaining materials.

[0033] In combination with the first aspect, in some implementations of the first aspect, when the second operation is an operation of performing text recognition on the circled content, the above-mentioned obtaining material based on the second operation includes: identifying the text included in the circled content, and copying, translating, sharing or searching the text to obtain the text as material.

[0034] In this implementation, when the circled content includes text, the electronic device can also display a text recognition control. If the user clicks on the text recognition control, the electronic device can automatically select all the text in the circled range and pop up copy, translate, share, and search controls (it can be understood that the operation on the text is performed through the above-mentioned transparent layer). The user can copy, translate, share or search the selected text, and the electronic device then obtains the material in text form.

[0035] In the above implementation, the user can use the stylus to perform global lasso (circle selection) at any location on the electronic device interface at any time and anywhere. After circling, the electronic device can automatically recognize the text and image information in the circled range, perform accurate text recognition or image clipping, and can quickly save the acquired materials, which is convenient for the user to use in subsequent creation, thereby improving the interaction efficiency.

[0036] In a second aspect, the present application provides a device, which is included in an electronic device, and the device has the function of implementing the electronic device behavior in the above-mentioned first aspect and the possible implementation of the above-mentioned first aspect. The function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above-mentioned functions. For example, a receiving module or unit, a processing module or unit, etc.

[0037] In a third aspect, the present application provides an electronic device, comprising: a processor, a memory, and an interface; the processor, the memory, and the interface cooperate with each other so that the electronic device executes any one of the methods in the technical solution of the first aspect.

[0038] In a fourth aspect, the present application provides a chip, including a processor. The processor is used to read and execute a computer program stored in a memory, and when the processor executes the computer program, the chip executes the method in the first aspect and any possible implementation thereof.

[0039] Optionally, the chip also includes a memory, and the memory is connected to the processor via a circuit or wire.

[0040] Further optionally, the chip also includes a communication interface.

[0041] In a fifth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor calls instructions so that the electronic device executes any one of the methods in the technical solution of the first aspect.

[0042] In a sixth aspect, the present application provides a computer program product, the computer program product comprising: a computer program code, when the computer program code runs on an electronic device, the electronic device executes any one of the methods in the technical solution of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0044] Figure 2 is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0045] Figure 3 is a schematic diagram of an example of establishing a connection between an electronic device and a stylus provided in an embodiment of the present application;

[0046] Figure 4 is a schematic diagram of another example of establishing a connection between an electronic device and a stylus provided in an embodiment of the present application;

[0047] Figure 5 This is a schematic diagram of an example of enabling the global lasso function provided in an embodiment of the present application;

[0048] Figure 6 is a schematic diagram of performing lassoing in a free lassoing mode provided in an embodiment of the present application;

[0049] Figure 7 is a schematic diagram of performing lassoing in a fixed lassoing mode provided in an embodiment of the present application;

[0050] Figure 8 is a schematic diagram of a text recognition scenario provided in an embodiment of the present application;

[0051] Fig. 9is a schematic diagram of an example of a smart cutout scene provided in an embodiment of the present application;

[0052] Fig.10 This is a schematic diagram of an example of dragging and saving circled content provided in an embodiment of the present application;

[0053] Fig.11 This is a schematic diagram of an example of superimposing a transparent layer under the global lasso function provided in an embodiment of the present application;

[0054] Fig.12 is a schematic diagram of an example of dragging circled content on a transparent layer provided in an embodiment of the present application;

[0055] Fig.13 is a schematic diagram of another example of superimposing a transparent layer under the global lasso function provided in an embodiment of the present application;

[0056] Fig.14 It is a flowchart of a method for obtaining materials provided in an embodiment of the present application;

[0057] Fig.15 This is a schematic diagram of an example of prompting to grant storage permission provided in an embodiment of the present application;

[0058] Fig.16 is a schematic diagram of an example of a stylus input event provided in an embodiment of the present application;

[0059] Fig.17 is a flowchart of another method for obtaining materials provided in an embodiment of the present application;

[0060] Fig.18 This is a schematic diagram of an operation of re-circling provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0062] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0063] Currently, when users use electronic devices for office work, reading, entertainment, etc., when they see content that interests them or important content, such as a paragraph of text or a picture, they usually want to save the corresponding content (or material) for later viewing when needed.

[0064] However, when users currently obtain corresponding content, they are usually unable to quickly obtain content materials and need to go through tedious operations such as screenshots, photo editing, text extraction, and new document creation. For example, if a user wants to save text in an interface, they need to first take a screenshot of the current interface, then trigger the electronic device to extract text and recognize text from the captured image, and save it after obtaining the corresponding text. Obviously, this process makes it impossible for users to obtain materials anytime and anywhere, resulting in problems such as low interaction efficiency.

[0065] In view of this, the embodiment of the present application provides a method for obtaining materials. The user can use a stylus to perform a global lasso (circle selection) at any location on the electronic device interface at any time and anywhere. After circling, the electronic device can automatically identify the text and image information in the circled range, perform accurate text recognition or image clipping, and quickly insert it into the created notes or system-level collections, which is convenient for users to use in subsequent creations and improves the interaction efficiency. It should be noted that the method for obtaining materials provided in the embodiment of the present application can be applied to mobile phones, tablet computers, vehicle-mounted equipment, augmented reality (AR) / virtual reality (VR) equipment, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA) and other electronic devices that can be connected to a stylus. The embodiment of the present application does not impose any restrictions on the specific type of electronic equipment.

[0066] For example, Figure 11 is a schematic diagram of the structure of an example of an electronic device 100 provided in an embodiment of the present application. Taking the electronic device 100 as a tablet computer as an example, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0067] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0068] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0069] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0070] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and / or a universal serial bus (USB) interface, etc.

[0071] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0072] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0073] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0074] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0075] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0076] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0077] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Figure 1 The structures of antenna 1 and antenna 2 in the electronic device 100 are only examples. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0078] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0079] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0080] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0081] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0082] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0083] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0084] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0085] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0086] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0087] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0088] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0089] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0090] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, notifications, etc.

[0091] In some embodiments, when the electronic device 100 is a mobile phone, such as a folding screen mobile phone, the electronic device 100 may also include a subscriber identification module (SIM) card interface, and the SIM card interface is used to connect a SIM card. The SIM card can be inserted into the SIM card interface or pulled out from the SIM card interface to achieve contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0092] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0093] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0094] Figure 2 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer. The application layer can include a series of application packages.

[0095] like Figure 2 As shown, the application package may include a floating ball (FloatMenu) application, a note (NotePad) application, a global collection (Collect) application, a global lasso (GlobalWrite) application, and a smart service application, etc.

[0096] Among them, the floating ball application can be used to turn on the global lasso function. The global lasso application is a system-level application that can realize lasso transparent view, screenshot service, identification of circled range, intelligent ability call, material result distribution and other functions; that is, when performing global lasso, a transparent layer can be covered on the current interface of the electronic device. The transparent layer can receive the input operation performed by the user using the stylus to perform circle selection lasso, and identify the content of the circled range, and then copy, search, translate, share, save, collect and distribute the recognition results. The note application and the global collection application provide application support for the preservation and distribution of material results of the global lasso application. The intelligent service application provides intelligent capabilities for identifying the content of the circled range of the global lasso application, such as text recognition (MagicText), intelligent cutout (ImageSegment), etc.

[0097] In one implementation, the global lasso application may include an input management module, a screenshot management module, a function implementation module, a smart text management module, a smart cutout management module, and a behavior marking module.

[0098] Among them, the input management module is used to process the input events of the stylus or finger, control the usage scenarios of the stylus or finger touch events, and can realize the transparent layer to transmit or intercept the input events of the stylus or finger. The screenshot management module is used to provide screenshot and area screenshot capabilities. The function implementation module is used to implement functions such as copying, saving, sharing, dragging, menu dragging, function mutual exclusion, abnormal prompts, and inserting the circled content (text or picture) into other supported applications. The smart text management module is used to call and manage the text recognition capabilities provided by the smart service application, and provide text data for the function implementation module. The smart cutout management module is used to call and manage the cutout capabilities provided by the smart service application, and provide image data for the function implementation module. The behavior dotting module is used to provide dotting reports of user behaviors (such as text recognition, copying, saving, sharing, etc.) during the lasso process, and provide basic data for user portraits.

[0099] In addition, the application package can also include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. ( Figure 2 not shown).

[0100] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0101] like Figure 2As shown, the application framework layer may include input management service, screenshot service, material result distribution service, etc. Among them, the input management service provides the ability to manage the input events of the global lasso application, including but not limited to the reception, interception, and distribution of input events. The screenshot service provides the ability to take a screenshot of the current interface when the user triggers the circle selection. The material result distribution service provides the ability to save and distribute the material results of the global lasso application.

[0102] In addition, the application framework layer can also include window managers, content providers, view systems, phone managers, resource managers, notification managers, etc. ( Figure 2 not shown).

[0103] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0104] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0105] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0106] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0107] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0108] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0109] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0110] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0111] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0112] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0113] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0114] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0115] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0116] A 2D graphics engine is a drawing engine for 2D drawings.

[0117] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, Bluetooth driver, audio driver, and sensor driver. The Bluetooth driver can provide the ability for electronic devices to establish a Bluetooth connection with the stylus.

[0118] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 1 and Figure 2 Taking the electronic device with the structure shown as an example, the method for obtaining materials provided in the embodiment of the present application is specifically described in combination with the accompanying drawings and application scenarios.

[0119] First of all, it should be noted that the electronic device in the embodiment of the present application needs to establish a connection with the stylus, and the user uses the stylus to circle on the interface of the electronic device. The following embodiment is explained using the electronic device as a tablet computer as an example. Of course, the electronic device can also be other types of devices such as mobile phones, such as foldable screen mobile phones, etc. The implementation principles are similar, and the same content will not be repeated here.

[0120] The electronic device can establish a connection with the stylus via a Bluetooth function. In one practicable manner, Figure 3 As shown, the user can open the setting interface of the electronic device, click the Bluetooth option, enter the Bluetooth setting interface, and turn on the Bluetooth function switch. At this time, the electronic device can search for available devices through Bluetooth signals and display the available devices in the available device list. If the stylus is within the Bluetooth signal search range of the electronic device, the name of the stylus will be displayed in the available device list, for example: M-Pencil. The user can then select the stylus, and then establish a Bluetooth connection between the electronic device and the stylus to complete the connection process.

[0121] In another possible implementation, Figure 4 As shown, the stylus pen can be paired with the electronic device through Bluetooth by adsorption pairing. For example, the electronic device has a built-in sensor and a magnet, and the stylus pen also has a built-in magnet. If the stylus pen is adsorbed on the top middle position of the electronic device (the side with the volume button), the sensor can identify the adsorption state. When the adsorption state is identified, the electronic device can pop up a connection prompt box on the display screen. If the user clicks the "Connect" option in the prompt box, the connection process between the electronic device and the stylus pen can be completed.

[0122] After the electronic device establishes a connection with the stylus, a floating ball with the stylus may be displayed on the display screen of the electronic device. If the user clicks the floating ball, the electronic device may pop up a function list of the stylus. Figure 5 As shown, the floating ball 51 is displayed on the edge of the display screen. After the user clicks the floating ball 51 with a stylus or a finger, the electronic device pops up a function list of the stylus, which may include but is not limited to opening notes, creating new notes, taking screenshots, global lasso and other function controls. When the user selects the global lasso function control, the electronic device turns on the global lasso function and enters the global lasso state. The user can circle and lasso in this state to obtain the corresponding material.

[0123] In an achievable manner, the global lasso function may include multiple modes, such as a free lasso mode and a fixed lasso mode. In the free lasso mode, a user may use a stylus to freely circle and select content of different shapes on the current interface of the electronic device. For example, Figure 6 As shown, the user clicks the free lasso icon 61 to enter the free lasso mode, and then uses the stylus to freely circle on the current interface, for example, circle out Figure 6 It is understood that when the electronic device is triggered to start the global lasso function, the electronic device can enter the free lasso mode by default.

[0124] In the fixed lasso mode, the user can use the stylus to circle the content of a fixed shape range on the current interface of the electronic device. Figure 7As shown, the user clicks the fixed lasso icon 71 to enter the fixed lasso mode, and then uses the stylus to circle the current interface, for example, Figure 7 The rectangular shape shown in the dashed line.

[0125] It can be understood that the embodiments of the present application can implement global lassoing on any display interface, which can be a desktop interface of an electronic device or a display interface of any application.

[0126] After the user has circled the content with the stylus pen, the electronic device can copy, save, and share the circled content, and the circled content is the acquired material. Furthermore, if the electronic device recognizes that the circled range includes text, it can perform text recognition operations. For example Figure 8 As shown, taking the free lasso mode as an example, after the selection is completed, the electronic device first displays the copy, save, and share controls above the circled range, and the user can copy, save, or share the circled content in the form of a picture; when the electronic device recognizes that the circled range includes text, it can also display a text recognition control. If the user clicks on the text recognition control, the electronic device can automatically select all the text in the circled range (for example, Figure 8 Select all the text in the music and calculator) and pop up the copy, translate, share, and search controls. The user can copy, translate, share, or search the selected text. Optionally, after the electronic device selects all the text in the circled range, the user can also select part of the selected text again, such as dragging Figure 8 The cursor on both sides of the Chinese characters selects part of the text.

[0127] Furthermore, if the electronic device recognizes that there is content that supports smart cutout within the circled range, it can also automatically perform smart cutout and identify the edge contour of the cutout. Fig. 9 As shown, after the selection is completed, the electronic device recognizes that the circled range includes portrait content, and automatically recognizes the portrait, and adsorbs the edge line of the circled range on the outline of the portrait, and displays the copy, save, and share controls. If the user clicks the save control, the electronic device can save the circled portrait image to the gallery application. In this scenario, the user can also drag the circled portrait image and drag the circled portrait image to other applications; optionally, the user can drag the circled portrait image by long pressing. For example, Fig.10 As shown in Figure (a) in the figure, the user can drag the portrait image to the global collection application; or Fig.10As shown in Figure (b) in the figure, when the electronic device is in split-screen display mode, the user can drag the portrait image to other applications (such as note applications) displayed in the split-screen, which greatly simplifies the process of users obtaining materials. Figure 8 In the scenario shown, if the user does not click the text recognition control, the user can also drag the circled content, such as dragging it to other applications.

[0128] It is understood that in the fixed lasso mode, the user can also copy, save, share and drag the content in the circled rectangular shape. It is also understood that after a circle selection operation is completed, as long as the global lasso function is still turned on, the user can continue to perform the next circle selection operation.

[0129] For the scenario where the user circles on a dynamic interface, assuming that when the user uses a stylus to write on the dynamic interface, the electronic device displays interface A, then the user circles based on the displayed content on interface A, but the original dynamic interface is still dynamically displayed, that is, the user's circle action does not affect the playback process of the dynamic interface on the electronic device. After the circle is completed, the circled content can also be copied, saved, shared, and dragged.

[0130] It should be noted that when the global lasso function is turned on, because the user needs to use the stylus to circle, the electronic device will no longer respond to other operations of the stylus input except circling and clicking. For example, when the user uses the stylus to swipe left or right to switch the display interface, the electronic device does not respond to the operation, that is, the display interface of the electronic device remains unchanged. If the user wants to exit the global lasso function, he can click the close control of the global lasso in the above embodiment, such as Figure 6 After the global lasso function is turned off, the electronic device can respond to any operation of the stylus input.

[0131] Based on the above scenario of performing global lasso to obtain materials, the embodiment of the present application is mainly implemented by overlaying a transparent layer on the layer of the current interface of the electronic device. The transparent layer is superimposed on the layer of the current interface of the electronic device. Because it is a transparent layer, it does not affect the content displayed on the current interface of the electronic device. At the same time, the transparent layer can obtain the operation events input by the user using the stylus in real time, identify and draw the circle selection track, and the drawn circle selection track and the circled content can be displayed on the transparent layer to present to the user.

[0132] For example, Fig.11 As shown, when the user triggers the global lasso function, the electronic device will cover a transparent layer O on the layer of the current interface. For ease of understanding, Fig.11In the figure, the transparent layer O is represented by a dotted line, but in actual application, the transparent layer O is not visible to the user. After the user uses the stylus to circle the transparent layer O, the circle track L and the circled content will be displayed on the transparent layer O, and the circle track L and the circled content are visible to the user. If the user drags the circled content, the dynamic effect of the dragging process can also be displayed on the transparent layer O. For example, Fig.12 In the example, when the circled content is dragged from position P to position Q, the dynamic process from position P to position Q can be displayed on the transparent layer O, and this process does not affect the display content on the current interface of the electronic device.

[0133] On the basis of the above transparent layer O, if the user clicks the text recognition control after circling, the electronic device can hide the transparent layer O, that is, no longer display the circled track L and the circled content, but add a new transparent layer W, such as Fig.13 As shown, the recognized text information is displayed on the transparent layer W. At the same time, the user can operate the text information on the transparent layer W, such as reselecting text, copying text, etc. If the user completes the text information operation or exits the text recognition mode, the electronic device can hide the transparent layer W and redisplay the transparent layer O so that the user can continue to circle the transparent layer O.

[0134] From the above description, it can be known that the user uses a stylus to circle and lasso. Then, after the electronic device turns on the global lasso function, the transparent layer O can be configured to intercept the input event of the stylus when receiving the input event of the stylus, so as to identify the input event of the stylus, and respond to the input event when it is identified that the stylus triggers the circle operation, such as displaying the circle track L in the transparent layer O. If the transparent layer O recognizes that the input event of the stylus is a click operation, the click operation can be encapsulated as a click event and transmitted to the layer of the current interface of the electronic device. The layer of the current interface responds to the click event, that is, it can perform a corresponding action, such as clicking to open an application. As for operations performed by the user using his fingers, the transparent layer O can, after receiving the finger input event, directly transmit the finger input event to the layer of the current interface of the electronic device, and the layer of the current interface will process the finger input event, that is, the transparent layer O does not process the finger input event; for example, if the user uses his finger to slide on the transparent layer O, the transparent layer O can transmit the sliding event to the layer of the current interface, and the layer of the current interface can respond to the sliding event, and then perform operations such as interface sliding browsing and application switching.

[0135] Optionally, after the user completes the selection with the stylus, if the selected content needs to be copied, saved, dragged, etc., to facilitate the user's operation, the transparent layer O can be configured to intercept and identify both the stylus input event and the finger input event. That is, the user can now operate the circled content with both the stylus and the finger.

[0136] Based on the above application scenario, the implementation process of the method for obtaining materials provided in the embodiment of the present application is introduced below. Fig.14 : is a flowchart of a method for obtaining material provided in an embodiment of the present application. The method can be executed by an electronic device and may specifically include:

[0137] S101: When a first operation input by a user is received, a global lasso function is enabled.

[0138] The first operation can be the above Figure 5 The electronic device can open the global lasso function when receiving the first operation by clicking the global lasso function control in the floating ball 51. It can be understood that in some embodiments, the global lasso function can also be called a material lasso function, a material interception function, etc., and the embodiment of the present application does not limit this naming.

[0139] Optionally, when the user triggers the global lasso function, the electronic device may also determine whether the global lasso function has corresponding storage permissions because running an application or opening a function in the electronic device requires storage permissions to run normally. If the global lasso function has corresponding storage permissions, the electronic device may execute subsequent steps; if the global lasso function does not have corresponding storage permissions, the electronic device may display a pop-up window for authorization of storage permissions to prompt the user to enable the corresponding storage permissions. Exemplarily, a schematic diagram of an electronic device displaying a pop-up window for authorization of storage permissions may be as follows: Fig.15 As shown, if the user selects the allow option, the electronic device can normally enable the global lasso function.

[0140] Optionally, when the electronic device determines that the global lasso function has corresponding storage permissions, the user may be prompted whether to display a guide animation, which is used to guide the user to correctly use the global lasso function. If the user chooses to display the guide animation, the electronic device may display the guide animation; if the user chooses not to display the guide animation, the electronic device may close the prompt to perform subsequent steps.

[0141] S102, initializing the global lasso interface and related controls.

[0142] After the electronic device turns on the global lasso function, the electronic device can initialize the global lasso interface on the current interface. The global lasso interface initialized here can be the transparent layer O mentioned above, that is, a transparent layer O is covered on the layer of the current interface. At the same time, the electronic device can also initialize related controls on the transparent layer O, such as the above Figure 6 or Figure 7 The free lasso icon 61, the fixed lasso icon 71 and the close control are shown; optionally, these related controls can be presented in the form of capsules.

[0143] Optionally, after initializing the global lasso interface, it can be seen from the above embodiment that the transparent layer O does not process finger input events. Therefore, the electronic device can also configure the global lasso interface to not process finger touch events or to transmit finger touch events, that is, it does not affect the user's use of fingers to browse content, switch applications, and other operations on the current interface, and only a stylus can be used to circle and lasso.

[0144] S103, receiving and identifying an input event of the user using a stylus, and determining the content circled by the stylus.

[0145] Among them, Fig.16 As shown, the input events of the user using the stylus may include but are not limited to pen down events (down events), move events (move events), pen up events (up events), long press and drag events (LongPress events), and single click operation events (SingleTapUp events).

[0146] For the down event, it means that the stylus starts to touch the display screen of the electronic device, that is, the event at the moment of pen drop. When the electronic device receives the down event, it can respond to the event and take a screenshot of the current interface (i.e., trigger a screenshot), as well as the screenshot obtained by the background cache. It should be noted that because a transparent layer is superimposed on the current interface, the user still sees the content of the current interface, so the screenshot obtained by the electronic device taking a screenshot will not be displayed on the current interface, but will only be saved in the background.

[0147] Optionally, when the electronic device takes a screenshot, it can also determine whether the screenshot is successful. If the screenshot is not successful, the electronic device can pop up a window prompt, such as prompting that the screenshot is not allowed due to copyright issues. If the screenshot is successful, the electronic device can also record the starting position of the stylus corresponding to the down event and continue to respond to subsequent events.

[0148] For a move event, that is, an event in which the stylus moves a circle on the transparent layer after the down event of the stylus, when the electronic device receives the move event, it can respond to the event and draw along the circle track of the stylus to draw the circle track on the transparent layer.

[0149] For the up event, i.e., the event of the stylus pen being lifted after the selection is completed on the transparent layer, after receiving the up event, the electronic device can not only display the entire selection trajectory, but also determine the selected content based on the starting position of the stylus pen, the screenshot of the current interface, and the selection trajectory, and recognize the selected content, such as the above-mentioned text recognition and smart cutout. Fig.17 As shown, it may specifically include:

[0150] S201, determining whether the number of drawing points of the stylus pen is less than a preset threshold, if so, executing S202, if not, executing S203.

[0151] Here, the drawing points of the stylus are the drawing points corresponding to the above-mentioned input move event, because in some scenarios, the user may not use the stylus to draw continuous lines, but to perform point (or click) operations, and these points cannot constitute the circle selection result. Therefore, if the number of drawing points input by the stylus is relatively small, for example, less than a preset threshold, the electronic device can process it as a click operation.

[0152] S202, transparently transmit the click event to the layer of the current interface, and respond to the click event.

[0153] Because the stylus is drawing points input on the above-mentioned transparent layer, when there are fewer drawing points, it is impossible to form a circle selection result. In this case, the transparent layer can encapsulate the click operation as a click event and transmit it to the layer of the current interface of the electronic device. The layer of the current interface can respond to the click event, that is, it can perform corresponding actions, such as clicking to open an application.

[0154] S203, configuring a global lasso interface to process finger input events.

[0155] From the above description, it can be seen that the global lasso interface configured at initialization does not process finger input events. However, after the user completes the selection with the stylus, in order to facilitate the user to operate the selected content, the global lasso interface can be configured to process finger input events.

[0156] S204, recognizing the content circled by the stylus pen, and obtaining and displaying the recognition result.

[0157] The electronic device can perform text recognition and intelligent image cutout recognition on the content circled by the stylus pen. When it is recognized that the circled content includes text, the electronic device can display the above-mentioned Figure 8 When the circled content is identified as including a portrait, the electronic device can automatically identify the portrait and attach the edge line of the circled range to the outline of the portrait (for example, Fig. 9 As shown). Optionally, the smart cutout recognition can not only recognize human portraits, but also recognize other objects such as animals and buildings, and automatically cut out these objects.

[0158] Optionally, during the process of the electronic device recognizing the content circled by the stylus pen, the electronic device may also dynamically display the circled track on the transparent layer, for example, displaying the circled track in a scrolling manner.

[0159] Through the above Fig.17 The electronic device can recognize and determine the content circled by the stylus through the process; the long press and drag event (LongPress event) and the single click operation event (SingleTapUp event) occur after the up event, see the following description for details.

[0160] S104, receiving a second operation of the user on the circled content, and acquiring corresponding materials.

[0161] From the above description, it can be seen that after the user completes the circle selection with the stylus, the electronic device can display the copy, save, and share controls, and the user can copy, save, or share the circled content in the form of a picture. If the user selects to copy, save, share, or long press the picture to drag (i.e., the second operation), the electronic device obtains the material in the form of a picture.

[0162] When the circled content includes text, the electronic device can also display a text recognition control. If the user clicks on the text recognition control, the electronic device can automatically select all the text in the circled range and pop up copy, translate, share, and search controls (it can be understood that the operation on the text is performed through the above-mentioned transparent layer W). The user can copy, translate, share or search the selected text, and the electronic device then obtains the material in text form.

[0163] When the circled content includes a portrait, the user can long press the portrait image to drag it, or copy, save or share it. The electronic device also obtains the portrait material in the form of a picture.

[0164] Optionally, the above operation of long pressing and dragging the circled content can also be understood as a long press and drag event (LongPress event). After the above transparent layer receives the long press and drag event, the drag effect can be displayed on the transparent layer.

[0165] It can be understood that when the user inputs the down event with the stylus, the electronic device takes a screenshot of the current interface, and the subsequent move events are all based on the screenshot, that is, the circled content is circled based on the screenshot. Here, the electronic device can correspond the corresponding position on the screenshot according to the starting position of the stylus and the circled track, and obtain a partial screenshot of the corresponding area in the screenshot. Then, when the user long presses and drags the circled content, the obtained partial screenshot is dragged, and the effect of the partial screenshot being dragged is displayed.

[0166] Optionally, since the global lasso interface has been configured to process finger input events, the second operation performed by the user on the circled content may be input through a stylus pen or a finger.

[0167] After the second operation, the electronic device can complete the global lasso process. If the user wants to circle again, the above S103 can be executed again. However, it should be noted that before executing S103 again, the electronic device needs to reconfigure the global lasso interface to not process finger input events.

[0168] In some embodiments, before S104, if the user feels that the circled content is incorrect or inappropriate, the user can re-circle it. Optionally, the user can perform a single-tap operation (i.e., a SingleTapUp event) outside the range of the circled content. After the electronic device receives the single-tap operation, it can exit the current circle selection mode and cancel the display of the previous copy, save, or share controls, so that the user can re-circle it. For example, Fig.18 As shown, the user has circled the range Y, but wants to circle it again, then the user can click anywhere outside the range Y, and then the electronic device returns to the uncircled state, and the user can circle it again.

[0169] Optionally, in this embodiment, if the user performs a single click operation within the range of the circled content, such as in Fig.18 If you click within the range Y in the image, the electronic device can flash controls such as copy, save or share.

[0170] It can be understood that the above method of obtaining materials can be Figure 2 The global lasso application is used to execute the process, or the processing modules in the global lasso application are used to execute the process interactively. The implementation principle is similar and will not be described in detail here.

[0171] With the above-mentioned method of acquiring materials, the user can use a stylus to perform a global lasso (circle selection) at any location on the electronic device interface at any time and anywhere. After circling, the electronic device can automatically identify the text and image information in the circled range, perform accurate text recognition or image clipping, and can quickly save the acquired materials, which is convenient for the user to use in subsequent creation, thereby improving the interaction efficiency.

[0172] The above describes in detail an example of a method for obtaining material provided by an embodiment of the present application. It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0173] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each function can be divided into various functional modules, such as a detection unit, a processing unit, a display unit, etc., or two or more functions can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0174] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0175] The electronic device provided in this embodiment is used to execute the above-mentioned method of acquiring materials, and thus can achieve the same effect as the above-mentioned implementation method.

[0176] In the case of an integrated unit, the electronic device may further include a processing module, a storage module and a communication module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device to execute stored program codes and data, etc. The communication module may be used to support the electronic device to communicate with other devices.

[0177] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP) and a microprocessor, and so on. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0178] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a Figure 1 Device of the structure shown.

[0179] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the method for acquiring materials in any of the above embodiments.

[0180] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the method for obtaining materials in the above-mentioned embodiment.

[0181] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory so that the chip executes the method of obtaining materials in the above-mentioned method embodiments.

[0182] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0183] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0184] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0185] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0186] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0187] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0188] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for acquiring material, the method being executed by an electronic device, characterized in that: A communication connection is established between the electronic device and the stylus, and the method includes: In the case of receiving a first operation input by the user, starting a global lasso function, wherein the global lasso function is used for the user to obtain materials from a current interface of the electronic device during use of the electronic device; Identifying and displaying the content circled by the user on the current interface using the stylus; A second operation of the user on the circled content is received, and the material is acquired according to the second operation.

2. The method according to claim 1, characterized in that The identifying and displaying the content circled by the user on the current interface using the stylus includes: Receiving an input event of the stylus on the current interface; The input event is identified, and the content circled by the stylus on the current interface is determined and displayed.

3. The method according to claim 2, characterized in that After the global lasso function is enabled, the method further includes: Covering a transparent layer on the layer of the current interface; The receiving an input event of the stylus on the current interface includes: An input event of the stylus is received through the transparent layer.

4. The method according to claim 3, characterized in that The input events include a pen-down event, a pen-movement event, and a pen-lifting event of the stylus.

5. The method according to claim 4, characterized in that The identifying the input event, determining and displaying the content circled by the stylus on the current interface, includes: When a pen-down event of the stylus is recognized, a screenshot of the current interface is taken to obtain a screenshot corresponding to the current interface, and a starting point position of the stylus is recorded; When a movement event of the stylus is recognized, a circle selection track is drawn on the transparent layer; When a pen-lifting event of the stylus is recognized, the content circled by the stylus on the current interface is determined and displayed based on the starting position of the stylus, the screenshot corresponding to the current interface, and the circled track.

6. The method according to claim 5, characterized in that After determining and displaying the content circled by the stylus on the current interface, the method further includes: Performing text recognition and / or intelligent image cutout recognition on the circled content; In the case where it is recognized that the circled content includes text, displaying a text recognition control; When it is recognized that the circled content includes an object that can be cut out, the object is automatically cut out.

7. The method according to any one of claims 3 to 6, characterized in that The transparent layer is configured not to process input events from the user's finger and to intercept input events from the stylus pen.

8. The method according to claim 7, characterized in that After identifying the content circled by the user on the current interface using the stylus, the method further includes: If it is identified that the number of points drawn by the stylus when circling on the transparent layer is less than a preset threshold, the points drawn by the stylus are encapsulated as click events through the transparent layer, and the click events are transparently transmitted to the layer of the current interface, and the content circled by the stylus on the current interface is not displayed; The click event is responded to through the layer of the current interface.

9. The method according to claim 7 or 8, characterized in that: Before receiving a second operation of the user on the circled content, the method further includes: The configuration of the transparent layer is modified to process input events of the user's finger.

10. The method according to any one of claims 1 to 9, characterized in that After identifying and displaying the content circled by the user on the current interface using the stylus, the method further includes: If a click operation of the user at the first position is received, the display of the circled content is canceled, the first position is a position outside the range of the second position on the current interface, and the second position is the position of the circled content on the current interface.

11. The method according to any one of claims 1 to 10, characterized in that The second operation includes copying, saving, sharing, text recognition and dragging the circled content.

12. The method according to claim 11, characterized in that In a case where the second operation is an operation of copying, saving or sharing the circled content, acquiring the material according to the second operation includes: The picture corresponding to the circled content is copied, saved or shared to obtain the picture corresponding to the circled content as the material.

13. The method according to claim 11, characterized in that In a case where the second operation is an operation of dragging the circled content, acquiring the material according to the second operation includes: Drag the picture corresponding to the circled content to the icon or display interface of the first application, and save the picture corresponding to the circled content as the material through the first application.

14. The method according to claim 13, characterized in that The first application includes a collection application or a note application.

15. The method according to claim 11, characterized in that In a case where the second operation is an operation of performing text recognition on the circled content, the acquiring the material according to the second operation includes: Identify the text included in the circled content, and copy, translate, share or search the text to obtain the text as the material.

16. The method according to any one of claims 1 to 15, characterized in that A floating ball corresponding to the stylus is displayed on the current interface of the electronic device, and the first operation is a touch operation on the floating ball.

17. An electronic device, characterized in that: include: one or more processors; one or more memories; The memory stores one or more programs, and when the one or more programs are executed by the processor, the electronic device executes the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: A computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the processor calls instructions so that the electronic device executes the method according to any one of claims 1 to 16.

Images