Method for determining target object, and electronic device

Abstract

Provided are a method for determining a target object, and an electronic device. By using the method, a target object can be determined that meets the expectation of a user, such that the user experience can be improved. For example, an electronic device displays a framing interface (303), the framing interface (303) comprises at least one object (401, 402, 403), and a first pattern (304) is displayed on an upper layer of the framing interface (303). In response to a first operation for adjusting the photographing range of a camera, the electronic device keeps the position of the first pattern (304) unchanged, such that the first object (402) in the framing interface (303) enters into the coverage range of the first pattern (304), and if the photographing range remains unchanged and the duration for which the photographing range remains unchanged is greater than or equal to a first duration, the first object (402) is determined as a target object.

Description

Method for determining target object and electronic device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on August 4, 2023, with application number 202310982981.3 and application name “A method and electronic device for determining a target object”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of terminal technology, and in particular to a method for determining a target object and an electronic device. Background Art

[0004] With the continuous advancement of terminal technology, mobile terminals are becoming increasingly versatile. For example, mobile terminals are equipped with cameras that can capture images of the surrounding environment and identify a target object from multiple objects contained in the image. Generally, after the mobile terminal activates the camera, a viewfinder is displayed. If the viewfinder includes multiple objects, the system automatically identifies a target object from among the multiple objects. However, the target object identified in this way often does not match the user's desired object, resulting in a poor user experience.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a method and electronic device for determining a target object, which help to improve the accuracy of determining the target object.

[0007] In a first aspect, a method for determining a target object is provided. The method can be applied to an electronic device. The electronic device can be, for example, a mobile phone, a tablet computer, or the like. The method includes: displaying a framing interface, the framing interface being the framing interface of a camera in the electronic device, the framing interface including at least one object, a first graphic displayed on an upper layer of the framing interface, the first graphic having an area smaller than the area of ​​the framing interface; in response to a first operation for adjusting the shooting range of the camera, the position of the first graphic remains unchanged, a first object in the framing interface enters a coverage range of the first graphic, the first object being any one of the at least one object, the coverage range of the first graphic being the coverage range of an area enclosed by an edge outline of the first graphic on the framing interface; and if the shooting range remains unchanged and the duration of the unchanged period is greater than or equal to a first duration, determining that the first object is a target object.

[0008] In an embodiment of the present application, after a user opens the viewfinder interface of an electronic device, a first graphic is displayed on the upper layer of the viewfinder interface. The user can use the first graphic to determine the target object. For example, the user adjusts the shooting range so that the user's desired object falls within the coverage of the first graphic, and then maintains the shooting range unchanged. When the electronic device detects that the shooting range remains unchanged and the duration of this change is greater than or equal to the first duration, it determines that the object within the coverage of the first graphic is the target object. The target object determined in this way is the object in the user's mind, and the user experience is better.

[0009] In one possible design, the first graphic satisfies at least one of the following: the shape of the first graphic is a circle; or, the area of ​​the first graphic is the system default; or, the first graphic is displayed at the center of the display screen of the electronic device; or, the first graphic is filled with a first color; or, the area within the first graphic is transparent; or, the edge of the first graphic is feathered, and the degree of feathering is the system default or positively correlated with the current object distance of the camera.

[0010] In an embodiment of the present application, the first graphic can have a variety of design styles, for example, the first graphic can be filled with color, have a certain degree of transparency, edge feathering, etc., so as to enhance the user experience during the operation as much as possible.

[0011] In one possible design, the area enclosed by the edge outline of the first graphic is the first area, and the other areas outside the first area are the second area. The first area and the second area satisfy at least one of the following: the color filled in the first area is different from the color filled in the first area; or, the transparency of the first area is higher than the transparency of the second area; or, the brightness of the first area is higher than the brightness of the second area.

[0012] In an embodiment of the present application, the area within the first graphic (i.e., the first area) and the area outside the first graphic (i.e., the second area) are displayed separately, which can more intuitively prompt the user to move the object the user wants into the first area, providing a better experience.

[0013] In one possible design, after determining that the first object is the target object, the method further includes: canceling the display of the first graphic. In the embodiment of the present application, the first graphic on the upper layer of the viewfinder interface is used to assist the user in determining the target object. After the target object is determined, the first graphic can be canceled to avoid blocking information.

[0014] In a possible design, before the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to the first time period, it also includes: when it is determined that the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to the second time period, determining that the first object is a candidate target object, and adjusting the first graphic to a second graphic, wherein the second graphic is a circumscribed circle or a circumscribed polygon of the first object.

[0015] In this embodiment of the present application, the first object is included in the coverage area of ​​the first graphic. If the electronic device determines that the shooting range remains unchanged for a period of 2 seconds, the first object is determined to be a candidate target object and the user is prompted that the first object is a candidate target object. If the electronic device determines that the shooting range continues to remain unchanged for a period of 4 seconds, the candidate target object is determined to be the target object. This method of determining the target object is more accurate.

[0016] In one possible design, after determining that the first object is the target object, the method further includes: canceling the display of the second graphic. In this embodiment of the present application, when the first object is determined to be a candidate target object, the second graphic is displayed to indicate that the first object is a candidate target object. If the candidate target object is determined to be the target object, the second graphic is canceled to avoid blocking information.

[0017] In one possible design, after determining that the first object is the target object, it also includes: the edge outline of the first object is highlighted; or, the area surrounded by the edge outline of the first object is filled with a third color; or, the brightness of the area surrounded by the edge outline of the first object is higher than that of other areas.

[0018] In an embodiment of the present application, after the electronic device determines the target object, it can highlight the target object to prompt the user which one is the target object.

[0019] In a possible design, after determining that the first object is a candidate target object, the method further includes: in response to a second operation for adjusting the shooting range of the camera, the positions of the first object and the second graphic in the viewfinder interface move synchronously.

[0020] In an embodiment of the present application, after the electronic device determines that the first object is a candidate target object, the second graphic will be attached to the first object. If the shooting range changes (for example, the user's hand shake causes the shooting range to change), the second graphic will move synchronously with the first object to ensure that the first object is still a candidate target object. This can avoid the need to re-determine the target object due to user shaking.

[0021] In one possible design, after the positions of the first object and the second graphic move synchronously, it also includes: determining that a second object appears at a first position in the viewfinder interface and the time period during which the second object remains at the first position reaches a third time period, where the first position is the display position of the first graphic before it is adjusted to the second graphic; restoring the second graphic to the first graphic, and the first graphic is displayed at the first position, and the first graphic includes the second object; and when determining that the shooting range remains unchanged and the time period during which it remains unchanged is greater than or equal to a fourth time period, determining that the second object is the target object.

[0022] In an embodiment of the present application, after the first object is determined as a candidate target object, the user can change the target object. Assuming that the user wants to change to a second object, the user can adjust the shooting angle so that the second object moves to the first position and the shooting range remains unchanged. In this way, the electronic device will cancel the candidacy of the first object, restore the second graphic to the first graphic, and re-determine the target object using the first graphic. For example, when the second object is included in the first graphic, if it is determined that the shooting range remains unchanged and the duration of the unchanged period is greater than or equal to the fourth duration, the second object is determined to be the target object.

[0023] In one possible design, after determining that a second object appears at a first position within the viewfinder interface, the method further includes: displaying a prompt box at the first position, the prompt box including the second object, and the prompt box being used to prompt that the second object will be determined as the target object; after determining that the time duration for which the second object remains at the first position reaches a third time duration, and before restoring the second graphic to the first graphic, the method further includes: canceling the display of the prompt box.

[0024] In the embodiment of the present application, after the first object is determined as a candidate target object, the user can change the target object. If the user wants to change to a second object, the user can adjust the shooting angle so that the second object moves to the first position. At this time, the electronic device will display a prompt box containing the second object to prompt the user that the second object will be determined as the target object. The user can use the prompt box to confirm whether the target object has been changed correctly.

[0025] In a possible design, the first graphic also includes a third object, and the third object is any object other than the first object among the at least one object; after the shooting range remains unchanged and the time length for which it remains unchanged is greater than or equal to the first time length, it also includes: determining that the third object is the target object.

[0026] In an embodiment of the present application, when the first graphic includes multiple objects, the electronic device may determine all of the multiple objects as target objects. Therefore, if the user only desires one object, the shooting range may be adjusted to include only that object as much as possible within the first graphic. If the user desires multiple objects, the shooting range may be adjusted to include multiple objects within the first graphic.

[0027] In one possible design, the method also includes: displaying the first object and the third object; when receiving a third operation, retaining the first object as the target object and canceling the third object as the target object; wherein the third operation includes an operation on the first object, or an operation on the third object.

[0028] In an embodiment of the present application, after the electronic device determines multiple target objects, the user can cancel a target object. For example, if there are two target objects, the user clicks on one of the objects to cancel it as the target object, while the other object remains as the target object. Alternatively, the user clicks on one of the objects to retain it as the target object, while the other object is canceled as the target object.

[0029] In a possible design, before the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to the first time period, it also includes: when it is determined that the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to a fourth time period, determining that the first object and the third object are candidate target objects, and adjusting the first graphic to a fourth graphic and a fifth graphic, the fourth graphic being the circumscribed circle or circumscribed polygon of the first object, and the fifth graphic being the circumscribed circle or circumscribed polygon of the third object.

[0030] In an embodiment of the present application, when the coverage area of ​​the first graphic includes multiple objects, if the electronic device determines that the shooting range remains unchanged for a period of 2 seconds, the multiple objects are determined to be candidate target objects, and the user is prompted to identify the candidate target objects. If the electronic device determines that the shooting range continues to remain unchanged for a period of 4 seconds, the multiple candidate target objects are determined to be the target object. This method of determining the target object is more accurate.

[0031] In a possible design, before the shooting range remains unchanged and the time period of the unchanged state is greater than or equal to the first time period, it also includes: if a fourth operation is received, retaining the third object as a candidate target object and retaining the fifth graphic, canceling the first object as a candidate target object and canceling the fourth graphic; wherein the fourth operation includes an operation on the fifth graphic, or an operation on the fourth graphic; after the shooting range remains unchanged and the time period of the unchanged state is greater than or equal to the first time period, it also includes: determining that the third object is the target object.

[0032] In an embodiment of the present application, after the electronic device determines multiple candidate target objects, the user can cancel the candidacy of a particular object. For example, if there are two candidate target objects, when the user clicks on one of the candidate target objects, that object's candidacy is canceled, while the other object's candidacy is retained. Alternatively, when the user clicks on one of the candidate target objects, that object's candidacy is retained, while the other object's candidacy is canceled.

[0033] In one possible design, the first graphic also includes a fourth object, and the fourth object is any object other than the first object among the at least one object. If the shooting range remains unchanged and the duration of the unchanged state is greater than or equal to the first duration, before determining that the first object is the target object, it also includes: receiving an operation of selecting the first object in the first graphic.

[0034] In the embodiment of the present application, when the first graphic includes multiple objects, the user can specify a certain object as the target object.

[0035] In a possible design, before receiving the operation of selecting the first object in the first graphic, it also includes: outputting prompt information, where the prompt information is used to prompt the user to specify a target object.

[0036] In an embodiment of the present application, when the first graphic includes multiple objects, the electronic device can prompt the user to specify a target object, and the user can select which object to use as the target object according to his or her needs.

[0037] In one possible design, after determining that the first object is the target object, it also includes: identifying the target object, identifying at least one of the name, category, purchase link, related objects of the target object, and purchase links of the related objects of the target object; or, translating the target object; or, copying the target object; or, cutting out the target object.

[0038] The method for determining a target object provided in the embodiment of the present application can be applied to any scenario where a target object needs to be determined in an image, such as a translation scenario, an object recognition scenario, etc. The embodiment of the present application does not give examples one by one.

[0039] In one possible design, after determining that the first object is the target object, it also includes: displaying a first button and a second button; wherein, when the first button is triggered, returning to the previous N steps, and when the second button is triggered, translating, identifying, copying or cropping the target object, where N is a positive integer.

[0040] In an embodiment of the present application, after the electronic device determines the target object, it can display a return button (i.e., the first button) and a confirmation button (i.e., the second button). When an operation is received on the confirmation button, the target object is translated, identified, copied or cropped. When an operation is received on the return button, it can return to the previous N steps to redetermine the target object.

[0041] In one possible design, the electronic device has a translation function, the viewfinder interface is the interface displayed when the translation function is turned on, and the first graphic is a circle; or, the electronic device has an object recognition function, the viewfinder interface is the interface displayed when the object recognition function is turned on, and the first graphic is a rectangle.

[0042] In an embodiment of the present application, the shape of the first graphic is related to the current scene. For example, if the current scene is an object recognition scene, the first graphic is a circle; if the current scene is a translation scene, the first shape is a square, which matches the shape of the text paragraph and can avoid missing the translation of certain texts.

[0043] In a second aspect, an electronic device is further provided, including:

[0044] a processor, a memory, and one or more programs;

[0045] The one or more programs are stored in the memory, and the one or more programs include instructions. When the instructions are executed by the processor, the electronic device executes the method steps provided in the first aspect above.

[0046] In a third aspect, a computer-readable storage medium is further provided, wherein the computer-readable storage medium is used to store a computer program. When the computer program is run on a computer, the computer is caused to execute the method according to the first aspect.

[0047] In a fourth aspect, a computer program product is further provided, comprising a computer program, which, when executed on a computer, enables the computer to execute the method provided in the first aspect.

[0048] In the fifth aspect, a chip is also provided, which is coupled to a memory in an electronic device and is used to call a computer program stored in the memory and execute the technical solution of the first aspect of the embodiment of the present application. In the embodiment of the present application, "coupling" refers to the direct or indirect combination of two components with each other.

[0049] For the technical effects that can be achieved in the above-mentioned second to fifth aspects, please refer to the description of the technical effects that can be achieved by the corresponding design scheme in the above-mentioned first aspect, and this application will not repeat them here. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] FIG1 is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application;

[0051] FIG2 is a schematic diagram of a software structure of an electronic device provided in an embodiment of the present application;

[0052] FIG3A is a schematic diagram of enabling the object recognition function according to an embodiment of the present application;

[0053] FIG3B is a schematic diagram of a first graphic filling color provided by an embodiment of the present application;

[0054] 3C to 3D are schematic diagrams of edge feathering of a first graphic according to an embodiment of the present application;

[0055] 3E to 3F are schematic diagrams of calling out a first graphic according to an embodiment of the present application;

[0056] FIG3G is a schematic diagram of a synthesis interface provided in an embodiment of the present application;

[0057] FIG4 is a first schematic diagram of determining a target object according to an embodiment of the present application;

[0058] FIG5 is a second schematic diagram of determining a target object according to an embodiment of the present application;

[0059] FIG6A is a first schematic diagram of replacing a target object according to an embodiment of the present application;

[0060] FIG6B is a second schematic diagram of replacing a target object according to an embodiment of the present application;

[0061] FIG6C is a third schematic diagram of replacing a target object according to an embodiment of the present application;

[0062] 7A and 7B are schematic diagrams of a first method of determining multiple target objects according to an embodiment of the present application;

[0063] 8A and 8B are schematic diagrams of a second method of determining multiple target objects according to an embodiment of the present application;

[0064] FIG9 is a schematic diagram of selecting a candidate target object according to an embodiment of the present application;

[0065] FIG10 is a schematic diagram of selecting a target object according to an embodiment of the present application;

[0066] FIG11 is a schematic diagram of opening a translation function according to an embodiment of the present application;

[0067] FIG12 is a schematic diagram of a translation process provided by an embodiment of the present application;

[0068] FIG13 is another schematic diagram of the translation process provided in one embodiment of the present application;

[0069] FIG14 is another structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] Below, some terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0071] The at least one involved in the embodiments of the present application includes one or more; wherein, more than one means greater than or equal to two. In addition, it should be understood that, in the description of this specification, words such as "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as expressing or implying relative importance, nor can they be understood as expressing or implying order. For example, the first figure and the second figure do not represent the importance of the two or the order of the two, but are only for distinguishing the description. In the embodiments of the present application, "and / or" is only a description of the association relationship, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects related to each other are in an "or" relationship.

[0072] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the specification. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0073] As used in this specification, the terms “when” or “after” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrases “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if it is determined that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0074] The method for determining the target object provided in the embodiment of the present application is applicable to electronic devices. Optionally, the electronic device has a camera. For example, the electronic device can be a portable electronic device such as a mobile phone, a tablet computer, a laptop computer, etc.; it can also be an image capture device such as a smart camera, it can also be office equipment such as a desktop computer, a projection line, etc., it can also be a wearable device such as a watch or a bracelet; it can also be a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, etc. In short, the embodiment of the present application does not limit the specific type of electronic device. For ease of understanding, this article mainly takes the electronic device as a mobile phone as an example for explanation.

[0075] FIG1 is a schematic diagram of the structure of an electronic device. The electronic device may be, for example, a mobile phone. As shown in FIG1 , the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0076] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may serve as the nerve center and command center of the electronic device. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a high-speed cache memory. This memory may store instructions or data that have just been used or are being recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces the processor 110's waiting time, and thus improves system efficiency.

[0077] Among them, the processor 110 can execute the process of the method for determining the target object provided in the embodiment of the present application. For example, the processor 100 can start the camera 193. The camera 193 is used to capture a preview image in real time. The processor 110 displays the viewfinder interface through the display screen 194, and updates the preview image in real time in the viewfinder interface. The preview image includes at least one object, and the upper layer of the viewfinder interface displays a first graphic (to be described later). It can be understood that when the user adjusts the shooting range, the preview image is updated in real time, so the position of the object in the viewfinder interface changes in real time.

[0078] Taking the example of a fixed arrangement of camera 193 and an electronic device, when the electronic device is stable and motionless, the camera's shooting range remains unchanged. When the electronic device's posture changes, the camera's shooting range changes. The gyroscope sensor 180B can detect in real time whether the body's posture has changed. When the gyroscope sensor 180B detects a change in the body's posture, it sends an instruction to the processor 110, which is used to indicate the change in the body's posture. The processor 110 can determine the change in the shooting range based on the instruction. In response to the adjustment of the shooting range, the processor 110 maintains the position of the first graphic unchanged and updates the preview image in real time in the viewfinder interface. Assuming that the position of the first object on the preview image moves into the coverage area of ​​the first graphic, in this case, if the processor 110 detects that the shooting range remains unchanged and the duration of the unchanged period is greater than or equal to the first duration, the first object is determined to be the target object. The specific implementation process of determining the target object will be described later.

[0079] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0080] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus lines. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, and the like via different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K via the I2C interface, enabling communication between the processor 110 and the touch sensor 180K via the I2C bus interface, thereby implementing the touch function of the electronic device 100.

[0081] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface, enabling the function of answering calls via a Bluetooth headset.

[0082] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0083] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0084] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 100. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0085] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0086] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0087] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0088] The wireless communication function of the electronic device can be implemented using antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, a modem processor, and a baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antenna can be used in conjunction with a tuning switch.

[0089] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied in electronic devices. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0090] The wireless communication module 160 can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0091] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0092] The display screen 194 is used to display the display interface of the application, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device may include one or more display screens 194.

[0093] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor, etc. Among them, the ISP is used to process the data fed back by the camera 193.

[0094] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, and the software code of at least one application, etc. The data storage area can store data (such as images, videos, etc.) generated during the use of the electronic device. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory, etc.

[0095] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as pictures and videos can be stored on the external memory card.

[0096] The electronic device can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0097] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0098] The speaker 170A, also called a "speaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to external speaker scenarios such as hands-free calls through one or more speakers 170A.

[0099] The receiver 170B, also called "earpiece", can be one or more and is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be heard by placing the receiver 170B close to the human ear.

[0100] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0101] The headphone jack 170D is used to connect a wired headphone.

[0102] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0103] The gyro sensor 180B can be used to determine the motion posture of the electronic device. In some embodiments, the gyro sensor 180B can be used to determine the angular velocity of the electronic device around three axes (i.e., the x, y, and z axes). The gyro sensor 180B can also be used for anti-shake photography.

[0104] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0105] The magnetic sensor 180D includes a Hall sensor. For example, the electronic device can use the magnetic sensor 180D to detect the opening and closing of a flip leather case.

[0106] The acceleration sensor 180E can detect the magnitude of the electronic device's acceleration in various directions (generally three axes) and the magnitude and direction of gravity when the electronic device is stationary.

[0107] The distance sensor 180F is used to measure distance. The electronic device can measure distance using infrared or laser.

[0108] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device emits infrared light through the light emitting diode. The electronic device uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device. When insufficient reflected light is detected, the electronic device can determine that there is no object near the electronic device.

[0109] The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device can adaptively adjust the brightness of the display screen 194 according to the sensed ambient light brightness.

[0110] The fingerprint sensor 180H is used to collect fingerprints.

[0111] The temperature sensor 180J is used to detect temperature.

[0112] The touch sensor 180K, also known as a "touch panel," can be mounted on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen." The touch sensor 180K is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event.

[0113] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone mass in a human vocal part.

[0114] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device may receive button inputs and generate key signal inputs related to the user settings and function control of the electronic device. The motor 191 may generate vibration prompts. The motor 191 may be used for incoming call vibration prompts or for touch vibration feedback. The indicator 192 may be an indicator light that may be used to indicate the charging status, power level changes, messages, missed calls, notifications, and the like. The SIM card interface 195 is used to connect a SIM card. The SIM card may be inserted into or removed from the SIM card interface 195 to connect to or disconnect from the electronic device.

[0115] It is understood that the components shown in FIG1 do not constitute a specific limitation on the electronic device. The electronic device in the embodiment of the present invention may include more or fewer components than those shown in FIG1. ​​In addition, the combination / connection relationship between the components in FIG1 can also be adjusted and modified.

[0116] Figure 2 is a diagram of the software structure of an electronic device provided in an embodiment of the present application. The electronic device may be a mobile phone. As shown in Figure 2, the software structure of the electronic device may be a layered structure, wherein the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. The layered architecture may be, for example, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application is based on a layered architecture. The system is used as an example to illustrate the software structure of the electronic device. In some embodiments, The system is divided into five layers, from top to bottom: application layer, application framework layer, Android runtime and system library, kernel layer and hardware layer.

[0117] The application layer can include a series of application packages. As shown in Figure 2, the application package can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0118] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 2, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and so on. The window manager manages window programs. It can obtain the display screen size, determine whether a status bar is present, lock the screen, take screenshots, and so on. The content provider stores and retrieves data and makes this data accessible to applications. This data may include video, images, audio, dialed and received calls, browsing history and bookmarks, and a phone book. The view system includes visual controls, such as controls for displaying text and images. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying images. The telephony manager provides communication functions for electronic device 100, such as managing call status (including connected and hung up). The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on. The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0119] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system. The core library consists of two parts: one for Java-based functions and the other for the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes Java files from the application layer and application framework layer as binary files. The virtual machine performs functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection. The system library includes multiple functional modules, such as the surface manager, media libraries, 3D graphics libraries (such as OpenGL ES), and 2D graphics engines (such as SGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various common audio and video formats, as well as still image files. The media library supports a variety of audio and video codecs, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0120] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0121] The hardware layer includes various hardware, such as cameras, displays, etc.

[0122] It should be understood that the software structure shown in FIG2 does not constitute a specific limitation on the software structure of an electronic device. The software structure of an electronic device in an embodiment of the present invention may include more or fewer modules than shown in FIG2 , for example, more or fewer layers than shown in FIG2 , or a layer may include more or fewer modules than shown in FIG2 , etc. Furthermore, the combination / connection relationship between the modules in FIG2 may also be adjusted and modified.

[0123] The following uses the camera application as an example to illustrate the processing flow of each component in Figure 2.

[0124] The touch sensor in the hardware layer receives a touch operation, such as a touch operation used to turn on the object recognition function in the electronic device (for example, Huawei Smart Vision). The hardware layer reports the input event to the upper layer (for example, the application framework layer), and the input event includes information about the touch operation, such as the operation type, operation time, coordinate position, etc. The application framework layer can determine the control targeted by the input operation. For example, if it is determined that it is an operation for Huawei Smart Vision, it will report it to the Smart Vision in the application layer. The application layer starts Smart Vision by calling the interface in the application framework layer, and starts the camera to capture images by calling the camera driver in the kernel layer. It also displays the viewfinder interface on the display screen by calling the display driver in the kernel layer. The viewfinder interface includes the image captured by the camera. The upper layer of the viewfinder interface also displays a first graphic. The area of ​​the first graphic is smaller than the area of ​​the viewfinder interface.

[0125] The motion sensor in the hardware layer detects a change in the posture of the electronic device and reports an instruction to the upper layer (such as the application framework layer), and the instruction is used to indicate the change in the posture of the electronic device. The application framework layer controls the position of the first graphic to remain unchanged and updates the image in real time in the viewfinder interface. The application framework layer detects in real time whether there is an object within the coverage of the first graphic. When the first object enters the coverage of the first graphic, if it is detected that the posture of the electronic device remains unchanged and the length of time it remains unchanged reaches a first length of time, the first object is determined to be the target object.

[0126] The following takes a mobile phone as an example and describes the method for determining a target object provided by an embodiment of the present application in conjunction with the accompanying drawings.

[0127] The mobile phone can have an object recognition function, that is, using the camera to collect images of the surrounding environment, determine the target object from multiple objects contained in the image, and then identify the target object, for example, identify the category, name, price, purchase link, price and link of the same product, etc. The object recognition function in the mobile phone includes Huawei Smart Vision, For easier understanding, the following uses the smart vision in a mobile phone as an example.

[0128] Please refer to (a) in Figure 3A , where the mobile phone displays interface 301. Interface 301 can be, for example, the negative one screen of the mobile phone. Interface 301 includes an icon 302 for smart vision. When the mobile phone receives an operation (for example, a click operation) on icon 302, it starts the camera and displays interface 303 as shown in (b) in Figure 3A . Interface 303 is the viewfinder interface of the camera (also known as the shooting interface). A preview image can be displayed in real time in interface 303. The preview image includes one or more objects. In an embodiment of the present application, a first graphic 304 is displayed on the upper layer of interface 303. The mobile phone can use the first graphic 304 to determine the target object, and the specific implementation process will be described later.

[0129] It is understandable that before displaying the first graphic 304, the mobile phone can determine the shape, area, display position, etc. of the first graphic 304. Among them, the shape of the first graphic 304 can be arbitrary, such as a circle, a square, a rectangle, etc. One possible way is that the shape of the first graphic 304 is the default of the mobile phone system (for example, the default is a circle). Another possible way is that the shape of the first graphic 304 is related to the current scene (object recognition scene or translation scene). For example, if the mobile phone currently turns on the object recognition function, that is, the current scene is the object recognition scene, then the first graphic 304 is a circle; or, if the mobile phone currently turns on the translation function, that is, the current scene is the translation scene, then the first graphic 304 can be a square or a rectangle (to be explained in Figure 11 later). This is because, generally speaking, text is presented line by line, and the entire paragraph of text is presented in a square or rectangle, so if the first graphic 304 is a square or a rectangle, it matches the shape of the entire paragraph of text, which is convenient for translation. Optionally, the display position of the first graphic 304 can be arbitrary, such as in the center of the display screen. One possible approach is that the display position of the first graphic 304 is the default position of the mobile phone system (eg, the center position). In addition, the area of ​​the first graphic 304 can be a fixed value of the system default, and the specific area size is not limited in this embodiment of the application.

[0130] FIG3A shows an example of a first graphic 304 that is hollow and has no fill color. Optionally, the first graphic 304 can also be filled with color. Here, "the first graphic 304" can be understood as the area enclosed by the edge outline of the first graphic 304. For example, referring to FIG3B (a), the first graphic 304 is filled with color, where the fill color can be any color, and this embodiment of the application is not limited thereto. It is understood that to prevent the fill color within the first graphic 304 from obscuring the preview image, the first graphic 304 can have a certain degree of transparency, as shown in FIG3B (a). Optionally, the outside of the first graphic 304 can also be filled with color. Here, "the outside of the first graphic 304" can be understood as other areas on the display screen outside the area enclosed by the edge outline of the first graphic 304. For ease of distinction, the fill color within the first graphic 304 can be different from the fill color outside the first graphic 304. For example, as shown in FIG3B (b), the first graphic 304 is filled with a bright color, while the outside of the first graphic 304 is filled with a dark color. Similarly, to avoid obstructing the preview image, the outside of first graphic 304 may also have a certain degree of transparency. The transparency inside first graphic 304 and the transparency outside first graphic 304 may be the same or different. For example, considering that an object inside first graphic 304 may be determined as a target object (described later), the transparency inside first graphic 304 may be higher than the transparency outside first graphic 304. This allows the user to more clearly see whether the desired object has been moved into first graphic 304 when adjusting the shooting range. Furthermore, the brightness inside first graphic 304 and outside first graphic 304 may be the same or different. For example, the brightness inside first graphic 304 is higher than the brightness outside first graphic 304.

[0131] One possible embodiment of (b) in FIG3B is as shown in (a) in FIG3C . After the mobile phone draws interface 303, it overlays an interface layer on top of interface 303, where the area of ​​the interface layer is equal to the area of ​​interface 303, as shown in (b) in FIG3C . The interface layer on the overlying layer includes a first graphic 304, and the interior of first graphic 304 and the exterior of first graphic 304 are filled with different colors, and the interior of first graphic 304 is more transparent than the exterior of first graphic 304.

[0132] Optionally, the edge of the first graphic 304 can be feathered. Edge feathering can be understood as edge blurring. For example, please refer to Figure 3D, which shows the effect of the edge of the first graphic 304 after feathering. Compared with (b) in Figure 3B, the edge of the first graphic 304 in Figure 3D is relatively blurred. One possible implementation method is that before the mobile phone displays the first graphic 304, it can first determine the feathering value of the first graphic 304, feather the edge of the first graphic 304 according to the determined feathering value, and then display the first graphic 304 after edge feathering. Among them, there are many ways for the mobile phone to determine the feathering value, for example, the feathering value is configured by default in the mobile phone system, or the mobile phone determines the feathering value according to the current object distance of the camera. For example, the mobile phone stores the correspondence between the camera object distance and the feathering value, and can determine the feathering value corresponding to the current object distance based on the correspondence. For example, the greater the object distance and the greater the feathering value, the greater the edge feathering degree and the blurrier the edge; the smaller the object distance and the smaller the feathering value, the smaller the edge feathering degree and the clearer the edge. As shown in Figure 3E, the edge feathering degree can be positively correlated with the current object distance of the camera. Optionally, the edge feathering degree of the first graphic 304 can be fixed or dynamically changing. Taking the fixed edge feathering degree of the first graphic 304 as an example, one possible way is that when the mobile phone opens the interface 303, the first graphic 304 automatically appears with the edge feathered, and the feathering degree is the default and does not change. Taking the dynamic change of the edge feathering degree of the first graphic 304 as an example, one possible way is that when the mobile phone opens the interface 303, the first graphic 304 automatically appears with the edge feathered, and the feathering degree is the default. Afterwards, when the mobile phone detects that the object distance of the camera has changed, the edge feathering degree of the first graphic 304 is adjusted according to the current object distance of the camera. For example, referring to (a) in FIG3E , when the mobile phone opens the interface 303, the first graphic 304 automatically appears and the edges are feathered and the feathering degree is the default. When the object distance of the camera is shortened, the edge feathering degree of the first graphic 304 is weakened, that is, the edge becomes clearer, as shown in (b) in FIG3E . When the object distance of the camera is increased, the edge feathering degree of the first graphic 304 is increased, that is, the edge is more blurred, as shown in (c) in FIG3E . It should be noted that in the three figures in FIG3E , the area of ​​the first graphic 304 looks different, but is actually the same, because the edge feathering causes the first graphic 304 to have a visually magnified effect.

[0133] In the above embodiment, first graphic 304 automatically appears when the phone opens interface 303. In other embodiments, first graphic 304 is not present when the phone opens interface 303. Upon receiving an operation to call out first graphic 304, first graphic 304 is displayed. Various operations can be used to call out first graphic 304, such as a long press, double-click, or clicking a specific button within interface 303. For example, referring to FIG3F (a), the phone displays interface 301. Interface 301 includes icon 302. When the phone receives an operation targeting icon 302, interface 303 as shown in FIG3F (b) is displayed. At this time, first graphic 304 is not yet displayed on the upper layer of interface 303. Interface 303 includes icon 309. When the phone receives an operation targeting icon 309, first graphic 304 is displayed, as shown in FIG3F (c). For another example, referring to FIG3G (a), the phone displays the main interface, which includes a camera icon 310. When the mobile phone receives an operation for icon 310, it displays the viewfinder interface 311 as shown in (b) of Figure 3G. The viewfinder interface 311 includes a more button 312. When the mobile phone receives an operation to click the more button 312, it displays the interface shown in (c) of Figure 3G, which includes a smart object recognition button 313. When the mobile phone receives an operation for the smart object recognition button 313, it displays the interface 311 as shown in (d) of Figure 3G, and the first graphic 304 is displayed on the upper layer of the interface 311. Optionally, after the mobile phone displays the first graphic 304, if it receives an operation for hiding the first graphic 304, it can cancel the display of the first graphic 304. Among them, the operation for calling out the first graphic 304 and the operation for hiding the first graphic 304 can be the same or different. Taking the double-click operation in the interface 303 as an example, if the first graphic 304 is not currently displayed, if the mobile phone receives a double-click operation in the interface 303, the first graphic 304 is displayed. When the first graphic 304 is currently displayed, if the phone receives a double-click operation within interface 303, the first graphic 304 is hidden. For example, in Figure 3F (c), if the phone receives an operation on icon 309, the interface returns to Figure 3F (b), hiding the first graphic 304. For example, in Figure 3G (d), if the phone receives an operation on More button 312, the interface in Figure 3G (c) is opened. If the phone receives an operation on icon 313, the interface in Figure 3G (b) is displayed, in which the first graphic 304 is no longer displayed.

[0134] The following describes a process in which the mobile phone uses the first graphic 304 to determine the target object.

[0135] Method A

[0136] Referring to FIG4(a), the mobile phone displays interface 303. Interface 303 includes three objects: object 401 (picture frame), object 402 (plant), and object 403 (cat). A first graphic 304 is displayed on the upper layer of interface 303. Assume that the user's desired target object is object 402 (plant). The user can adjust the shooting range so that object 402 enters first graphic 304. Object 402 entering first graphic 304 can be understood as entering the coverage area of ​​first graphic 304, that is, entering the area enclosed by the edge outline of first graphic 304 within the coverage area on interface 303. As shown in FIG4(b), first graphic 304 includes object 402. When the mobile phone detects that the shooting range remains unchanged for a period of time that reaches threshold 1 (e.g., 2 seconds, 3 seconds, or 4 seconds), it determines that the object within first graphic 304 (i.e., object 402) is the target object. Among them, the shooting range remains unchanged, which may include: the field of view (FOV) remains unchanged (i.e., the shooting is not zoomed in or out), and the shooting angle remains unchanged (i.e., the shooting angle is not moved in various directions such as up, down, left, and right). Taking into account that shaking is inevitable when the user holds the electronic device, that is, the shooting range will inevitably change, so in the embodiment of the present application, the shooting range remains unchanged, and may also include: the change in the field of view angle is smaller than the first angle, and / or the change in the shooting angle when moving in various directions such as up, down, left, and right is smaller than the second angle. Among them, the specific values ​​of the first angle and the second angle are not limited in the embodiment of the present application. It should be understood that if the camera on the mobile phone is fixed, when the posture of the mobile phone changes, the camera shooting range changes, and when the mobile phone remains stable, the camera shooting range remains unchanged. Therefore, in this case, "the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 1" may mean that the mobile phone remains stable and the stable length reaches a threshold value 1. Among them, the mobile phone remains stable may mean that the posture of the mobile phone remains stable. Assuming that the posture of a mobile phone is described using posture angles (including pitch, yaw, and roll), the mobile phone remaining stable may include: the change in pitch angle being less than angle 1, and / or the change in yaw angle being less than angle 2, and / or the change in roll angle being less than angle 3. The specific values ​​of angle 1, angle 2, and angle 3 are not limited in this embodiment of the application.

[0137] Furthermore, it should be noted that if there is only one object within first graphic 304, the mobile phone determines that object as the target object. If there are multiple objects within first graphic 304, the mobile phone determines that all of the objects are target objects (as described later in FIG. 7A ). Therefore, if the user only desires a single object, the capture range should be adjusted to the extent possible so that only that object is included within first graphic 304.

[0138] Optionally, in FIG4(b), after the mobile phone determines that object 402 within first graphic 304 is the target object, an interface such as FIG4(c) may be displayed, in which first graphic 304 is not displayed. In order to prompt the user that the currently determined target object is object 402, object 402 may be highlighted. The highlighting of object 402 may include: highlighting the edge outline of object 402 (for example, highlighting or bolding), filling the area enclosed by the edge outline of object 402 with a specific color, and the brightness of the area enclosed by the edge outline of object 402 being higher than that of other areas, etc., which is not limited in the embodiments of the present application.

[0139] Optionally, in Figure 4 (c), the phone may also display a confirmation button and a return button. When the phone receives an action on the confirmation button, it identifies the target object and displays the recognition result (not shown in the figure). For example, the recognition result may include: the target object's category, name, price, purchase link, price and link of similar products, etc. When the phone receives an action on the return button, it returns to the previous step, that is, returns to the interface of Figure 4 (b).

[0140] Method B

[0141] Referring to FIG. 5 (a), the mobile phone displays interface 303. Interface 303 includes three objects: object 401 (picture frame), object 402 (plant), and object 403 (cat). A first graphic 304 is displayed on the upper layer of interface 303. Assume that the user's desired target object is object 402 (plant). The user can adjust the shooting range so that object 402 enters first graphic 304. The principle of object 402 entering first graphic 304 is described in Method A above. As shown in FIG. 5 (b), first graphic 304 includes object 402. When the mobile phone detects that the shooting range remains unchanged for a period of time that reaches threshold 2, it determines that the object within first graphic 304 (i.e., object 402) is a candidate target object. It should be understood that if there is only one object within first graphic 304, the mobile phone determines that object as a candidate target object. If there are multiple objects within first graphic 304, the mobile phone determines that all multiple objects are candidate target objects (as described in FIG. 8A below).

[0142] Optionally, in FIG5(b), after the mobile phone determines that object 402 within first graphic 304 is a candidate target object, it displays an interface as shown in FIG5(c), in which first graphic 304 is adjusted to second graphic 305. The area of ​​second graphic 305 is smaller than that of first graphic 304. For example, second graphic 305 is a circumscribed circle or a circumscribed polygon of object 402, which is used to prompt the user that object 402 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the duration of this unchanged range reaches threshold 3, the candidate target object is determined to be the target object, that is, object 402 within second graphic 305 is determined to be the target object.

[0143] Optionally, in (c) of Figure 5 , after the mobile phone determines that object 402 is the target object, an interface as shown in (d) of Figure 5 is displayed, and the second graphic 305 in the interface is canceled. In order to prompt the user that the currently determined target object is object 402, object 402 is highlighted. For the method of highlighting object 402, please refer to method A above, which will not be repeated here. Optionally, in (d) of Figure 5 , a return button and a confirmation button can also be displayed on the mobile phone. When the mobile phone receives an operation for the confirmation button, the target object can be identified and the identification result can be displayed. When the mobile phone receives an operation for the return button, it can return to the previous step, that is, the interface as shown in (b) of Figure 5 is displayed, and the user can adjust the shooting range again to make other objects enter the first graphic 304.

[0144] The difference between Method A and Method B is that in Method A, after the user adjusts the shooting range so that object 402 (the plant) is within first graphic 304, as long as the shooting range remains unchanged for a period of time exceeding threshold 1 (e.g., 5 seconds), object 402 is determined as the target object. In Method B, after the user adjusts the shooting range so that object 402 (the plant) is within first graphic 304, the shooting range remains unchanged for a period of time exceeding threshold 2, at which point object 402 is determined as a candidate target object. Furthermore, if the shooting range remains unchanged for a period of time exceeding threshold 3, object 402 is determined as the target object. Therefore, Method A has fewer steps and is more efficient. Method B first determines candidate target objects and then determines the target object, resulting in higher accuracy. A mobile phone can use either Method A or Method B. For example, the mobile phone system may default to using Method A or Method B. Of course, Method A and Method B can be switched, for example, by providing a toggle button on the mobile phone that allows switching between Method A and Method B.

[0145] Among them, threshold 1, threshold 2 and threshold 3 can satisfy: threshold 1 is equal to threshold 2; or, threshold 1 is equal to the sum of threshold 2 and threshold 3. For example, threshold 2 is 2s and threshold 3 is also 2s, then threshold 1 is 4s. The embodiment of the present application does not limit the specific value of each threshold.

[0146] The following mainly uses the above method B (i.e., Figure 5) as an example for explanation. It is understandable that during the process of Figure 5, the user may want to change the target object. For example, the user may change the target object when the phone displays the interface (b) in Figure 5, or may change the target object when the phone displays the interface (c) in Figure 5, or may change the target object when the phone displays the interface (d) in Figure 5. The following describes different examples.

[0147] Example 1: When the mobile phone displays the interface shown in (b) of FIG5 , the user changes the target object.

[0148] Referring to (a) in FIG. 6A , object 402 is located within first graphic 304 . At this point, if the user wants to change the target object (for example, to object 403 ), the user can adjust the shooting range so that object 403 enters first graphic 304 . As shown in (b) in FIG. 6A , object 403 is included within first graphic 304 . When the mobile phone detects that the shooting range remains unchanged and the duration of this unchanged period reaches a threshold of 4, object 403 is determined to be a candidate target object. The mobile phone displays an interface as shown in (c) in FIG. 6A , in which first graphic 304 is adjusted to second graphic 305 . Second graphic 305 is a circumscribed circle or circumscribed polygon of object 403 , indicating that object 403 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the duration of this unchanged period reaches a threshold of 5, the candidate target object is determined to be the target object, i.e., object 403 within second graphic 305 is determined to be the target object. After determining that object 403 is the target object, the mobile phone displays an interface as shown in (d) of FIG. 6A , in which the second graphic 305 disappears and object 403 is highlighted, prompting the user that the currently determined target object is object 403 .

[0149] Among them, threshold 4 can be equal to threshold 2 in the previous text, and threshold 5 can be equal to threshold 3 in the previous text.

[0150] Example 2: When the mobile phone displays the interface (c) in Figure 5, the user changes the target object.

[0151] Please refer to (a) in Figure 6B , object 402 (green plants) is located in the second graphic 305, indicating that object 402 (green plants) has been determined as a candidate target object. At this time, if the user wants to change the target object (for example, to object 403), the shooting range can be adjusted. It is understandable that when the shooting range is adjusted, the position of object 402 (green plants) that has been determined as a candidate target object will move. In an embodiment of the present application, since object 402 has been determined as a candidate target object, when the position of object 402 moves, the position of the second graphic 305 moves synchronously, as shown in (b) in Figure 6B . It can be understood that after object 402 is determined as a candidate target object, the second graphic 305 is attached to the upper layer of object 402 and moves synchronously with object 402.

[0152] As mentioned above, the second graphic 305 is obtained by transforming the first graphic 304. Before being adjusted to the second graphic 305, the first graphic 304 is displayed at the center. After the second graphic 305 moves synchronously with the object 402, the center becomes vacant.

[0153] The mobile phone can detect whether a new object appears at the center position. Assume that a new object appears at the center position, for example, in FIG6B (b), object 403 appears at the center position. Optionally, when a new object appears at the center position, a graphic 601 can be displayed to indicate that the new object, i.e., object 403, is about to be determined as the target object. Of course, graphic 601 may not be displayed. If a new object appears at the center position and the shooting range remains unchanged for a period of time that reaches a threshold value 6, an interface such as FIG6B (c) is displayed, in which the second graphic 305 is restored to the first graphic 304 and displayed at the center position. It is understandable that since object 403 is already at the center position, after the first graphic 304 is restored, object 403 may be completely within the first graphic 304, or may be partially within the first graphic 304 and partially outside the first graphic 304. In this way, the user can adjust the shooting range so that object 403 is as completely within the first graphic 304 as possible. When object 403 is within first graphic 304, if the mobile phone detects that the shooting range remains unchanged and the duration of this remains unchanged reaches threshold 7, object 403 is determined to be a candidate target object and an interface as shown in (d) of FIG6B is displayed, in which first graphic 304 is adjusted to second graphic 305. Second graphic 305 is the circumscribed circle or circumscribed polygon of object 403, which is used to indicate that object 403 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the duration of this remains unchanged reaches threshold 8, object 403 within second graphic 305 is determined to be a target object. Therefore, the mobile phone displays an interface as shown in (e) of FIG6B, in which second graphic 305 disappears and object 403 is highlighted to prompt the user that the currently determined target object is object 403. Among them, threshold 7 can be equal to threshold 2 mentioned above, and threshold 8 can be equal to threshold 3 mentioned above.

[0154] If the mobile phone detects that no new object appears at the center, object 402, which has been determined as a candidate target object, remains a candidate target object. When the mobile phone detects that the shooting range remains unchanged and the duration of this change reaches a threshold of 9, object 402 is determined to be a target object. Threshold 9 may be equal to the aforementioned threshold of 3.

[0155] Example 3: When the mobile phone displays the interface (d) in Figure 5, the user changes the target object.

[0156] Referring to (a) in FIG6C , the phone has determined that object 402 (the plant) is the target object, so object 402 is highlighted. The phone also displays a return button 602 and a confirmation button 603. When the phone receives an operation on the return button 602, it displays an interface as shown in (b) in FIG6C , which displays a first graphic 304 on top of the screen, and the first graphic 304 includes object 402. If the user wants to change the target object (for example, to object 403), the user can adjust the shooting range so that object 403 is within the first graphic 304, as shown in (c) in FIG6C . If the phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold of 10, it can display an interface as shown in (d) in FIG6C , in which the first graphic 304 is adjusted to the second graphic 305. The second graphic 305 is the circumscribed circle or circumscribed polygon of object 403, which prompts the user that object 403 has been determined as a candidate target object. If the mobile phone detects that the shooting range remains unchanged for a period of time that reaches threshold 11, an interface as shown in FIG6C (e) is displayed, in which second graphic 305 disappears and object 403 is highlighted, prompting the user that the currently determined target object is object 403. Threshold 10 may be equal to threshold 2 mentioned above, and threshold 11 may be equal to threshold 3 mentioned above.

[0157] In the above embodiment, the number of target objects is 1 as an example. It is understandable that the number of target objects can be multiple. The following describes the case where the mobile phone determines multiple target objects.

[0158] Method C

[0159] Please refer to (a) in Figure 7A , the mobile phone display interface 303. Assume that there are two target objects desired by the user, namely object 402 and object 403. The user adjusts the shooting range so that both object 402 and object 403 enter the first graphic 304, as shown in (b) in Figure 7A , and the first graphic 304 includes object 402 and object 403. It can be understood that in order to ensure that object 402 and object 403 are as close as possible to the first graphic 304, when the user adjusts the shooting range, the user may step back to move away from the object, that is, the object distance increases, so the edge feathering of the first graphic 304 in (b) in Figure 7A increases. When the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold of 12, it determines that object 402 and object 403 in the first graphic 304 are both target objects. Optionally, after the mobile phone determines that both object 402 and object 403 are target objects, it may display an interface as shown in (c) of FIG. 7A , in which the first graphic 304 is canceled and object 402 and object 403 are highlighted, respectively, to prompt the user that object 402 and object 403 are determined as target objects. Optionally, in (c) of FIG. 7A , the mobile phone also displays a return button and a confirmation button. When the mobile phone receives an operation on the return button, it returns to the interface of (b) of FIG. 7A . When the mobile phone receives an operation on the confirmation button, it recognizes object 402 and object 403 and displays the recognition results (not shown in the figure).

[0160] For easier understanding, please compare Figure 4 and Figure 7A above. In Figure 4 (b), first graphic 304 includes only one object 402. When the phone detects that the shooting range remains unchanged for a period of time exceeding threshold 1, it determines that object 402 is the target object. In Figure 7A (b), the first graphic includes two objects, namely object 402 and object 403. When the phone detects that the shooting range remains unchanged for a period of time exceeding threshold 12, it determines that both object 402 and object 403 are the target objects. Threshold 12 can be equal to threshold 1. Therefore, if the user expects only one object, the shooting range is adjusted to ensure that only that object is included in first graphic 304. If the user expects multiple objects, the shooting range is adjusted to ensure that all objects are within first graphic 304.

[0161] Optionally, in FIG7A (b), when first graphic 304 includes two objects, the user can select one of the two objects. For example, referring to FIG7B (a), first graphic 304 includes object 402 and object 403. After receiving the operation to select object 402, the mobile phone can immediately determine that object 402 is the target object or determine that object 402 is the target object when it is determined that the shooting range remains unchanged and the duration of the unchanged state reaches threshold 13. Since object 403 is not selected, object 403 is not the target object. The operation for selecting object 402 can be an operation on object 402 (e.g., single-click, double-click, long press, etc.), or a click operation on object 403 (e.g., single-click, double-click, or long press). After the mobile phone determines that object 402 is the target object, it can display an interface such as FIG7B (b), in which object 402 is highlighted and object 403 is not highlighted. Threshold 13 can be equal to threshold 12.

[0162] Optionally, before receiving the user's operation of selecting an object 402, the mobile phone can output a prompt message to prompt the user to specify a target object (not shown in the figure). Optionally, when the electronic device determines that the first graphic 304 includes multiple objects, it can immediately display the prompt message or display the prompt message when it is determined that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold 14. The prompt message can be text, an icon, etc., or a dotted box. For example, the dotted box can have the same shape and position as the first graphic 304. Optionally, after determining the target object, the mobile phone can cancel the display of the prompt message. Among them, the sum of threshold 13 and threshold 14 can be greater than or equal to threshold 12.

[0163] Method D

[0164] Please refer to (a) in Figure 8A, where the mobile phone displays interface 303. Assume that the user has two target objects, namely object 402 and object 403. The user adjusts the shooting range so that both object 402 and object 403 enter the first graphic 304. As shown in (b) in Figure 8A, the first graphic 304 includes object 402 and object 403. When the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 15, it determines that the objects in the first graphic 304 (i.e., object 402 and object 403) are candidate target objects. Optionally, after the mobile phone determines that object 402 and object 403 are candidate target objects, it can display an interface as shown in (c) in Figure 8A, in which the first graphic 304 is adjusted to two small graphics, namely second graphic 304a and third graphic 304b. Among them, the second graphic 304a is the circumscribed circle or circumscribed polygon of object 402, which is used to indicate that object 402 is a candidate target object. The third graphic 304b is the circumscribed circle or circumscribed polygon of object 403, which is used to prompt that object 403 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 16, it determines that objects 402 and 403 are target objects. Optionally, after the mobile phone determines that objects 402 and 403 are target objects, it can display an interface as shown in (d) of Figure 8A, in which the second graphic 304a and the third graphic 304b are canceled, and objects 402 and 403 are highlighted respectively to prompt the user that objects 402 and 403 are target objects. Optionally, in (c) of Figure 8A, the mobile phone also displays a return button and a confirmation button. When the mobile phone receives an operation for the return button, it returns to the interface of (b) of Figure 8A. When the mobile phone receives an operation for the confirmation button, it recognizes objects 402 and 403 and displays the recognition results (not shown in the figure).

[0165] For easier understanding, please compare Figure 5 and Figure 8A above. In Figure 5 (b), the first graphic 304 includes only one object 402. When the mobile phone detects that the shooting range remains unchanged and the duration of this unchanged period reaches threshold 2, object 402 is determined to be a candidate target object, as shown in Figure 5 (c). When the mobile phone detects that the shooting range remains unchanged and the duration of this unchanged period reaches threshold 3, the candidate target object is determined to be the target object, as shown in Figure 5 (d). In Figure 8A (b), the first graphic 304 includes two objects, namely object 402 and object 403. When the mobile phone detects that the shooting range remains unchanged and the duration of this unchanged period reaches threshold 15, both object 402 and object 403 are determined to be candidate target objects, as shown in Figure 8A (c). When the mobile phone detects that the shooting range remains unchanged and the duration of this unchanged period reaches threshold 16, the candidate target object is determined to be the target object, as shown in Figure 8A (d). Therefore, threshold 15 can be equal to threshold 2. Threshold 16 can be equal to threshold 3. It can be understood that in (b) in Figure 8A, if the first graphic 304 includes more objects, when the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches the threshold 15, it is determined that all objects in the first graphic 304 are candidate target objects, and the first graphic 304 will be adjusted to more small graphics, each small graphic corresponding to an object.

[0166] Optionally, in FIG8A(b), when first graphic 304 includes multiple objects, the user can select one of the multiple objects. For example, referring to FIG8B(a), first graphic 304 includes object 402 and object 403. After receiving the operation to select object 402, the mobile phone can immediately determine object 402 as a candidate target object or determine object 402 as a candidate target object when it is determined that the shooting range remains unchanged and the duration of this change reaches a threshold of 17. Since object 403 is not selected, object 403 is not a candidate target object. The operation for selecting object 402 can be an operation on object 402 or a click operation on object 403. After the mobile phone determines that object 402 is a candidate target object, it can display an interface as shown in FIG8B(b), in which first graphic 304 is adjusted to second graphic 304a, which is a circumscribed circle or circumscribed polygon of object 402. If the mobile phone detects that the shooting range remains unchanged and the duration of this change reaches a threshold of 18, it determines object 402 as a target object. After the mobile phone determines that object 402 is the target object, it displays an interface as shown in FIG8B (c), in which object 402 is highlighted and object 403 is not highlighted.

[0167] Optionally, before receiving the user's operation of selecting object 402, the mobile phone can output prompt information to prompt the user to specify a target object (not shown in the figure). Please refer to the above for the display method of the prompt information, which will not be repeated.

[0168] The difference between Method C and Method D is that Method C involves fewer steps and is more efficient. Method D offers greater accuracy. A mobile phone can use either Method C or Method D. For example, a mobile phone system may default to either Method C or Method D. Of course, Method C and Method D can be switched, for example, by using a toggle button on the phone. It should be noted that Methods A and B, as well as Methods C and D here, can be used in combination.

[0169] For example, when the mobile phone determines that the first graphic 304 includes only one object, method A, that is, the method shown in FIG4 , is used. When the mobile phone determines that the first graphic 304 includes multiple objects, method C, that is, the method shown in FIG7A or FIG7B , is used.

[0170] Alternatively, method A and method D may be combined. For example, when the mobile phone determines that the first graphic 304 includes only one object, method A, i.e., the method shown in FIG4 , is used. When the mobile phone determines that the first graphic 304 includes multiple objects, method D, i.e., the method shown in FIG8A or FIG8B , is used.

[0171] Alternatively, method B and method C may be combined. For example, when the mobile phone determines that the first graphic 304 includes only one object, method B, i.e., the method shown in FIG5 , is used. When the mobile phone determines that the first graphic 304 includes multiple objects, method C, i.e., the method shown in FIG7A or FIG7B , is used.

[0172] Alternatively, method B and method D may be combined. For example, when the mobile phone determines that the first graphic 304 includes only one object, method B, i.e., the method shown in FIG5 , is used. When the mobile phone determines that the first graphic 304 includes multiple objects, method D, i.e., the method shown in FIG8A or FIG8B , is used.

[0173] The following description uses method D (i.e., FIG8A ) as an example. It is understood that during the process of FIG8A , the user can change the target object. For example, when the mobile phone displays FIG8A(b), the user can change the target object. The principle of changing the target object is the same as in Example 1 above, and will not be repeated. For another example, when the mobile phone displays FIG8A(c), the user can change the target object. The implementation principle of changing the target object is the same as in Example 2 above, and will not be repeated. For another example, when the mobile phone displays FIG8A(d), the user can change the target object. The principle of changing the target object is the same as in Example 3 above, and will not be repeated. In addition, in FIG8A(c), after the mobile phone determines two candidate target objects, the user can select one of the two candidate target objects. For the specific implementation process, please refer to Example 4 below. In FIG8A(d), after the mobile phone determines two target objects, the user can select one of the two target objects. For the specific implementation process, please refer to Example 5 below.

[0174] Example 4

[0175] For example, see (a) in Figure 9 , where the mobile phone displays interface 303. Interface 303 displays a second graphic 304a and a third graphic 304b on top. Second graphic 304a is the circumcircle of object 402, indicating that object 402 is a candidate target object. Third graphic 304b is the circumcircle of object 403, indicating that object 403 is a candidate target object. When the mobile phone receives a first preset operation, object 403 is removed from the candidacy list, leaving only object 402 as a candidate. The first preset operation can be an operation on third graphic 304b or an operation on second graphic 304a, and can be a single click, double click, or long press. For example, if the first preset operation is an operation on third graphic 304b, when the user clicks on third graphic 304b, object 403 within third graphic 304b is removed from the candidacy list, while object 402 within second graphic 304a remains as a candidate resource. In other words, whichever graphic the user clicks on removes the candidate resource from the object within that graphic, leaving the candidate resource within the other graphic. For example, if the first preset operation is for second shape 304a, after the user clicks second shape 304a, object 402 within second shape 304a remains as a candidate resource, while object 403 within third shape 304b is removed from the candidacy pool. In other words, the object within the shape that the user clicks remains as a candidate, while the object within the other shape is removed from the candidacy pool.

[0176] In Figure 9 (a), after the mobile phone cancels the candidacy of object 403 and retains the candidacy of object 402, an interface as shown in Figure 9 (b) may be displayed, in which the third graphic 304b is canceled and the second graphic 304a remains unchanged. After the third graphic 304b is canceled, a dotted box may be displayed to indicate that the candidacy of object 403 has been canceled. Optionally, the dotted box may not be displayed. If the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 19, it determines that object 402 is the target object. Since the candidacy of object 403 has been canceled, object 403 will not be determined as the target object. Optionally, after the mobile phone determines that object 402 is the target object, an interface as shown in Figure 9 (c) may be displayed, in which the second graphic 304a is canceled and object 402 is highlighted to prompt the user that object 402 is the target object.

[0177] It should be noted that in Figure 9 (b), the phone has removed object 403 from the candidacy list, leaving only object 402 as a candidate. At this point, if the phone receives a second preset action, it can return to the interface shown in Figure 9 (a), allowing the user to reselect a candidate target object. The second preset action can be clicking a blank area in interface 303, where the blank area is defined as the area without the second graphic 304a or the third graphic 304b.

[0178] Example 5

[0179] For example, referring to (a) in FIG10 , object 402 and object 403 are highlighted, indicating that object 402 and object 403 have been determined as target objects. When the mobile phone receives the third preset operation, only object 402 is retained as the target object, and object 403 is no longer the target object. The mobile phone displays an interface as shown in (b) in FIG10 , in which only object 402 is highlighted, and object 403 is no longer highlighted. The third preset operation can be an operation for object 402 or an operation for object 403, and the operation can be a single-click, double-click, or long-press operation. Taking the third preset operation as an example of an operation for object 402, that is, after the user clicks on object 402, object 402 is retained as the target object, and object 403 is canceled as the target object. In other words, whichever object the user clicks on is retained as the target object, and the other object is canceled as the target object. Taking the third preset operation as an example, which is an operation on object 403, after the user clicks on object 403, object 403 is cancelled as the target object and object 402 is retained as the target object. In other words, whichever object the user clicks on is cancelled as the target object and another object is retained as the target object.

[0180] In the above embodiment, the object recognition function in the mobile phone is taken as an example, and the object recognition function is Huawei Smart Vision. It is understandable that the mobile phone may also include other object recognition functions, such as etc., can use the method for determining the target object provided by the embodiment of the present application. In addition, the embodiment of the present application can also be applicable to translation scenarios. For example, a mobile phone can have a translation function, that is, using a camera to capture an image, and then translate the text in the image, such as English to Chinese or Chinese to English. The translation function in the mobile phone includes, for example, The translation and camera applications have text translation functions, etc. This application does not give examples one by one. For the sake of ease of understanding, the following is an example of the translation application on the mobile phone.

[0181] Please refer to (a) in Figure 11, the mobile phone displays the main interface. The main interface includes an icon 1101 of the translation application. When the mobile phone receives an operation for icon 1101, it displays the viewfinder interface shown in (b) in Figure 11, which includes multiple paragraphs of text, and the first graphic 304 is displayed on the upper layer of the viewfinder interface. It can be understood that before the mobile phone displays the first graphic 304, the shape, area, display position, etc. of the first graphic 304 can be determined, please refer to the previous description. It should be noted that the first graphic 304 in the previous text is circular, and here, the first graphic 304 is square or rectangular. As mentioned above, when the current scene is object recognition, the first graphic 304 is circular, and when the current scene is translation, the first graphic 304 is square or rectangular. The reason has been described in the previous text and will not be repeated here. The following describes the process of the mobile phone using the first graphic 304 to translate the target object.

[0182] Method E

[0183] As shown in (a) of Figure 12 , the mobile phone displays a framing interface. The framing interface includes multiple paragraphs of text. A first graphic 304 is displayed on the upper layer of the framing interface. Suppose the user wishes to translate paragraph 1. The user can adjust the shooting range so that paragraph 1 falls within the range of first graphic 304. As shown in (b) of Figure 12 , first graphic 304 includes paragraph 1. When the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 20, it determines that the object within first graphic 304 (i.e., paragraph 1) is the target object. Optionally, after the mobile phone determines that paragraph 1 is the target object, it displays an interface as shown in (c) of Figure 12 , in which paragraph 1 is highlighted to indicate that paragraph 1 has been determined as the target object. The highlighting of paragraph 1 may include: highlighting or bolding each character in paragraph 1. In (c) of Figure 12 , the mobile phone also displays a confirmation button and a return button. If the mobile phone receives an operation on the confirmation button, the mobile phone translates paragraph 1 and displays the translation result (not shown in the figure). If the return button is pressed, the interface returns to FIG. 12 ( b ), and the user can adjust the shooting range to translate other paragraphs.

[0184] The threshold 20 may be equal to the threshold 1 mentioned above.

[0185] Method F

[0186] As shown in (a) of Figure 13 , the mobile phone displays a framing interface. The framing interface includes multiple paragraphs of text. A first graphic 304 is displayed on the upper layer of the framing interface. Suppose the user wishes to translate paragraph 1. The user can adjust the shooting range so that paragraph 1 falls within the range of first graphic 304. As shown in (b) of Figure 13 , first graphic 304 includes paragraph 1. If the mobile phone receives a signal that the shooting range remains unchanged and the duration of this change reaches a threshold value 21, the mobile phone displays a dotted box to prompt the user to select a portion of the text for translation, as shown in (c) of Figure 13 . If the mobile phone receives a signal that the user clicks a punctuation mark (e.g., a colon), a sentence (e.g., sentence 1301) is determined based on the punctuation mark. The starting point of the sentence is the clicked punctuation mark (i.e., the colon) and the end point is the punctuation mark following the clicked punctuation mark (i.e., the colon). After determining sentence 1301, the mobile phone adjusts first graphic 304 to second graphic 304a, as shown in (d) of Figure 13 . Second graphic 304a indicates that sentence 1301 has been determined as a candidate target object. The threshold 21 may be less than or equal to the threshold 20 mentioned above.

[0187] In Figure 13 (d), the user can continue to select sentences. For example, when the phone receives a click on a punctuation mark (e.g., a question mark), it identifies a sentence (e.g., sentence 1302) based on the punctuation mark. The sentence starts at the clicked punctuation mark (i.e., the question mark) and ends at the punctuation mark following the clicked punctuation mark (i.e., the question mark). After identifying sentence 1302, the phone displays a third graphic 304b, as shown in Figure 13 (e). Third graphic 304b indicates that sentence 1302 has been identified as a candidate target object.

[0188] After the mobile phone determines that sentences 1301 and 1302 are candidate target objects, if it is detected that the shooting range remains unchanged and the duration of the unchanged shooting range reaches a threshold value 22, then sentences 1301 and 1302 are determined to be target objects, and the mobile phone displays an interface as shown in (e) in Figure 13, in which the dotted box is canceled and sentences 1301 and 1302 are highlighted. In (e) in Figure 13, the mobile phone also displays a confirmation button and a cancel button. If the mobile phone receives an operation for the confirmation button, the mobile phone translates the selected sentence and displays the translation result (not shown in the figure). If the return button operation is received, the mobile phone returns to the previous step, for example, returns to the interface of (d) in Figure 13, and the user selects other sentences in paragraph 1 for translation; or, returns to the interface of (c) in Figure 13, and the user can adjust the shooting range to translate other paragraphs.

[0189] This article mainly uses the object recognition scene and the translation scene as examples for explanation. However, it should be noted that the technical solution provided by the embodiment of the present application can be applied to any scene where the target object needs to be determined in the image. After determining the target object, the electronic device can identify the target object (i.e., the object recognition scene in the previous article), translate the target object (i.e., the translation scene in the previous article), and perform other operations on the target object, such as copying (i.e., copying the target object), cutting out (i.e., cutting out the target object), and so on.

[0190] Figure 14 is a schematic diagram of the structure of an electronic device 1400 provided in an embodiment of the present application. The electronic device 1400 may be the mobile phone mentioned above. As shown in Figure 14, the electronic device 1400 may include: one or more processors 1401; one or more memories 1402; a communication interface 1403, and one or more computer programs 1404. The above-mentioned components may be connected via one or more communication buses 1405. The one or more computer programs 1404 are stored in the above-mentioned memories 1402 and are configured to be executed by the one or more processors 1401. The one or more computer programs 1404 include instructions. For example, when the electronic device 1400 is the mobile phone mentioned above, the instructions may be used to execute the relevant steps of the mobile phone in the corresponding embodiment above. For example, the mobile phone may display a viewfinder interface, which is a viewfinder interface of a camera. The viewfinder interface includes at least one object. The upper layer of the viewfinder interface displays a first graphic, and the area of ​​the first graphic is smaller than the area of ​​the viewfinder interface. The mobile phone can also respond to the first operation for adjusting the shooting range of the camera, keep the position of the first graphic unchanged, and the first object in the viewfinder interface enters the coverage of the first graphic, the first object is any object of at least one object, and the coverage of the first graphic is the coverage of the area surrounded by the edge outline of the first graphic on the viewfinder interface. The mobile phone can also determine that the first object is the target object when it is determined that the shooting range remains unchanged and the time length for which it remains unchanged is greater than or equal to the first time length. Exemplarily, the process of the mobile phone determining the target object can be found in the relevant descriptions of Figures 3A to 13 above, and for the sake of brevity of the specification, it will not be repeated. The communication interface 1403 is used to realize communication between the electronic device 1400 and other devices. For example, the communication interface can be a transceiver.

[0191] In the embodiments provided in the present application above, the method provided in the embodiment of the present application is introduced from the perspective of an electronic device (such as a mobile phone, etc.) as the execution subject. In order to implement the various functions in the method provided in the embodiment of the present application above, the electronic device may include a hardware structure and / or a software module to implement the above functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether a function of the above functions is executed in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application and design constraints of the technical solution.

[0192] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)). In the absence of conflict, the solutions of the above embodiments can be used in combination.

[0193] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0194] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.

[0195] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0196] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0197] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include these modifications and variations.

Claims

1. A method for determining a target object, characterized in that: Applied to electronic equipment, the method comprises: Displaying a framing interface, the framing interface is a framing interface of a camera in the electronic device, the framing interface includes at least one object, a first graphic is displayed on an upper layer of the framing interface, and an area of ​​the first graphic is smaller than an area of ​​the framing interface; In response to a first operation for adjusting the shooting range of the camera, the position of the first graphic remains unchanged, a first object in the viewfinder interface enters a coverage range of the first graphic, the first object is any object of the at least one object, and the coverage range of the first graphic is a coverage range of an area enclosed by an edge contour of the first graphic on the viewfinder interface; If the shooting range remains unchanged and the time length for which the range remains unchanged is greater than or equal to the first time length, the first object is determined to be the target object.

2. The method according to claim 1, characterized in that: The first graph satisfies at least one of the following: The shape of the first figure is a circle; or, The area of ​​the first figure is a system default; or, The first graphic is displayed at the center of the display screen of the electronic device; or, The first graphic is filled with a first color; or, The area within the first graphic has transparency; or, The edge of the first graphic is feathered, and the feathering degree is a system default or positively correlated with the current object distance of the camera.

3. The method according to claim 1 or 2, characterized in that: The area enclosed by the edge contour of the first graphic is the first area, and the other areas outside the first area are the second area. The first area and the second area satisfy at least one of the following conditions: The color filled in the first area is different from the color filled in the first area; or, The transparency of the first area is higher than the transparency of the second area; or, The brightness of the first area is higher than the brightness of the second area.

4. The method according to any one of claims 1 to 3, characterized in that: After determining that the first object is a target object, the method further includes: The first graphic is canceled from being displayed.

5. The method according to any one of claims 1 to 3, characterized in that: Before the shooting range remains unchanged and the duration of the unchanged shooting range is greater than or equal to the first duration, the method further includes: When it is determined that the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to the second time period, the first object is determined to be a candidate target object, and the first graphic is adjusted to a second graphic, where the second graphic is a circumscribed circle or a circumscribed polygon of the first object.

6. The method according to claim 5, characterized in that After determining that the first object is a target object, the method further includes: The second graphic is canceled from being displayed.

7. The method according to any one of claims 1 to 6, characterized in that: After determining that the first object is a target object, the method further includes: The edge contour of the first object is highlighted; or, The area surrounded by the edge outline of the first object is filled with a third color; or The brightness in the area surrounded by the edge contour of the first object is higher than the brightness in other areas.

8. The method according to claim 5, characterized in that After determining that the first object is a candidate target object, the method further includes: In response to a second operation for adjusting the shooting range of the camera, the positions of the first object and the second graphic in the viewfinder interface move synchronously.

9. The method according to claim 8, characterized in that After the positions of the first object and the second graphic are synchronously moved, the method further includes: Determining that a second object appears at a first position in the framing interface and that the second object remains at the first position for a third time period, wherein the first position is a display position of the first graphic before the first graphic is adjusted to the second graphic; Restoring the second graphic to the first graphic, and displaying the first graphic at the first position, wherein the first graphic includes the second object; When it is determined that the shooting range remains unchanged and the time length for which the shooting range remains unchanged is greater than or equal to a fourth time length, the second object is determined to be a target object.

10. The method according to claim 9, characterized in that After determining that the second object appears at the first position in the viewfinder interface, the method further includes: Displaying a prompt box at the first position, wherein the prompt box includes the second object, and the prompt box is used to prompt that the second object will be determined as a target object; After determining that the time duration for which the second object remains at the first position reaches a third time duration and before restoring the second graphic to the first graphic, the method further includes: canceling display of the prompt box.

11. The method according to any one of claims 1 to 10, characterized in that: The first graphic also includes a third object, which is any object other than the first object among the at least one object; after the shooting range remains unchanged and the time length for which it remains unchanged is greater than or equal to the first time length, it also includes: determining that the third object is a target object.

12. The method according to claim 11, characterized in that The method further comprises: displaying the first object and the third object; When receiving the third operation, retaining the first object as the target object and canceling the third object as the target object; The third operation includes an operation on the first object, or an operation on the third object.

13. The method according to claim 11, characterized in that Before the shooting range remains unchanged and the duration of the unchanged shooting range is greater than or equal to the first duration, the method further includes: When it is determined that the shooting range remains unchanged and the time period of remaining unchanged is greater than or equal to a fourth time period, it is determined that the first object and the third object are candidate target objects, and the first figure is adjusted to a fourth figure and a fifth figure, the fourth figure is a circumscribed circle or a circumscribed polygon of the first object, and the fifth figure is a circumscribed circle or a circumscribed polygon of the third object.

14. The method according to claim 13, characterized in that Before the shooting range remains unchanged and the duration of the unchanged shooting range is greater than or equal to the first duration, the method further includes: If a fourth operation is received, the third object is retained as a candidate target object and the fifth graphic is retained, the first object is cancelled as a candidate target object and the fourth graphic is cancelled; wherein the fourth operation includes an operation on the fifth graphic, or an operation on the fourth graphic; After the shooting range remains unchanged and the time period for which the range remains unchanged is greater than or equal to the first time period, the method further includes: determining that the third object is a target object.

15. The method according to any one of claims 1 to 10, characterized in that: The first graphic further includes a fourth object, and the fourth object is any object other than the first object among the at least one object. If the shooting range remains unchanged and the time length for which the shooting range remains unchanged is greater than or equal to the first time length, before determining that the first object is a target object, the method further includes: An operation of selecting the first object in the first graphic is received.

16. The method according to claim 15, characterized in that Before receiving an operation of selecting the first object in the first graphic, the method further includes: Output prompt information, where the prompt information is used to prompt the user to specify a target object.

17. The method according to any one of claims 1 to 16, characterized in that: After determining that the first object is a target object, the method further includes: Identify the target object, and identify at least one of the name, category, purchase link, related objects of the target object, and purchase links of the related objects of the target object; or, Translating the target object; or, Copy the target object; or, Cut out the target object.

18. The method according to any one of claims 1 to 16, characterized in that: After determining that the first object is a target object, the method further includes: A first button and a second button are displayed, wherein when the first button is triggered, the process returns to the previous N steps, where N is a positive integer, and when the second button is triggered, the target object is translated, identified, copied, or cropped.

19. The method according to any one of claims 1 to 18, characterized in that: The electronic device has a translation function, the viewfinder interface is an interface displayed when the translation function is turned on, and the first graphic is a circle; or, The electronic device has an object recognition function, the viewfinder interface is an interface displayed when the object recognition function is turned on, and the first graphic is a rectangle.

20. An electronic device, characterized in that: include: a processor, a memory, and one or more programs; The one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the processor, enable the electronic device to perform the method steps as described in any one of claims 1 to 19.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 19.

22. A computer program product, characterized in that The method comprises a computer program, which, when being run on a computer, causes the computer to execute the method according to any one of claims 1 to 19.

Images