Method for determining target object and electronic equipment

Abstract

The embodiment of the invention provides a method for determining a target object and electronic equipment, the target object determined by adopting the method meets user expectation, and the user experience can be improved. For example, the electronic equipment displays a view-finding interface, the view-finding interface comprises at least one object, and a first graph is displayed on the upper layer of the view-finding interface. The electronic device responds to a first operation used for adjusting the shooting range of the camera, the position of the first graph is kept unchanged, a first object in the view finding interface enters the coverage range of the first graph, and if the shooting range is kept unchanged and the duration for keeping the shooting range unchanged is longer than or equal to the first duration, the first object is determined as a target object.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method and electronic device for determining a target object. Background Art

[0002] With the continuous development of terminal technology, the functions of mobile terminals are becoming more and more abundant. For example, a mobile terminal is provided with a camera, which can be used to collect images of the surrounding environment and determine the target object from multiple objects contained in the image. Generally speaking, after the mobile terminal starts the camera, the viewfinder interface is displayed. If the viewfinder interface includes multiple objects, the system will automatically determine a target object from the multiple objects. The target object determined in this way often does not match the object expected by the user, resulting in a poor user experience. Summary of the invention

[0003] A method for determining a target object and an electronic device provided in an embodiment of the present application are helpful to improve the accuracy of determining the target object.

[0004] In a first aspect, a method for determining a target object is provided. The method may be applicable to an electronic device. The electronic device may be, for example, a mobile phone, a tablet computer, and the like. The method comprises: displaying a framing interface, the framing interface being a framing interface of a camera in the electronic device, the framing interface comprising at least one object, a first graphic being displayed on an upper layer of the framing interface, and an area of ​​the first graphic being smaller than an area of ​​the framing interface; in response to a first operation for adjusting a shooting range of the camera, a position of the first graphic remains unchanged, a first object in the framing interface enters a coverage range of the first graphic, the first object is any one of the at least one object, and a coverage range of the first graphic is a coverage range of an area enclosed by an edge contour of the first graphic on the framing interface; if the shooting range remains unchanged and the duration of the unchanged period is greater than or equal to the first duration, determining that the first object is a target object.

[0005] In an embodiment of the present application, after the user opens the viewfinder interface of the electronic device, a first graphic is displayed on the upper layer of the viewfinder interface. The user can determine the target object with the help of the first graphic. For example, the user adjusts the shooting range so that the object desired by the user enters the coverage of the first graphic, and then keeps the shooting range unchanged. When the electronic device detects that the shooting range remains unchanged and the duration of the unchanged state is greater than or equal to the first duration, it determines that the object within the coverage of the first graphic is the target object. The target object determined in this way is the object in the user's mind, and the experience is better.

[0006] In one possible design, the first graphic satisfies at least one of the following: the shape of the first graphic is a circle; or, the area of ​​the first graphic is the system default; or, the first graphic is displayed at the center of the display screen of the electronic device; or, the first graphic is filled with a first color; or, the area within the first graphic is transparent; or, the edge of the first graphic is feathered, and the degree of feathering is the system default or is positively correlated with the current object distance of the camera.

[0007] In an embodiment of the present application, the first graphic may have a variety of design styles, for example, the first graphic may be filled with color, have a certain degree of transparency, have edge feathering, etc., so as to enhance the user experience during the operation as much as possible.

[0008] In one possible design, the area enclosed by the edge outline of the first figure is the first area, and the other areas outside the first area are the second area. The first area and the second area satisfy at least one of the following: the color filled in the first area is different from the color filled in the first area; or, the transparency of the first area is higher than the transparency of the second area; or, the brightness of the first area is higher than the brightness of the second area.

[0009] In an embodiment of the present application, the area within the first graphic (i.e., the first area) and the area outside the first graphic (i.e., the second area) are displayed separately, which can more intuitively prompt the user to move the object the user desires into the first area, providing a better experience.

[0010] In a possible design, after determining that the first object is the target object, the method further includes: canceling the display of the first graphic. In the embodiment of the present application, the first graphic on the upper layer of the viewfinder interface is used to assist the user in determining the target object. After determining the target object, the first graphic can be canceled to avoid blocking information.

[0011] In a possible design, before the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to a first time period, it also includes: when it is determined that the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to a second time period, determining that the first object is a candidate target object, and adjusting the first graphic to a second graphic, wherein the second graphic is a circumscribed circle or a circumscribed polygon of the first object.

[0012] In the embodiment of the present application, the coverage of the first graphic includes the first object. If the electronic device determines that the shooting range remains unchanged and remains unchanged for 2 seconds, the first object is determined to be a candidate target object, and the user is prompted that the first object is a candidate target object. If the electronic device determines that the shooting range continues to remain unchanged and remains unchanged for 4 seconds, the candidate target object is determined to be the target object. The target object determined in this way is more accurate.

[0013] In a possible design, after determining that the first object is the target object, the second graphic is also displayed. In an embodiment of the present application, when the first object is determined to be a candidate target object, the second graphic is displayed to prompt that the first object is a candidate target object. If the candidate target object is determined to be the target object, the second graphic is displayed to avoid blocking information.

[0014] In a possible design, after determining that the first object is the target object, it also includes: the edge outline of the first object is highlighted; or, the area surrounded by the edge outline of the first object is filled with a third color; or, the brightness of the area surrounded by the edge outline of the first object is higher than the brightness of other areas.

[0015] In the embodiment of the present application, after the electronic device determines the target object, it can highlight the target object to prompt the user which one is the target object.

[0016] In a possible design, after determining that the first object is a candidate target object, the method further includes: in response to a second operation for adjusting a shooting range of the camera, the positions of the first object and the second graphic in the viewfinder interface are synchronously moved.

[0017] In an embodiment of the present application, after the electronic device determines that the first object is a candidate target object, the second graphic will be attached to the first object. If the shooting range changes (for example, the shooting range changes due to user hand shaking), the second graphic will move synchronously with the first object to ensure that the first object is still a candidate target object. This can avoid the need to re-determine the target object due to user shaking.

[0018] In a possible design, after the positions of the first object and the second graphic move synchronously, it also includes: determining that a second object appears at a first position in the framing interface and that the second object remains at the first position for a third time period, wherein the first position is the display position of the first graphic before being adjusted to the second graphic; restoring the second graphic to the first graphic, and the first graphic is displayed at the first position, and the first graphic includes the second object; and determining that the second object is the target object when the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to a fourth time period.

[0019] In an embodiment of the present application, after the first object is determined as a candidate target object, the user can change the target object. Assuming that the user wants to change to a second object, the user can adjust the shooting angle so that the second object moves to the first position and keeps the shooting range unchanged. In this way, the electronic device will cancel the candidacy of the first object, restore the second graphic to the first graphic, and use the first graphic to redefine the target object. For example, when the second object is included in the first graphic, if it is determined that the shooting range remains unchanged and the duration of the unchanged range is greater than or equal to the fourth duration, the second object is determined to be the target object.

[0020] In one possible design, after determining that a second object appears at a first position in the framing interface, the method further includes: displaying a prompt box at the first position, the prompt box including the second object, and the prompt box being used to prompt that the second object will be determined as a target object; after determining that the time period during which the second object remains at the first position reaches a third time period, and before restoring the second graphic to the first graphic, the method further includes: canceling the display of the prompt box.

[0021] In the embodiment of the present application, after the first object is determined as a candidate target object, the user can change the target object. Assuming that the user wants to change to the second object, the user can adjust the shooting angle to move the second object to the first position. At this time, the electronic device will display a prompt box, which includes the second object to prompt the user that the second object will be determined as the target object, so the user can determine whether the change of the target object is correct through the prompt box.

[0022] In a possible design, the first graphic also includes a third object, and the third object is any object other than the first object among the at least one object; after the shooting range remains unchanged and the time length of the unchanged range is greater than or equal to the first time length, it also includes: determining that the third object is the target object.

[0023] In the embodiment of the present application, when the first graphic includes multiple objects, the electronic device can determine all of the multiple objects as target objects. Therefore, if the user expects only one object, the shooting range can be adjusted to include only the object as much as possible in the first graphic; if the user expects multiple objects, the shooting range can be adjusted to include multiple objects in the first graphic.

[0024] In one possible design, the method also includes: displaying the first object and the third object; when receiving a third operation, retaining the first object as the target object and canceling the third object as the target object; wherein the third operation includes an operation on the first object, or an operation on the third object.

[0025] In an embodiment of the present application, after the electronic device determines multiple target objects, the user can cancel a target object. For example, if there are two target objects, the user clicks on one of the objects, and the object is canceled as the target object, and the other object is retained as the target object. Alternatively, the user clicks on one of the objects, and the object is retained as the target object, and the other object is canceled as the target object.

[0026] In a possible design, before the shooting range remains unchanged and the time length of the unchanged state is greater than or equal to the first time length, it also includes: when it is determined that the shooting range remains unchanged and the time length of the unchanged state is greater than or equal to a fourth time length, determining that the first object and the third object are candidate target objects, and adjusting the first graphic to a fourth graphic and a fifth graphic, the fourth graphic being the circumscribed circle or circumscribed polygon of the first object, and the fifth graphic being the circumscribed circle or circumscribed polygon of the third object.

[0027] In an embodiment of the present application, when the coverage of the first graphic includes multiple objects, if the electronic device determines that the shooting range remains unchanged and remains unchanged for 2 seconds, it determines that the multiple objects are candidate target objects, and prompts the user which objects are candidate target objects. If the electronic device determines that the shooting range continues to remain unchanged and remains unchanged for 4 seconds, it determines that the multiple candidate target objects are all target objects. The target object determined in this way is more accurate.

[0028] In a possible design, before the shooting range remains unchanged and the time length of the unchanged state is greater than or equal to the first time length, it also includes: if a fourth operation is received, retain the third object as a candidate target object and retain the fifth graphic, cancel the first object as a candidate target object and cancel the fourth graphic; wherein the fourth operation includes an operation on the fifth graphic, or an operation on the fourth graphic; after the shooting range remains unchanged and the time length of the unchanged state is greater than or equal to the first time length, it also includes: determining that the third object is the target object.

[0029] In an embodiment of the present application, after the electronic device determines multiple candidate target objects, the user can cancel the candidacy of a certain object. Taking two candidate target objects as an example, when the user clicks on one of the candidate target objects, the candidate resource of the object is canceled and the candidacy of the other object is retained. Alternatively, when the user clicks on one of the candidate target objects, the candidacy of the object is retained and the candidacy of the other object is canceled.

[0030] In a possible design, the first graphic also includes a fourth object, and the fourth object is any object other than the first object among the at least one object. If the shooting range remains unchanged and the time length for which it remains unchanged is greater than or equal to the first time length, before determining that the first object is the target object, it also includes: receiving an operation of selecting the first object in the first graphic.

[0031] In the embodiment of the present application, when the first graphic includes multiple objects, the user can specify a certain object as the target object.

[0032] In a possible design, before receiving the operation of selecting the first object in the first graphic, it also includes: outputting prompt information, where the prompt information is used to prompt the user to specify a target object.

[0033] In an embodiment of the present application, when the first graphic includes multiple objects, the electronic device can prompt the user to specify a target object, and the user can select which object to use as the target object according to his or her needs.

[0034] In a possible design, after determining that the first object is the target object, it also includes: identifying the target object, identifying at least one of the name, category, purchase link, related objects of the target object, and purchase links of the related objects of the target object; or, translating the target object; or, copying the target object; or, cutting out the target object.

[0035] The method for determining a target object provided in the embodiment of the present application can be applied to any scenario where a target object needs to be determined in an image, such as a translation scenario, an object recognition scenario, etc. The embodiment of the present application does not give examples one by one.

[0036] In a possible design, after determining that the first object is the target object, it also includes: displaying a first button and a second button; wherein, when the first button is triggered, returning to the previous N steps, and when the second button is triggered, translating, identifying, copying or cutting out the target object, N is a positive integer.

[0037] In an embodiment of the present application, after the electronic device determines the target object, it can display a return button (i.e., the first button) and a confirmation button (i.e., the second button). When an operation is received on the confirmation button, the target object can be translated, identified, copied or cut out. When an operation is received on the return button, it can return to the previous N steps to redefine the target object.

[0038] In one possible design, the electronic device has a translation function, the viewfinder interface is the interface displayed when the translation function is turned on, and the first graphic is a circle; or, the electronic device has an object recognition function, the viewfinder interface is the interface displayed when the object recognition function is turned on, and the first graphic is a rectangle.

[0039] In an embodiment of the present application, the shape of the first graphic is related to the current scene. For example, if the current scene is an object recognition scene, the first graphic is a circle; if the current scene is a translation scene, the first shape is a square, which matches the shape of the text paragraph and can avoid missing the translation of certain texts.

[0040] In a second aspect, an electronic device is also provided, comprising:

[0041] a processor, a memory, and one or more programs;

[0042] The one or more programs are stored in the memory, and the one or more programs include instructions. When the instructions are executed by the processor, the electronic device executes the method steps provided in the first aspect above.

[0043] In a third aspect, there is also provided: a computer-readable storage medium for storing a computer program, which enables the computer to execute the method described in the first aspect when the computer program is run on the computer.

[0044] According to a fourth aspect, a computer program product is provided, comprising a computer program. When the computer program is run on a computer, the computer is enabled to execute the method provided in the first aspect.

[0045] In a fifth aspect, a chip is also provided, which is coupled to a memory in an electronic device and is used to call a computer program stored in the memory and execute the technical solution of the first aspect of the embodiment of the present application. In the embodiment of the present application, "coupling" refers to the direct or indirect combination of two components with each other.

[0046] For the technical effects that can be achieved by the above-mentioned second to fifth aspects, please refer to the description of the technical effects that can be achieved by the corresponding design scheme in the above-mentioned first aspect, and this application will not repeat them here. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application;

[0048] Figure 2 A schematic diagram of a software structure of an electronic device provided in an embodiment of the present application;

[0049] Figure 3AA schematic diagram of opening the object recognition function provided in one embodiment of the present application;

[0050] Figure 3B A schematic diagram of a first graphic filling color provided in an embodiment of the present application;

[0051] FIG. 3C to FIG. 3D A schematic diagram of edge feathering of a first graphic provided in an embodiment of the present application;

[0052] FIG. 3E to FIG. 3F A schematic diagram of calling out a first graphic provided by an embodiment of the present application;

[0053] Figure 3G A schematic diagram of a synthesis interface provided by an embodiment of the present application;

[0054] Figure 4 A first schematic diagram of determining a target object provided by an embodiment of the present application;

[0055] Figure 5 A second schematic diagram of determining a target object provided in an embodiment of the present application;

[0056] Fig. 6A A first schematic diagram of replacing a target object provided by an embodiment of the present application;

[0057] Figure 6B A second schematic diagram of replacing a target object provided in an embodiment of the present application;

[0058] Figure 6C A third schematic diagram of replacing a target object provided in an embodiment of the present application;

[0059] FIG. 7A to FIG. 7B A first schematic diagram of determining multiple target objects provided by an embodiment of the present application;

[0060] FIG. 8A to FIG. 8B A second schematic diagram of determining multiple target objects provided by an embodiment of the present application;

[0061] Fig. 9 A schematic diagram of selecting a candidate target object provided by an embodiment of the present application;

[0062] Fig.10 A schematic diagram of selecting a target object provided by an embodiment of the present application;

[0063] Fig.11 A schematic diagram of opening a translation function provided in an embodiment of the present application;

[0064] Fig.12 A schematic diagram of a translation process provided in an embodiment of the present application;

[0065] Fig.13 Another schematic diagram of a translation process provided in an embodiment of the present application;

[0066] Fig.14 Another structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0067] Below, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0068] The at least one involved in the embodiments of the present application includes one or more; wherein, a plurality refers to greater than or equal to two. In addition, it should be understood that in the description of this specification, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as an explicit or implicit relative importance, nor can they be understood as an explicit or implicit order. For example, the first figure and the second figure do not represent the importance of the two or the order of the two, but are only for distinguishing the description. In the embodiments of the present application, "and / or" is only a description of the association relationship, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated with each other are in an "or" relationship.

[0069] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the specification. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0070] As used in this specification, the term “when” or “after” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrase “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if determining that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0071] The method for determining the target object provided in the embodiment of the present application is applicable to electronic devices. Optionally, the electronic device has a camera. For example, the electronic device can be a portable electronic device such as a mobile phone, a tablet computer, a laptop computer, etc.; it can also be an image capture device such as a smart camera, it can also be an office equipment such as a desktop computer, a projection line, etc., it can also be a wearable device such as a watch, a bracelet, etc.; it can also be a virtual reality (Virtual Reality, VR) device, an augmented reality (AugmentedReality, AR) device, a mixed reality technology (Mixed Reality, MR) device and other devices. In short, the embodiment of the present application does not limit the specific type of electronic device. For ease of understanding, this article mainly takes the electronic device as a mobile phone as an example for explanation.

[0072] Figure 1 FIG. 1 shows a schematic diagram of the structure of an electronic device. The electronic device may be, for example, a mobile phone. Figure 1 As shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0073] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the electronic device. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a high-speed cache memory. The memory may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.

[0074] Among them, the processor 110 can execute the process of the method for determining the target object provided by the embodiment of the present application. For example, the processor 100 can start the camera 193. The camera 193 is used to collect preview images in real time. The processor 110 displays the viewfinder interface through the display screen 194, and updates the preview image in real time in the viewfinder interface. The preview image includes at least one object, and the upper layer of the viewfinder interface displays a first graphic (which will be described later). It can be understood that in the process of the user adjusting the shooting range, the preview image is updated in real time, so the position of the object in the viewfinder interface changes in real time.

[0075] Taking the camera 193 and the electronic device as an example, when the electronic device is stable and motionless, the camera shooting range remains unchanged, and when the electronic device posture changes, the camera shooting range changes. The gyroscope sensor 180B can detect whether the body posture changes in real time. When the gyroscope sensor 180B detects the change of the body posture, it sends an instruction to the processor 110, and the instruction is used to indicate the change of the body posture. The processor 110 can determine the change of the shooting range according to the instruction. In response to the adjustment of the shooting range, the processor 110 keeps the position of the first graphic unchanged, and updates the preview image in real time in the viewfinder interface. Assuming that the position of the first object on the preview image moves into the coverage range of the first graphic, in this case, if the processor 110 detects that the shooting range remains unchanged and the duration of the unchanged period is greater than or equal to the first duration, the first object is determined to be the target object. Among them, the specific implementation process of determining the target object will be described later.

[0076] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0077] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0078] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0079] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0080] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0081] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0082] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0083] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0084] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0085] The wireless communication function of the electronic device can be implemented by antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0086] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0087] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to electronic devices. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0088] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0089] The display screen 194 is used to display the display interface of the application, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device may include one or more display screens 194.

[0090] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor, etc. Among them, the ISP is used to process the data fed back by the camera 193.

[0091] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, and the software code of at least one application program, etc. The data storage area can store data (such as images, videos, etc.) generated during the use of the electronic device, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory, etc.

[0092] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as pictures and videos are stored in the external memory card.

[0093] The electronic device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0094] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0095] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to external speaker scenarios such as hands-free calls through one or more speakers 170A.

[0096] The receiver 170B, also called a "earpiece", may be one or more and is used to convert an audio electrical signal into a sound signal. When the electronic device 100 receives a call or a voice message, the voice can be received by placing the receiver 170B close to the ear.

[0097] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0098] The earphone jack 170D is used to connect a wired earphone.

[0099] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0100] The gyro sensor 180B can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting.

[0101] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0102] The magnetic sensor 180D includes a Hall sensor. For example, the electronic device can use the magnetic sensor 180D to detect the opening and closing of the flip leather case.

[0103] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes), and can detect the magnitude and direction of gravity when the electronic device is stationary.

[0104] Distance sensor 180F, used to measure distance. Electronic devices can measure distance through infrared or laser.

[0105] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device emits infrared light outward through the light emitting diode. The electronic device uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device. When insufficient reflected light is detected, the electronic device can determine that there is no object near the electronic device.

[0106] The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device can adaptively adjust the brightness of the display screen 194 according to the sensed brightness of the ambient light.

[0107] The fingerprint sensor 180H is used to collect fingerprints.

[0108] The temperature sensor 180J is used to detect the temperature.

[0109] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.

[0110] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of a human vocal part.

[0111] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The electronic device can receive the button input and generate a key signal input related to the user settings and function control of the electronic device. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device.

[0112] Understandably, Figure 1 The components shown do not constitute a specific limitation on the electronic device. The electronic device in the embodiment of the present invention may include Figure 1 In addition, Figure 1 The combination / connection relationship between the components can also be adjusted and modified.

[0113] Figure 2 The software structure diagram of the electronic device provided in the embodiment of the present application. The electronic device may be a mobile phone. Figure 2 As shown, the software structure of the electronic device can be a layered structure, and the layered architecture is to divide the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. The layered architecture can be, for example, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application is based on the layered architecture. The system is taken as an example to illustrate the software structure of the electronic device. The system is divided into five layers, from top to bottom: application layer, application framework layer, Android runtime and system library, kernel layer and hardware layer.

[0114] The application layer can include a series of application packages. Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0115] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 2As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like. The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, and the like. The content provider is used to store and obtain data and make the data accessible to the application. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, and the like. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and the like. The view system can be used to build applications. The display interface may be composed of one or more views. For example, a display interface including a text notification icon may include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for the electronic device 100. For example, the management of call status (including connecting, hanging up, and the like). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of an icon or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0116] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one is the function functions that need to be called by the Java language, and the other is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection. The system library can include multiple functional modules. For example: surface manager, media library, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc. The surface manager is used to manage the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats, as well as static image files, etc. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0117] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0118] The hardware layer includes various hardware, such as cameras, displays, etc.

[0119] Understandably, Figure 2 The software structure shown does not constitute a specific limitation on the software structure of the electronic device. The software structure of the electronic device in the embodiment of the present invention may include: Figure 2 More or fewer modules in Figure 2 More or fewer layers, or a layer contains more Figure 2 More or fewer modules, etc. In addition, Figure 2 The combination / connection relationship between modules can also be adjusted and modified.

[0120] The following takes the camera application as an example to explain Figure 2 The processing flow of each component.

[0121] The touch sensor in the hardware layer receives a touch operation, such as a touch operation used to turn on the object recognition function in the electronic device (for example, Huawei Smart Vision). The hardware layer reports the input event to the upper layer (for example, the application framework layer), and the input event includes information about the touch operation, such as the operation type, operation time, coordinate position, etc. The application framework layer can determine the control targeted by the input operation. For example, if it is determined that it is an operation for Huawei Smart Vision, it will report it to the Smart Vision in the application layer. The application layer starts Smart Vision by calling the interface in the application framework layer, and starts the camera to collect images by calling the camera driver in the kernel layer. It also displays the viewfinder interface on the display screen by calling the display driver in the kernel layer, and the viewfinder interface includes the image collected by the camera. The upper layer of the viewfinder interface also displays a first graphic. The area of ​​the first graphic is smaller than the area of ​​the viewfinder interface.

[0122] The motion sensor in the hardware layer detects the change in the posture of the electronic device and reports instructions to the upper layer (such as the application framework layer), and the instructions are used to indicate the change in the posture of the electronic device. The application framework layer controls the position of the first graphic to remain unchanged, and updates the image in real time in the viewfinder interface. The application framework layer detects in real time whether there is an object within the coverage of the first graphic. When the first object enters the coverage of the first graphic, if it is detected that the posture of the electronic device remains unchanged and the duration of the unchanged posture reaches a first duration, the first object is determined to be the target object.

[0123] Taking a mobile phone as an example, the method for determining a target object provided in an embodiment of the present application is described below in conjunction with the accompanying drawings.

[0124] The mobile phone can have an object recognition function, that is, using the camera to collect images of the surrounding environment, determine the target object from multiple objects contained in the image, and then identify the target object, for example, identify the category, name, price, purchase link, price and link of the same product, etc. recognize things, recognize things, Recognize objects, etc. For ease of understanding, the following article takes the smart vision in a mobile phone as an example.

[0125] See also Figure 3A (a) The mobile phone displays an interface 301. The interface 301 may be, for example, the negative first screen of the mobile phone. The interface 301 includes an icon 302 of smart vision. When the mobile phone receives an operation (for example, a click operation) on the icon 302, the camera is activated and the following image is displayed: Figure 3A(b) Interface 303. Interface 303 is a camera viewfinder interface (also referred to as a shooting interface). A preview image can be displayed in real time in interface 303. The preview image includes one or more objects. In the embodiment of the present application, a first graphic 304 is displayed on the upper layer of interface 303. The mobile phone can use the first graphic 304 to determine the target object, and the specific implementation process will be described later.

[0126] It is understandable that before displaying the first graphic 304, the mobile phone can determine the shape, area, display position, etc. of the first graphic 304. Among them, the shape of the first graphic 304 can be arbitrary, such as a circle, a square, a rectangle, etc. One possible way is that the shape of the first graphic 304 is the default of the mobile phone system (for example, the default is a circle). Another possible way is that the shape of the first graphic 304 is related to the current scene (object recognition scene or translation scene). For example, the mobile phone currently has the object recognition function turned on, that is, the current scene is the object recognition scene, then the first graphic 304 is a circle; or, the mobile phone currently has the translation function turned on, that is, the current scene is the translation scene, then the first graphic 304 can be a square or a rectangle (to be described later). Fig.11 ). This is because, generally speaking, text is presented line by line, and the entire text is presented in a square or rectangular shape, so if the first graphic 304 is a square or rectangular shape, it matches the shape of the entire text and is convenient for translation. Optionally, the display position of the first graphic 304 can be arbitrary, such as at the center of the display screen. One possible way is that the display position of the first graphic 304 is the default of the mobile phone system (e.g., the center position). In addition, the area of ​​the first graphic 304 can be a fixed value that the system defaults to, and the specific area size is not limited in the embodiments of the present application.

[0127] Figure 3A In the example, the first graphic 304 is hollow and has no fill color. Optionally, the first graphic 304 can also be filled with color. Here, "the first graphic 304" can be understood as the area surrounded by the edge outline of the first graphic 304. For example, see Figure 3B (a), the first graphic 304 is filled with a color, wherein the filled color can be any color, which is not limited in the present embodiment. It is understandable that in order to prevent the filled color in the first graphic 304 from blocking the preview image, the first graphic 304 can have a certain degree of transparency, such as Figure 3B (a). Optionally, the outside of the first graphic 304 may also be filled with a color. Here, "outside the first graphic 304" may be understood as other areas outside the area surrounded by the edge contour of the first graphic 304 on the display screen. For the sake of convenience, the filling color inside the first graphic 304 may be different from the filling color outside the first graphic 304. For example, Figure 3B(b), the first graphic 304 is filled with a bright color, and the first graphic 304 is filled with a dark color. Similarly, in order to avoid blocking the preview image, the first graphic 304 may also have a certain degree of transparency. The transparency of the first graphic 304 and the transparency of the first graphic 304 may be the same or different. For example, considering that the object in the first graphic 304 may be determined as a target object (described later), the transparency of the first graphic 304 may be higher than the transparency of the first graphic 304, so that the user can see more clearly whether the desired object is moved to the first graphic 304 when adjusting the shooting range. In addition, the brightness of the first graphic 304 and the brightness of the first graphic 304 may be the same or different, for example, the brightness of the first graphic 304 is higher than the brightness of the first graphic 304.

[0128] Figure 3B One possible approach for (b) is, Figure 3C (a), after the mobile phone draws the interface 303, a layer of interface is covered on the interface 303, and the area of ​​the interface is equal to the area of ​​the interface 303, such as Figure 3C (b) The interface at the upper layer includes a first graphic 304 , and the inside of the first graphic 304 and the outside of the first graphic 304 are filled with different colors, and the transparency inside the first graphic 304 is higher than the transparency outside the first graphic 304 .

[0129] Optionally, the edge of the first graphic 304 may be feathered. Edge feathering may be understood as edge blurring. For example, see Figure 3D , which is the effect of feathering the edge of the first graphic 304. Figure 3B (b) Compared with Figure 3D The edge of the first graphic 304 in the image is relatively blurred. One possible implementation method is that before the mobile phone displays the first graphic 304, it can first determine the feathering value of the first graphic 304, feather the edge of the first graphic 304 according to the determined feathering value, and then display the first graphic 304 after the edge feathering. There are many ways for the mobile phone to determine the feathering value. For example, the feathering value is configured by default in the mobile phone system, or the mobile phone determines the feathering value according to the current object distance of the camera. For example, the mobile phone stores the correspondence between the object distance of the camera and the feathering value, and can determine the feathering value corresponding to the current object distance based on the correspondence. For example, the larger the object distance and the larger the feathering value, the greater the degree of edge feathering and the blurrier the edge; the smaller the object distance and the smaller the feathering value, the smaller the degree of edge feathering and the clearer the edge. Figure 3E, the degree of edge feathering may be positively correlated with the current object distance of the camera. Optionally, the degree of edge feathering of the first graphic 304 may be fixed or dynamically changing. Taking the fixed degree of edge feathering of the first graphic 304 as an example, one possible way is that when the mobile phone opens the interface 303, the first graphic 304 automatically appears and the edge is feathered, and the degree of feathering is default and does not change. Taking the dynamic change of the degree of edge feathering of the first graphic 304 as an example, one possible way is that when the mobile phone opens the interface 303, the first graphic 304 automatically appears and the edge is feathered, and the degree of feathering is default at this time. Afterwards, when the mobile phone detects that the object distance of the camera has changed, the degree of edge feathering of the first graphic 304 is adjusted according to the current object distance of the camera. For example, see Figure 3E (a), when the mobile phone opens the interface 303, the first graphic 304 automatically appears and the edge is feathered and the feathering degree is the default. When the object distance of the camera is shortened, the edge feathering degree of the first graphic 304 is weakened, that is, the edge becomes clearer, such as Figure 3E (b) When the object distance of the camera increases, the edge feathering degree of the first graphic 304 increases, that is, the edge becomes more blurred, such as Figure 3E (c). It should be noted that Figure 3E In the three figures, the areas of the first graphic 304 appear different, but are actually the same because edge feathering causes the first graphic 304 to be visually enlarged.

[0130] In the above embodiment, when the mobile phone opens the interface 303, the first graphic 304 automatically appears. In other embodiments, when the mobile phone opens the interface 303, the first graphic 304 is temporarily absent, and when an operation for calling out the first graphic 304 is received, the first graphic 304 is displayed. There are multiple operations for calling out the first graphic 304, such as long pressing, double clicking, clicking a specific button in the interface 303, etc. For example, see Figure 3F (a), the mobile phone displays interface 301. Interface 301 includes icon 302. When the mobile phone receives an operation for icon 302, it displays the following Figure 3F (b) interface 303, at this time, the first graphic 304 is not displayed on the upper layer of the interface 303. The interface 303 includes an icon 309. When the mobile phone receives an operation for the icon 309, the first graphic 304 is displayed, such as Figure 3F (c). For another example, see Figure 3G As shown in (a), the mobile phone displays the main interface, which includes a camera icon 310. When the mobile phone receives an operation for the icon 310, the following is displayed: Figure 3G (b) The viewfinder interface 311 includes a more button 312. When the mobile phone receives an operation of clicking the more button 312, the following is displayed: Figure 3G(c) shows an interface including a smart object recognition button 313. When the mobile phone receives an operation on the smart object recognition button 313, the following is displayed: Figure 3G (d) interface 311, the first graphic 304 is displayed on the upper layer of the interface 311. Optionally, after the mobile phone displays the first graphic 304, if an operation for hiding the first graphic 304 is received, the display of the first graphic 304 can be canceled. The operation for calling out the first graphic 304 and the operation for hiding the first graphic 304 can be the same or different. Taking the double-click operation in the interface 303 as an example, when the first graphic 304 is not currently displayed, if the mobile phone receives a double-click operation in the interface 303, the first graphic 304 is displayed. When the first graphic 304 is currently displayed, if the mobile phone receives a double-click operation in the interface 303, the first graphic 304 is hidden. Figure 3F (c) as an example, when the mobile phone receives an operation on icon 309, it returns to Figure 3F (b) interface, hide the first graphic 304. Figure 3G (d) as an example, when the mobile phone receives an operation for more buttons 312, open Figure 3G (c) interface, if an operation is received on icon 313, it will be displayed Figure 3G (b) shows an interface in which the first graphic 304 is not displayed.

[0131] The following describes a process in which the mobile phone uses the first graphic 304 to determine the target object.

[0132] Method A

[0133] See also Figure 4 (a), the mobile phone displays interface 303. Interface 303 includes three objects, namely object 401 (picture frame), object 402 (green plant) and object 403 (cat). The upper layer of interface 303 displays a first graphic 304. Assume that the user's desired target object is object 402 (green plant). The user can adjust the shooting range to make object 402 enter the first graphic 304. Among them, object 402 entering the first graphic 304 can be understood as object 402 entering the coverage of the first graphic 304, that is, entering the area surrounded by the edge outline of the first graphic 304 within the coverage of the interface 303. As shown Figure 4(b), the first graphic 304 includes an object 402. When the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 1 (for example, 2s, 3s or 4s), it determines that the object in the first graphic 304 (i.e., object 402) is the target object. Among them, the shooting range remains unchanged, which may include: the field of view (FOV) remains unchanged (i.e., the shooting is not enlarged or reduced), and the shooting angle remains unchanged (i.e., the shooting angle is not moved in various directions such as up, down, left, and right). Considering that it is inevitable that the user will shake when holding the electronic device, that is, the shooting range will inevitably change, so in the embodiment of the present application, the shooting range remains unchanged, and may also include: the change in the field of view angle is less than the first angle, and / or, the change in the shooting angle when moving in various directions such as up, down, left, and right is less than the second angle. Among them, the specific values ​​of the first angle and the second angle are not limited in the embodiment of the present application. It should be understood that if the camera on the mobile phone is fixedly set, when the posture of the mobile phone changes, the camera shooting range changes, and when the mobile phone remains stable, the camera shooting range remains unchanged. Therefore, in this case, "the shooting range remains unchanged and the length of time it remains unchanged reaches threshold 1" may mean that: the mobile phone remains stable and the stability time reaches threshold 1. Among them, the mobile phone remaining stable may mean that the posture of the mobile phone remains stable. Assuming that the posture of the mobile phone is described using posture angles (including pitch angle, yaw angle and roll angle), then the mobile phone remains stable, which may include: the change in pitch angle is less than angle 1, and / or the change in yaw angle is less than angle 2, and / or the change in roll angle is less than angle 3. Among them, the specific values ​​of Angle 1, Angle 2 and Angle 3 are not limited in the embodiments of this application.

[0134] In addition, it should be noted that if there is only one object in the first graphic 304, the mobile phone determines that the object is the target object. If there are multiple objects in the first graphic 304, the mobile phone determines that the multiple objects are all target objects (described later). Fig. 7A Therefore, if the user expects only one object, the first graphic 304 should include only the object as much as possible when adjusting the shooting range.

[0135] Optional, Figure 4 In (b), after the mobile phone determines that the object 402 in the first graphic 304 is the target object, it can display the following Figure 4 (c) shows an interface in which the first graphic 304 is not displayed. In order to prompt the user that the currently determined target object is object 402, object 402 may be highlighted. The highlighting of object 402 may include: highlighting the edge contour of object 402 (for example, highlighting or bolding), filling a specific color in the area surrounded by the edge contour of object 402, and the brightness of the area surrounded by the edge contour of object 402 is higher than that of other areas, etc., which is not limited in the embodiments of the present application.

[0136] Optional, Figure 4 In (c), the mobile phone can also display a confirmation button and a return button. When the mobile phone receives an operation on the confirmation button, it identifies the target object and displays the identification result (not shown in the figure). For example, the identification result includes: the category, name, price, purchase link, price and link of the same product, etc. of the target object. When the mobile phone receives an operation on the return button, it returns to the previous step, that is, returns to Figure 4 (b) Interface.

[0137] Method B

[0138] See also Figure 5 (a), the mobile phone displays interface 303. Interface 303 includes three objects, namely object 401 (picture frame), object 402 (green plant) and object 403 (cat). The upper layer of interface 303 displays first graphic 304. Assume that the target object desired by the user is object 402 (green plant). The user can adjust the shooting range to make object 402 enter the first graphic 304. The principle of object 402 entering the first graphic 304 can be found in method A above. Figure 5 (b), the first graphic 304 includes the object 402. When the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 2, it determines that the object in the first graphic 304 (i.e., the object 402) is a candidate target object. It should be understood that if there is only one object in the first graphic 304, the mobile phone determines that the object is a candidate target object. If there are multiple objects in the first graphic 304, the mobile phone determines that the multiple objects are all candidate target objects (described later). Fig. 8A (instructions).

[0139] Optional, Figure 5 In (b), after the mobile phone determines that the object 402 in the first graphic 304 is a candidate target object, the following is displayed: Figure 5 (c) shows an interface in which the first graphic 304 is adjusted to the second graphic 305. The area of ​​the second graphic 305 is smaller than that of the first graphic 304. For example, the second graphic 305 is a circumscribed circle or a circumscribed polygon of the object 402, which is used to prompt the user that the object 402 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 3, the candidate target object is determined to be the target object, that is, the object 402 in the second graphic 305 is determined to be the target object.

[0140] Optional, Figure 5 In (c), after the mobile phone determines that the object 402 is the target object, the following is displayed: Figure 5 (d) interface, in which the second graphic 305 is not displayed. In order to prompt the user that the currently determined target object is object 402, object 402 is highlighted. The method of highlighting object 402 can be found in the above method A, which will not be repeated here. Optionally, Figure 5In (d), the mobile phone can also display a return button and a confirmation button. When the mobile phone receives an operation on the confirmation button, it can identify the target object and display the recognition result. When the mobile phone receives an operation on the return button, it can return to the previous step, that is, display Figure 5 (b) , the user can adjust the shooting range again to allow other objects to enter the first graphic 304 .

[0141] The difference between method A and method B is that in method A, after the user adjusts the shooting range so that the object 402 (green plant) is located within the first graphic 304, as long as the shooting range is kept unchanged and the duration of the unchanged state reaches a threshold value 1 (for example, 5s), the object 402 is determined as the target object. In method B, after the user adjusts the shooting range so that the object 402 (green plant) is located within the first graphic 304, the shooting range is kept unchanged and the duration of the unchanged state reaches a threshold value 2, and the object 402 is determined as a candidate target object, and the shooting range is kept unchanged and the duration of the unchanged state reaches a threshold value 3, and the object 402 is determined as a target object. Therefore, method A has fewer processes and higher efficiency. Method B first determines the candidate target object and then determines the target object, and has higher accuracy. The mobile phone can use either method A or method B. For example, the mobile phone system uses method A or method B by default. Of course, method A and method B can be switched, for example, a switching button is set on the mobile phone, and the switching between method A and method B can be achieved through the button.

[0142] Among them, threshold 1, threshold 2 and threshold 3 can satisfy: threshold 1 is equal to threshold 2; or, threshold 1 is equal to the sum of threshold 2 and threshold 3. For example, threshold 2 is 2s and threshold 3 is also 2s, then threshold 1 is 4s. The embodiment of the present application does not limit the specific value of each threshold.

[0143] The following mainly uses the above method B (i.e. Figure 5 ) is used as an example. It can be understood that Figure 5 During the process of changing the target object, the user may want to change the target object. Figure 5 (b) When changing the target object, it may also be displayed on the phone Figure 5 (c) When changing the target object, or, when the mobile phone displays Figure 5 (d) The target object is changed when the interface is displayed. The following is an explanation using different examples.

[0144] Example 1, mobile phone display Figure 5 (b) When the user changes the target object.

[0145] See also Fig. 6A(a), object 402 is located in the first graphic 304. At this time, if the user wants to change the target object (for example, to object 403), the shooting range can be adjusted so that object 403 enters the first graphic 304. Fig. 6A (b), the first graphic 304 includes an object 403. When the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches a threshold 4, the object 403 is determined to be a candidate target object. The mobile phone displays the following Fig. 6A (c) shows an interface in which the first graphic 304 is adjusted to the second graphic 305. The second graphic 305 is the circumscribed circle or circumscribed polygon of the object 403, and is used to indicate that the object 403 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 5, the candidate target object is determined to be the target object, that is, the object 403 in the second graphic 305 is determined to be the target object. After determining that the object 403 is the target object, the mobile phone displays the following Fig. 6A (d) In the interface, the second graphic 305 disappears and the object 403 is highlighted to prompt the user that the currently determined target object is the object 403.

[0146] Among them, threshold 4 can be equal to threshold 2 in the previous text, and threshold 5 can be equal to threshold 3 in the previous text.

[0147] Example 2, mobile phone display Figure 5 (c) When the user changes the target object.

[0148] See also Figure 6B (a), object 402 (green plant) is located in the second graphic 305, indicating that object 402 (green plant) has been determined as a candidate target object. At this time, if the user wants to change the target object (for example, to object 403), the shooting range can be adjusted. It is understandable that when the shooting range is adjusted, the position of object 402 (green plant) that has been determined as a candidate target object will move. In the embodiment of the present application, since object 402 has been determined as a candidate target object, when the position of object 402 moves, the position of the second graphic 305 moves synchronously, such as Figure 6B (b) It can be understood that after the object 402 is determined as a candidate target object, the second graphic 305 is attached to the upper layer of the object 402 and moves synchronously with the object 402 .

[0149] As mentioned above, the second graphic 305 is obtained by transforming the first graphic 304. Before being adjusted to the second graphic 305, the first graphic 304 is displayed at the center. After the second graphic 305 moves synchronously with the object 402, the center becomes vacant.

[0150] The phone can detect whether a new object appears in the center. Suppose a new object appears in the center, for example Figure 6B(b), object 403 appears at the center. Optionally, when a new object appears at the center, graphic 601 may be displayed to indicate that the new object, i.e. object 403, is to be determined as the target object. Of course, graphic 601 may not be displayed. If a new object appears at the center and the shooting range remains unchanged for a period of time that reaches threshold 6, then the following graphic 601 may be displayed: Figure 6B (c) shows an interface in which the second graphic 305 is restored to the first graphic 304 and displayed at the center. It is understandable that since the object 403 is already at the center, after the first graphic 304 is restored, the object 403 may be completely located within the first graphic 304, or may be partially located within the first graphic 304 and partially located outside the first graphic 304. In this way, the user can adjust the shooting range so that the object 403 is as completely located as possible within the first graphic 304. When the object 403 is within the first graphic 304, if the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 7, the object 403 is determined to be a candidate target object, and the display is as follows: Figure 6B (d) shows an interface in which the first graphic 304 is adjusted to the second graphic 305. The second graphic 305 is the circumscribed circle or circumscribed polygon of the object 403, and is used to indicate that the object 403 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 8, the object 403 in the second graphic 305 is determined to be the target object. Therefore, the mobile phone displays the following Figure 6B (e) shows an interface in which the second graphic 305 disappears and the object 403 is highlighted to prompt the user that the currently determined target object is the object 403. The threshold 7 may be equal to the threshold 2 mentioned above, and the threshold 8 may be equal to the threshold 3 mentioned above.

[0151] If the mobile phone detects that no new object appears at the center position, the object 402 that has been determined as a candidate target object is still a candidate target object, and the mobile phone determines that the object 402 is a target object when it detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 9. The threshold value 9 may be equal to the threshold value 3 mentioned above.

[0152] Example 3, mobile phone display Figure 5 (d) When the user changes the target object.

[0153] See also Figure 6C (a), the mobile phone has determined that object 402 (green plant) is the target object, so object 402 is highlighted. The mobile phone also displays a return button 602 and a confirmation button 603. When the mobile phone receives the operation of the return button 602, it displays the following Figure 6C(b) shows an interface in which a first graphic 304 is displayed on the upper layer, and the first graphic 304 includes an object 402. If the user wants to change the target object (for example, to object 403), the shooting range can be adjusted so that the object 403 is located in the first graphic 304, as shown in FIG. Figure 6C (c). If the mobile phone detects that the shooting range remains unchanged and the time it remains unchanged reaches a threshold of 10, it can display the following Figure 6C (d) shows an interface in which the first graphic 304 is adjusted to the second graphic 305. The second graphic 305 is the circumscribed circle or circumscribed polygon of the object 403, which is used to prompt the user that the object 403 is determined as a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 11, the display is as follows: Figure 6C (e) shows an interface in which the second graphic 305 disappears and the object 403 is highlighted to prompt the user that the currently determined target object is the object 403. The threshold 10 may be equal to the threshold 2 mentioned above, and the threshold 11 may be equal to the threshold 3 mentioned above.

[0154] In the above embodiment, the number of target objects is 1 as an example, and it can be understood that the number of target objects can be multiple. The following describes the case where the mobile phone determines multiple target objects.

[0155] Method C

[0156] See also Fig. 7A (a), mobile phone display interface 303. Assume that the user expects two target objects, namely object 402 and object 403. The user adjusts the shooting range so that both object 402 and object 403 enter the first graphic 304, as shown in FIG. Fig. 7A (b), the first graphic 304 includes the object 402 and the object 403. It is understandable that in order to ensure that the object 402 and the object 403 are as far as possible within the first graphic 304, when the user adjusts the shooting range, he may step back to get away from the object, that is, the object distance increases, so Fig. 7A (b) The edge feathering of the first graphic 304 increases. When the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 12, it determines that the object 402 and the object 403 in the first graphic 304 are both target objects. Optionally, after the mobile phone determines that the object 402 and the object 403 are both target objects, it can display the following Fig. 7A (c) interface, in which the first graphic 304 is not displayed, and the object 402 and the object 403 are highlighted respectively, to prompt the user that the object 402 and the object 403 are determined as the target object. Fig. 7A In (c), the mobile phone also displays a return button and a confirmation button. When the mobile phone receives an operation on the return button, it returns to Fig. 7A(b) interface, when the mobile phone receives an operation on the confirmation button, it recognizes the objects 402 and 403 and displays the recognition results (not shown in the figure).

[0157] For ease of understanding, please compare Figure 4 and Fig. 7A , Figure 4 In (b), the first graphic 304 includes only one object 402. When the mobile phone detects that the shooting range remains unchanged and the duration of the unchanged range reaches a threshold 1, it determines that the object 402 is the target object. Fig. 7A In (b), the first figure includes two objects, namely, object 402 and object 403. When the mobile phone detects that the shooting range remains unchanged and the time for which the shooting range remains unchanged reaches a threshold value 12, it is determined that object 402 and object 403 are both target objects. Among them, threshold value 12 can be equal to threshold value 1. Therefore, if the user expects only one object, then when adjusting the shooting range, the first figure 304 should include only the object as much as possible; if the user expects multiple objects, then when adjusting the shooting range, multiple objects should be located in the first figure 304 as much as possible.

[0158] Optional, Fig. 7A In (b), when the first graphic 304 includes two objects, the user can select one of the two objects. Figure 7B (a), the first graphic 304 includes an object 402 and an object 403. After the mobile phone receives the operation of selecting object 402, it can immediately determine that object 402 is the target object or determine that object 402 is the target object when it is determined that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 13. Since object 403 is not selected, object 403 is not the target object. Among them, the operation for selecting object 402 can be an operation on object 402 (for example, single-click, double-click, long press, etc.), or a click operation on object 403 (for example, single-click, double-click or long press). After the mobile phone determines that object 402 is the target object, it can display the following. Figure 7B (b) shows an interface in which object 402 is highlighted and object 403 is not highlighted. The threshold 13 may be equal to the threshold 12.

[0159] Optionally, before receiving the user's operation of selecting an object 402, the mobile phone may output a prompt message to prompt the user to specify a target object (not shown in the figure). Optionally, when the electronic device determines that the first graphic 304 includes multiple objects, the prompt message may be displayed immediately or when it is determined that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 14, the prompt message may be displayed. The prompt message may be text, an icon, etc., or a dotted line box, for example, the dotted line box may have the same shape and position as the first graphic 304. Optionally, after determining the target object, the mobile phone may cancel the display of the prompt message. Among them, the sum of threshold value 13 and threshold value 14 may be greater than or equal to threshold value 12.

[0160] Method D

[0161] See also Fig. 8A (a), the mobile phone displays interface 303. Assume that the user expects two target objects, namely object 402 and object 403. The user adjusts the shooting range so that both object 402 and object 403 enter the first graphic 304. Fig. 8A (b), the first graphic 304 includes object 402 and object 403. When the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 15, it determines that the objects in the first graphic 304 (i.e., object 402 and object 403) are candidate target objects. Optionally, after the mobile phone determines that object 402 and object 403 are candidate target objects, it can display the following Fig. 8A (c) interface, in which the first graphic 304 is adjusted to two small graphics, namely the second graphic 304a and the third graphic 304b. Among them, the second graphic 304a is the circumscribed circle or circumscribed polygon of object 402, which is used to prompt that object 402 is a candidate target object. The third graphic 304b is the circumscribed circle or circumscribed polygon of object 403, which is used to prompt that object 403 is a candidate target object. If the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 16, it determines that object 402 and object 403 are target objects. Optionally, after the mobile phone determines that object 402 and object 403 are target objects, it can display the following Fig. 8A (d) interface, in which the second graphic 304a and the third graphic 304b are not displayed, and the object 402 and the object 403 are highlighted respectively, to prompt the user that the object 402 and the object 403 are the target objects. Fig. 8A In (c), the mobile phone also displays a return button and a confirmation button. When the mobile phone receives an operation on the return button, it returns to Fig. 8A (b) interface, when the mobile phone receives an operation on the confirmation button, it recognizes the objects 402 and 403 and displays the recognition results (not shown in the figure).

[0162] For ease of understanding, please compare Figure 5 and Fig. 8A , Figure 5 In (b), the first graphic 304 includes only one object 402. When the mobile phone detects that the shooting range remains unchanged and the time period remains unchanged reaches a threshold 2, it determines that the object 402 is a candidate target object, such as Figure 5 (c). When the mobile phone detects that the shooting range remains unchanged and the time it remains unchanged reaches a threshold of 3, the candidate target object is determined to be the target object, such as Figure 5 (d). Fig. 8A In (b), the first graphic 304 includes two objects, namely, object 402 and object 403. When the mobile phone detects that the shooting range remains unchanged and the time period remains unchanged reaches a threshold value 15, it determines that object 402 and object 403 are both candidate target objects, such as Fig. 8A (c). When the mobile phone detects that the shooting range remains unchanged and the duration of the unchanged shooting range reaches a threshold value 16, the candidate target object is determined to be the target object, such as Fig. 8A (d). Therefore, threshold 15 may be equal to threshold 2. Threshold 16 may be equal to threshold 3. It is understood that Fig. 8A In (b), if the first graphic 304 includes more objects, when the mobile phone detects that the shooting range remains unchanged and the duration of the unchanged period reaches the threshold 15, it is determined that all objects in the first graphic 304 are candidate target objects, and the first graphic 304 will be adjusted to more small graphics, each small graphic corresponding to an object.

[0163] Optional, Fig. 8A In (b), when the first graphic 304 includes multiple objects, the user can select one of the multiple objects. Figure 8B (a), the first graphic 304 includes an object 402 and an object 403. After the mobile phone receives the operation of selecting object 402, it can immediately determine that object 402 is a candidate target object or determine that object 402 is a candidate target object when it is determined that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 17. Since object 403 is not selected, object 403 is not a candidate target object. Among them, the operation for selecting object 402 can be an operation on object 402 or a click operation on object 403. After the mobile phone determines that object 402 is a candidate target object, it can display the following Figure 8B (b) shows an interface in which the first graphic 304 is adjusted to the second graphic 304a, which is the circumscribed circle or circumscribed polygon of the object 402. If the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 18, the object 402 is determined to be the target object. After the mobile phone determines that the object 402 is the target object, the following is displayed: Figure 8B (c) is an interface in which object 402 is highlighted and object 403 is not highlighted. Threshold 17 may be equal to threshold 15, and threshold 18 may be equal to threshold 16.

[0164] Optionally, before receiving the user's operation of selecting object 402, the mobile phone can output prompt information to prompt the user to specify a target object (not shown in the figure). Please refer to the above text for the display method of the prompt information, which will not be repeated.

[0165] The difference between method C and method D is that: method C has fewer processes and higher efficiency. Method D has higher accuracy. The mobile phone can use either method C or method D. For example, the mobile phone system uses method C or method D by default. Of course, method C and method D can be switched. For example, a switching button is set on the mobile phone, and the switching between method C and method D can be achieved through the button. It should be noted that the above method A and method B, as well as the method C and method D here can be used in combination.

[0166] For example, when the mobile phone determines that the first graphic 304 includes only one object, the method A is used. Figure 4 When the mobile phone determines that the first graphic 304 includes multiple objects, the method C is used. Fig. 7A or Figure 7B way.

[0167] Alternatively, method A is combined with method D. For example, when the mobile phone determines that the first graphic 304 includes only one object, method A is used. Figure 4 When the mobile phone determines that the first graphic 304 includes multiple objects, the method D is used. Fig. 8A or Figure 8B way.

[0168] Alternatively, method B is combined with method C. For example, when the mobile phone determines that the first graphic 304 includes only one object, method B is used. Figure 5 When the mobile phone determines that the first graphic 304 includes multiple objects, the method C is used. Fig. 7A or Figure 7B way.

[0169] Alternatively, method B is combined with method D. For example, when the mobile phone determines that the first graphic 304 includes only one object, method B is used. Figure 5 When the mobile phone determines that the first graphic 304 includes multiple objects, the method D is used. Fig. 8A or Figure 8B way.

[0170] The following is in the form of D (i.e. Fig. 8A ) is used as an example. It can be understood that Fig. 8ADuring the process, the user can change the target object. For example, when the mobile phone displays 8A (b), the user can change the target object. The principle of changing the target object is the same as that of Example 1 in the previous text, and will not be repeated. For another example, when the mobile phone displays Fig. 8A (c) The user can change the target object. The implementation principle of changing the target object is the same as in Example 2 above, so it will not be repeated. Fig. 8A (d) The user can change the target object. The principle of changing the target object is the same as in Example 3 above and will not be repeated. Fig. 8A In (c), after the mobile phone determines two candidate target objects, the user can select one of the two candidate target objects. For the specific implementation process, please refer to Example 4 below. Fig. 8A In (d), after the mobile phone determines two target objects, the user can select one of the two target objects. For the specific implementation process, please refer to Example 5 below.

[0171] Example 4

[0172] For example, see Fig. 9 (a), the mobile phone displays interface 303. The upper layer of interface 303 displays the second graphic 304a and the third graphic 304b. Among them, the second graphic 304a is the circumscribed circle of object 402, which is used to prompt that object 402 is a candidate target object. The third graphic 304b is the circumscribed circle of object 403, which is used to prompt that object 403 is a candidate target object. When the mobile phone receives the first preset operation, the candidate qualification of object 403 is cancelled, and only the candidate qualification of object 402 is retained. Among them, the first preset operation can be an operation for the third graphic 304b or an operation for the second graphic 304a, and the operation can be a single click, double click or long press operation. Take the first preset operation as an example, that is, after the user clicks on the third graphic 304b, the candidate qualification of object 403 in the third graphic 304b is cancelled, and the candidate resource of object 402 in the second graphic 304a is retained. In other words, whichever graphic the user clicks, the candidate qualification of the object in the graphic is cancelled, and the candidate qualification of the object in the other graphic is retained. For example, the first preset operation is an operation on the second graphic 304a, that is, after the user clicks the second graphic 304a, the candidate resource of the object 402 in the second graphic 304a is retained, and the candidate qualification of the object 403 in the third graphic 304b is cancelled. In other words, whichever graphic the user clicks, the candidate qualification of the object in the graphic is retained, and the candidate qualification of the object in the other graphic is cancelled.

[0173] Fig. 9 In (a), after the mobile phone cancels the candidacy of object 403 and retains the candidacy of object 402, the following may be displayed: Fig. 9(b) interface, in which the third graphic 304b is canceled and the second graphic 304a remains unchanged. After the third graphic 304b is canceled, a dotted box may be displayed to indicate that the candidacy of object 403 is canceled. Optionally, the dotted box may not be displayed. If the mobile phone detects that the shooting range remains unchanged and the length of time it remains unchanged reaches a threshold value 19, object 402 is determined to be the target object. Since the candidacy of object 403 is canceled, object 403 will not be determined as the target object. Optionally, after the mobile phone determines that object 402 is the target object, it may display the following Fig. 9 (c) is an interface in which the second graphic 304a is not displayed and the object 402 is highlighted to prompt the user that the object 402 is the target object.

[0174] It should be noted that in Fig. 9 In (b), the mobile phone cancels the candidacy of object 403 and only retains the candidacy of object 402. At this time, if the mobile phone receives the second preset operation, it can return to Fig. 9 (a), the user can reselect the candidate target object. The second preset operation can be clicking on a blank area in the interface 303, where the blank area refers to an area without the second graphic 304a and the third graphic 304b.

[0175] Example 5

[0176] For example, see Fig.10 (a), object 402 and object 403 are highlighted, indicating that object 402 and object 403 have been determined as target objects. When the mobile phone receives the third preset operation, only object 402 is retained as the target object, and object 403 is no longer the target object. The mobile phone displays the following Fig.10 (b) is an interface in which only object 402 is highlighted and object 403 is no longer highlighted. The third preset operation may be an operation on object 402 or an operation on object 403, and the operation may be a single-click, double-click or long-press operation. Take the case where the third preset operation is an operation on object 402 as an example, that is, after the user clicks on object 402, object 402 is retained as the target object and object 403 is cancelled as the target object. In other words, whichever object the user clicks on is retained as the target object and the other object is cancelled as the target object. Take the case where the third preset operation is an operation on object 403 as an example, that is, after the user clicks on object 403, object 403 is cancelled as the target object and object 402 is retained as the target object. In other words, whichever object the user clicks on is cancelled as the target object and the other object is retained as the target object.

[0177] In the above embodiments, the object recognition function in the mobile phone is taken as an example, and the object recognition function is Huawei Smart Vision. It can be understood that the mobile phone can also include other object recognition functions, such as recognize things, recognize things, The method for determining the target object provided by the embodiment of the present application can be used for identifying objects, etc. In addition, the embodiment of the present application can also be applicable to translation scenarios. For example, a mobile phone can have a translation function, that is, using a camera to capture an image, and then translate the text in the image, such as English to Chinese or Chinese to English. The translation function in the mobile phone includes, for example, The translation and camera applications have text translation functions, etc. This application does not give examples one by one. For the sake of ease of understanding, the following is an example of the translation application on the mobile phone.

[0178] See also Fig.11 (a), the mobile phone displays the main interface. The main interface includes the translation application icon 1101. When the mobile phone receives an operation on the icon 1101, the following is displayed: Fig.11 (b) shows a framing interface, which includes multiple paragraphs of text, and the first graphic 304 is displayed on the upper layer of the framing interface. It is understandable that before the mobile phone displays the first graphic 304, the shape, area, display position, etc. of the first graphic 304 can be determined, please refer to the previous description. It should be noted that the first graphic 304 in the previous text is circular, and here, the first graphic 304 is square or rectangular. As mentioned above, when the current scene is object recognition, the first graphic 304 is circular, and when the current scene is translation, the first graphic 304 is square or rectangular. The reason has been described in the previous text and will not be repeated here. The following describes the process of the mobile phone using the first graphic 304 to translate the target object.

[0179] Method E

[0180] like Fig.12 (a), the mobile phone displays a viewfinder interface. The viewfinder interface includes multiple paragraphs of text. The first graphic 304 is displayed on the upper layer of the viewfinder interface. Assume that the user wants to translate paragraph 1. The user can adjust the shooting range so that paragraph 1 falls within the range of the first graphic 304. Fig.12 (b), the first graphic 304 includes paragraph 1. When the mobile phone detects that the shooting range remains unchanged and the time period of the unchanged range reaches the threshold 20, it determines that the object in the first graphic 304 (i.e., paragraph 1) is the target object. Optionally, after the mobile phone determines that paragraph 1 is the target object, it displays the following Fig.12 (c) An interface in which paragraph 1 is highlighted to indicate that paragraph 1 is determined as a target object. The highlighting of paragraph 1 may include: highlighting or bolding each word in paragraph 1. Fig.12In (c), the mobile phone also displays a confirmation button and a return button. If the mobile phone receives an operation on the confirmation button, the mobile phone translates paragraph 1 and displays the translation result (not shown in the figure). If the mobile phone receives an operation on the return button, it returns to Fig.12 (b) The user can adjust the shooting range to translate other paragraphs.

[0181] The threshold 20 may be equal to the threshold 1 mentioned above.

[0182] Method F

[0183] like Fig.13 (a), the mobile phone displays a viewfinder interface. The viewfinder interface includes multiple paragraphs of text. The first graphic 304 is displayed on the upper layer of the viewfinder interface. Assume that the user wants to translate paragraph 1. The user can adjust the shooting range so that paragraph 1 falls within the range of the first graphic 304. Fig.13 (b), the first graphic 304 includes paragraph 1. If the mobile phone receives that the shooting range remains unchanged and the duration of the unchanged period reaches the threshold 21, the mobile phone displays a dotted box to prompt the user to select part of the text for translation, such as Fig.13 (c). If the mobile phone receives an operation that the user clicks a punctuation mark (e.g., a colon), a sentence (e.g., sentence 1301) is determined based on the punctuation mark. The starting point of the sentence is the clicked punctuation mark (i.e., the colon), and the end point is the punctuation mark after the clicked punctuation mark (i.e., the colon). After the mobile phone determines sentence 1301, it adjusts the first graphic 304 to the second graphic 304a, such as Fig.13 (d) The second graphic 304a is used to indicate that the sentence 1301 is determined as a candidate target object. The threshold 21 may be less than or equal to the threshold 20 mentioned above.

[0184] Fig.13 (d), the user can continue to select sentences. For example, when the mobile phone receives the user's click operation on a punctuation mark (e.g., a question mark), a sentence (e.g., sentence 1302) is determined based on the punctuation mark. The starting point of the sentence is the clicked punctuation mark (i.e., the question mark), and the end point is the punctuation mark after the clicked punctuation mark (i.e., the question mark). After the mobile phone determines sentence 1302, it displays the third graphic 304b, as shown in FIG. Fig.13 (e) The third graphic 304b is used to indicate that the sentence 1302 is determined as a candidate target object.

[0185] After the mobile phone determines that sentences 1301 and 1302 are candidate target objects, if it detects that the shooting range remains unchanged and the unchanged time reaches the threshold 22, it determines that sentences 1301 and 1302 are target objects, and the mobile phone displays the following Fig.13(e) is an interface in which the dotted box is not displayed and sentences 1301 and 1302 are highlighted. Fig.13 In (e), the mobile phone also displays a confirmation button and a cancel button. If the mobile phone receives an operation on the confirmation button, the mobile phone translates the selected sentence and displays the translation result (not shown in the figure). If the mobile phone receives an operation on the return button, it returns to the previous step, for example, returns to Fig.13 (d) interface, the user selects other sentences in paragraph 1 for translation; or returns to Fig.13 (c) The user can adjust the shooting range to translate other paragraphs.

[0186] This article mainly uses the object recognition scene and the translation scene as examples for explanation. However, it should be noted that the technical solution provided in the embodiment of the present application can be applied to any scene where the target object needs to be determined in the image. After determining the target object, the electronic device can identify the target object (i.e., the object recognition scene in the previous article), translate the target object (i.e., the translation scene in the previous article), and perform other operations on the target object, such as copying (i.e., copying the target object), cutting out (i.e., cutting out the target object), and so on.

[0187] Fig.14 The electronic device 1400 provided in the embodiment of the present application is a schematic diagram of the structure. The electronic device 1400 may be the mobile phone mentioned above. Fig.14 As shown, the electronic device 1400 may include: one or more processors 1401; one or more memories 1402; a communication interface 1403, and one or more computer programs 1404, and the above-mentioned devices may be connected via one or more communication buses 1405. The one or more computer programs 1404 are stored in the above-mentioned memory 1402 and are configured to be executed by the one or more processors 1401, and the one or more computer programs 1404 include instructions. For example, when the electronic device 1400 is the mobile phone in the above text, the instruction can be used to execute the relevant steps of the mobile phone in the corresponding embodiment above, for example, the mobile phone can display a viewfinder interface, the viewfinder interface is the viewfinder interface of the camera, the viewfinder interface includes at least one object, the upper layer of the viewfinder interface displays a first graphic, and the area of ​​the first graphic is smaller than the area of ​​the viewfinder interface. The mobile phone can also keep the position of the first graphic unchanged in response to the first operation for adjusting the shooting range of the camera, and the first object in the viewfinder interface enters the coverage range of the first graphic. The first object is any object of the at least one object, and the coverage range of the first graphic is the coverage range of the area surrounded by the edge contour of the first graphic on the viewfinder interface. The mobile phone may also determine that the first object is the target object when it is determined that the shooting range remains unchanged and the time period for which the shooting range remains unchanged is greater than or equal to the first time period. FIG. 3A to FIG. 13The communication interface 1403 is used to implement communication between the electronic device 1400 and other devices. For example, the communication interface may be a transceiver.

[0188] In the embodiments provided by the present application, the method provided by the embodiments of the present application is introduced from the perspective of an electronic device (such as a mobile phone, etc.) as an execution subject. In order to implement the functions in the method provided by the embodiments of the present application, the electronic device may include a hardware structure and / or a software module, and implement the above functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether a function of the above functions is executed in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application and design constraints of the technical solution.

[0189] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc. In the absence of conflict, the solutions of the above embodiments can be used in combination.

[0190] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0191] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0192] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0193] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0194] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining a target object, characterized in that: Applied to electronic equipment, the method comprises: Displaying a framing interface, the framing interface is a framing interface of a camera in the electronic device, the framing interface includes at least one object, a first graphic is displayed on an upper layer of the framing interface, and an area of ​​the first graphic is smaller than an area of ​​the framing interface; In response to a first operation for adjusting the shooting range of the camera, the position of the first graphic remains unchanged, a first object in the viewfinder interface enters a coverage range of the first graphic, the first object is any object of the at least one object, and the coverage range of the first graphic is a coverage range of an area enclosed by an edge contour of the first graphic on the viewfinder interface; If the shooting range remains unchanged and the time length for which the range remains unchanged is greater than or equal to the first time length, the first object is determined to be the target object.

2. The method according to claim 1, characterized in that The first graph satisfies at least one of the following: The shape of the first figure is a circle; or, The area of ​​the first figure is a system default; or, The first graphic is displayed at the center of the display screen of the electronic device; or, The first graphic is filled with a first color; or, The area within the first graphic has transparency; or, The edge of the first graphic is feathered, and the feathering degree is a system default or positively correlated with the current object distance of the camera.

3. The method according to claim 1 or 2, characterized in that: The area enclosed by the edge contour of the first graphic is the first area, and the other areas outside the first area are the second area. The first area and the second area satisfy at least one of the following conditions: The color filled in the first area is different from the color filled in the first area; or, The transparency of the first area is higher than the transparency of the second area; or, The brightness of the first area is higher than the brightness of the second area.

4. The method according to any one of claims 1 to 3, characterized in that: After determining that the first object is a target object, the method further includes: The first graphic is canceled from being displayed.

5. The method according to any one of claims 1 to 3, characterized in that: Before the shooting range remains unchanged and the duration of the unchanged shooting range is greater than or equal to the first duration, the method further includes: When it is determined that the shooting range remains unchanged and the time period for which it remains unchanged is greater than or equal to the second time period, the first object is determined to be a candidate target object, and the first graphic is adjusted to a second graphic, where the second graphic is a circumscribed circle or a circumscribed polygon of the first object.

6. The method according to claim 5, characterized in that After determining that the first object is a target object, the method further includes: The second graphic is canceled from being displayed.

7. The method according to any one of claims 1 to 6, characterized in that: After determining that the first object is a target object, the method further includes: The edge contour of the first object is highlighted; or, The area surrounded by the edge outline of the first object is filled with a third color; or The brightness in the area surrounded by the edge contour of the first object is higher than the brightness in other areas.

8. The method according to claim 5, characterized in that After determining that the first object is a candidate target object, the method further includes: In response to a second operation for adjusting the shooting range of the camera, the positions of the first object and the second graphic in the viewfinder interface move synchronously.

9. The method according to claim 8, characterized in that After the positions of the first object and the second graphic are synchronously moved, the method further includes: Determining that a second object appears at a first position in the framing interface and that the second object remains at the first position for a third time period, wherein the first position is a display position of the first graphic before the first graphic is adjusted to the second graphic; Restoring the second graphic to the first graphic, and displaying the first graphic at the first position, wherein the first graphic includes the second object; When it is determined that the shooting range remains unchanged and the time length for which the shooting range remains unchanged is greater than or equal to a fourth time length, the second object is determined to be a target object.

10. The method according to claim 9, characterized in that After determining that the second object appears at the first position in the viewfinder interface, the method further includes: Displaying a prompt box at the first position, wherein the prompt box includes the second object, and the prompt box is used to prompt that the second object will be determined as a target object; After determining that the time duration for which the second object remains at the first position reaches a third time duration and before restoring the second graphic to the first graphic, the method further includes: canceling display of the prompt box.

11. The method according to any one of claims 1 to 10, characterized in that: The first graphic also includes a third object, which is any object other than the first object among the at least one object; after the shooting range remains unchanged and the time length for which it remains unchanged is greater than or equal to the first time length, it also includes: determining that the third object is a target object.

12. The method according to claim 11, characterized in that The method further comprises: displaying the first object and the third object; When receiving the third operation, retaining the first object as the target object and canceling the third object as the target object; The third operation includes an operation on the first object, or an operation on the third object.

13. The method according to claim 11, characterized in that Before the shooting range remains unchanged and the duration of the unchanged shooting range is greater than or equal to the first duration, the method further includes: When it is determined that the shooting range remains unchanged and the time period of remaining unchanged is greater than or equal to a fourth time period, it is determined that the first object and the third object are candidate target objects, and the first figure is adjusted to a fourth figure and a fifth figure, the fourth figure is a circumscribed circle or a circumscribed polygon of the first object, and the fifth figure is a circumscribed circle or a circumscribed polygon of the third object.

14. The method according to claim 13, characterized in that Before the shooting range remains unchanged and the duration of the unchanged shooting range is greater than or equal to the first duration, the method further includes: If a fourth operation is received, the third object is retained as a candidate target object and the fifth graphic is retained, the first object is cancelled as a candidate target object and the fourth graphic is cancelled; wherein the fourth operation includes an operation on the fifth graphic, or an operation on the fourth graphic; After the shooting range remains unchanged and the time period for which the range remains unchanged is greater than or equal to the first time period, the method further includes: determining that the third object is a target object.

15. The method according to any one of claims 1 to 10, characterized in that: The first graphic further includes a fourth object, and the fourth object is any object other than the first object among the at least one object. If the shooting range remains unchanged and the time length for which the shooting range remains unchanged is greater than or equal to the first time length, before determining that the first object is a target object, the method further includes: An operation of selecting the first object in the first graphic is received.

16. The method according to claim 15, characterized in that Before receiving an operation of selecting the first object in the first graphic, the method further includes: Output prompt information, where the prompt information is used to prompt the user to specify a target object.

17. The method according to any one of claims 1 to 16, characterized in that: After determining that the first object is a target object, the method further includes: Identify the target object, and identify at least one of the name, category, purchase link, related objects of the target object, and purchase links of the related objects of the target object; or, Translating the target object; or, Copy the target object; or, Cut out the target object.

18. The method according to any one of claims 1 to 16, characterized in that: After determining that the first object is a target object, the method further includes: A first button and a second button are displayed, wherein when the first button is triggered, the process returns to the previous N steps, where N is a positive integer, and when the second button is triggered, the target object is translated, identified, copied, or cropped.

19. The method according to any one of claims 1 to 18, characterized in that: The electronic device has a translation function, the viewfinder interface is an interface displayed when the translation function is turned on, and the first graphic is a circle; or, The electronic device has an object recognition function, the viewfinder interface is an interface displayed when the object recognition function is turned on, and the first graphic is a rectangle.

20. An electronic device, characterized in that: include: a processor, a memory, and one or more programs; The one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the processor, enable the electronic device to perform the method steps as described in any one of claims 1 to 19.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 19.

22. A computer program product, characterized in that The method comprises a computer program, which, when being run on a computer, causes the computer to execute the method according to any one of claims 1 to 19.

Images