Terminating image classification based on user familiarity

Abstract

A computing system may: perform image classification within a displayed image; while performing the image classification, determine that a first object of an object category is present within the displayed image; present a prompt within the displayed image; receive a selection of a first object from a user in response to the prompt; in response to receiving the selection of the first object, perform a function on the first object; determine that the user is familiar with the function based on the selection of the first object in response to the prompt; based on determining that the user is familiar with the function, terminate performing the image classification within the displayed image; and, in response to the user selecting a second object of the object category within the displayed image, perform the function on the second object.

Description

Technical Field

[0001] This description deals with performing functions on images. Background Art

[0002] Various functions can be performed on image files, such as optical character recognition or scanning. Continuously performing image classification on files to determine which functions can be performed consumes computing resources. Summary of the Invention

[0003] According to an example, a non-transitory computer-readable storage medium includes instructions stored thereon. When executed by at least one processor, the instructions may be configured to cause a computing system to: perform image classification within a displayed image; while performing the image classification, determine that a first object of an object category is present within the displayed image; generate a prompt; in response to receiving the selection of the first object, receive a selection of the first object from a user; perform a function on the first object based on the selection of the first object; determine that the user is familiar with the function based on the selection of the first object; terminate performing the image classification within the displayed image based on the determination that the user is familiar with the function; and, in response to receiving a selection of a second object of the object category within the displayed image, perform the function on the second object.

[0004] According to an example, a computing system may include: at least one processor; and a non-transitory computer-readable storage medium including instructions stored thereon. When executed by the at least one processor, the instructions may be configured to cause the computing system to: perform image classification within a displayed image; while performing the image classification, determine that a first object of an object category is present within the displayed image; generate a prompt; in response to receiving the selection of the first object, receive a selection of the first object from a user; perform a function on the first object based on the selection of the first object; determine that the user is familiar with the function; based on determining that the user is familiar with the function, terminate performing the image classification within the displayed image; and, in response to receiving a selection of a second object of the object category within the displayed image, perform the function on the second object.

[0005] A method may include: performing image classification within a displayed image by a computing system; while performing the image classification, determining that a first object of an object category is present within the displayed image; generating a prompt; receiving a selection of a first object from a user in response to receiving the selection of the first object; performing a function on the first object based on the selection of the first object; determining that the user is familiar with the function; terminating performing the image classification within the displayed image based on determining that the user is familiar with the function; and, in response to receiving a selection of a second object of the object category within the displayed image, performing the function on the second object.

[0006] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1A An electronic device presenting a first object within a display is shown.

[0008] Figure 1B An electronic device is shown presenting a prompt associated with a first object.

[0009] Figure 1C An electronic device is shown with a user selecting a first object.

[0010] Figure 1D The electronic device is shown after performing a function on a first object.

[0011] Figure 1E An electronic device is shown presenting a second object within a display.

[0012] Figure 1F The electronic device is shown with a user selecting a second object.

[0013] Figure 1G The electronic device is shown after performing the function on the second object.

[0014] Figure 2 It is a block diagram of an electronic device.

[0015] Figure 3 is a flow chart of a method performed by an electronic device.

[0016] Figure 4 is a flow chart of a method performed by an electronic device.

[0017] Figure 5 Examples of computer devices and mobile computer devices that can be used to implement the techniques described herein are shown.

[0018] Like reference numerals refer to like elements. DETAILED DESCRIPTION

[0019] An electronic device, such as a smartphone, either by itself or in communication with a server, can perform functions on objects and / or images displayed by the electronic device. These functions can include, for example: performing optical character recognition (OCR) on text included in an image; scanning a document included in an image; identifying a person, animal, or monument included in an image; initiating a communication (e.g., an email or phone call) based on contact information included in the image; or decoding a barcode included in an image. One or more of these functions can be performed in a visual search application. One or more of these functions can be performed in conjunction with an augmented reality environment and / or application.

[0020] The user may initially be unaware of what functions the electronic device can perform on objects within an image (e.g., an image captured during an augmented reality interaction and / or session). The electronic device can teach and / or train the user that the electronic device can perform functions on objects by performing image classification on the objects, and if the electronic device can perform the function on the object category to which the electronic device determines the object belongs, presenting a prompt for the user to select the object. The user can interact with and / or select one or both of the prompt and the object. The electronic device can perform the function on the object in response to the user selecting the object. One or more of these features described above can be performed in conjunction with visual search (e.g., augmented reality visual search).

[0021] Performing image classification consumes computing resources, such as processor resources, memory resources, and / or drains the battery of the electronic device. In order to reduce the consumption of computing resources, the electronic device can stop and / or terminate the execution of image classification when the user is already familiar with the function. The electronic device can determine that the user is familiar with the function based on the history of the electronic device receiving the selection of the object to prompt the electronic device to perform the function and / or the history of the electronic device presenting prompts to the user. The user can still select the object without the prompt and / or the electronic device does not perform image classification, and the electronic device can respond to receiving the selection of the object by executing the function corresponding to the object category of the selected object. In some examples, the electronic device can perform image classification on the selected object after receiving the selection of the object.

[0022] Figure 1A The electronic device 100 is shown presenting a first object 104A within the display 102. The electronic device 100 may comprise, for example, a smartphone, a tablet computer, a phablet, a notebook or laptop computer, or a desktop computer, as non-limiting examples.

[0023] Electronic device 100 may include a display 102. Display 102 may present, display, and / or output graphical output and / or one or more images. In some examples, display 102 may include a touch screen that receives and / or processes touch input from a user.

[0024] The image displayed and / or presented by the display 102 may include one or more objects 104A. The objects 104A may include, for example, as non-limiting examples, text, barcodes and / or Quick Response (QR) codes, documents such as text documents or image documents, people and animals, or monuments.

[0025] Electronic device 100 may perform image classification on a portion of an image presented by display 102 and / or one or more objects (e.g., object 104A) included within the image presented by display 102. Electronic device 100 may determine whether an object, such as object 104A, belongs to an object category for which electronic device 100 may perform a function. In some examples, electronic device 100 may determine whether any objects of multiple object categories are present within an image displayed by display 102. In some examples, each object category may correspond to a single function that electronic device 100 may perform on an object. In some examples, one or more functions may be performed on objects of more than one object category. In this example, electronic device 100 determines that object 104A is a member of an object category and / or that object 104 of the object category is present within an image displayed by display 102.

[0026] Figure 1B The electronic device 100 is shown presenting a prompt 106 associated with the first object 104A. Presenting a prompt within a display is an example of generating a prompt. Other examples of generated prompts include auditory and / or verbal prompts (e.g., voice that speaks text instead of and / or in addition to text) and / or tactile feedback from the display 102. The prompt 106 can indicate a function that the electronic device 100 will perform on the first object 104A in response to the user selecting the first object 104A. The prompt can be informational, indicating a function that the electronic device 100 can perform. For example, prompt 106 can indicate that electronic device 100 will initiate a phone call or email to a phone number or email address included in object 104A, will scan and / or decode a barcode or QR code included in object 104A, perform optical character recognition (OCR) on text included in object 104A, scan a document included in object 104A, identify a person and / or human being included in object 104A, identify the type and / or breed of animal included in object 104A, and / or identify a monument or point of interest included in object 104A.

[0027] The user can respond to prompt 106 by selecting object 104A. In examples where display 102 is a touch screen, the user can select object 104A by inputting a predetermined gesture into display 102. Examples of predetermined gestures include clicking on the portion of display 102 where object 104A is presented, and / or clicking and holding a finger on the portion of display 102 where object 104A is presented. When an input device such as a computer mouse is available, examples of selecting object 104A include clicking the mouse and / or double-clicking the mouse when a cursor is positioned over the portion of display 102 where object 104A is presented.

[0028] Figure 1C The electronic device 100 is shown with a user selecting a first object 104A. The user's selection 108A of the object 104A can be performed by the user clicking and / or clicking and holding the portion of the display 102 presenting the object 104A. The selection 108A can easily disambiguate and / or specify the object on which the function is to be performed. The electronic device 100 can receive and / or process the selection 108A. In response to receiving and / or processing the selection 108A, the electronic device 100 can perform a function. The function performed by the electronic device 100 can be based on the object category to which the electronic device 100 determines the object 104A belongs when performing image classification. In some examples, the electronic device 100 can transform the object 104A by performing the function.

[0029] Figure 1D It is shown that after the first object 104A ( Figure 1DThe electronic device 100 (not shown) after executing a function. The function can convert the first object 104A into a first converted object 110A. In some examples where the object class of the first object 104A is a phone number, the function can include initiating a call to the phone number, and the converted object 110A can include a visual indicator that the phone call is being made. In some examples where the object class of the first object 104A is an email address, the function can include starting to draft an email to the email address, and the converted object 110A can include a blank email with the email address from the first object 104A in the "To" line. In some examples where the object class of the first object 104A is a barcode or QR code, the function can include decoding the barcode or QR code, and the converted object 110A can include a text description of the item identified by the barcode or QR code included in the first object 104A and / or an image of the item identified by the barcode or QR code included in the first object 104A. In some examples where the object category of the first object 104A includes text, the functionality may include performing optical character recognition on the text, and the converted object 110A may include the recognized text from the first object 104A in a different, more readable format. In some examples where the object category of the first object 104A is an indicator of a document, the functionality may include scanning the document, and the converted object 110A may include an indicator that the document from the first object 104A was scanned. In some examples where the object category of the first object 104A is a person, an animal, or a monument or landmark, the functionality may include identifying a person and / or the person, animal, monument, or landmark, and the converted object 110A may include a textual description and / or name of the person, animal, monument, or landmark included in the first object 104A.

[0030] After being presented with prompt 106, performing selection 108A of first object 104A, and viewing the result of the function and / or the converted object 110A, the user may become familiar with the function and no longer need prompt 106 to select an object for the electronic device 100 to perform the function on. The electronic device 100 may determine that the user is familiar with the function based on receiving selection 108A of an object 104A included in the object category associated with the function. In one embodiment, the determination of familiarity may depend on the number of selections 108A of first object 104A, particularly the number of selections 108A of first object 104A within a predetermined time interval. If the electronic device 100 receives selections 108A multiple times, the electronic device 100 may determine and / or assume that the user has a certain degree of familiarity with the selection. Another criterion for determining familiarity may be the time between the display of object 104A and the selection 108A, or the user's potential familiarity with the object. These criteria may be used in combination, and these are examples used in conjunction with the familiarity determiner 210 described further below.

[0031] Based on determining that the user is familiar with the functionality, the electronic device 100 may terminate and / or stop performing image classification and / or not present the prompt 106 to the user.

[0032] Figure 1E The electronic device 100 is shown presenting a second object 104B within the display 102 . Figure 1E The image presented by the display 102 in FIG. 1 (including the second object 104B) may be different from Figure 1A 104B. The second object 104B may belong to the same object category as the first object 104A, and / or the object category of the second object 104B may be associated with the same function performed on the first object 104A. Based on the electronic device 100 determining that the user is familiar with the object category and / or the function performed on the first object 104A, the electronic device 100 will terminate the image classification and / or will not present the prompt 106 for the user to request the function to be performed on the second object 104B. Based on the user's familiarity with the function, the user may select the object 104B without the prompt 106.

[0033] Figure 1FElectronic device 100 is shown with a user selecting second object 104B. Based on the user's familiarity with the functionality, the user may enter selection 108B without prompt 106 in a manner similar to entering selection 108A. Electronic device 100 may receive the selection of object 104B. Based on receiving the selection of object 104B, electronic device 100 may perform image classification on object 104B and / or determine that object 104B belongs to the same object category as object 104A. Based on determining that object 104B belongs to the same object category, electronic device 100 may receive selection 108B and execute the functionality associated with the object category of object 104B in a manner similar to the functionality executed for object 104A.

[0034] Figure 1G The electronic device 100 is shown after performing a function on the second object 104B. The electronic device 100 may perform a function on the object 104B to generate the converted object 110A in a manner similar to the manner in which the function was performed on the object 104A to generate the converted object 110A described above. Figure 1G The display 102 presents the transformed object 110B.

[0035] Figure 2 is a block diagram of electronic device 100. Electronic device 100 may include image renderer 202. Image renderer 202 may generate and / or render an image for presentation and / or display by display 102. The image may be generated based on data processed by an application running and / or executing on electronic device 100, such as, for example, a web browser, a camera application, a video player, or a social media application, as non-limiting examples. The image generated and / or rendered by image renderer 202 may include an object, such as objects 104A, 104B and / or transformed objects 110A, 110B.

[0036] The electronic device 100 may include an image classifier 204. The image classifier 204 may classify objects within an image and / or determine whether an object within an image is a member of an object class. In some examples, the image classifier 204 may determine whether an object within each frame of a displayed image is a member of an object class. In some examples, to reduce the consumption of computing resources, the image classifier 204 may determine whether an object within every nth frame of a displayed image (e.g., an object within every fourth frame) is a member of an object class. The image classifier 204 may determine whether an object within a displayed image is a member of an object class by comparing features of the object with features of the object class. The image classifier 204 may determine whether an object within a displayed image is a member of an object class by executing on-device machine learning techniques, such as, for example, an artificial neural network, a convolutional neural network, K-nearest neighbor classification, a decision tree, or a support vector machine (SVM), as non-limiting examples. In some examples, the image classifier 204 may determine whether each object within a given displayed image and / or multiple portions of a given displayed image are members of multiple classes. In some examples, image classifier 204 may sequentially determine whether an object and / or portion of an image is a member of multiple different object classes.

[0037] Image classifier 204 may include and / or have access to category library 206. Category library 206 may include and / or store a plurality of object categories. Image classifier 204 may compare features of an object (e.g., objects 104A, 104B) presented on display 102 with features of the object categories stored in category library 206 to determine whether the object is a member of the object category stored in category library 206.

[0038] Image classifier 204 may include image detector 208. Image detector 208 may detect locations of objects on display 102 that image classifier 204 has determined are members of an object class. Hint generator 214 may use the locations determined by image detector 208 to determine where to present a hint (e.g., hint 106).

[0039] The electronic device 100 may include a familiarity determiner 210. The familiarity determiner 210 may determine whether a user is familiar with an object category and / or functions associated with the object category, and / or the user's familiarity therewith. Familiarity may be associated with a user and / or account that is logged in, active, and / or interacting with the electronic device 100. In some examples, the familiarity determiner 210 may generate a binary value for the user, indicating that the user is unfamiliar with the object category and / or associated functions and should be prompted to request to perform a function on an object of the object category, or indicating that the user is familiar with the object category and / or associated functions and should not be prompted to request to perform a function on an object of the object category. In some examples, the familiarity determiner 210 may generate one of a plurality of familiarity values ​​within a range for the user, and the electronic device 100 may determine whether to present a prompt to the user based on a combination of the familiarity value and context information.

[0040] Familiarity determiner 210 may determine a user's familiarity value for each object category and / or function. The familiarity value may be based on the number and / or frequency of prompts for performing a function and / or prompts for objects of a given object category presented to the user, with a greater number and / or frequency of prompts for objects and / or object categories increasing the familiarity value. The familiarity value may be based on the number of times a user has selected an object of a given object category and / or selected a given function, with a greater number of times a user has selected an object for a function to be performed increasing the familiarity value for the object's function and / or object category. The familiarity value may be based on the length of time since the user selected an object of a given object category and / or selected a given function, with a greater length of time since the user selected an object of a given object category and / or selected a given function decreasing the familiarity value. In some examples, electronic device 100 may determine that the user is sufficiently familiar with a function and / or an object category associated with the function, and may stop and / or terminate performing image classification for the object category and cease presenting prompts for the function associated with the object category. Then, after the user has not selected an object of the object category and / or the function associated with the object category for a period of time, may determine that the user is no longer familiar with the function and / or the object category associated with the function. Based on determining that the user is no longer familiar with the function and / or the object category associated with the function, electronic device 100 may resume performing image classification for the image category and present a prompt for the user to select an object of the object category to perform the function associated with the object category. If a predetermined first threshold of familiarity value is reached, familiarity determiner 210 may control the termination of classification, and if a predetermined second threshold of familiarity value is reached, familiarity determiner 210 may control the resumption of classification. This takes into account that a user may become familiar with an image after some use, but lose familiarity after a period of non-use. In one embodiment, the first and second thresholds may be the same.

[0041] The electronic device 100 may include a context processor 212. The context processor 212 may process context information, which the electronic device 100 may combine with a familiarity value to determine whether to perform image classification and / or present a prompt. The context information may include, for example, the state of computing resources, such as the charge level of a battery included in the electronic device 100 (other examples of computing resources include memory availability, processor resources, connection strength to a network, or a photographic modality), the time of day, audio signals received by a microphone included in the electronic device 100, the location of the electronic device 100, and / or a user's interest in images of a particular object category (e.g., a person or a particular type of animal, such as a dog or cat), as indicated by the user viewing images of the object category and / or storing images of the object category. If the battery charge level is low, the electronic device 100 may not perform image classification to conserve battery power, even though the user is unfamiliar with the function and / or object category. If the time of day is one at which the user is not likely to be interested in performing specific functions, the electronic device 100 may not perform image classification for object categories associated with those specific functions, despite the user's unfamiliarity with those specific functions and / or object categories, unless the familiarity value is very low, in which case the electronic device 100 may perform image classification. If the audio signal indicates a specific environment, such as a coffee shop, the electronic device 100 may perform image classification for object categories associated with the specific environment, such as coffee or menus, despite the user's relatively high familiarity value for the object categories associated with the specific environment. If the user is in a specific location, such as a restaurant area, the electronic device may perform image classification for objects of an object category associated with the specific location, such as signs and / or menus, despite the user's relative familiarity with the object category associated with the specific location. If the electronic device 100 has determined that the user has a high interest in images of a specific object category, the electronic device 100 may perform image classification for objects of the specific object category, despite the user's relative familiarity with the object category and / or the functions associated with the object category.

[0042] The electronic device 100 may include a prompt generator 214. The prompt generator 214 may generate prompts, such as Figure 1B 1 and 2 may be used to display a prompt 106 for a user to select an object on which to perform a function. Prompt generator 214 may generate and / or display a prompt on display 102 based on and / or in response to image classifier 204 determining that the object is a member of a class of objects on which electronic device 100 may perform a function. Prompt generator 214 may generate the prompt near the object on display 102 based on the position of the object determined by image detector 208. The prompt may identify and / or describe (e.g., in text) the function to be performed on the object. The function to be performed may be associated with the object class of the object on which the function is to be performed.

[0043] Electronic device 100 may include a selection processor 216. Selection processor 216 may process the selection of an object, such as any of objects 104A and 104B, such as selecting any of objects 108A and 108B, that is a member of an object class associated with a function. In examples where display 102 is a touch screen, selection may include a gesture on a portion of display 102 presenting and / or displaying the selected object, such as a tap or tap-and-hold gesture, or a single or double-click of a mouse while a cursor is over the portion of display 102 presenting and / or displaying the selected object. Selection processor 216 may determine whether the input meets criteria for a predefined gesture with respect to an object presented and / or displayed by display 102. If selection processor 216 determines that the input does meet criteria for a predefined gesture with respect to an object presented and / or displayed by display 102, selection processor 216 may determine that the user has selected the object. Based on determining that the user has selected the object, selection processor 216 may prompt and / or instruct function processor 218 to perform a function on and / or with respect to the object.

[0044] Electronic device 100 may include a function processor 218. Function processor 218 may execute a function on an object determined by selection processor 216 to be user-selected. Function processor 218 may execute and / or select a function based on the object category determined by image classifier 204 to which the object belongs. Function processor 218 may pass the object as a parameter to the function. The object passed as a parameter to the function may include a portion of an image presented by display 102. The portion of the image passed as a parameter by function processor 218 may be based on the location of the object determined by image detector 208. Functions executed and / or selected by function processor 218 may include, for example: initiating a phone call to a phone number included in the object; starting an email to an email address included in the object; decoding a barcode or QR code included in the object; performing optical character recognition on text included in the object; scanning a document included in and / or identified by the object; or identifying a person and / or a human, animal, monument, or landmark included in the object. In some examples, function processor 218 may execute the function locally on electronic device 100. In some examples, the function processor 218 can execute the function by sending a request to the server, the request including the object, such as by calling an application programming interface (API) with the object as a parameter, and receiving a converted object from the server in response to the request. After executing the function, the function processor 218 can generate a converted object, such as the converted objects 110A, 110B, for presentation and / or display by the display 102.

[0045] The electronic device 100 may include at least one processor 220. The at least one processor 220 may execute instructions, such as instructions stored in at least one memory device 222, to cause the electronic device 100 to perform any combination of the methods, functions, and / or techniques described herein.

[0046] The electronic device 100 may include at least one memory device 222. The at least one memory device 222 may include a non-transitory computer-readable storage medium. The at least one memory device 222 may store thereon data and instructions that, when executed by at least one processor (e.g., processor 220), are configured to cause a computing system (e.g., electronic device 100) to perform any combination of the methods, functions, and / or techniques described herein. Therefore, in any embodiment described herein (even if not explicitly stated with respect to a particular embodiment), software (e.g., processing modules, stored instructions) and / or hardware (e.g., processors, storage devices, etc.) associated with or included in the electronic device 100 may be configured to perform any combination of the methods, functions, and / or techniques described herein, either alone or in combination with the electronic device 100.

[0047] The electronic device 100 may include at least one input / output node 224. At least one input / output node 224 may receive and / or send data, and / or may receive input from a user and provide output to the user. The input and output functions may be combined into a single node, or may be divided into separate input and output nodes. The input / output node 224 may include, for example, a display (which may be a touch screen display, such as the display 102), a camera, a speaker, a microphone, one or more buttons, a motion detector and / or an accelerometer, a thermometer, a light sensor, and / or one or more wired or wireless interfaces for communicating with other computing devices.

[0048] Figure 3 is a flowchart of a method performed by the electronic device 100 . Figure 3 The electronic device 100 is shown looping through image classification based on whether the user is familiar with a function and / or an object category associated with the function. The image presenter 202 may present an image (302). The image presenter 202 may present the image on the display 102. The image may be based on an application running on the electronic device, such as a web browser, a camera, or a social media application.

[0049] Image classifier 204 can classify an image (304) and / or a portion of an image presented by display 102. Image classifier 204 can classify an image while simultaneously initiating a loop for classifying the image for a particular object class. Portions of an image can be considered objects, such as objects 104A, 104B. Image classifier 204 can classify an image and / or an object by determining whether the image and / or object is a member of one or more object classes stored in a class library.

[0050] The familiarity determiner 210, in conjunction with the context processor 212, can determine 306 whether the user is familiar with the object category of the object and / or the functionality associated with the object. The familiarity determiner 210, in conjunction with the context processor 212, can determine whether the user is familiar with the object category of the object and / or the functionality associated with the object based on a combination of one or more of the following, as non-limiting examples: the number of times a prompt (e.g., prompt 106) has been presented to the user to perform the functionality for an object of the object category; the number of times the user has selected an object of the object category; the amount of time that has passed since the user selected an object of the object category; the charge level of a battery included in the electronic device 100; the time of day; an audio signal received by a microphone included in the electronic device 100; the location of the electronic device 100; and / or the user's interest in images of a particular object category, as indicated by the user viewing images of the object category and / or storing images of the object category.

[0051] If the familiarity determiner 210, in conjunction with the context processor 212, determines that the user is familiar with the object category and / or function, the electronic device 100 may close the loop of image classification or function with respect to the object category and / or function. Based on the familiarity determiner's determination that the user is familiar with the object category and / or function, and / or closing the loop of image classification with respect to the object category and / or function, the selection processor 216 may determine whether the electronic device 100 receives a selection of an object (308), as described below. In some examples, if the familiarity determiner 210, in conjunction with the context processor 212, determines that the user is familiar with the object category and / or function, the image classifier 204 may stop determining whether an object familiar with the object category exists based on the electronic device 100 closing the loop of image classification with respect to the object category and / or function. If the familiarity determiner 210, in conjunction with the context processor 212, determines that the user is not familiar with the object category and / or function, and / or opening the loop of image classification with respect to the object category and / or function, the prompt generator 214 may present a prompt (312) (and / or the electronic device 100 may open the loop of image classification with respect to the object category and / or function). A prompt such as prompt 106 may indicate and / or describe a function to be performed on an object. The representation of the prompt may be based on the position of the object and the object class of the object determined by the image detector 208 and / or the function to be performed on the object.

[0052] After presenting the prompt (312) and / or determining that the user is familiar with the object category and / or function, the electronic device 100 and / or the selection processor 216 can determine whether the electronic device 100 receives a selection of the object. The selection processor 216 can determine whether the electronic device 100 receives a selection of the object based on whether the electronic device 100 receives a predetermined gesture, such as a click or click and hold on the area of ​​the display 102 where the object is presented, or a single click or double click of a mouse when the cursor is over the area of ​​the display 102 where the object is presented.

[0053] If the selection processor 216 determines that the electronic device 100 did not receive a selection of an object, the electronic device 100 may continue to present the image (302). If the selection processor 216 determines that the electronic device 100 did receive a selection of an object, the function processor 218 may perform a function on the object (310). After performing the function (310), the electronic device 100 may continue to present the image (302).

[0054] Figure 4 is a flow chart of a method performed by an electronic device 100. The method may include performing image classification within a displayed image (402). The method may include, while performing the image classification, determining that a first object 104A of an object category is present within the displayed image (404). The method may include generating a prompt, such as presenting a prompt 106 within the displayed image (406). The method may include, for example, receiving a selection 108A of the first object 104A from a user in response to the prompt 106 (408). The method may include, in response to receiving the selection of the first object 104A, performing a function on the first object 104A (410). The method may include, based on the selection of the first object 104A in response to the prompt 106, determining that the user is familiar with the function (412). The method may include, based on determining that the user is familiar with the function, terminating performing the image classification within the displayed image (414). The method may include, in response to the user selecting a second object 104B of the object category within the displayed image, performing the function on the second object 104B (416).

[0055] In some examples, performing the function on the first object may include performing a local image recognition function on the displayed image.

[0056] In some examples, performing the function on the first object may include sending the displayed image to a server, and receiving the transformed object from the server.

[0057] In some examples, the object category can include text.In some examples, the function can include performing optical character recognition on the first object.

[0058] In some examples, the object category can include a barcode.In some examples, the function can include decoding the first object.

[0059] In some examples, the object category can include humans.In some examples, the function can include determining a name associated with the first object.

[0060] In some examples, the method may also include determining the object category based on the location of the computing system.

[0061] In some examples, the method may also include determining the object category based on images previously viewed by the user.

[0062] In some examples, determining that the user is familiar with the function may include determining that the user is familiar with the function relative to the object category, and terminating performing image classification within the displayed image may include terminating performing image classification within the displayed image relative to the object category.

[0063] In some examples, terminating performance of image classification within the displayed image can be based on determining that the user is familiar with the function and the time of day.

[0064] In some examples, terminating performance of image classification within the displayed image can be based on determining that the user is familiar with the functionality and a charge level of a battery included in the computing system.

[0065] In some examples, terminating performance of image classification within the displayed image can be based on determining that the user is familiar with the function and an audio signal received by a microphone included in the computing system.

[0066] In some examples, the method may further include, in response to receiving a selection of the second object, performing image classification on the second object and determining that the second object belongs to an object category, and performing the function on the second object may be based on determining that the second object belongs to the object category.

[0067] In some examples, the method may further include, after determining that a first object of the object category is present within the displayed image, performing image detection on the first object. Presenting the prompt within the displayed image may include presenting the prompt at a location within the displayed image based on a location at which the first object is detected within the displayed image.

[0068] In some examples, the method may also include: displaying multiple additional images; determining that the user is no longer familiar with the function based on the user not selecting a portion of the displayed multiple additional images; and based on determining that the user is no longer familiar with the function, performing image classification within a subsequently displayed image; while performing image classification within the subsequently displayed image, determining that a subsequent object of the object category exists within the subsequently displayed image; and performing a subsequent prompt within the subsequently displayed image.

[0069] In some examples, the object category may include a first object category, the prompt may include a first prompt, and the function may include a first function. The method may further include: performing image classification within a subsequently displayed image; while performing image classification within the subsequently displayed image, detecting a second object of a second object category within the subsequently displayed image, the second object category being different from the first object category; presenting a second prompt within the subsequently displayed image; receiving a selection of the second object from a user; and in response to receiving the selection of the second object, performing a second function on the second object, the second function being different from the first function.

[0070] Figure 5 Examples of a general-purpose computer device 500 and a general-purpose mobile computer device 550 are shown, which can be used with the techniques described herein. Computing device 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, tablet computers, workstations, personal digital assistants, televisions, servers, blade servers, mainframes, and other suitable computing devices that can communicate with electronic device 100. Computing device 550 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices that can be examples of electronic device 100. The components shown here, their connections and relationships, and their functions are exemplary only and are not meant to limit the embodiments of the inventions described and / or claimed in this document.

[0071] Computing device 500 includes a processor 502, a memory 504, a storage device 506, a high-speed interface 508 connected to the memory 504 and a high-speed expansion port 510, and a low-speed interface 512 connected to a low-speed bus 514 and the storage device 506. Processor 502 can be a semiconductor-based processor. Memory 504 can be a semiconductor-based memory. Each of components 502, 504, 506, 508, 510, and 512 is interconnected using various buses and can be mounted on a common motherboard or in other suitable ways. Processor 502 can process instructions for execution within computing device 500, including instructions stored in memory 504 or on storage device 506 for displaying graphical information for a GUI on an external input / output device (e.g., a display 516 coupled to high-speed interface 508). In other embodiments, multiple processors and / or multiple buses, as well as multiple memories and types of memories, can be used as appropriate. Furthermore, multiple computing devices 500 may be connected, with each device providing portions of the necessary operations (eg, as a server bank, a group of blade servers, or a multi-processor system).

[0072] Memory 504 stores information within computing device 500. In one embodiment, memory 504 is one or more volatile memory units. In another embodiment, memory 504 is one or more non-volatile memory units. Memory 504 can also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0073] Storage device 506 can provide mass storage for computing device 500. In one embodiment, storage device 506 can be or include a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a magnetic tape device, a flash memory or other similar solid-state storage device, or an array of devices, including devices in a storage area network or other configuration. A computer program product can be tangibly embodied in an information carrier. A computer program product can also include instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 504, storage device 506, or memory on processor 502.

[0074] The high-speed controller 508 manages bandwidth-intensive operations of the computing device 500, while the low-speed controller 512 manages less bandwidth-intensive operations. This allocation of functions is exemplary only. In one embodiment, the high-speed controller 508 is coupled to the memory 504, the display 516 (e.g., via a graphics processor or accelerator), and to the high-speed expansion ports 510 that can accept various expansion cards (not shown). In this embodiment, the low-speed controller 512 is coupled to the storage device 506 and the low-speed expansion ports 514. The low-speed expansion ports, which can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as a keyboard, pointing device, scanner, or network device, such as a switch or router, for example, via a network adapter.

[0075] Computing device 500 can be implemented in a variety of different forms, as shown. For example, it can be implemented as a standard server 520, or multiple implementations in a group of such servers. It can also be implemented as part of a rack server system 524. In addition, it can be implemented in a personal computer such as laptop computer 522. Alternatively, components from computing device 500 can be combined with other components in a mobile device (not shown), such as device 550. Each of these devices can include one or more of computing devices 500, 550, and the entire system can be composed of multiple computing devices 500, 550 communicating with each other.

[0076] Computing device 550 includes a processor 552, a memory 564, an input / output device such as a display 554, a communication interface 566, and a transceiver 568, among other components. Device 550 may also be equipped with a storage device, such as a micro drive or other device, to provide additional storage. Each of components 550, 552, 564, 554, 566, and 568 is interconnected using various buses, and several of the components may be mounted on a common motherboard or in other suitable ways.

[0077] Processor 552 can execute instructions within computing device 550, including instructions stored in memory 564. The processor can be implemented as a chipset including separate or multiple analog and digital processors. The processor can provide, for example, coordination of other components of device 550, such as control of a user interface, applications running on device 550, and wireless communications of device 550.

[0078] The processor 552 can communicate with the user via a control interface 558 and a display interface 556 coupled to the display 554. The display 554 can be, for example, a TFT LCD (thin film transistor liquid crystal display) or an OLED (organic light emitting diode) display or other appropriate display technology. The display interface 556 may include appropriate circuitry for driving the display 554 to present graphics and other information to the user. The control interface 558 can receive commands from the user and convert them for submission to the processor 552. In addition, an external interface 562 can be provided in communication with the processor 552 to enable near-area communication of the device 550 with other devices. The external interface 562 can, for example, provide wired communication in some embodiments, or wireless communication in other embodiments, and multiple interfaces can also be used.

[0079] Memory 564 stores information within computing device 550. Memory 564 can be implemented as one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Expansion memory 574 can also be provided and connected to device 550 via expansion interface 572, which can include, for example, a SIMM (Single In-Line Memory Module) card interface. Such expansion memory 574 can provide additional storage space for device 550 or store applications or other information for device 550. Specifically, expansion memory 574 can include instructions for executing or supplementing the above-described processes and can also include security information. Thus, for example, expansion memory 574 can be provided as a security module for device 550 and can be programmed with instructions that allow for secure use of device 550. Furthermore, secure applications and additional information can be provided via a SIMM card, such as placing identification information on the SIMM card in an unhackable manner.

[0080] The memory may include, for example, flash memory and / or NVRAM memory, as discussed below. In one embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 564, expansion memory 574, or memory on processor 552, which may be received, for example, via transceiver 568 or external interface 562.

[0081] Device 550 can perform wireless communications via a communication interface 566, which may include digital signal processing circuitry, if desired. Communication interface 566 can provide for communications in various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS. Such communications can occur, for example, via a radio frequency transceiver 568. In addition, short-range communications can occur, for example, using Bluetooth, WiFi, or other such transceivers (not shown). In addition, a GPS (Global Positioning System) receiver module 570 can provide additional navigation and location-related wireless data to device 550, which can be used as appropriate by applications running on device 550.

[0082] Device 550 may also perform voice communications using audio codec 560, which may receive spoken information from a user and convert it into usable digital information. Audio codec 560 may similarly generate audible sounds for the user, such as through a speaker in a handset of device 550. Such sounds may include sounds from voice phone calls, may include recorded sounds (e.g., voice messages, music files, etc.), and may also include sounds generated by applications running on device 550.

[0083] Computing device 550 can be implemented in many different forms, as shown in the figure. For example, it can be implemented as a cellular phone 580. It can also be implemented as part of a smart phone 582, a personal digital assistant, or other similar mobile device.

[0084] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs executable and / or interpretable on a programmable system comprising at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from and send data and instructions to a storage system, at least one input device, and at least one output device.

[0085] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium," "computer-readable medium," and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor that includes a machine-readable medium that receives the machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0086] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0087] The systems and techniques described herein can be implemented in a computing system that includes a back-end component (such as a data server), or includes a middleware component (such as an application server), or includes a front-end component (such as a client computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or includes any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0088] A computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0089] A number of embodiments have been described, however, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0090] Some examples are described below.

[0091] Example 1: A non-transitory computer-readable storage medium comprising instructions stored thereon, the instructions being configured to cause a computing system, when executed by at least one processor, to:

[0092] performing image classification within the displayed image;

[0093] while performing said image classification, determining that a first object of an object class is present within the displayed image;

[0094] presenting a prompt within the displayed image;

[0095] receiving, in response to the prompt, a selection of the first object from a user;

[0096] in response to receiving the selection of the first object, performing a function on the first object;

[0097] determining that the user is familiar with the functionality based on the selection of the first object in response to the prompt;

[0098] based on determining that the user is familiar with the function, terminating performing image classification within the displayed image; and

[0099] In response to the user selecting a second object of the object category within the displayed image, the function is performed on the second object.

[0100] Example 2: The non-transitory computer-readable storage medium of Example 1, wherein performing the function on the first object comprises performing a local image recognition function on the displayed image.

[0101] Example 3: The non-transitory computer-readable storage medium of Example 1 or 2, wherein performing the function on the first object comprises:

[0102] sending the displayed image to a server; and

[0103] A transformed object is received from the server.

[0104] Example 4: The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the object category comprises text.

[0105] Example 5: The non-transitory computer-readable storage medium of Example 4, wherein the function comprises performing optical character recognition on the first object.

[0106] Example 6: The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the object category comprises a barcode.

[0107] Example 7: The non-transitory computer-readable storage medium of Example 6, wherein the function comprises decoding the first object.

[0108] Example 8: The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the class of objects includes humans.

[0109] Example 9: The non-transitory computer-readable storage medium of Example 8, wherein the function comprises determining a name associated with the first object.

[0110] Example 10 The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the instructions are further configured to cause the computing system to determine the object category based on a location of the computing system.

[0111] Example 11 The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the instructions are further configured to cause the computing system to determine the object category based on images previously viewed by the user.

[0112] Example 12: The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the instructions are further configured to cause the computing system to determine whether the object belongs to a category of objects for which the electronic device is capable of performing a function.

[0113] Example 13: A non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the determination of the familiarity by the familiarity determiner depends on the number of selections of the object, in particular the number of selections of the object in a predetermined time interval, and / or on the time between the display of the object and the selection, and in particular a familiarity value is generated.

[0114] Example 14: The non-transitory computer-readable storage medium of Example 13, wherein the familiarity determiner controls termination of the classification if a predetermined first threshold of familiarity value is reached, and controls resumption of the classification if a predetermined second threshold of familiarity value is reached.

[0115] Example 15: The non-transitory computer-readable storage medium of Example 13 or 14, wherein the context processor processes context information, and the electronic device is capable of combining the context information with the familiarity value to determine whether to perform image classification and / or present a prompt, the context information being particularly the charge level of a battery included in the electronic device, the time of day, an audio signal received by a microphone included in the electronic device, the location of the electronic device, and / or the user's interest in images of a particular object category as shown by the user viewing images of the object category and / or storing images of the object category.

[0116] Example 16: The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein:

[0117] The determining that the user is familiar with the function comprises determining that the user is familiar with the function with respect to the object category; and

[0118] The terminating performing image classification within the displayed image includes terminating performing image classification within the displayed image with respect to the object category.

[0119] Example 17: The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the instructions are configured to cause the computing system to terminate performing image classification within the displayed image based on determining that the user is familiar with the function and the time of day.

[0120] Example 18: A non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the instructions are configured to cause the computing system to terminate performing image classification within the displayed image based on determining that the user is familiar with the functionality and a charge level of a battery included in the computing system.

[0121] Example 19: A non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the instructions are configured to cause the computing system to terminate performing image classification within the displayed image based on determining that the user is familiar with the function and an audio signal received by a microphone included in the computing system.

[0122] Example 20: The non-transitory computer-readable storage medium according to at least one of the preceding claims, wherein:

[0123] The instructions are further configured to cause the computing system to perform image detection on the first object after determining that the first object of the object class is present within the displayed image; and

[0124] Presenting the hint within the displayed image includes presenting the hint at a location within the displayed image based on a location at which the first object was detected within the displayed image.

[0125] Example 21: The non-transitory computer-readable storage medium of at least one of the preceding examples, wherein the instructions are further configured to cause the computing system to:

[0126] Display multiple additional images;

[0127] determining that the user is no longer familiar with the functionality based on the user not selecting a portion of the displayed plurality of additional images; and

[0128] Based on determining that the user is no longer familiar with the function:

[0129] performing image classification within subsequently displayed images;

[0130] while performing the image classification within the subsequently displayed image, determining that a subsequent object of the object class is present within the subsequently displayed image; and

[0131] A subsequent prompt is presented within the subsequently displayed image.

[0132] Example 22: The non-transitory computer-readable storage medium according to at least one of the preceding claims, wherein:

[0133] The object categories include a first object category;

[0134] The prompts include a first prompt;

[0135] The functions include a first function; and

[0136] The instructions are further configured to cause the computing system to:

[0137] performing image classification within subsequently displayed images;

[0138] while performing the image classification within the subsequently displayed image, detecting a second object of a second object class within the subsequently displayed image, the second object class being different from the first object class;

[0139] presenting a second prompt within the subsequently displayed image;

[0140] receiving a selection of the second object from the user; and

[0141] In response to receiving the selection of the second object, a second function is performed on the second object, the second function being different from the first function.

[0142] Example 23: A computing system comprising:

[0143] at least one processor; and

[0144] The non-transitory computer-readable storage medium of any of Examples 1-22.

[0145] Example 24: A method comprising:

[0146] performing, by a computing system, image classification within the displayed image;

[0147] while performing said image classification, determining that a first object of an object class is present within the displayed image;

[0148] presenting a prompt within the displayed image;

[0149] receiving said first selection from a user;

[0150] in response to receiving the selection of the prompt, performing a function on the first object;

[0151] determining, based on the selection of the prompt, that the user is familiar with the function;

[0152] terminating image classification within the displayed image based on determining that the user is familiar with the function; and

[0153] In response to the user selecting a second object of the object category within the displayed image, the function is performed on the second object.

[0154] Furthermore, the logic flows depicted in the figures do not require the particular order shown or sequential order to achieve the desired results. Furthermore, other steps may be provided or eliminated relative to the described flows, and other components may be added to or removed from the described systems. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. A non-transitory computer-readable storage medium comprising instructions stored thereon, the instructions being configured to, when executed by at least one processor, cause a computing system to: performing image classification within the displayed image to determine that a first object of an object class is present within the displayed image; receiving, from a user, a selection of the first object in response to a prompt within the displayed image; in response to receiving the selection of the first object, performing a function on the first object; determining that the user is familiar with the function based on the selection of the first object in response to the prompt; based on determining that the user is familiar with the function, terminating performing image classification within the displayed image; as well as In response to the user selecting a second object of the object category within the displayed image, the function is performed on the second object.

2. The non-transitory computer-readable storage medium according to claim 1, wherein Performing the function on the first object includes performing a local image recognition function on the displayed image.

3. The non-transitory computer-readable storage medium of claim 1, wherein: Executing the function on the first object includes: sending the displayed image to a server; and A transformed object is received from the server.

4. The non-transitory computer-readable storage medium of claim 1, wherein: The object categories include text.

5. The non-transitory computer-readable storage medium of claim 4, wherein: The function includes performing optical character recognition on the first object.

6. The non-transitory computer-readable storage medium of claim 1, wherein: The object class includes barcodes.

7. The non-transitory computer-readable storage medium of claim 6, wherein: The functions include decoding the first object.

8. The non-transitory computer-readable storage medium of claim 1, wherein: The class of objects includes humans.

9. The non-transitory computer-readable storage medium of claim 8, wherein: The functions include determining a name associated with the first object.

10. The non-transitory computer-readable storage medium of claim 1, wherein: The instructions are further configured to cause the computing system to determine the object category based on a location of the computing system.

11. The non-transitory computer-readable storage medium of claim 1 , wherein: The instructions are further configured to cause the computing system to determine the object category based on images previously viewed by the user.

12. The non-transitory computer-readable storage medium of claim 1, wherein: The instructions are further configured to cause the computing system to determine whether the object belongs to a class of objects for which the computing system is capable of performing a function.

13. The non-transitory computer-readable storage medium of claim 1, wherein: The familiarity determiner determines the familiarity depending on a number of selections of the object including a number of selections of the object in a predetermined time interval and / or depending on a time between the display and the selection of the object, and a familiarity value is generated.

14. The non-transitory computer-readable storage medium of claim 13, wherein: The familiarity determiner controls the termination of the classification if a predetermined first threshold of the familiarity value is reached and controls the resumption of the classification if a predetermined second threshold of the familiarity value is reached.

15. The non-transitory computer-readable storage medium of claim 13, wherein: A context processor processes context information, which the computing system can combine with the familiarity value to determine whether to perform image classification and / or present a prompt, the context information being a charge level of a battery included in the computing system, a time of day, an audio signal received by a microphone included in the computing system, a location of the computing system, and / or an interest of the user in images of a particular object category as shown by the user viewing images of the object category and / or storing images of the object category.

16. The non-transitory computer-readable storage medium of claim 1, wherein: The determining that the user is familiar with the function comprises determining that the user is familiar with the function with respect to the object category; and Terminating performing image classification within the displayed image includes terminating performing image classification within the displayed image with respect to the object category.

17. The non-transitory computer-readable storage medium of claim 1, wherein: The instructions are configured to cause the computing system to terminate performing image classification within the displayed image based on determining that the user is familiar with the function and time of day.

18. The non-transitory computer-readable storage medium of claim 1, wherein: The instructions are configured to cause the computing system to terminate performing image classification within the displayed image based on determining that the user is familiar with the functionality and a charge level of a battery included in the computing system.

19. The non-transitory computer-readable storage medium of claim 1, wherein: The instructions are configured to cause the computing system to terminate performing image classification within the displayed image based on a determination that the user is familiar with the function and an audio signal received by a microphone included in the computing system.

20. The non-transitory computer-readable storage medium of claim 1, wherein: The instructions are further configured to cause the computing system to perform image detection on the first object after determining that the first object of the object category is present within the displayed image; as well as Presenting the cue within the displayed image includes presenting the cue at a location within the displayed image based on a location at which the first object is detected within the displayed image.

21. The non-transitory computer-readable storage medium of claim 1, wherein: The instructions are further configured to cause the computing system to: Display multiple additional images; determining that the user is no longer familiar with the functionality based on the user not selecting a portion of the displayed plurality of additional images; and Based on determining that the user is no longer familiar with the function: performing image classification within subsequently displayed images; while performing said image classification within said subsequently displayed image, determining that a subsequent object of said object class is present within said subsequently displayed image; as well as A subsequent prompt is presented within the subsequently displayed image.

22. The non-transitory computer-readable storage medium of claim 1, wherein: The object categories include a first object category; The prompts include a first prompt; The functions include a first function; and The instructions are further configured to cause the computing system to: performing image classification within subsequently displayed images; while performing the image classification within the subsequently displayed image, detecting a second object of a second object class within the subsequently displayed image, the second object class being different from the first object class; presenting a second prompt within the subsequently displayed image; receiving a selection of the second object from the user; as well as In response to receiving the selection of the second object, a second function is performed on the second object, the second function being different from the first function.

23. A computing system comprising: at least one processor; as well as The non-transitory computer-readable storage medium according to any one of claims 1 to 22.

24. A method for performing a function on an image, comprising: performing, by a computing system, image classification within a displayed image to determine that a first object of an object class is present within the displayed image; receiving, from a user, a selection of the first object in response to a prompt within the displayed image; in response to receiving the selection of the first object, performing a function on the first object; determining that the user is familiar with the function based on the selection of the first object in response to the prompt; terminating image classification within the displayed image based on determining that the user is familiar with the function; as well as In response to the user selecting a second object of the object category within the displayed image, the function is performed on the second object.

Images