Electronic device and control method thereof
Through image analysis and server update mechanisms, the reduction in accuracy and user environment adaptability of the artificial intelligence model when identifying cooking objects is solved, achieving high accuracy and user satisfaction.
Patent Information
- Application Number
- CN202380027306.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-06-27
- Filing Date
- 2023-03-10
- Publication Date
- 2024-11-15
AI Technical Summary
Artificial intelligence models have problems with decreasing recognition accuracy when identifying cooking objects, especially when cooking materials or user practices change, it is difficult to maintain high accuracy for all food ingredients, and the identification results are inconsistent due to user diversified environment.
Through image analysis, the images captured by the camera are input into the trained artificial intelligence model, and the identification information of the object and the first accuracy information of the recognition degree are obtained, and based on the second accuracy information corresponding to the identification information The difference, updating the artificial intelligence model includes sending request update signals to the server, receiving and applying update information to improve recognition.
The model update is realized based on the real-time accuracy of the cooking objects frequently used by users, and the recognition accuracy and user satisfaction are improved, and the changes in cooking materials and user environment are adapted to.
Smart Images

Figure CN118973449A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an electronic device for analyzing an image including a cooking object using an artificial intelligence model and a control method thereof. More specifically, the present disclosure relates to an electronic device for determining whether to update an artificial intelligence model based on the recognition accuracy of a cooking object and a control method thereof. Background Art
[0002] The artificial intelligence model can be used to specify a cooking object included in an image and provide the user with information associated with the specified cooking object. If an input image is applied to the artificial intelligence model as input data, the artificial intelligence model can output various information associated with the cooking object included in the input image.
[0003] The artificial intelligence model can not only provide identification information that simply shows what cooking objects are included in the image, but can also provide users with additional information associated with the cooking objects, such as (but not limited to) cooking methods, preservation methods, preventive measures, etc.
[0004] However, since cooking materials or users' cooking practices may change over time, there may be a problem of decreased recognition accuracy if the AI model is not updated.
[0005] In addition, it may be difficult for the AI model to maintain high accuracy for all food ingredients. For example, in the case of bread, eggs, etc., the food ingredients can be recognized well, but in the case of atypical food ingredients (e.g., shark fin) or food ingredients that are difficult to classify (e.g., chicken breast, chicken thigh), it may be difficult to recognize.
[0006] Due to the diverse environments of users (nationality, race, eating habits, etc.), it may be difficult for the artificial intelligence model to provide all users with analysis results of cooking objects that meet the same accuracy. Summary of the invention
[0007] Technical issues
[0008] Provided are an electronic device and a control method thereof for performing updating of an artificial intelligence model based on a difference between first accuracy information obtained through an image and pre-stored second accuracy information.
[0009] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments.
[0010] According to one aspect of the present disclosure, an electronic device includes: a camera; and at least one processor configured to: use the camera to obtain an image captured by the camera; input the obtained image into a trained artificial intelligence model; use the artificial intelligence model to obtain identification information of an object in the image and first accuracy information indicating the recognition degree of the identification information; and update the artificial intelligence model based on the difference between the obtained first accuracy information and second accuracy information corresponding to the identification information, wherein the second accuracy information indicates a reference recognition degree associated with the identification information.
[0011] The second accuracy information may indicate a reference recognition used in a training process of the updated artificial intelligence model.
[0012] The electronic device may also include: at least one memory configured to store the artificial intelligence model; and a communication interface configured to communicate with a server, wherein the at least one processor may also be configured to: based on the difference between the first accuracy information and the second accuracy information being greater than or equal to a threshold, control the communication interface to send a signal to the server requesting update information of the artificial intelligence model; and based on receiving the update information from the server through the communication interface, update the artificial intelligence model based on the received update information.
[0013] The electronic device may also include: a display, wherein at least one processor may also be configured to: control the display to display a user interface (UI) guiding the update of the artificial intelligence model based on the difference being greater than or equal to a threshold; and control the communication interface to send a signal to the server requesting the update of the artificial intelligence model based on user input received through the displayed UI.
[0014] The UI may include at least one of identification information, first accuracy information, second accuracy information, a difference value, or information asking for updating the artificial intelligence model.
[0015] At least one processor can also be configured to: based on obtaining the identification information and first accuracy information of the object, control the communication interface to send a signal to the server requesting second accuracy information corresponding to the identification information; and based on receiving the second accuracy information corresponding to the identification information from the server through the communication interface, store the received second accuracy information in at least one memory.
[0016] At least one processor can also be configured to: based on the difference being greater than or equal to a threshold, control the communication interface to send the obtained image to the server to train the artificial intelligence model stored in the server.
[0017] At least one processor may also be configured to: control the communication interface to send a signal to the server requesting first update information corresponding to the identification information of the artificial intelligence model based on the identification information of the object obtained based on multiple captured images obtained from the camera being counted greater than or equal to a first threshold number of times; and update the artificial intelligence model based on the received first update information based on the first update information corresponding to the identification information received from the server through the communication interface.
[0018] At least one processor can also be configured to: control the communication interface to send a signal to the server requesting second update information corresponding to the identification information of the artificial intelligence model based on the identification information of the object obtained based on at least one captured image obtained from the camera being counted less than a second threshold number of times; and update the artificial intelligence model based on the received second update information based on the second update information corresponding to the identification information received from the server through the communication interface.
[0019] The electronic device may also include: a chamber, in which an object is arranged, wherein the at least one processor may also be configured to: obtain a first image of the interior of the chamber captured from a first viewpoint using a camera; obtain a second image of the interior of the chamber captured from a second viewpoint different from the first viewpoint using the camera; and obtain an average of a first accuracy value corresponding to the first image of the object and a second accuracy value corresponding to the second image of the object as the first accuracy information.
[0020] According to one aspect of the present disclosure, a control method for an electronic device includes: obtaining an image; inputting the obtained image into a trained artificial intelligence model; using the artificial intelligence model to obtain identification information of an object in the image and first accuracy information indicating the recognition degree of the identification information; and updating the artificial intelligence model based on the difference between the obtained first accuracy information and second accuracy information corresponding to the identification information, wherein the second accuracy information indicates a reference recognition degree associated with the identification information.
[0021] The second accuracy information may indicate a reference recognition used in a training process of the updated artificial intelligence model.
[0022] The control method may also include: sending a signal to the server requesting update information of the artificial intelligence model based on the difference between the first accuracy information and the second accuracy information being greater than or equal to a threshold, wherein updating the artificial intelligence model may include: updating the artificial intelligence model based on the update information received from the server.
[0023] The control method may further include: displaying a user interface (UI) guiding the update of the artificial intelligence model based on the difference being greater than or equal to a threshold; and sending a signal to the server requesting the update of the artificial intelligence model based on receiving user input through the displayed UI.
[0024] The UI may include at least one of identification information, first accuracy information, second accuracy information, a difference value, or information asking for updating the artificial intelligence model. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:
[0026] Figure 1 is a block diagram showing an electronic device;
[0027] Figure 2 It shows in detail Figure 1 A block diagram of an electronic device;
[0028] Figure 3 is a perspective view showing an electronic device;
[0029] Figure 4 is a diagram showing an internal configuration of a cooking cavity for cooking a cooking object;
[0030] Figure 5 is a diagram showing a system including an electronic device, a server, a terminal device, etc.;
[0031] Figure 6 is a diagram showing an operation of updating an artificial intelligence model based on an analysis result of a cooking object;
[0032] Figure 7 is a diagram showing input data and output data of an artificial intelligence model;
[0033] Figure 8 is a diagram showing identification information;
[0034] Fig. 9 is a diagram showing accuracy information obtained from an electronic device and accuracy information obtained from a server;
[0035] Fig.10 is a flow chart illustrating operations for updating an artificial intelligence model based on accuracy information;
[0036] Fig.11 is a flowchart illustrating an operation of updating an artificial intelligence model taking into account the number of times identification information is counted;
[0037] Fig.12is a diagram showing information on a cooking object consisting of an analysis result of the cooking object and a plurality of depths;
[0038] Fig.13 is a flow chart showing the operation of receiving update information from a server;
[0039] Fig.14 is a flow chart showing the operations of various modules included in the electronic device;
[0040] Fig.15 is a flowchart showing an operation of an electronic device displaying a UI guiding an update;
[0041] Fig.16 is a flowchart showing an operation of a terminal device displaying a UI guiding an update;
[0042] Fig.17 is a diagram showing a UI for guiding updates;
[0043] Fig.18 is a flow chart illustrating an operation of receiving second accuracy information from a server;
[0044] Fig.19 is a flow chart showing the operation of sending an image to a server;
[0045] Fig. 20 is a diagram showing an operation of obtaining user feedback;
[0046] Fig.21 is a flow chart showing the operation of sending a signal to a server requesting an image and an update;
[0047] Fig. 22 is a flowchart showing an operation of performing an update by counting identification information;
[0048] Fig.23 is a diagram showing an operation of displaying a UI guiding an update when identification information is counted greater than or equal to a first threshold number of times;
[0049] Fig.24 is a diagram showing an operation of displaying a UI guiding an update when identification information is counted less than a second threshold number of times;
[0050] Fig.25 is a flow chart showing the operation of updating an artificial intelligence model when identification information is preliminarily recognized; and
[0051] Fig.26 is a flowchart illustrating a control method of an electronic device. DETAILED DESCRIPTION
[0052] The present disclosure will be described in detail below with reference to the accompanying drawings.
[0053] The terms used when describing the various embodiments of the present disclosure are currently widely used general terms selected in consideration of their functions in this article. However, the terms may vary according to the intentions of technicians in the relevant field, legal or technical interpretations, the emergence of new technologies, etc. In addition, in some cases, there may be arbitrarily selected terms, and in this case, the meaning of the terms will be disclosed in more detail in the corresponding description. Therefore, the terms used in this article should not be simply understood as their names, but should be understood based on the meaning of the terms and the overall context of the present disclosure.
[0054] In the present disclosure, expressions such as “having”, “may have”, “including”, “may include”, etc. are used to indicate the existence of corresponding features (for example, elements such as values, functions, operations or components), and do not exclude the existence or possibility of additional features.
[0055] The expression at least one of A and / or B should be understood to mean any one of “A” or “B” or “A and B”.
[0056] As used herein, expressions such as "first", "second", "first", "second", etc. may be used to refer to various elements, regardless of order and / or importance. In addition, it should be noted that the expressions are only used to distinguish one element from another element, and are not used to limit the relevant elements.
[0057] When an element (for example, a first element) is indicated as being “(operably or communicatively) coupled to” or “connected to” another element (for example, a second element), it can be understood that the element is directly coupled to the other element or coupled through other elements (for example, a third element).
[0058] Unless otherwise specified, singular expressions include plural expressions. It should be understood that terms such as "consisting of" or "comprising" are used herein to indicate the presence of features, quantities, steps, operations, elements, components or combinations thereof, without excluding the possibility of the presence or addition of one or more other features, quantities, steps, operations, elements, components or combinations thereof.
[0059] The term "module" or "component" used in the embodiments herein performs at least one function or operation, and can be implemented by hardware or software, or by a combination of hardware and software. In addition, in addition to the "module" or "component" that needs to be implemented as specific hardware, multiple "modules" or multiple "components" can be integrated into at least one module and implemented in at least one processor.
[0060] In the present disclosure, the term "user" may refer to a person using an electronic device or an apparatus (eg, an artificial intelligence electronic device) using an electronic device.
[0061] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.
[0062] Figure 1 is a block diagram showing the electronic device 100 .
[0063] Reference Figure 1 , the electronic device 100 may include at least one of a camera 110 or a processor 120 .
[0064] The electronic device 100 may be a data processing device that stores an artificial intelligence model for analyzing an image. Here, the electronic device 100 may store the artificial intelligence model itself.
[0065] The electronic device 100 may be a cooking device for cooking a cooking object, for example, a cooking device may be an oven, an electric oven, a multi-function oven, an electric stove, a microwave oven, etc.
[0066] The electronic device 100 according to various embodiments of the present disclosure may include, for example, at least one of a smart phone, a tablet personal computer (PC), a mobile phone, a desktop PC, a laptop PC, a personal digital assistant (PDA), and a portable multimedia player (PMP). In some embodiments, the electronic device 100 may include, for example, at least one of a television, a digital video disk (DVD) player, and a media box (e.g., HomeSyncTM, Apple TVTM, or Google TVTM, etc.).
[0067] The camera 110 may be a configuration for capturing a subject and generating a captured image, and here, the captured image may be a concept including both a moving image and a still image. The camera 110 may obtain an image of at least one external device and may be implemented as a camera, a lens, an infrared sensor, etc.
[0068] The camera 110 may include a lens and an image sensor. The type of the lens may include a typical general lens, a wide-angle lens, a zoom lens, etc., and may be determined according to the type, characteristics, use environment, etc. of the electronic device 100. For the image sensor, a complementary metal oxide semiconductor (CMOS), a charge coupled device (CCD), etc. may be used.
[0069] The camera 110 may output incident light as an image signal. Specifically, the camera 110 may include a lens, a pixel, and an AD converter. The lens may collect light of a subject and focus an optical image in an imaging area, and the pixel may output light incident through the lens as an image signal in analog form. Then, the AD converter may convert and output the image signal in analog form into an image signal in digital form.
[0070] The camera 110 may generate a captured image by capturing a cooking object located inside the electronic device 100 .
[0071] In describing the electronic device 100 according to an embodiment of the present disclosure, the camera 110 has been described as one, but a plurality of cameras may be provided in actual implementation.
[0072] The at least one processor 120 may perform overall control operations of the electronic device 100. Specifically, the processor 120 may play a role in controlling overall operations of the electronic device 100. The at least one processor 120 is described as the processor 120 below.
[0073] The processor 120 may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) for processing digital signals. However, the embodiment is not limited thereto, and may include, for example, (but not limited to) one or more of a central processing unit (CPU), a microcontroller unit (MCU), a microprocessing unit (MPU), a controller, an application processor (AP), a graphics processing unit (GPU), or a communication processor (CP), an advanced reduced instruction set computer (RISC) machine (ARM) processor, or may be defined by corresponding terms. In addition, the processor 120 may be implemented as a system on chip (SoC) or a large-scale integration (LSI) embedded with a processing algorithm, and may be implemented in the form of a field programmable gate array (FPGA). In addition, the processor 120 may perform various functions by executing computer executable instructions stored in a memory.
[0074] The processor 120 can obtain the captured image from the camera 110, obtain identification information of the cooking object included in the image and first accuracy information showing the recognition degree (recognition level) of the identification information by inputting the obtained image into the trained artificial intelligence model, and update the artificial intelligence model based on the difference between the obtained first accuracy information and the second accuracy information corresponding to the identification information, and the second accuracy information can be information showing a reference recognition degree associated with the identification information.
[0075] Here, the processor 120 may obtain an image. Then, the processor 120 may perform an analysis operation based on the obtained image. Specifically, the processor 120 may input (or apply) the image as input data into the artificial intelligence model. Here, the processor 120 may obtain identification information of a cooking object included in the image (e.g., salmon fillet) and first accuracy information (91.6%) corresponding to the identification information as output data.
[0076] According to an embodiment, the processor 120 may directly obtain an image through the camera 110 included in the electronic device 100 .
[0077] According to another embodiment, the processor 120 may obtain an image capturing a cooking object through an external device, etc. That is, the electronic device 100 may not include the camera 110, and the image input (or applied) to the artificial intelligence model may be obtained through an external device, etc.
[0078] Here, the artificial intelligence model may receive an input of an image and output an analysis result of a cooking object included in the image. The analysis result may include identification information and first accuracy information. Figure 7 Describes the analytical operations performed by the artificial intelligence model.
[0079] Here, the first accuracy information may refer to the recognition degree or recognition reliability of determining the cooking object included in the image as identification information (eg, salmon fillet). The accuracy information may be described as accuracy, accuracy value, recognition, recognition value, recognition rate, etc.
[0080] Here, the identification information may refer to information for specifying a cooking object. The identification information may include at least one of a unique number, a name, and a representative image. Figure 8 Description associated with the identification information.
[0081] Here, when obtaining the identification information of the cooking object included in the image, the processor 120 may obtain second accuracy information different from the first accuracy information. Here, the second accuracy information may be a reference value compared with the first accuracy information. In addition, the second accuracy information may vary according to the identification information (or the cooking object). For example, the second accuracy information corresponding to a salmon fillet may be different from the second accuracy information corresponding to half a potato.
[0082] The second accuracy information may be described as a reference recognition, a reference accuracy, a comparable reference value, or the like.
[0083] According to an embodiment, the second accuracy information may be information pre-stored in the electronic device 100. The processor 120 may store the second accuracy information in the memory 130. Then, the processor 120 may search (or identify) the second accuracy information according to a specific event (an event in which specific identification information is obtained).
[0084] According to another embodiment, the second accuracy information may be information received through an external device (e.g., server 200). The processor 120 may request the second accuracy information from the external device according to a specific event (e.g., an event of obtaining specific identification information). Then, the processor 120 may receive the second accuracy information from the external device.
[0085] The second accuracy information may be information showing a reference recognition degree used in a learning process of the updated artificial intelligence model.
[0086] According to an embodiment, the second accuracy information may display the reference recognition used in the artificial intelligence model after the pre-update. There may be a new second artificial intelligence model (e.g., an updated model) that is different from the first artificial intelligence model (e.g., the model before the update) being used in the electronic device 100. Here, the processor 120 may store the second accuracy information used in the new second artificial intelligence model. Then, the processor 120 may obtain the first accuracy information by using the first artificial intelligence model currently being used. In addition, the processor 120 may obtain the second accuracy information used in the second artificial intelligence model that has been updated. Then, the processor 120 may determine whether to update the first artificial intelligence model to the second artificial intelligence model by comparing the first accuracy information with the second accuracy information. If the difference obtained by subtracting the first accuracy information from the second accuracy information is greater than or equal to the threshold, the processor 120 may update the first artificial intelligence model to the second artificial intelligence model. Here, the second artificial intelligence model has been described as having been stored in the electronic device 100. According to another embodiment, the electronic device 100 may store only the second accuracy information, and the server 200 may store all information associated with the second artificial intelligence model. In this case, the processor 120 may request the second artificial intelligence model from the server 200.
[0087] According to another embodiment, the second accuracy information may be an average value of the accuracy values of the cooking object obtained from a plurality of electronic devices (e.g., user terminal devices) respectively. For example, the second accuracy information may be an average value of the accuracy value of the cooking object obtained from a first electronic device among the plurality of electronic devices and an average value of the accuracy value of the cooking object obtained from a second electronic device. The plurality of electronic devices may refer to cooking devices used by a plurality of users respectively. Here, the second accuracy information may be information generated based on data received from a plurality of users. For example, the average accuracy information of a plurality of users may be calculated based on usage history information of the plurality of users. The second accuracy information may refer to the average accuracy of the plurality of users associated with specific identification information.
[0088] The electronic device 100 may also include a communication interface 140 that communicates with the memory 130 storing the trained artificial intelligence model and the server 200, and at least one processor 120 may send a signal requesting update information of the artificial intelligence model to the server 200 through the communication interface 140 based on the difference between the first accuracy information and the second accuracy information being greater than or equal to a threshold, and based on receiving the update information from the server 200, update the artificial intelligence model based on the received update information.
[0089] The electronic device 100 may store the trained artificial intelligence model in the memory 130. Then, when a specific event (e.g., an image analysis event) is recognized, the processor 120 may perform an image analysis operation by using the artificial intelligence model stored in the memory 130. Here, the processor 120 may determine whether to update the artificial intelligence model based on the result of the image analysis operation.
[0090] Specifically, the processor 120 may determine whether to update the artificial intelligence model based on the difference between the first accuracy information and the second accuracy information. Fig.10 The operation of determining whether to update based on the difference value is described in.
[0091] Here, the processor 120 may obtain update information for updating. The update information may refer to information necessary for updating the artificial intelligence model. Here, the update operation may include the meaning of changing, modifying, replacing, updating, generating, etc. of the artificial intelligence model.
[0092] The scope of information referred to by update information may vary.
[0093] According to an embodiment, the update information may refer to the entire artificial intelligence model. The processor 120 may delete the existing artificial intelligence model by obtaining the update information and install a new artificial intelligence model.
[0094] According to another embodiment, the update information may refer to information used in identifying a specific cooking object. The processor 120 may modify a portion of data of an existing artificial intelligence model into new data by obtaining the update information.
[0095] According to still another embodiment, the update information may refer to information corresponding to a specific depth among information used in recognizing the cooking object.
[0096] like Fig.12 The cooking objects in the table 1220 in FIG. 1204 may be divided into a plurality of depths. Each depth may refer to a category used in dividing the cooking objects.
[0097] The artificial intelligence model can identify the cooking object as one of the items belonging to the first depth (e.g., seafood, meat, vegetables, etc.) based on information corresponding to the first depth having the widest range.
[0098] In addition, the artificial intelligence model can identify the cooking object as one of the items belonging to the second depth (for example, if the first depth is seafood, the second depth can be salmon, shrimp, etc.) based on information corresponding to the second depth that is narrower in range than the first depth.
[0099] In addition, the artificial intelligence model can identify the cooking object as one of the items belonging to the third depth (for example, if the second depth is salmon, the third depth can be salmon fillet, smoked salmon, etc.) based on information corresponding to the third depth which is narrower in range than the second depth.
[0100] The update information may include information corresponding to at least one depth among the plurality of depths. Here, which depth information among the plurality of depths is being obtained as the update information may vary according to user settings.
[0101] The electronic device 100 may also include a display 150, and at least one processor 120 may control the display 150 to display a user interface (UI) guiding the update of the artificial intelligence model based on the difference being greater than or equal to a threshold, and based on receiving user input through the displayed UI, send a signal requesting the update of the artificial intelligence model to the server 200 through the communication interface 140.
[0102] Here, the user input (or user feedback) may be directly received from the electronic apparatus 100. According to another embodiment, the user input (or user feedback) may be received by the terminal device 300 and transmitted to the electronic apparatus 100.
[0103] Here, if the difference between the first accuracy information and the second accuracy information is greater than or equal to the threshold, the processor 120 may provide the user with a screen including a UI for guiding updating of the artificial intelligence model.
[0104] According to an embodiment, a UI may be provided to a user through the electronic device 100. Fig.15 In addition to the UI for guiding the update, various UIs for user input or user feedback (e.g., Fig.17 UI 1700, Fig. 20 UI 2000, Fig.23 UI 2300 and Fig.24 UI 2400).
[0105] According to another embodiment, a UI may be provided to the user through the terminal device 300. Fig.16 In addition to the UI for guiding the update, various UIs for user input or user feedback (e.g., Fig.17 UI 1700, Fig. 20 UI 2000, Fig.23 UI 2300 and Fig.24 UI2400).
[0106] The UI may include at least one of identification information, first accuracy information, second accuracy information, a difference value, or information asking for updating the artificial intelligence model.
[0107] Will be in Fig.17 A screen including a UI guiding the update is described in. A description associated with the identification information, the first accuracy information, and the second accuracy information may be included in Fig.17 The information for inquiring about updating the artificial intelligence model may refer to at least one of information 1730 for guiding updating or information 1740 for receiving user input.
[0108] The electronic device 100 may further include a memory 130, and the at least one processor 120 may send a signal requesting second accuracy information corresponding to the identification information to the server 200 through the communication interface 140 based on obtaining the identification information and the first accuracy information of the cooking object, and store the received second accuracy information in the memory 130 based on receiving the second accuracy information corresponding to the identification information from the server 200. Fig.18 The description associated with it is described in.
[0109] At least one processor 120 may send the obtained image to the server 200 through the communication interface 140 for the obtained image to be used in training the artificial intelligence model stored in the server 200 based on the difference being greater than or equal to the threshold.
[0110] Here, the obtained image may be included in the learning data.
[0111] According to an embodiment, the processor 120 may send learning data to the server 200. The server 200 may perform a training (or retraining) operation based on the learning data received from the electronic device 100. Here, the learning data may include at least one of an image, identification information, or user feedback. Fig.17 The description associated with it is described in.
[0112] According to another embodiment, the processor 120 can use the learning data to retrain the artificial intelligence model by itself. Here, the retraining operation may include an operation of changing (or adjusting) the weight value.
[0113] According to another embodiment, the processor 120 may send learning data to an edge device. The edge device may be a device that provides edge computing. Edge computing may refer to the operation of processing data generated from a terminal device based on a specific purpose. The edge device may be disposed at the periphery of the electronic device 100, and may perform data analysis functions, data processing functions, and the like. For example, the edge device may use learning data to retrain an artificial intelligence model. Here, the edge device may be described as an edge node or an edge device. The edge device may exhibit lower performance than the server 200, but may easily send and receive data to and from the electronic device 100. The edge device may be implemented as a locally based network device, such as a gateway, a router, and the like.
[0114] The processor 120 may perform the image analysis operation multiple times. Then, the processor 120 may count the identification information obtained each time the image analysis operation is performed. Then, whether to update may be determined based on the number of times the identification information is counted.
[0115] The counting operation may count the number of events in which specific identification information is recognized by the artificial intelligence model. Here, the counting operation may count the number of events in which a user input is received selecting specific identification information, even if the event is not recognized by the artificial intelligence model. For example, the counting operation may refer to counting the identification information with the highest accuracy or the identification information directly selected by the user (via the UI) by the artificial intelligence model.
[0116] According to an embodiment, if the number of times the specific identification information is counted is greater than or equal to the first threshold number of times, the processor 120 may determine that the user frequently uses the specific identification information. If the device of the user who frequently uses the specific identification information is updated to better recognize the specific identification information, the user satisfaction may become higher.
[0117] Specifically, the at least one processor 120 may send a signal requesting first update information corresponding to the identification information of the artificial intelligence model to the server 200 through the communication interface 140 based on the identification information of the cooking object obtained according to the plurality of captured images obtained from the camera 110 being counted to be greater than or equal to the first threshold number of times, and based on receiving the first update information corresponding to the identification information from the server 200, update the artificial intelligence model based on the received first update information. Fig. 22 and Fig.23 The description associated with it is described in.
[0118] According to another embodiment, the processor 120 can identify that the user does not frequently use the specific identification information based on the number of times the specific identification information is counted being less than the second threshold number of times. If the device of the user who does not frequently use the specific identification information is updated to accurately identify the specific identification information in one attempt, the user can have a sense of satisfaction in the image analysis function.
[0119] Specifically, at least one processor 120 may send a signal requesting second update information corresponding to the identification information of the artificial intelligence model to the server 200 through the communication interface 140 based on the identification information of the cooking object obtained according to at least one captured image obtained from the camera 110 being counted less than the second threshold number of times, and based on receiving the second update information corresponding to the identification information from the server 200, update the artificial intelligence model based on the received second update information. Fig. 22 and Fig.24 The description associated with it is described in.
[0120] The electronic device 100 may also include a cooking cavity for accommodating a cooking object, and the at least one processor 120 may obtain a first image of the interior of the cooking cavity captured from a first viewpoint from the camera 110, obtain a second image of the interior of the cooking cavity captured from a second viewpoint different from the first viewpoint from the camera 110, and obtain an average of a first accuracy value of the cooking object corresponding to the first image and a second accuracy value of the cooking object corresponding to the second image as the first accuracy information.
[0121] Will be in Figure 3 The description associated with the cooking chamber is described in Figure 4 The capturing operation by the camera 110 is described in FIG.
[0122] Here, the processor 120 may capture the same cooking object from different viewpoints. The processor 120 may obtain analysis results based on the images captured at different viewpoints. Then, the processor 120 may obtain and store an average value of the accuracy values obtained at different viewpoints as the first accuracy information.
[0123] The electronic device 100 may be a device that performs image analysis using an artificial intelligence model, and may determine whether to update the artificial intelligence model based on the difference between the accuracy value (first accuracy information) obtained through the image and the pre-stored accuracy value (second accuracy information). Therefore, the electronic device 100 may perform an update operation based on the real-time accuracy of cooking objects frequently used by the user. In addition, the user may experience a higher satisfaction than when updating at a preset cycle.
[0124] In the above, only a simple configuration constituting the electronic device 100 is shown and described, but various configurations may be additionally provided in implementation. Figure 2 Description The description associated with it.
[0125] Figure 2 It shows Figure 1 1 is a block diagram of a detailed configuration of the electronic device 100.
[0126] Reference Figure 2 , the electronic device 100 may include a camera 110 , a processor 120 , a memory 130 , a communication interface 140 , a display 150 , an operation interface 160 , an input and output interface 170 , a speaker 180 , and a microphone 190 .
[0127] For operations identical to those previously described among operations of the camera 110 and the processor 120 , redundant descriptions will be omitted.
[0128] The memory 130 may be implemented as an internal memory included in the processor 120 (e.g., but not limited to, a read-only memory (ROM; e.g., an electrically erasable programmable read-only memory (EEPROM)), a random access memory (RAM), etc.), or as a memory separate from the processor 120. In this case, the memory 130 may be implemented in the form of a memory embedded in the electronic device 100, or in the form of a memory attachable to or detachable from the electronic device 100 according to data storage purposes. For example, data for driving the electronic device 100 may be stored in a memory embedded in the electronic device 100, and data for extended functions of the electronic device 100 may be stored in a memory attachable to or detachable from the electronic device 100.
[0129] The memory embedded in the electronic device 100 can be implemented as at least one of a volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)) or a non-volatile memory (e.g., one-time programmable ROM (OTPROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash memory or NOR flash memory), hard disk drive (HDD), or solid state drive (SSD)), and in the case of a memory that is connectable to and removable from the electronic device 100, the memory can be implemented in the form of, for example, (but not limited to) a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (micro-SD), mini secure digital (mini-SD), extreme digital (xD), multimedia card (MMC)), etc.), an external memory connectable to a USB port (e.g., USB memory), etc.
[0130] The memory 130 may store at least one instruction. The processor 120 may perform various operations based on the instruction stored in the memory 130.
[0131] The communication interface 140 may be a configuration that performs communication with various types of external devices according to various types of communication methods. The communication interface 140 may include a wireless communication module or a wired communication module. Here, each communication module may be implemented in the form of at least one hardware chip.
[0132] The wireless communication module may be a module for communicating with an external device by wireless means. For example, the wireless communication module may include at least one module among a Wi-Fi module, a Bluetooth module, an infrared module or other communication modules.
[0133] The Wi-Fi module and the Bluetooth module can perform communication in the Wi-Fi method and the Bluetooth method, respectively. When using the Wi-Fi module or the Bluetooth module, various connection information (such as (but not limited to) service set identifier (SSID), session key, etc.) can be sent and received first, and after being connected to communicate using the various connection information, various information can be sent and received.
[0134] The infrared communication module may perform communication according to an Infrared Data Association (IrDA) technology for wirelessly transmitting data over a short distance using infrared rays existing between visible rays and millimeter waves.
[0135] Other communication modules may include at least one communication chip for performing communication according to various wireless communication standards other than the above-mentioned communication methods (for example, but not limited to, ZigBee, third generation (3G), third generation partnership project (3GPP), long term evolution (LTE), advanced LTE (LTE-A), fourth generation (4G), fifth generation (5G), etc.).
[0136] The wired communication module may be a module that communicates with an external device by wired means. For example, the wired communication module may include at least one of a local area network (LAN) module, an Ethernet module, a twisted pair cable, a coaxial cable, an optical fiber cable, or an ultra-wideband (UWB) module.
[0137] The display 150 may be implemented as various types of displays, such as (but not limited to) a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. In the display 150, a driving circuit, a backlight unit, etc. may be included, and the driving circuit may be implemented in the form of an amorphous silicon thin film transistor (a-si TFT), a low temperature polycrystalline silicon (LTPS) TFT, an organic TFT (OTFT), etc. The display 150 may be implemented as a touch screen coupled to a touch sensor, a flexible display, a three-dimensional display (3D display), etc. In addition, according to an embodiment of the present disclosure, the display 150 may include not only a display panel that outputs an image, but also a frame that accommodates the display panel. Specifically, according to an embodiment of the present disclosure, the frame may include a touch sensor for sensing user interaction.
[0138] The operation interface 160 may be implemented as a device such as a button, a touch panel, a mouse, and a keyboard, or may be implemented as a touch screen capable of performing the above-mentioned display function and operation input function together therewith. Here, the button may be various types of buttons formed at random areas of the front surface part or the side surface part, the rear surface part, etc., of the outside of the main body of the electronic device 100, for example, a mechanical button, a touch panel, or a scroll wheel.
[0139] The input and output interface 170 may be any one of a high-definition multimedia interface (HDMI), a mobile high-definition link (MHL), a universal serial bus (USB), a display port (DP), a thunderbolt port (Thunderbolt), a video graphics array (VGA) port, an RGB port, a D-sub (D-SUB), or a digital video interface (DVI). The input and output interface 170 may input and output at least one of an audio signal and a video signal. According to an embodiment, the input and output interface 170 may include a port for inputting and outputting only an audio signal and a port for inputting and outputting only a video signal as separate ports, or may be implemented with one port for inputting and outputting both an audio signal and a video signal. The electronic device 100 may send at least one of an audio signal and a video signal to an external device (e.g., an external display device or an external speaker) through the input and output interface 170. Specifically, the output port included in the input and output interface 170 may be connected to an external device, and the electronic device 100 may send at least one of an audio signal and a video signal to the external device through the output port.
[0140] Here, the input and output interface 170 may be connected to the communication interface. The input and output interface 170 may transmit information received from an external device to the communication interface, or transmit information received through the communication interface to the external device.
[0141] The speaker 180 may be an element that outputs not only various audio data but also various notification sounds, voice messages, and the like.
[0142] The microphone 190 may be a configuration for receiving an input of a user voice or other sound and converting it into audio data. The microphone 190 may receive a user voice in an activated state. For example, the microphone 190 may be formed on the upper side of the electronic device 100, or may be integrally formed in the front surface direction, the side direction, etc. The microphone 190 may include various configurations, such as (but not limited to) a microphone for collecting a user voice in an analog form, an amplifying circuit for amplifying the collected user voice, an A / D converter circuit for sampling the amplified user voice and converting it into a digital signal, a filter circuit for removing noise components from the converted digital signal, etc.
[0143] Figure 3 is a perspective view showing the electronic device 100 .
[0144] Reference Figure 3 , the electronic device 100 is merely one example, and according to various embodiments of the present disclosure, the electronic device may be implemented in various forms.
[0145] like Figure 3 As shown, the electronic device 100 may include a body 10 forming an exterior.
[0146] In addition, the electronic device 100 may include a cooking cavity 20 opened toward one side. Here, the cooking cavity 20 may refer to a space (i.e., a receiving space) for receiving a cooking object (or a food object). The cooking cavity 20 of the main body 10 may be a cooking cavity 20 for cooking a cooking object, and the cooking cavity 20 may be opened toward the front of the electronic device 100.
[0147] The cooking cavity 20 may be formed in a box-like shape, and the front surface may be opened for putting in or taking out the cooking object. To this end, the front surface of the body 10 may include an opening connected to the cooking cavity 20.
[0148] In this case, the front surface of the cooking cavity 20 can be opened and closed by a door 21 connected to the body 10. To this end, the door 21 may be hinge-coupled to the lower portion of the body 10 so as to be rotatable around the body 10. In addition, at the upper front surface of the door 21, a door handle 23 may be provided for the user to grasp, and the user may grasp the door handle 23 to open and close the cooking cavity 20.
[0149] In addition, a heater for heating the cooking object may be provided in the cooking cavity 20. In this case, the heater may be an electric heater including a resistor. However, the heater is not limited to the electric heater and may be a gas heater that generates heat by burning gas.
[0150] At an upper portion of the body 10, a control panel 30 may be provided. The control panel 30 may include a display 31 that displays various operation information of the electronic device 100 and receives an input of a user command for controlling the operation of the electronic device 100. In addition, the control panel 30 may include a button 32 that receives an input of various user commands for controlling the operation of the electronic device 100.
[0151] According to an embodiment of the present disclosure, the electronic device 100 may perform cooking of a cooking object in consideration of the size and cooking state of the cooking object inserted into the cooking cavity.
[0152] Specifically, the electronic device 100 may determine the cooking time of the cooking object according to the type of the cooking object selected by the user through the control panel 30. At this time, the electronic device 100 may determine the size of the cooking object and determine the cooking time according to the determined size. For example, even if the cooking objects are the same type of cooking objects, the cooking time may be determined to be relatively shorter when the size of the cooking object is smaller, and the cooking time may be determined to be relatively longer when the size of the cooking object is larger.
[0153] Then, the electronic device 100 may determine a cooking state of the cooking object while cooking of the cooking object is in progress, and control the progress of cooking based on the cooking state.
[0154] Specifically, the electronic device 100 may end cooking even if the set cooking time has not been completed based on the cooking of the cooking object being determined to be completed according to the cooking degree of the cooking object. For example, the electronic device 100 may determine the cooking degree of the cooking object, and when the cooking of the cooking object is determined to be completed according to the cooking degree of the cooking object, end cooking even if the cooking time has not been completed.
[0155] As described above, according to various embodiments of the present disclosure, the cooking time is not determined by considering only the type of the cooking object, but even if the cooking objects are the same type of cooking objects, different cooking times can be determined according to the sizes of the cooking objects, and the time when cooking will be completed can be determined according to the cooking state of the cooking objects while cooking is in progress. Therefore, even if the accurate cooking method is unknown due to the difference in the size of the cooking objects, etc., the user can optimally cook the cooking objects based on selecting only the type of cooking objects that the user desires to cook without monitoring the cooking progress.
[0156] Figure 4 is a diagram showing an internal configuration of a cooking cavity in which a cooking object is cooked.
[0157] Reference Figure 4, an embodiment in which the food object 410 exists inside the cooking cavity 20 of the electronic device 100 may be assumed.
[0158] The food object 410 may exist inside the container 405, and the container 405 may be placed above the shelf 22. Then, the camera 110 of the electronic device 100 may capture at least one of the shelf 22, the container 405, or the food object 410.
[0159] The viewing angle may vary according to the type of the camera 110 , but the camera 110 may be arranged to capture the entire area inside the cooking cavity 20 .
[0160] According to an embodiment, the camera 110 may be Figure 4 The camera 110 is attached to the wall surface of the cooking cavity 20 and is set to a tilted state toward the lower direction based on a specific angle. For example, the camera 110 can be set to a tilted state at a 45 degree angle toward the lower direction in a state of being attached to the wall surface of the cooking cavity 20.
[0161] According to another embodiment, the camera 110 may be attached to the upper plate (or ceiling) of the cooking cavity 20 and disposed toward the lower direction based on a specific angle. For example, the camera 110 may be disposed toward the lower direction while being attached to the upper plate of the cooking cavity 20.
[0162] Although Figure 4 The camera 110 in FIG. 1 is shown as being in a protruding state, but in actual implementation, the camera may be implemented in a form of being disposed inside a wall surface of the cooking cavity 20 or inside an upper plate of the cooking cavity 20 to protect the camera lens.
[0163] Figure 5 is a diagram showing a system 1000 including an electronic device 100 , a server 200 , a terminal apparatus 300 , and the like.
[0164] The system 1000 may include an electronic device 100, a server 200, a terminal device 300, or at least one of a plurality of electronic devices 100-b to 100-n.
[0165] Here, the electronic device 100 may perform communication with the server 200. Here, the server 200 may communicate even with a plurality of electronic devices 100-b to 100-n in addition to the electronic device 100. The server 200 may collect and analyze information of various devices by performing communication not only with the electronic device 100 but also with the plurality of electronic devices 100-b to 100-n.
[0166] The server 200 may perform communication with the terminal device 300. Here, the terminal device 300 may refer to a user device (eg, a smart phone, a smart watch, etc.).
[0167] The electronic device 100 or the terminal device 300 may display various information associated with the electronic device 100. For example, the electronic device 100 or the terminal device 300 may display a UI requesting user feedback or user input associated with an update. Whether to display the UI in the electronic device 100 or to display the UI in the terminal device 300 may be determined according to various embodiments.
[0168] Figure 6 is a diagram illustrating an operation of updating an artificial intelligence model based on an analysis result of a cooking object.
[0169] Reference Figure 6 , the electronic device 100 may obtain an image (S605). Here, the image may include a cooking object. The electronic device 100 may obtain an image including a cooking object. In an example, the electronic device 100 may obtain an image captured by a camera 110 provided in the electronic device 100. In another example, the electronic device 100 may receive an image including a cooking object from an external device (e.g., the server 200).
[0170] In addition, the electronic device 100 may analyze the cooking object by inputting the image into the terminal device 300 (S610). The electronic device 100 may analyze the cooking object included in the image through the artificial intelligence model. The electronic device 100 may obtain the image analysis result (or the cooking object analysis result) through the artificial intelligence model. Specifically, the electronic device 100 may input the image as input data into the artificial intelligence model, and obtain the identification information of the cooking object included in the image and the accuracy information corresponding to the identification information as output data.
[0171] In addition, the electronic device 100 may update the artificial intelligence model based on the image analysis result (cooking object analysis result) (S615). The electronic device 100 may obtain identification information showing (indicating) the cooking object and accuracy information corresponding to the identification information as the cooking object analysis result. The electronic device 100 may determine whether to update the artificial intelligence model based on the accuracy information. Fig.10 A detailed description associated therewith is described in.
[0172] Figure 7 is a diagram showing input data and output data of an artificial intelligence model.
[0173] Reference Figure 7 , the electronic device 100 may obtain the image 710. Then, the electronic device 100 may input the image 710 as input data into the artificial intelligence model. The artificial intelligence model may obtain the identification information and the accuracy information corresponding to the identification information as output data through the input layer, the hidden layer, the output layer, etc.
[0174] For example, as shown in Table 720, if the input image is a cooking object including salmon, the artificial intelligence model can receive an image including a cooking object (salmon) as input data and obtain output data. Here, the output data can be identification information (salmon fillet) and accuracy information (91.6%) corresponding to the identification information (salmon fillet). The output data may include multiple identification information. The electronic device 100 can obtain identification information (chicken) and accuracy information (8.2%), obtain identification information (steak) and accuracy information (0.5%), obtain identification information (bread) and accuracy information (0.4%), and identification information (pizza) and accuracy information (0.2%). The electronic device 100 can identify that the probability that the cooking object included in the image 710 as input data is a salmon fillet is 91.6%, the probability that the cooking object is a chicken is 8.2%, the probability that the cooking object is a steak is 0.5%, the probability that the cooking object is bread is 0.4%, and the probability that the cooking object is pizza is 0.2%.
[0175] Figure 8 is a diagram showing identification information.
[0176] Reference Figure 8 In table 810, the identification information may include at least one of a unique number, a name, and a representative image. Figure 7 In the description, the identification information has been described as a name described in text, but the actual identification information may include various information other than the name to specify the subject.
[0177] For example, the unique number of a salmon fillet may be #001, the unique number of a chicken may be #002, the unique number of a steak may be #003, the unique number of a bread may be #004, and the unique number of a pizza may be #005.
[0178] The electronic device 100 may specify the cooking object based on at least one of a unique number, a name, or a representative image. The electronic device 100 may store the analysis result associated with the image based on the identification information. In addition, the electronic device 100 may display so as to provide the identification information to the user.
[0179] Fig. 9 is a diagram showing accuracy information obtained from the electronic device 100 and accuracy information obtained from the server 200 .
[0180] Reference Fig. 9, the electronic device 100 may perform the analysis operation on the image including the cooking object multiple times. The electronic device 100 may recognize the images different from each other as shown in the analysis results 901, 902, and 903 as salmon fillets. In addition, the electronic device 100 may recognize the images different from each other as shown in the analysis results 911, 912, and 913 as half a potato.
[0181] The electronic device 100 may perform counting of identification information having the highest accuracy when obtaining the analysis result.
[0182] The electronic device 100 may obtain multiple analysis results 901, 902, 903, ..., 911, 912, and 913 based on multiple image analysis operations, and obtain a cooking object recognition result 920 by combining the obtained multiple analysis results. The cooking object recognition result may be described as an overall analysis result, an overall result, etc.
[0183] For example, the electronic device 100 can identify that a salmon fillet has been identified 15 times, and the accuracy average is 73.9%. In addition, the electronic device 100 can identify that half a potato has been identified 12 times, and the accuracy average is 83.2%. The cooking object recognition result 920 may store information that a specific cooking object (steak) has never been recognized. According to another embodiment, when in a state where information associated with a specific cooking object (steak) is not stored, the electronic device 100 may identify that the steak has not been recognized based on the cooking object recognition result. In the example, each of the analysis results 901, 902, 903, ..., 911, 912, and 913 may be described as first accuracy information. In another example, the cooking object recognition result 920 may be described as first accuracy information.
[0184] Here, the electronic device 100 may send the cooking object recognition result 920 to the server 200. The server 200 may store the reference information 930 based on the cooking object recognition results received from the plurality of electronic devices. The server 200 may obtain the reference information 930 based on the identification information and accuracy information of the cooking objects recognized from the plurality of electronic devices (plural users). Here, the reference information 930 may be described as the second accuracy information. The server 200 may send the reference information 930 to the electronic device 100.
[0185] Fig.10 is a flow chart illustrating the operation of updating an artificial intelligence model based on accuracy information.
[0186] Reference Fig.10 , the electronic device 100 may obtain an image (S1005). Here, the image may include a cooking object.
[0187] Here, the electronic device 100 may obtain identification information of a cooking object included in an image and first accuracy information corresponding to the identification information (S1010). Here, the first accuracy information may refer to a likelihood value that the cooking object will be recognized as the identification information.
[0188] Here, the electronic device 100 may obtain second accuracy information corresponding to the identification information (S1015). The second accuracy information may show a reference recognition degree of a specific cooking object (specific identification information).
[0189] Here, the electronic device 100 may obtain a difference between the first accuracy information and the second accuracy information (S1020). The electronic device 100 may obtain a value obtained by subtracting the first accuracy information from the second accuracy information as the difference.
[0190] Here, the electronic device 100 may identify whether the difference value obtained from step S1020 is greater than or equal to a threshold value (S1025). If the difference value is greater than or equal to the threshold value (S1025-Yes), the electronic device 100 may update the artificial intelligence model (S1030). If the difference value is less than the threshold value (S1025-No), the electronic device 100 may obtain a new image and perform steps S1005 to S1025.
[0191] For example, if the difference value is a value (eg, 11.2%) greater than a threshold value (eg, 10%), the electronic device 100 may update the artificial intelligence model.
[0192] according to Fig.10 In the embodiment described in which the first accuracy information is subtracted from the second accuracy information, the smaller the difference is, the higher the model applicability may be.
[0193] According to another embodiment, the electronic device 100 may obtain the difference value according to a different calculation method in step S1020. The electronic device 100 may obtain a value obtained by subtracting the second accuracy information from the first accuracy information as the difference value. Here, if the difference value is less than the threshold value, the electronic device 100 may update the artificial intelligence model. According to an embodiment of subtracting the second accuracy information from the first accuracy information, the greater the difference value, the higher the model applicability may be.
[0194] Fig.11 is a flowchart showing an operation of updating an artificial intelligence model in consideration of the number of times identification information is counted.
[0195] Reference Fig.11 , steps S1105, S1110, S1115, S1120, S1125 and S1130 may correspond to Fig.10Therefore, redundant description will be omitted.
[0196] After obtaining the identification information of the cooking object included in the image and the first accuracy information (after step S1110), the electronic device 100 may identify whether the number of times the identification information is counted is greater than or equal to a threshold number of times (S1111). The electronic device 100 may specify the identification information of the cooking object included in the image based on the artificial intelligence model, and obtain the cumulative number of times the specified identification information is counted.
[0197] For example, refer to Fig.12 As shown in table 1210 in FIG. 1 , the cumulative count times may be stored for each identification information in the electronic device 100. For example, the number of times a salmon fillet is recognized may be 15 times, the number of times a half potato is recognized may be 12 times, and the number of times a pumpkin pie is recognized may be 1 time.
[0198] If the number of times the identification information is counted is greater than or equal to the threshold number of times ( S1111 —No), the electronic device 100 may obtain second accuracy information corresponding to the identification information ( S1115 ).
[0199] If the number of times the identification information is counted is less than the threshold number (S1111-No), the electronic device 100 may obtain a new image and perform steps S1105 and S1110.
[0200] According to another embodiment, step S1111 may be performed after step S1115 , step S1120 , or step S1125 .
[0201] Fig.12 is a diagram showing information on a cooking object consisting of an analysis result of the cooking object and a plurality of depths.
[0202] Fig.12 Table 1210 in the figure may include a cooking object analysis result. Here, the cooking object analysis result may include a model suitability based on a difference between the first accuracy information and the second accuracy information. Model suitability may refer to a degree showing how accurately an artificial intelligence model currently installed (stored) in the electronic device recognizes a cooking object.
[0203] If an image including a salmon fillet is obtained, the electronic device 100 can use the artificial intelligence model and recognize that the identification information is a salmon fillet, the first accuracy information is 73.9%, and the second accuracy information is 85.1%. Here, the electronic device 100 can identify the difference (11.2%) by subtracting the first accuracy information from the second accuracy information. Since the difference (11.2%) is greater than or equal to the threshold value (e.g., 10%), the electronic device 100 can determine that the applicability of the artificial intelligence model is low.
[0204] If an image including half a potato is obtained, the electronic device 100 can use the artificial intelligence model and recognize that the identification information is half a potato, the first accuracy information is 83.2%, and the second accuracy information is 75.5%. Here, the electronic device 100 can identify the difference (-7.7%) by subtracting the first accuracy information from the second accuracy information. Since the difference (-7.7%) is less than the threshold value (e.g., 10%), the electronic device 100 can determine that the applicability of the artificial intelligence model is high.
[0205] If an image including pumpkin pie is obtained, the electronic device 100 may use the artificial intelligence model and recognize that the identification information is pumpkin pie, and the first accuracy information is 98%. Here, the electronic device 100 may not obtain the second accuracy information and the difference value. If the number of recognitions is less than the threshold number of times, the electronic device 100 may not determine the applicability of the artificial intelligence model. Fig.11 The description associated with it is described in.
[0206] Fig.12 The table 1220 in may include cooking object information having a plurality of depths. The first depth may refer to a category of the widest range, the second depth may refer to a category of a narrower range than the first depth, and the third depth may refer to a category of the narrowest range.
[0207] The first depth may be categorized into seafood, meat, vegetables, etc.
[0208] The second depth of information associated with seafood may include salmon, shrimp, etc. The third depth of information associated with salmon may include salmon fillet, smoked salmon, etc. The third depth of information associated with shrimp may include fried shrimp, etc.
[0209] The second depth information associated with meat may include pork, chicken, etc. The third depth information associated with pork may include grilled meat or braised spareribs (kebabs). The third depth information associated with chicken may include fried chicken, etc.
[0210] The second depth information associated with vegetables may include tomatoes or garlic. The third depth information of tomatoes may include tomato soup, tomato stew, etc. The third depth information of garlic may include roasted garlic, etc.
[0211] The description associated with the above-mentioned depth information is merely an example and may include additional classification references.
[0212] Fig.13 is a flowchart showing an operation of receiving update information from the server 200 .
[0213] Reference Fig.13 , steps S1305, S1310, S1315, S1320, S1325 and S1330 may correspond to Fig.10 Therefore, redundant description will be omitted.
[0214] If the difference between the first accuracy information and the second accuracy information is greater than or equal to the threshold value ( S1325 -Yes), the electronic device 100 may transmit a signal requesting update information to the server 200 ( S1329 - 1 ).
[0215] The server 200 may receive a signal requesting update information from the electronic device 100. The server 200 may then identify update information corresponding to the electronic device 100 (S1329-2). Specifically, the server 200 may store a plurality of update information associated with the artificial intelligence model. The electronic device 100 may then identify (or obtain) update information suitable for the electronic device 100 from among the plurality of update information. The server 200 may send the identified update information to the electronic device 100 (S1329-3).
[0216] The electronic device 100 may receive the update information from the server 200. Then, the electronic device 100 may update the artificial intelligence model based on the received update information (S1330).
[0217] Fig.14 is a flowchart illustrating operations of various modules included in the electronic device 100.
[0218] Reference Fig.14 , the electronic device 100 may include at least one of an image storage module 100 - 1 , a cooking object recognition module 100 - 2 , a recognition result storage module 100 - 3 , or an update module 100 - 4 .
[0219] The image storage module 100-1 may obtain an image (S1405). Then, the image storage module 100-1 may transmit the obtained image to the cooking object recognition module 100-2 (S1410).
[0220] The cooking object recognition module 100-2 may receive an image from the image storage module 100-1. Then, the cooking object recognition module 100-2 may apply the received image to the artificial intelligence model and recognize the image (S1415). The cooking object recognition module 100-2 may obtain recognition result information. The recognition result information may include at least one of the image (the image obtained from step S1405), the identification information, or the first accuracy information corresponding to the identification information. The cooking object recognition module 100-2 may send the recognition result information to the recognition result storage module 100-3 (S1420).
[0221] The recognition result storage module 100-3 may store the recognition result information received from the cooking object recognition module 100-2 (S1425). The recognition result storage module 100-3 may additionally update (or modify) the number of times the specific identification information is counted. For example, if a salmon fillet is identified in the cooking object recognition module 100-2, the number of times the salmon fillet is counted may be increased by +1. The recognition result storage module 100-3 may send the recognition result information to the server 200 (S1430). Here, the recognition result storage module 100-3 may only generate a sending instruction, and the actual sending operation may be performed through the communication interface 140.
[0222] The server 200 may receive the recognition result information from the recognition result storage module 100-3. Then, the server 200 may train (or retrain) the artificial intelligence model based on the received recognition result information (S1431).
[0223] According to an embodiment, steps S1430 and S1431 may be omitted. After storing the recognition result information, the recognition result storage module 100-3 may send a storage notification of the recognition result information to the update module 100-4 (S1435).
[0224] If a notification is received from the recognition result storage module 100-3 (notification of step S1435), the update module 100-4 may request the recognition rate statistics (second accuracy information) from the server 200 (S1440). Here, the update module 100-4 may only generate a request instruction, and may perform an operation of sending an actual request through the communication interface 140.
[0225] The server 200 may receive a request for the recognition rate statistics information (second accuracy information) from the update module 100-4. Then, the server 200 may send the recognition rate statistics information (second accuracy information) to the update module 100-4 (S1445).
[0226] The update module 100-4 may receive the recognition rate statistics (second accuracy information) from the server 200. The update module 100-4 may determine the applicability of the artificial intelligence model based on the recognition rate statistics (second accuracy information) (S1450). Then, the update module 100-4 may determine whether to update the artificial intelligence model according to the applicability determination result (S1455). If it is identified as updating the artificial intelligence model, the update module 100-4 may request update information from the server 200 (S1460).
[0227] The server 200 may receive a request for update information from the update module 100-4. Then, the server 200 may send the update information to the update module 100-4 (S1465).
[0228] The update module 100-4 may receive update information from the server 200. Then, the update module 100-4 may update the artificial intelligence model based on the received update information (S1470).
[0229] Fig.15 is a flowchart illustrating an operation of the electronic device 100 displaying a UI guiding an update.
[0230] Reference Fig.15 , steps S1505, S1510, S1515, S1520, S1525, S1529-1, S1529-2, S1529-3, and S1530 may correspond to Fig.13 Therefore, redundant description will be omitted.
[0231] If the difference between the first accuracy information and the second accuracy information is greater than or equal to the threshold value (S1525-Y), the electronic device 100 may display a UI guiding the update (S1526). Then, the electronic device 100 may identify whether a user input for the update is received (S1527).
[0232] If a user input for update is received through the displayed UI (S1527-Y), the electronic device 100 may transmit a signal requesting update information to the server 200 (S1529-1).
[0233] If no user input for updating is received through the displayed UI (S1527-N), the electronic device 100 may obtain a new image and perform steps S1505 to S1527.
[0234] Fig.16is a flowchart showing an operation of the terminal device 300 displaying a UI guiding an update.
[0235] Reference Fig.16 , steps S1605, S1610, S1615, S1620, S1625 and S1630 may correspond to Fig.13 Therefore, redundant description will be omitted.
[0236] If the difference between the first accuracy information and the second accuracy information is greater than or equal to the threshold value (S1625-Yes), the electronic device 100 may transmit a signal requesting update information to the server 200 (S1626-1).
[0237] The server 200 may receive a signal requesting update information from the electronic device 100. Then, the server 200 may determine whether update information corresponding to the electronic device 100 can be recognized (S1626-2).
[0238] If the update information corresponding to the electronic device 100 cannot be recognized (S1626-2-No), the server 200 may not perform a separate operation, or may transmit information notifying that the update information cannot be recognized to the electronic device 100. The electronic device 100 may display information notifying that the update information cannot be recognized.
[0239] If the update information corresponding to the electronic apparatus 100 can be identified (S1626-2-Yes), the server 200 may transmit a signal requesting a user input to the terminal device 300 (S1626-3).
[0240] The terminal device 300 may receive a signal requesting a user input from the server 200. Then, the terminal device 300 may display a UI guiding an update (S1627-1).
[0241] Then, the terminal device 300 may identify whether a user input for updating is received (S1627-2).
[0242] If no user input for update is received through the displayed UI (S1627-2-No), the electronic device 100 may wait for reception of a new signal.
[0243] If a user input for update is received through the displayed UI (S1627-2-Yes), the electronic device 100 may transmit the user input for update to the server (S1627-3).
[0244] If a user input for updating is recognized, the server 200 may transmit update information to the electronic device 100 (S1628).
[0245] The electronic device 100 may receive the update information from the server 200. Then, the electronic device 100 may update the artificial intelligence model based on the received update information (S1630).
[0246] Fig.17 is a diagram showing a UI that guides updating.
[0247] Reference Fig.17 , the electronic device 100 may display a UI 1700 guiding an update. Here, the UI 1700 may include at least one of an image 1710, a cooking object analysis result 1720, information 1730 guiding an update, or information 1740 for receiving a user input.
[0248] Image 1710 may be an image including a cooking object, and may be an image used as input data in an artificial intelligence model.
[0249] The cooking object analysis result 1720 may correspond to Fig.12 Table 1210 in .
[0250] The information 1730 guiding the update may include information associated with the update of the artificial intelligence model. Specifically, the information 1730 may include text information showing that the recognition of the identified cooking object (e.g., salmon fillet) can be improved.
[0251] The information 1740 for receiving the user input may be information for selecting the user input for updating (“Yes”) or the user input for not performing the updating (“No”).
[0252] Fig.18 is a flowchart showing an operation of receiving the second accuracy information from the server 200 .
[0253] Fig.18 Steps S1805, S1810, S1815, S1820, S1825, S1829-1, S1829-2, S1829-3, and S1830 in Fig.13 Therefore, redundant description will be omitted.
[0254] After obtaining the identification information and the first accuracy information ( S1810 ), the electronic device 100 may transmit a signal requesting second accuracy information corresponding to the identification information to the server 200 ( S1811 ).
[0255] The server 200 may receive a signal requesting the second accuracy information corresponding to the identification information from the electronic device 100. Then, the server 200 may identify the second accuracy information corresponding to the identification information (identification information identified in the electronic device 100) from among the plurality of stored information (S1812). Then, the server 200 may send the second accuracy information to the electronic device 100 (S1813).
[0256] The electronic device 100 may receive the second accuracy information from the server 200. Then, the electronic device 100 obtains the second accuracy information corresponding to the transmitted identification information (S1815).
[0257] Fig.19 is a flowchart showing an operation of transmitting an image to the server 200 .
[0258] Fig.19 Steps S1905, S1910, S1915, S1920, S1925, and S1930 in may correspond to Fig.13 Therefore, redundant description will be omitted.
[0259] If the difference between the first accuracy information and the second accuracy information is greater than or equal to the threshold value ( S1925 -Yes), the electronic device 100 may transmit learning data to the server 200 ( S1926 ).
[0260] According to an embodiment, the learning data may include an image (e.g., the image obtained from step S1905). The electronic device 100 may simply send the image to the server so that the image is simply used in the training of the artificial intelligence model. The server 200 may simply use the image received from the electronic device 100 as learning data.
[0261] According to another embodiment, the learning data may include images and identification information. The electronic device 100 may send both the images used in the training and the identification information analyzed by the artificial intelligence model to the server 200. The server 200 may train the artificial intelligence model in consideration of the images and identification information analyzed in the electronic device 100.
[0262] According to another embodiment, the learning data may include an image, identification information, and user feedback information. The electronic device 100 may display a UI including identification information and obtain user feedback. Here, the user feedback may be a user input showing whether the identification information as the analysis result of the image is accurately analyzed. The electronic device 100 may receive user feedback showing whether the identification information is consistent with the cooking object. Here, the electronic device 100 may send user feedback together with the image and identification information to the server 200. The server 200 may train the artificial intelligence model by considering user feedback showing whether the analysis result (identification information) of the image is accurate.
[0263] The server 200 may receive the learning data from the electronic device 100. Then, the server 200 may train (or retrain) the artificial intelligence model using the received learning data (S1927). Then, the server 200 may send the trained (or retrained) artificial intelligence model to the electronic device 100 (S1928).
[0264] The electronic device 100 may receive the trained artificial intelligence model from the server 200. Then, the electronic device 100 may update the existing artificial intelligence model based on the artificial intelligence model trained by the server 200.
[0265] Here, the artificial intelligence model trained by the server 200 may be described as update information.
[0266] Fig. 20 is a diagram showing an operation of obtaining user feedback.
[0267] Reference Fig. 20 , the electronic device 100 may display a UI 2000 for obtaining user feedback. The UI 2000 may include at least one of an image 2010, identification information 2020 recognized from the image, or information 2030 for receiving user feedback.
[0268] Image 2010 may refer to input data analyzed from an artificial intelligence model.
[0269] The identification information recognized from the image 2020 may refer to the identification information of the cooking object analyzed by the artificial intelligence model.
[0270] The information 2030 for receiving user feedback may refer to a user option for displaying whether identification information recognized from the image 2020 matches an actual cooking object.
[0271] Fig.21 is a flowchart showing an operation of sending a signal requesting an image and an update to the server 200 . Fig.21Steps S2105, S2110, S2115, S2120, S2125, and S2130 in may correspond to Fig.13 Therefore, redundant description will be omitted.
[0272] If the difference between the first accuracy information and the second accuracy information is greater than or equal to the threshold value ( S2125 -Yes), the electronic device 100 may transmit a signal requesting learning data and update information to the server 200 ( S2126 ).
[0273] like Fig.19 The learning data described in may include at least one of an image, identification information, or user feedback information.
[0274] The server 200 may receive a signal requesting the learning data and the update information from the electronic device 100. Then, when receiving the signal requesting the update information, the server 200 may obtain the update information based on the learning data (S2127).
[0275] According to an embodiment, the server 200 may identify an artificial intelligence model corresponding to the learning data from among a plurality of stored artificial intelligence models, and then the server 200 may obtain the identified artificial intelligence model as update information.
[0276] According to another embodiment, the server 200 may use the learning data to update one of the plurality of stored artificial intelligence models. The server 200 may then obtain the updated artificial intelligence model as update information.
[0277] The server 200 may transmit update information to the electronic device 100 (S2128).
[0278] The electronic device 100 may receive the update information from the server 200. Then, the electronic device 100 may update the artificial intelligence model based on the received update information (S2130).
[0279] Fig. 22 is a flowchart showing an operation of performing update by counting identification information.
[0280] Reference Fig. 22 , the electronic device 100 may obtain an image (S2205). Then, the electronic device 100 may obtain identification information and first accuracy information of a cooking object included in the image (S2210). Then, the electronic device 100 may count the identification information obtained in step S2210 (S2215).
[0281] The electronic device 100 may perform the image analysis operation multiple times. Here, the electronic device 100 may count the identification information obtained in the image analysis operation. Here, the counting operation may be an operation of accumulating and calculating the number of times the identification information is counted.
[0282] The electronic device 100 may identify whether the count number is greater than or equal to the first threshold number (S2220). If the count number is greater than or equal to the first threshold number (S2220-Yes), the electronic device 100 may transmit a signal requesting the first update information (S2225).
[0283] The server 200 may receive a signal requesting the first update information from the electronic device 100. Then, the server 200 may identify the first update information corresponding to the electronic device 100 (S2230). Then, the server 200 may transmit the first update information to the electronic device 100 (S2235).
[0284] The electronic device 100 may receive the first update information from the server 200. Then, the electronic device 100 may update the artificial intelligence model based on the received first update information (S2240).
[0285] If the counted number is less than the first threshold number (S2220-No), the electronic device 100 can identify whether the counted number is less than the second threshold number (S2245). If the counted number is greater than or equal to the second threshold number (S2245-Yes), the electronic device 100 can obtain a new image and execute steps S2205 to S2245.
[0286] If the counted number of times is less than the second threshold number of times (S2245-Y), the electronic device 100 may transmit a signal requesting the second update information to the server 200 (S2250).
[0287] The server 200 may receive a signal requesting the second update information from the electronic device 100. Then, the server 200 may identify the second update information corresponding to the electronic device 100 (S2255). Then, the server 200 may transmit the second update information to the electronic device 100 (S2260).
[0288] The electronic device 100 may receive the second update information from the server 200. Then, the electronic device 100 may update the artificial intelligence model based on the received second update information (S2265).
[0289] According to an embodiment, the first update information and the second update information may vary. The first update information may include update information for improving the recognition of a cooking object included in an image. In addition, the second update information may include an artificial intelligence model dedicated to a specific cooking object (identification information recognized by the image). For example, if a cooking object is identified with a count number greater than or equal to a first threshold number of times, the electronic device 100 may perform an update operation for improving the recognition of the corresponding cooking object. In addition, if a cooking object is identified with a count number less than a second threshold number of times, the electronic device 100 may receive and store a new artificial intelligence model that specifically identifies the corresponding cooking object. Fig.23 The UI display operation associated with the first update information is described in Fig.24 The UI display operation associated with the second update information is described in.
[0290] According to another embodiment, the first update information and the second update information may be the same. The server 200 may store the most recently updated artificial intelligence model. If a request associated with update information is received from the electronic device 100, the server 200 may send update information including the most recently updated artificial intelligence model to the electronic device 100. The electronic device 100 may update the existing artificial intelligence model based on the update information received from the server 200.
[0291] Fig.23 is a diagram illustrating an operation of displaying a UI guiding an update when identification information is counted greater than or equal to a first threshold number of times.
[0292] Reference Fig.23 , the electronic device 100 may display a UI 2300 for requesting the first update information based on the number of times the cooking object recognized from the image is counted being greater than or equal to the first threshold number of times.
[0293] The UI 2300 may include at least one of an image 2310 , information 2320 for requesting update information corresponding to identification information, or information 2330 for receiving a user input.
[0294] Image 2310 may refer to input data that is analyzed in an artificial intelligence model.
[0295] The information 2320 for requesting update information corresponding to the identification information may include at least one of the cooking object analyzed by the artificial intelligence model (e.g., salmon fillet), a parent category of the cooking object (e.g., seafood), or information guiding updates for improving the identification of the analyzed cooking object.
[0296] The information 2330 for receiving user input may refer to a user option for determining whether to perform an update.
[0297] Fig.24 is a diagram illustrating an operation of displaying a UI guiding an update when identification information is counted less than a second threshold number of times.
[0298] Reference Fig.24 , the electronic device 100 may display a UI 2400 for requesting the second update information based on the number of times the cooking object recognized from the image is counted being less than the second threshold number of times.
[0299] The UI 2400 may include at least one of an image 2410 , information 2420 for requesting update information corresponding to identification information, or information 2430 for receiving a user input.
[0300] Image 2410 may refer to input data that is analyzed in an artificial intelligence model.
[0301] The information 2420 for requesting update information corresponding to the identification information may include at least one of the cooking object analyzed by the artificial intelligence model (e.g., barbecue), the upper-level category of the cooking object (e.g., meat), or information guiding updates for improving the recognition of the analyzed cooking object.
[0302] The information for receiving the user input 2430 may refer to a user option for determining whether to perform an update.
[0303] Fig.25 is a flowchart showing the operation of updating the artificial intelligence model when identification information is preliminarily recognized.
[0304] Fig.25 Steps S2505, S2510, S2515, S2520, S2525, and S2530 in may correspond to Fig.10 Therefore, redundant description will be omitted.
[0305] After obtaining the difference between the first accuracy information and the second accuracy information (S2520), the electronic device 100 may identify whether the identification information is preliminarily identified (S2521). Here, the identification information may be the information obtained in step S2510.
[0306] If the identification information is not preliminarily recognized (S2521-No), the electronic device 100 may identify whether the difference is greater than or equal to the threshold value (S2525). Then, if the difference is greater than or equal to the threshold value (S2525-Yes), the electronic device 100 may update the artificial intelligence model (S2530). If the difference is less than the threshold value (S2525-No), the processor 120 may repeat and execute steps S2505 to S2521.
[0307] If the identification information is preliminarily recognized (S2521-Yes), the electronic device 100 may determine whether update information corresponding to the identification information can be obtained (S2522). If the identification information is preliminarily recognized, the reliability of the obtained accuracy information may be determined to be low. Therefore, the electronic device 100 may additionally request update information corresponding to the preliminarily recognized identification information (cooking object).
[0308] If update information corresponding to the identification information can be obtained (S2522-Yes), the electronic device 100 can obtain the update information and update the artificial intelligence model based on the obtained update information (S2530).
[0309] If the update information corresponding to the identification information cannot be obtained (S2522-No), the electronic device 100 may obtain a new image and perform steps S2505 to S2522.
[0310] Fig.26 is a flowchart illustrating a control method of an electronic device.
[0311] Reference Fig.26 , the control method of the electronic device may include: obtaining a captured image (S2605); obtaining identification information of a cooking object included in the image and first accuracy information showing the recognition of the identification information by inputting the obtained image into a trained artificial intelligence model (S2610); and updating the artificial intelligence model based on the difference between the obtained first accuracy information and second accuracy information corresponding to the identification information (S2615), and the second accuracy information may be information showing a reference recognition associated with the identification information.
[0312] The second accuracy information may be an average of an accuracy value of the cooking object obtained from the first electronic device among the plurality of electronic devices and an accuracy value of the cooking object obtained from the second electronic device.
[0313] The control method may also include: sending a signal to the server requesting update information of the artificial intelligence model based on the difference between the first accuracy information and the second accuracy information being greater than or equal to a threshold, and updating the artificial intelligence model (S2615) may include: based on receiving the update information from the server, updating the artificial intelligence model based on the received update information.
[0314] The control method may further include: displaying a user interface (UI) guiding the update of the artificial intelligence model based on the difference being greater than or equal to a threshold; and sending a signal to the server requesting the update of the artificial intelligence model based on receiving user input through the displayed UI.
[0315] The UI may include at least one of identification information, first accuracy information, second accuracy information, a difference value, or inquiring about information on updating the artificial intelligence model.
[0316] The control method may also include: based on obtaining the identification information and the first accuracy information of the cooking object, sending a signal to the server requesting second accuracy information corresponding to the identification information; and based on receiving the second accuracy information corresponding to the identification information from the server, storing the received second accuracy information.
[0317] The control method may further include: based on the difference being greater than or equal to a threshold, sending the obtained image to a server for use in training an artificial intelligence model stored in the server.
[0318] The control method may also include: based on the identification information of the cooking object obtained according to multiple captured images being counted greater than or equal to a first threshold number of times, sending a signal to the server requesting first update information corresponding to the identification information of the artificial intelligence model, and updating the artificial intelligence model (S2615) may include: based on receiving the first update information corresponding to the identification information from the server, updating the artificial intelligence model based on the received first update information.
[0319] The control method may also include: based on the identification information of the cooking object obtained according to at least one captured image being counted less than a second threshold number of times, sending a signal to the server requesting second update information corresponding to the identification information of the artificial intelligence model, and updating the artificial intelligence model (S2615) may include: based on receiving the second update information corresponding to the identification information from the server, updating the artificial intelligence model based on the received second update information.
[0320] Obtaining the captured image (S2605) may include: obtaining a first image capturing the interior of a cooking cavity accommodating a cooking object from a first viewpoint; and obtaining a second image capturing the interior of the cooking cavity from a second viewpoint different from the first viewpoint, and obtaining first accuracy information (S2610) may include: obtaining an average of a first accuracy value of the cooking object corresponding to the first image and a second accuracy value of the cooking object corresponding to the second image as the first accuracy information.
[0321] The methods according to various embodiments of the present disclosure described above may be implemented in the form of an application installable in an electronic device of the related art.
[0322] In addition, the methods according to the various embodiments of the present disclosure described above may be implemented by only performing software upgrade or hardware upgrade on the electronic device of the related art.
[0323] Furthermore, the various embodiments of the present disclosure described above may be performed by an embedded server provided in the electronic device or at least one external server among the electronic device and the display apparatus.
[0324] The various embodiments described above can be implemented with software including instructions stored in a machine-readable storage medium (e.g., a computer). The machine can call instructions stored in a storage medium, and as a device operable according to the called instructions, an electronic device according to the above-mentioned embodiments can be included. Based on the instructions being executed by the processor, the processor can directly or use other elements under the control of the processor to perform the function corresponding to the instruction. The instruction may include code generated by a compiler or executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. In this article, "non-transitory" only means that the storage medium is tangible and does not include signals, and the term does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.
[0325] In addition, the methods according to the various embodiments described above may be configured to include a computer program product. The computer program product may be exchanged between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or through an application store (e.g., PLAYSTORE TM ) Online distribution. In the case of online distribution, at least a part of the computer program product may be at least temporarily stored in a storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server, or may be temporarily generated.
[0326] In addition, each element (e.g., module or program) according to the various embodiments described above can be formed as a single entity or multiple entities, and some of the above-mentioned sub-elements can be omitted, or other sub-elements can also be included in various embodiments. Alternatively or additionally, some elements (e.g., modules or programs) can be integrated into one entity to perform the same or similar functions performed by the corresponding elements before integration. According to various embodiments, the operations performed by a module, program or another element can be performed sequentially, in parallel, repeatedly or in a heuristic manner, or at least some operations can be performed in different orders, omitted, or different operations can be added.
[0327] Although the present disclosure has been shown and described with reference to various exemplary embodiments of the present disclosure, it should be understood that the various exemplary embodiments are intended to be illustrative rather than restrictive. It should be understood by those skilled in the art that various changes in form and details may be made therein without departing from the true spirit and full scope of the present disclosure including the appended claims and their equivalents.
Claims
1. An electronic device, include: camera; as well as At least one processor configured to: obtaining, using the camera, an image captured by the camera, The obtained images are input into the trained artificial intelligence model. using the artificial intelligence model to obtain identification information of an object in the image and first accuracy information indicating a degree of recognition of the identification information, and updating the artificial intelligence model based on a difference between the obtained first accuracy information and the second accuracy information corresponding to the identification information, The second accuracy information indicates a reference recognition degree associated with the identification information.
2. The electronic device according to claim 1, in, The second accuracy information indicates the reference recognition used in the training process of the updated artificial intelligence model.
3. The electronic device according to claim 1, further comprising: include: at least one memory configured to store the artificial intelligence model; as well as a communication interface configured to communicate with the server, Wherein, the at least one processor is further configured to: Based on the difference between the first accuracy information and the second accuracy information being greater than or equal to a threshold, controlling the communication interface to send a signal to the server requesting update information of the artificial intelligence model, and Based on receiving the update information from the server through the communication interface, the artificial intelligence model is updated based on the received update information.
4. The electronic device according to claim 3, further comprising: include: monitor, Wherein, the at least one processor is further configured to: Based on the difference being greater than or equal to the threshold, controlling the display to display a user interface UI guiding the update of the artificial intelligence model, and Based on receiving user input through the displayed UI, controlling the communication interface to send a signal to the server requesting an update of the artificial intelligence model.
5. The electronic device according to claim 4, in, The UI includes at least one of the identification information, the first accuracy information, the second accuracy information, the difference, or information inquiring about updating the artificial intelligence model.
6. The electronic device according to claim 3, in, The at least one processor is further configured to: Based on obtaining the identification information of the object and the first accuracy information, controlling the communication interface to send a signal requesting second accuracy information corresponding to the identification information to the server, and Based on receiving second accuracy information corresponding to the identification information from the server through the communication interface, the received second accuracy information is stored in the at least one memory.
7. The electronic device according to claim 3, in, The at least one processor is further configured to: Based on the difference being greater than or equal to the threshold, the communication interface is controlled to send the obtained image to the server to train the artificial intelligence model stored in the server.
8. The electronic device according to claim 3, in, The at least one processor is further configured to: controlling the communication interface to send a signal requesting first update information of the artificial intelligence model corresponding to the identification information to the server based on the identification information of the object obtained according to the plurality of captured images obtained from the camera being counted greater than or equal to a first threshold number of times; and Based on receiving the first update information corresponding to the identification information from the server through the communication interface, the artificial intelligence model is updated based on the received first update information.
9. The electronic device according to claim 3, in, The at least one processor is further configured to: controlling the communication interface to send a signal requesting second update information of the artificial intelligence model corresponding to the identification information to the server based on the identification information of the object obtained according to at least one captured image obtained from the camera being counted less than a second threshold number of times; and Based on receiving the second update information corresponding to the identification information from the server through the communication interface, the artificial intelligence model is updated based on the received second update information.
10. The electronic device according to claim 1, further comprising: include: a chamber in which the object is arranged, Wherein, the at least one processor is further configured to: obtaining, using the camera, a first image capturing the interior of the chamber from a first viewpoint, obtaining, using the camera, a second image capturing the interior of the chamber from a second viewpoint different from the first viewpoint, and An average value of a first accuracy value of the object corresponding to the first image and a second accuracy value of the object corresponding to the second image is obtained as the first accuracy information.
11. A control method for an electronic device, the control method include: Obtaining images; The obtained images are fed into the trained AI model; using the artificial intelligence model to obtain identification information of an object in the image and first accuracy information indicating a degree of recognition of the identification information; as well as updating the artificial intelligence model based on a difference between the obtained first accuracy information and the second accuracy information corresponding to the identification information, The second accuracy information indicates a reference recognition degree associated with the identification information.
12. The control method according to claim 11, in, The second accuracy information indicates the reference recognition used in the training process of the updated artificial intelligence model.
13. The control method according to claim 11, further comprising: include: Based on the difference between the first accuracy information and the second accuracy information being greater than or equal to a threshold, sending a signal to the server requesting update information of the artificial intelligence model, Wherein, updating the artificial intelligence model includes: based on receiving the update information from the server, updating the artificial intelligence model based on the received update information.
14. The control method according to claim 13, further comprising: include: Based on the difference being greater than or equal to the threshold, displaying a user interface UI guiding an update of the artificial intelligence model; as well as Based on receiving user input through the displayed UI, a signal is sent to the server requesting an update of the artificial intelligence model.
15. The control method according to claim 14, in, The UI includes: At least one of the identification information, the first accuracy information, the second accuracy information, the difference, or information inquiring about updating the artificial intelligence model.
