Server device and control method therefor

Abstract

A server device is disclosed. The server device according to the present disclosure comprises: a communication unit for communicating with a plurality of refrigerators, each equipped with a first artificial intelligence model; a memory on which a second artificial intelligence model is stored; and a processor, wherein the processor receives, from the plurality of refrigerators, result data obtained using the first artificial intelligence model, and information about an object through the communication unit and stores same in the memory, inputs the information about the object into the second artificial intelligence model so as to obtain a result value, diagnoses the performance of the first artificial intelligence model of each of the plurality of refrigerators on the basis of the result value and the result data, configures, on the basis of the diagnosis result, update data for updating the first artificial intelligence model for at least one refrigerator from among the plurality of refrigerators, and controls the communication unit such that the update data is transmitted to the at least one refrigerator.

Description

Server device and its control method

[0001] The present disclosure relates to a server device and a control method thereof, and more particularly, to a server device for managing the performance of an artificial intelligence model of a refrigerator and a control method thereof.

[0002] A refrigerator is a home appliance used to store or transport items, and to store and manage items for long periods of time.

[0003] Recent refrigerators can take images of items being received or shipped and use artificial intelligence models to identify and manage the items.

[0004] However, since refrigerators may have lower hardware performance than server devices, it was difficult to install and use an artificial intelligence model at the level of a server device running on a refrigerator.

[0005] Therefore, even when using an AI model embedded in a refrigerator, it was difficult to compensate for performance degradation caused by various environmental conditions, or the performance did not meet user satisfaction. Furthermore, when images were transmitted to a server for object management, there were issues with delayed reflection.

[0006] Therefore, it is necessary to find a way to apply and manage a customized artificial intelligence model according to the characteristics of the refrigerator.

[0007] According to at least one embodiment of the present disclosure, a server device includes a communication unit for performing communication with a plurality of refrigerators each equipped with a first artificial intelligence model, a memory in which a second artificial intelligence model is stored, and a processor, wherein the processor receives result data and information on objects using the first artificial intelligence model in the plurality of refrigerators through the communication unit and stores the data in the memory, inputs the information on the objects into the second artificial intelligence model to obtain a result value, diagnoses the performance of the first artificial intelligence model of each of the plurality of refrigerators based on the result value and the result data, configures update data for updating the first artificial intelligence model for at least one refrigerator among the plurality of refrigerators based on the diagnosis result, and controls the communication unit to transmit the update data to the at least one refrigerator.

[0008] According to at least one embodiment of the present disclosure, a refrigerator includes a communication unit for communicating with a camera server device, a memory for storing an artificial intelligence model, and a processor, wherein the processor controls the camera to capture an object being brought into or taken out of the refrigerator, thereby obtaining an image of the object.

[0009] A refrigerator is provided that inputs the image into the artificial intelligence model, performs management of objects stored in the refrigerator based on the output result of the artificial intelligence model and the entry / exit time of the objects, transmits the output result of the artificial intelligence model to the server device through the communication unit, and updates the artificial intelligence model using the transmitted update data when update data for the artificial intelligence model is transmitted from the server device.

[0010] According to at least one embodiment of the present disclosure, a control method of a server device is provided, the control method comprising: a step of receiving result data and information on an object using a first artificial intelligence model from a plurality of refrigerators, each of which has a first artificial intelligence model installed; a step of inputting the information on the object into a second artificial intelligence model installed in the server device to obtain a result value; a step of diagnosing the performance of the first artificial intelligence model of each of the plurality of refrigerators based on the result value and the result data; a step of configuring update data for updating the first artificial intelligence model for at least one refrigerator among the plurality of refrigerators based on the diagnosis result; and a step of transmitting the update data to the at least one refrigerator.

[0011] FIG. 1 is a diagram illustrating a performance management system according to at least one embodiment of the present disclosure.

[0012] FIG. 2 is a flowchart illustrating the operation of a performance management system according to at least one embodiment of the present disclosure.

[0013] FIG. 3 is a block diagram illustrating a server device according to at least one embodiment of the present disclosure.

[0014] FIG. 4 and FIG. 5 are diagrams for explaining a method for updating an artificial intelligence model according to various embodiments of the present disclosure.

[0015] Figure 6 is a diagram for explaining a classification module according to one embodiment.

[0016] FIG. 7 is a diagram for explaining a method for updating a classification module according to one embodiment.

[0017] FIG. 8 is a block diagram illustrating a configuration of a refrigerator according to at least one embodiment of the present disclosure.

[0018] FIG. 9 is a diagram for explaining the addition of classification targets by weight of objects according to one embodiment.

[0019] Figure 10 is a diagram showing an example of the diagnosis process of the first artificial intelligence model when an object is worn.

[0020] Figure 11 is a diagram showing an example of the diagnostic process of the first artificial intelligence model when an object is shipped.

[0021] FIG. 12 is a diagram illustrating a process of transmitting result data from a refrigerator to a server device according to at least one embodiment of the present disclosure.

[0022] FIG. 13 is a flowchart for explaining a method for controlling a server device according to at least one embodiment of the present disclosure.

[0023] It should be understood that the various embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, but rather to encompass various modifications, equivalents, or alternatives of the embodiments.

[0024] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0025] The singular form of a noun corresponding to an item may include one or more of said items, unless the relevant context clearly indicates otherwise.

[0026] In this disclosure, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0027] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0028] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0029] In addition, terms such as 'front', 'rear', 'top', 'bottom', 'side', 'left', 'right', 'upper', and 'lower' used in the present disclosure are defined based on the drawings, and the shape and position of each component are not limited by these terms.

[0030] Terms such as "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the present disclosure, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0031] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0032] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0033] Hereinafter, various embodiments will be described in detail with reference to the attached drawings. FIG. 1 is a diagram illustrating the configuration of a performance management system according to at least one embodiment of the present disclosure.

[0034] The performance management system may include a refrigerator (200), a server device (100), and an external device (300), as illustrated in FIG. 1. The performance management system may be a network system that is interconnected and communicates with each other so that the server device (100) can manage the performance of various electronic devices. Although FIG. 1 illustrates a refrigerator (200), the performance management system according to various embodiments of the present disclosure may be implemented as a system for managing the performance of various electronic devices capable of using an artificial intelligence model, such as a washing machine or an air conditioner, in addition to the refrigerator (200).

[0035] A refrigerator (200) may be a cooling device for storing and preserving food. According to one embodiment of the present disclosure, the refrigerator (200) is not limited to a typical household refrigerator, and may include devices for storing and transporting items, such as a kimchi refrigerator, a liquor refrigerator, a cosmetics refrigerator, and a freezer.

[0036] Refrigerators can take various forms depending on their type. Typically, the refrigerator body may include an inner case, an outer case positioned outside the inner case, and insulation provided between the inner cases. The inner case may form at least one storage compartment. Each storage compartment is partitioned from one another by a partition wall and can store various items, such as food, medicine, and cosmetics. The storage compartment may be designed with at least one open side for storing and removing items. The opening of each storage compartment may be opened and closed by a door.

[0037] The door may be provided to open and close one or more storage compartments, or one door may be provided to open and close multiple storage compartments. The door may be installed on the front of the main body in a pivotal or sliding manner.

[0038] The refrigerator (200) of Fig. 1 can capture images of objects being brought into or taken out of each storage room using a camera installed on the inside of the door or on one side of the inner case. The refrigerator (200) analyzes the images of the captured objects using an artificial intelligence model, and then displays information about the objects on its own display or transmits the information to an external device (300).

[0039] The external device (300) in FIG. 1 is illustrated as a mobile phone, but is not limited thereto and may be implemented as various electronic devices such as a PC, laptop PC, tablet PC, or kiosk. The user can check information about an object through the refrigerator's (200) own display or the external device (300).

[0040] For this operation, the refrigerator (200) according to the present disclosure may be equipped with an artificial intelligence model. For convenience of explanation, the artificial intelligence model equipped in the refrigerator (200) is hereinafter referred to as a first artificial intelligence model (210).

[0041] The first artificial intelligence model (210) may be a model learned to identify and manage items being received or shipped.

[0042] In the present disclosure, “management” may include various tasks such as identifying the type of items being received or shipped, tracking expiration dates, checking quantities, monitoring the temperature and humidity inside the refrigerator to maintain the freshness of food, predicting food consumption habits and expiration dates to provide warnings or recommendations, or monitoring the status of the refrigerator in real time.

[0043] Items being received or shipped can be identified through the camera of the refrigerator (200). The camera may be configured to capture images or video of the items.

[0044] The refrigerator (200) can input images of objects being received or shipped into the first artificial intelligence model (210) to identify the objects being received or shipped. Identifying objects may mean not only identifying the type of objects, but also differentiating objects of the same type based on characteristic information of the objects. For example, let's assume that three apples and two bunches of bananas have been received. The refrigerator (200) can identify the types of apples and bananas. In addition, it can store characteristic information for apple A, apple B, and apple C within the same apple. Here, if one apple has been shipped, the refrigerator (200) can manage the objects received in the refrigerator by reducing the number of apples received.

[0045] Additionally, the refrigerator (200) can obtain information related to the object by inputting an image of the photographed object into the first artificial intelligence model (210).

[0046] Information related to an object may include information about the object's type, object's characteristics, and object's deformation status. As described above, information about the object's type and object's characteristics may be used to identify the object's type and even different objects of the same type.

[0047] Information on the degree of deformation of an object may be information on the difference between the characteristic information of the object at the time of receipt and the characteristic information of the object at the time of shipment, if the object has been deformed over a certain period of time since being received. If one of the received bananas has passed the average appropriate consumption period since being received, the refrigerator (200) may provide guidance to the user. In addition, if the banana is initially received, then shipped, and then re-received, the refrigerator (200) may recognize the banana as the same product. If the similarity between the characteristic information of the banana at the time of shipment and the characteristic information of previously stored received objects is below a threshold, the refrigerator (200) may input an image of the banana into the object enhancement module and identify the object based on the output value of the module.

[0048] The first artificial intelligence model (210) may include a classification module, a feature matching module, and an object deformation compensation module. The classification module may be an artificial intelligence model that identifies the type of objects being received or shipped. The refrigerator (200) may acquire images of objects being received or shipped and input them into the classification module. The classification module may extract image features from the input image of the object and identify the type of object based on the extracted features.

[0049] The feature matching module may be an artificial intelligence model that stores feature information of items being received or shipped and compares similarity. When an image of an item being received or shipped is input into the feature matching module, the feature matching module can extract feature information from the image of the item. In addition, the refrigerator (200) can store the extracted feature information. For example, in the above example, if apples a, b, and c are received, the feature matching module installed in the refrigerator can extract and store feature information consisting of the size, color, shape, etc. of each apple, and when shipped, the apple with the most similar feature information can be shipped.

[0050] The object deformation compensation module can be an AI model that identifies objects by compensating for the characteristics of objects that deform or discolor during delivery or shipment. If the similarity between the feature information of the "shipped object" and the previously stored feature information of the "incoming object" is below a threshold, the output of the object deformation compensation module can be used together.

[0051] In other words, if the similarity between the feature information of the outgoing object and the feature information of the incoming object does not exceed a threshold, the image of the outgoing object can be input into the object deformation compensation module to obtain a result value that takes the object deformation into account. The object can be identified based on the similarity between the obtained result value and the feature information of the incoming object.

[0052] A similarity score that does not exceed a threshold value may indicate that none of the images of previously received objects are similar to the images of the outgoing objects. Here, the images of the outgoing and incoming objects can be enhanced (color, brightness, etc.) to extract multiple feature information. By comparing the similarity of the extracted features with the features of the incoming objects, the altered objects can be identified.

[0053] For example, let's assume that broccoli is received and then shipped. Here, the refrigerator (200) can acquire an image of the broccoli through a camera and input it into the first artificial intelligence model (210) to acquire the type of object and feature information at the time of receipt. Then, when the broccoli is shipped, the refrigerator (200) can take an image of the broccoli through a camera and input it into the first artificial intelligence model (210) to acquire feature information at the time of shipment. Here, the object may be deformed or discolored during the storage period from receipt to shipment. In other words, there may be a deformation in the image of the broccoli at the time of shipment compared to the image of the broccoli at the time of receipt. Here, the processor (120) can identify that the similarity between the feature information corresponding to the image of the broccoli at the time of shipment and the feature information corresponding to the object at the time of receipt does not exceed a threshold value. In this case, the image of the broccoli at the time of shipment can be input into the object deformation compensation module to acquire feature information while taking the deformation into account. If the similarity between the acquired feature information and the feature information corresponding to the broccoli image at the time of receipt is higher than the threshold, the object can be identified as the same object.

[0054] Meanwhile, the refrigerator (200) can store a large number of different objects. The type and number of objects may vary depending on the user, and may also vary depending on the country or region where the refrigerator (200) is installed, the season, the purpose of use, etc. In this way, the types of objects that can be stored in the refrigerator (200) may vary infinitely, but the performance of the first artificial intelligence model (210) installed in the refrigerator (200) remains at the state at the time of installation. Therefore, the first artificial intelligence model (210) cannot identify and manage all objects. Furthermore, if the hardware and software specifications of the refrigerator (200) are lower than those of a server device or a PC, the first artificial intelligence model (210) must also be relatively lightweight. Therefore, the number of types of objects that the first artificial intelligence model (210) can identify is also inevitably limited. Therefore, objects exceeding a certain number cannot be identified and managed.

[0055] Furthermore, even if various objects are stored in the refrigerator (200), the frequency of use may vary depending on the purpose of each object. For example, some objects, such as water or beverages, may be removed frequently, while others, such as frozen foods, may be stored for long periods of time and removed only occasionally. Therefore, among the types of objects that the first artificial intelligence model (210) can manage, some objects may require frequent management, while others may require little or no management. In this case, there is a need to replace objects that are rarely managed with objects that are used relatively frequently.

[0056] However, since there was no way to update the first artificial intelligence model (210) installed in the refrigerator (200) in the past, its usability may decrease over time.

[0057] Accordingly, in various embodiments of the present disclosure, the server device (100) can diagnose the performance of the first artificial intelligence model of the refrigerator (200) and then update it when necessary, thereby increasing its usability.

[0058] Specifically, the refrigerator (200) can transmit or receive data with the server device (100). The refrigerator (200) can transmit to the server device (100) location information of the refrigerator, information about the HW model, information about the SW model, date, season, information about result data using an artificial intelligence model, etc. The result data may include an output value of the artificial intelligence model or information obtained from the output value. For example, when a photographed image is input, the artificial intelligence model can output a plurality of object names corresponding to object images included in the photographed image, along with reliability information for each object name. In addition, various examples of result data will be described again in the following section.

[0059] Alternatively, the refrigerator (200) may transmit data, such as setting information of the refrigerator (200) input or modified by a user through an external device, or control information that controls the operation of the refrigerator (200), to the server device (100).

[0060] *49 The server device (100) can transmit update data and processing result data of the server device to provide a customized artificial intelligence model based on information received from the refrigerator (200). Data transmission or reception between the refrigerator (200) and the server device (100) can occur periodically or at any time. The server device (100) can also be equipped with its own artificial intelligence model. For convenience of explanation, the artificial intelligence model installed in the server device (100) is referred to as a second artificial intelligence model (110). The second artificial intelligence model (110) can diagnose the performance of the artificial intelligence model equipped in the refrigerator (200) based on data received from the refrigerator (200). The second artificial intelligence model (110) can be implemented as a high-performance, high-capacity model compared to the first artificial intelligence model (210), but is not necessarily limited thereto.

[0061] The second artificial intelligence model (110) may include a classification module, a feature matching module, and an object deformation supplementation module. Detailed descriptions of the classification module, feature matching module, and object deformation supplementation module have been described above, so redundant details are omitted.

[0062] The external device (300) can provide a function to modify the results of the user's artificial intelligence model. The external device (300) is connected to the refrigerator (200) and the server device (100), and can provide the user with the ability to remotely control and monitor the devices. The external device (300) can receive feedback on the results of the artificial intelligence model from the user through the UI.

[0063] For example, let's say an apple is placed in the refrigerator (200). If the refrigerator (200) identifies the placed object as a banana, the user can correct it to an apple through the external device (300). Alternatively, even if the refrigerator (200) identifies the placed object as an apple, the user can determine whether the refrigerator (200) has correctly identified it through the external device (300). The external device (300) can be implemented as various electronic devices other than a mobile phone as described above, and is not limited thereto, and can also be implemented as a display or remote control provided on the refrigerator body.

[0064] An external device (300) can transmit information input from a user to a server device (100) or a refrigerator (200).

[0065] The refrigerator (200) can update the first artificial intelligence model based on update data received from the server device (100). Accordingly, the first artificial intelligence model can be upgraded periodically to suit the user's usage conditions.

[0066] FIG. 2 is a diagram illustrating the operation of a performance management system according to at least one embodiment of the present disclosure.

[0067] According to FIG. 2, when an object is received or delivered to a refrigerator (200) (S210), the refrigerator (200) can acquire an image of the object (S220). The refrigerator (200) can input the acquired image into an artificial intelligence model (210) installed in the refrigerator (200) to acquire result data (S230).

[0068] The refrigerator can transmit the acquired result data and images of objects, etc., to the server device (100) from time to time or periodically.

[0069] The server device (100) can diagnose the first artificial intelligence model (210) installed in the refrigerator based on the transmitted result data (S240). Specifically, the server device (100) can diagnose the performance of the first artificial intelligence model (210) by inputting at least one of the result data and the object image into the second artificial intelligence model (210) it has, and then comparing the result with the received result data.

[0070] The server device (100) can obtain update data based on the received external information (S250) and the diagnosis results of the first artificial intelligence model (210) (S260). The external information may include information on the location, time, season, and country of the refrigerator. The refrigerator (200) can obtain optimal update data based on the external information and the diagnosis results of the first artificial intelligence model (210). The server device (100) can transmit the obtained update data to the refrigerator (200). The refrigerator (200) can update the first artificial intelligence model (210) installed in the refrigerator based on the transmitted update data (S270).

[0071] FIG. 3 is a drawing for explaining the configuration of a server device according to one embodiment of the present disclosure.

[0072] According to one embodiment of the present disclosure, a server device (100) may include a memory (140), a processor (120), and a communication unit (130).

[0073] The server device (100) can communicate with various devices, such as a plurality of refrigerators (200) and external devices (300), each of which is equipped with a first artificial intelligence model (210), through a communication unit (130).

[0074] The communication unit (130) can transmit and receive various signals and data to and from a refrigerator or other various external devices through various wired and wireless communication methods such as Bluetooth, AP-based Wi-Fi (Wi-Fi, Wireless LAN network), Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, and coaxial.

[0075] The memory (140) is a configuration that stores programs, data, commands, etc. required for the operation of the server device (100). A second artificial intelligence model (110) may be installed in the memory (140). The second artificial intelligence model (110) may include a classification module, a feature matching module, and an object transformation supplementation module, as well as a module learned to update the first artificial intelligence model (210).

[0076] The memory (140) may be implemented with at least one of various memories, such as DRAM (dynamic RAM), SRAM (static RAM), SDRAM (synchronous dynamic RAM), OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory, a hard drive, or a solid state drive (SSD).

[0077] The processor (120) is a configuration for controlling the overall operation of the server device (100). The processor (120) may be implemented as a digital signal processor (DSP) for processing digital signals, a microprocessor. However, the processor (120) is not limited thereto, and may include one or more of a central processing unit (CPU), a microcontroller unit (MCU), a microprocessor (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, and an artificial intelligence (AI) processor, or may be defined by the relevant terms. In addition, the processor (120) may be implemented as a system on chip (SoC) having a built-in processing algorithm, a large scale integration (LSI), or may be implemented in the form of a field programmable gate array (FPGA). The processor (120) may perform various functions by executing computer executable instructions stored in the memory (140).

[0078] When data and information about objects are transmitted using the first artificial intelligence model (210) in multiple refrigerators, the processor (120) can receive them through the communication unit (130) and store them in the memory (140). Since the first artificial intelligence model (210) installed in multiple refrigerators has been described above, a detailed description thereof will be omitted.

[0079] Information about objects may include images of objects being stored or shipped from multiple refrigerators. The resulting data may include at least one of feature information corresponding to the object images, the reliability of the classification module of the first AI model, information about the object's deformation status, and a request for a change to the AI ​​model.

[0080] Feature information corresponding to an image of an object may be a result of inputting the image of the object into the first artificial intelligence model feature matching module. The processor (120) can distinguish the object based on the feature information.

[0081] The reliability of the first AI model classification module may be the result value of inputting the image of an object into the first AI model classification module. The reliability of the classification module may be extracted as a probability value for each type of product. For example, let's assume that an image of an object is input into the classification module. The processor (120) may obtain result values ​​such as a probability of 0.7 for a banana, a probability of 0.1 for an apple, and a probability of 0.1 for a tomato based on the image of the input object. Here, the processor (120) may identify the type of an object whose reliability is greater than a threshold value as the type of the object in the image of the input object. Here, if the threshold value is 0.6, the type of the input object may be identified as a banana.

[0082] Information about the deformation state of an object can be the result of inputting the image of the object into the object enhancement module, the image of the deformed object, and information about reliability.

[0083] The processor (120) can input information about an object into the second artificial intelligence model (110) to obtain a result value. In addition, based on the result value and the result data using the first artificial intelligence model, the performance of the first artificial intelligence model (210) of each of the plurality of refrigerators can be diagnosed.

[0084] Diagnosing performance can mean analyzing or examining the operation and functionality of a processor or AI model. The results obtained through performance diagnosis can serve as a benchmark for improving the performance of the AI ​​model. While the term "diagnosis" is used in this disclosure, other terms, such as "assessment," "evaluation," "appraisal," "analysis," or "review," could also be used.

[0085] When multiple refrigerators are connected, the processor (120) can diagnose the performance of the first artificial intelligence model for each refrigerator and, based on the results, identify at least one refrigerator that requires updating.

[0086] The processor (120) may configure update data for updating the first artificial intelligence model (210) for at least one refrigerator that requires updating. Here, updating the first artificial intelligence model (210) may mean improving the performance of the existing first artificial intelligence model (210) by adding new functions, bug fixes, etc.

[0087] The update data may be data for updating the first artificial intelligence model (210) installed in the refrigerator (200). Specifically, the update data may include information on the type and characteristics of objects acquired from the second artificial intelligence model (110) of the server device (100) based on the result data received from the refrigerator (200), information on updating the refrigerator classification module, information on updating the feature matching module, and information on updating the object supplementation module.

[0088] The processor (120) can control the communication unit to transmit update data to at least one refrigerator.

[0089] The first artificial intelligence model (210) of a refrigerator may be updated through the following process. The processor (120) sets an update flag on a refrigerator requiring an update, periodically monitors the update flag, and initiates transmission of update data when the flag is turned on. Here, the update flag is a function used to coordinate software development and management, and may be a function used to activate or deactivate a specific function.

[0090] The processor (120) can transmit individual files and dynamically apply the files in the refrigerator (200). Alternatively, the processor (120) can transmit a configuration that bundles the updated individual files and the refrigerator's artificial intelligence configuration program as a package.

[0091] According to one embodiment of the present disclosure, the processor (120) may input an image of an object into a second artificial intelligence model (110) to obtain a result value of the object. Here, the result value may be a reliability of the type of the object obtained by inputting the image of the object into the second artificial intelligence model (110) and characteristic information of the object.

[0092] The processor (120) can obtain update data for the first artificial intelligence model (210) based on the acquired result value and the received result data. For example, the processor (120) can obtain similarity with the first artificial intelligence model (210) by comparing the reliability of the type of object acquired from the second artificial intelligence model (110) with the reliability of the classification module acquired from the first artificial intelligence model (210). Here, if the similarity is below a threshold, the processor (120) can configure update data for updating the classification module of the first artificial intelligence model.

[0093] Meanwhile, while only a simple configuration of the server device (100) has been illustrated and described above, various additional configurations may be provided during implementation. These will be described below with reference to FIGS. 4 to 7.

[0094] FIG. 4 is a diagram illustrating a method for updating a feature matching module according to at least one embodiment of the present disclosure.

[0095] The processor (120) can input an image of an object received from a refrigerator (200) into a feature matching module included in a second artificial intelligence model (110) to obtain second feature information. The processor (120) can determine whether to update the first artificial intelligence model based on the similarity between the first feature information obtained using the feature matching module of the first artificial intelligence and the second feature information obtained using the feature matching module of the second artificial intelligence (S410).

[0096] Here, the similarity of feature information can be measured through Euclidean Distance, Cosine Similarity, Manhattan Distance, and Jaccard Similarity.

[0097] If the similarity of the acquired first artificial intelligence model is below a threshold (S420), the processor (120) can generate update data for updating the first artificial intelligence model based on the feature matching module of the second artificial intelligence model (110) (S420).

[0098] Here, the processor (120) can evaluate the performance of the updated data by grouping images of objects having high similarity in first feature information (S430). The performance of the updated data can be reliability. That is, if the reliability does not exceed a preset threshold, the processor (120) can train the feature matching module to exceed the threshold and then use the training data as the updated data of the first artificial intelligence model. The threshold used to evaluate the performance can be a value determined separately from the threshold used for comparison with similarity.

[0099] Here, if the performance exceeds the threshold, the processor (120) can transmit update data to the refrigerator (200).

[0100] For example, let's assume that a plurality of apples are stored in a refrigerator (200). The refrigerator (200) can obtain images of the plurality of apples. The obtained images of the apples can be input into a first artificial intelligence model (210) to obtain first feature information. Here, the processor (120) can receive the images of the plurality of apples and the plurality of pieces of first feature information from the refrigerator (200). The processor (120) can input the images of the plurality of apples into a second artificial intelligence model (110) to obtain a plurality of pieces of second feature information. The processor (120) can obtain the similarity between the first feature information and the second feature information corresponding to each image. If the similarity is below a threshold, the processor (120) can generate update data of the first artificial intelligence model (210). Here, the processor (120) can select images corresponding to feature information having a similarity below the threshold, and generate update data of the feature matching module based on the selected images.

[0101] After inputting the image of the selected object into the feature matching module of the second artificial intelligence model (110) to obtain reliability, if the reliability exceeds a threshold, the processor (120) can generate update data for updating the feature matching module of the first artificial intelligence model (210) based on the feature matching module of the second artificial intelligence model (110). That is, the processor (120) can generate update data for changing the feature matching module of the first artificial intelligence model (210) so that it has the same parameter values ​​as the various parameter values ​​used in the feature matching module of the second artificial intelligence model (110).

[0102] FIG. 5 is a diagram for explaining the update of an object deformation supplement module according to one embodiment.

[0103] The second artificial intelligence model (110) may include an object deformation compensation module that identifies objects by considering changes in characteristic information related to the object's deformation. The object deformation compensation module can be defined as an artificial intelligence model that identifies objects by reducing weights in consideration of the characteristics of objects that may be deformed or discolored during the storage period from receipt to shipment.

[0104] The processor (120) can receive result data including information on the product deformation status from the refrigerator (200).

[0105] The processor (120) can input images of multiple objects into the second object deformation module of the second artificial intelligence model (110) to obtain second feature information. Based on the second feature information and the first feature information obtained from the first object deformation module mounted on the first artificial intelligence model (210), similarity is obtained (S510), and when the similarity is below a threshold, update data for updating the first artificial intelligence model (210) can be generated based on the second object deformation supplement module (S520).

[0106] The processor (120) can evaluate the similarity of feature information before and after a change in the same product in the updated data and the performance of differentiating the same product from other products (S530). If the reliability of the performance evaluation result does not exceed a threshold (S540), the processor (120) can train the AI ​​model to be updated based on the updated data to increase the similarity of feature information before and after a change in the same product and to differentiate the same product from other products, and then re-evaluate the performance. Accordingly, the updated data can be regenerated.

[0107] FIG. 6 is a drawing for explaining a classification module according to at least one embodiment of the present disclosure.

[0108] According to one embodiment, the memory (140) can store a plurality of classification modules that are differently generated based on at least one criterion among the type of object, country, region, season, date, consumption trend, and user characteristics.

[0109] Each classification module may be an artificial intelligence model trained to identify different objects. Figure 6 illustrates multiple classification modules (600) generated based on similar shapes, country / region, and consumption trends. Each classification module (610) comprises multiple object items (620). Each classification module may be trained to identify the type of object items registered to the classification module.

[0110] The processor (120) can select a classification module from among the multiple classification modules registered in the memory (140) based on the country in which the refrigerator is installed, the age or gender of the user, etc. For example, if the refrigerator is used in Korea, classifier 4 can be selected from the classification modules classified by country and region. On the other hand, if the refrigerator is installed in a household with two children, classifier 2 can be selected from the consumption trend-based classifiers.

[0111] Each user can create an account on the server device (100) by executing an application installed on an electronic device, such as a mobile phone. During the account creation process, each user can store not only information such as the type and model name of the electronic device they own, but also various personal information such as the user's address and number of family members. The processor (120) can select an appropriate classification module based on various information entered during the user account registration process as well as information entered separately by the user.

[0112] In addition, when the processor (120) receives at least one data among the location, season, and unique information of the users of the plurality of refrigerators through the communication unit, the processor (120) may select classification modules corresponding to the received data from among the plurality of classification modules to configure an ensemble model (630). The ensemble model (630) may be a model reconstructed by combining a plurality of classification modules. FIG. 6 illustrates a case in which an ensemble model is configured by combining a module selected from a country-region classifier or a module selected from a consumption trend-based classifier and a classifier that groups objects by similar shape.

[0113] When the ensemble model (630) is configured, the processor (120) can generate update data for changing the classification module of the first artificial intelligence model to the ensemble model.

[0114] Figure 7 is a diagram for explaining the update of a classification module according to one embodiment.

[0115] According to one embodiment, the memory (140) can store information on a plurality of items. The processor (120) can identify the types of items being stored in or shipped from the refrigerator (200) based on the information on the plurality of items stored in the memory (140). Here, the information on the plurality of items can be updated on a regular or periodic basis.

[0116] The first artificial intelligence model may be a model trained to manage a preset number and type of objects. The preset number and type of objects may refer to the number of types of objects (e.g., 30 types) that the first artificial intelligence model (210) installed in the refrigerator (200) can identify, as well as the types of each food (e.g., apples, bananas, etc.).

[0117] The processor can review the list of items in the classification model (S710). In addition, based on the result data, if the number of items among the items managed by the first artificial intelligence model (210) whose reliability is below a threshold is below a certain percentage, the processor (120) can determine whether to maintain the item among multiple item items (S720) and determine a replacement item (S730).

[0118] Here, the result data may include the reliability of the first artificial intelligence model classification module, the image of the object, information about objects that failed to be recognized, transaction frequency, and whether or not they are weighted targets.

[0119] Additionally, the processor (120) can generate update data for updating the first artificial intelligence model to manage at least one of the preset number and items of objects by replacing it with a substitute object (S760).

[0120] The server device (100) can not only individually manage the performance of multiple refrigerators, but can also group multiple refrigerators and manage their performance by group. The refrigerators can be grouped based on refrigerator information (HW model, SW model, location information, date, weather, etc.) and user characteristics (feedback input cycle, number of inputs, consumption characteristics, etc.).

[0121] Additionally, the above-described embodiments may be performed individually or all at once.

[0122] FIG. 8 is a drawing for explaining the configuration of a refrigerator according to at least one embodiment of the present disclosure.

[0123] The refrigerator (200) may include a processor (220), memory (230), communication unit (240), and camera (250).

[0124] The processor (220) can control the camera (250) to capture images of objects being stored in or shipped from the refrigerator (200), thereby obtaining images of the objects. The processor (220) can input the obtained images into the first artificial intelligence model (210) and perform management of objects stored in the refrigerator based on the output results and the time of storage and shipment of the objects.

[0125] In addition, the processor (220) can transmit the output result of the first artificial intelligence model (210) and information about objects, etc. to the server device (100) through the communication unit (240). In addition, when update data for the first artificial intelligence model is transmitted from the server device (100), the processor (220) can update the first artificial intelligence model using the transmitted update data.

[0126] In addition, the refrigerator (200) may include a display (260) and a user interface (270). The display (260) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (260) may also include a driving circuit, a backlight unit, etc., which may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0127] The processor (220) can receive user input or modification details through the user interface (270). The user interface (270) is configured to directly receive various user commands from the user. The user interface (270) can be implemented as a touch screen, a touch pad, a touch button, a push button, a wheel, etc. For example, if implemented as a touch screen, the user can directly input a user command by drawing by touching the touch screen with his or her hand or a touch pen, or can input a user command through a soft keyboard displayed on the touch screen.

[0128] According to one embodiment, the refrigerator (200) can communicate with an external device via a communication unit. The external device may include a user's terminal device or a display (260) mounted on the refrigerator. Additionally, the artificial intelligence model may be a model trained to manage a preset number and type of items.

[0129] Additionally, the processor (220) can identify at least one of the items of the items being received or shipped, the quantity of the items being received or shipped, and the change in status based on the output result of the artificial intelligence model.

[0130] The processor (220) can transmit information to an external device to indicate the management status of the object based on the identification results and the time of receipt and shipment of the object. The information to indicate the management status of the object may include information on the type of food, expiration date, quantity management, consumption habits of the food, and predictions regarding the expiration date.

[0131] If the update data includes replacement item information for changing the above item, the processor (220) can change at least one of the items of the object managed by the artificial intelligence model based on the replacement item information. Here, the replacement item information may be information for identifying the type of object.

[0132] According to one embodiment, the AI ​​model includes a classification module that identifies the type of object and a feature matching module that identifies the features of the object. Descriptions of the classification module and the feature matching module have been described above, and thus a detailed description thereof will be omitted.

[0133] The processor (220) can input an image of an object into a classification module and obtain the type of the object based on the output value of the classification module. In addition, the processor can input an image of an object into a feature matching module and obtain feature information of the object.

[0134] The output result of the artificial intelligence model may include at least one of the confidence level for the type of identified object, characteristic information of the object, state change of the object, and weight.

[0135] In one embodiment, the AI ​​model may include an object deformation compensation module that identifies objects by considering variations in feature information related to object deformation. Furthermore, if the similarity between the feature information of an outgoing object and the previously stored feature information of an incoming object is below a threshold, the processor (220) may input the image into the object deformation compensation module to obtain reliability.

[0136] Reliability can be determined based on the similarity between feature information obtained from the image of the received object and feature information obtained by inputting the image of the shipped object into the object deformation compensation module. Here, if the reliability is below a threshold, the output of the AI ​​model can be transmitted to the server device via the communication unit.

[0137] The output results may include model change requests, object feature information, and information about object state changes.

[0138] FIG. 9 is a diagram for explaining the addition of classification targets by weight of objects according to one embodiment.

[0139] According to one embodiment, the processor (220) may determine weights for all items being put into or taken out of the refrigerator based on the frequency of entering and leaving the refrigerator and store the weights in memory.

[0140] Here, the weighting may be determined based on various criteria, including not only the frequency of receipt or shipment, but also the average amount of items received based on the entered user information (gender, age, number of people), whether fruits and vegetables are received above the average for processed foods, and whether items are received continuously on a daily basis, and is not limited to the above-mentioned criteria.

[0141] Additionally, objects whose weights meet preset threshold conditions can be identified as classification targets for an artificial intelligence model. Furthermore, information about the classification targets can be transmitted to the server device (100).

[0142] The processor (220) may add items with high incoming and outgoing weights as classification targets. However, the criteria described above are not limited to these items, and items with only high incoming weights or only high outgoing weights may also be classified. The classification target may refer to an item that can identify the type of item being incoming or outgoing from the refrigerator (200) based on information about the item.

[0143] Figures 10 to 12 are drawings showing a process of transmitting performance diagnosis of the first artificial intelligence model (210) and result data according to use of the first artificial intelligence model to a server device (100) when a product is received or shipped from a refrigerator (200).

[0144] Figure 10 is a diagram showing a diagnosis of the first artificial intelligence model when a product is received according to one embodiment.

[0145] When an object is received, the processor (220) can capture an image of the product. The image of the object can be input into a classification module to obtain a reliability rating for the object type (S1010). Here, if the reliability rating for the highest object type (TOP1) is less than a threshold (S1020) or the difference between the reliability rating for the highest object type (TOP1) and the reliability rating for the second highest object type (TOP2) is less than a threshold (S1030), the object can be identified as a storage target (S1060).

[0146] For example, let's assume that an object A is delivered. The processor (220) can take a picture of object A and input it into the classification module and the feature matching module. Here, the classification module can obtain the result value as a reliability value by type. That is, it can obtain values ​​for the probability of being pork, the probability of being beef, and the probability of being chicken. In this case, let's say that the probability of being pork is the highest value (TOP1) and the probability of being beef is the second highest value (TOP2). If the probability of being pork is less than a threshold value or the difference between the probability of being pork and the probability of being beef is less than a threshold value, the processor (220) can determine that the type of the object has not been identified and identify it as a storage target. Here, the processor (220) can store the feature information extracted from the image of the object as classification failure feature information.

[0147] Figure 11 is a diagram illustrating a diagnosis of the first AI model when a product is shipped, according to one embodiment. Specifically, this diagram illustrates the diagnosis of the classification module and the product modification supplement module when a product is shipped.

[0148] Since the diagnosis of the classification module has been described above, any duplicate content will be omitted.

[0149] The processor (220) can input the image of the object into the product deformation supplement module (S1120) when the similarity between the feature information acquired from the image of the object and the previously stored feature information photographed when the object is received is less than a threshold (S1110).

[0150] Here, if the similarity is less than the threshold based on the result value input to the object supplementation module and the object characteristic information stored at the time of receipt (S1130), the processor (220) can store it as classification failure characteristic information (S1140).

[0151] FIG. 12 is a diagram illustrating a process of transmitting result data to a server device (100) according to one embodiment.

[0152] If the feature information saved as a classification failure is similar to the feature information saved as a classification failure due to a transfer failure (S1210) and the object is a weighting target (S1220), the processor (220) can transmit the result data to the server device (100) (S1240).

[0153] In addition, if the feature information that is not a weighting target or is saved as a classification failure is not similar to the feature information that is saved as a classification failure due to a transfer failure, the processor (220) can complete the receipt or shipment of goods without transmitting the corresponding result data to the server device (100) (S1230).

[0154] FIG. 13 is a flowchart for explaining a method for controlling a server device according to at least one embodiment of the present disclosure.

[0155] According to FIG. 13, the server device can receive result data using the first artificial intelligence model from a plurality of refrigerators, each of which has the first artificial intelligence model installed (S1310).

[0156] In addition, information about an object is input into a second artificial intelligence model installed in a server device to obtain a result value (S1320), and the performance of the first artificial intelligence model of each of a plurality of refrigerators can be diagnosed based on the obtained result value and result data (S1330). Specifically, the accuracy of the first artificial intelligence model can be obtained. In addition, based on the diagnosis result, update data for updating the first artificial intelligence model of at least one refrigerator among the plurality of refrigerators can be configured (S1340), and the update data can be transmitted to the plurality of refrigerators (S1350). That is, the server device can configure the update data when the obtained accuracy is below a threshold.

[0157] Here, accuracy can be information indicating how reliable the output values ​​of the first AI model are. For example, as in the various embodiments described above, the similarity between the feature information output by the first AI model and the feature information output by the second AI model based on images of the same object can be used as accuracy.

[0158] FIG. 13 can be performed by a server device having the configuration of FIGS. 3 to 7, but is not necessarily limited thereto and can be performed by a server device having various other configurations.

[0159] The methods according to the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing server device.

[0160] The methods according to the various embodiments of the present disclosure described above can be implemented with only a software upgrade or a hardware upgrade for an existing server device.

[0161] The various embodiments of the present disclosure described above may also be performed through an embedded server provided in a server device, or an external server of at least one of the server device and the display device.

[0162] According to an example embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include a server device according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When the instructions are executed by a processor, the processor may directly or under the control of the processor perform a function corresponding to the instructions using other components. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily in the storage medium.

[0163] According to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0164] Each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of one or more entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0165] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea of ​​the present disclosure.

Claims

1. For server devices, A communication unit for communicating with multiple refrigerators, each equipped with a first artificial intelligence model; Memory in which the second artificial intelligence model is stored; and Processor; including; The above processor, Receive data and information about objects using the first artificial intelligence model in the plurality of refrigerators through the communication unit and store them in the memory, Inputting information about the above object into the second artificial intelligence model to obtain a result value, Diagnose the performance of the first artificial intelligence model of each of the plurality of refrigerators based on the above result value and the above result data, Based on the diagnosis results, update data is configured to update the first artificial intelligence model for at least one refrigerator among the plurality of refrigerators, A server device that controls the communication unit to transmit the update data to at least one refrigerator.

2. In paragraph 1, Information about the above items, Includes images of items being received or shipped, The above result data is, It includes at least one of first feature information corresponding to the image of the object, reliability of the first artificial intelligence model classification module, and information on the deformation state of the object. The above processor, Inputting the image of the above object into the second artificial intelligence model to obtain the result value, The accuracy of the first artificial intelligence model is obtained based on the above result value and the above result data, A server device that configures update data for updating the first artificial intelligence model when the accuracy is below a threshold.

3. In paragraph 2, The above second artificial intelligence model is, Includes a feature matching module for obtaining feature information of an object from an image of the object, The above processor, By inputting the image of the above object into the feature matching module, second feature information is obtained, Obtain the similarity between the second feature information and the first feature information, If the above similarity is below the threshold, A server device that generates the update data for updating the first artificial intelligence model based on the feature matching module.

4. In paragraph 2, The above second artificial intelligence model is, It includes an object deformation supplement module for identifying the object by considering the change in the above characteristic information regarding the deformation of the object, The above processor, By inputting the images of the plurality of objects into the object deformation supplementation module, second feature information corresponding to the deformation state of each of the plurality of objects is obtained, and the similarity between the second feature information and the first feature information is obtained. If the above similarity is below the threshold, A server device that generates the update data for updating the first artificial intelligence model based on the object deformation supplement module.

5. In paragraph 1, The above memory is, Store multiple classification modules that are generated differently based on at least one of the following criteria: type of object, country, region, season, date, consumption trend, and user characteristics; The above processor When at least one of the data of the location, season, and unique information of the user of the plurality of refrigerators is received through the communication unit, An ensemble model is configured by selecting at least one classification module corresponding to the received data from among the plurality of classification modules, A server device that generates the update data for changing the classification module of the first artificial intelligence model to the ensemble model.

6. In paragraph 1, The above memory is, Store information about multiple object items, The above first artificial intelligence model is a model trained to manage objects of a preset number and item, The above processor, Based on the above-described received result data, if the number of objects among the items managed by the first artificial intelligence model whose reliability is below a threshold is below a certain percentage, Determine a substitute item from among the above multiple items, A server device that generates the update data for updating the first artificial intelligence model to manage at least one of the objects of the above-described number and items by replacing it with the substitute object.

7. In the refrigerator, camera; A communication unit for communicating with a server device; Memory for storing artificial intelligence models; and Processor; including; The above processor, Control the camera to capture an object being received or delivered from the refrigerator, thereby obtaining an image of the object; By inputting the image into the artificial intelligence model, management of the objects stored in the refrigerator is performed based on the output result of the artificial intelligence model and the entry / exit time of the objects. Transmitting the output result of the artificial intelligence model to the server device through the communication unit, A refrigerator that updates the artificial intelligence model using the transmitted update data when update data for the artificial intelligence model is transmitted from the server device.

8. In paragraph 7, The above communication unit performs communication with a user terminal device, The above artificial intelligence model is a model trained to manage objects of a preset number and type. The above processor, Based on the output result of the artificial intelligence model, at least one of the items of the goods being received or shipped, the quantity of the goods being received or shipped, and the change in status is identified, Based on the identification result and the receipt time and shipment time of the object, information for notifying the management status of the object is transmitted to the user terminal device, A refrigerator that changes at least one of the items of the object managed by the artificial intelligence model according to the replacement item information, if the above update data includes replacement item information for changing the above item.

9. In paragraph 7, The above artificial intelligence model includes a classification module that identifies the type of the object and a feature matching module that identifies the features of the object. The above processor, Inputting the image of the object into the classification module and identifying the type of the object based on the result value of the classification module, By inputting the image of the above object into the feature matching module, feature information of the above object is obtained, The output result of the above artificial intelligence model is: A refrigerator comprising at least one of a model change request, a confidence level for the type of the identified object, characteristic information of the object, a state change of the object, and a weight.

10. In paragraph 9, The above artificial intelligence model is, It includes an object deformation supplement module that identifies an object by considering the change in the above characteristic information regarding the deformation of the object, The above processor, If the similarity between the characteristic information of the shipped item and the characteristic information of the stored incoming item is below the threshold, The image of the above-mentioned object is input into the object deformation supplementation module to obtain second feature information, Reliability is obtained based on the similarity between the second feature information and the feature information of the stored item, A refrigerator that transmits the output result of the artificial intelligence model to the server device through the communication unit when the reliability is below a threshold.

11. In paragraph 9, The above processor, Input the above image into the above classification module to obtain reliability, A refrigerator that transmits data according to the use of the artificial intelligence model to a server device when the reliability is below a threshold.

12. In paragraph 9, The above processor, For all items being received or delivered to the refrigerator, weights are determined based on the frequency of receipt and delivery and stored in the memory. The above weight identifies objects that meet the preset threshold conditions as classification targets of the artificial intelligence model. A refrigerator that transmits information about the above classification target to the server device.

13. In the method of controlling the server device A step of receiving data and information about objects using the first artificial intelligence model in a plurality of refrigerators, each of which has the first artificial intelligence model installed; A step of inputting information about the above object into a second artificial intelligence model installed in the server device and obtaining a result value; A step of diagnosing the performance of the first artificial intelligence model of each of the plurality of refrigerators based on the result value and the result data; A step of configuring update data for updating the first artificial intelligence model for at least one refrigerator among the plurality of refrigerators based on the diagnosis results; and A control method comprising: a step of transmitting the update data to at least one refrigerator; 14. In paragraph 13, Information about the above items, Includes images of items being received or shipped, The above result data is, It includes at least one of first feature information corresponding to the image of the object, reliability of the classification model of the first artificial intelligence model, and information on the deformation state of the object. The above control method is, A step of inputting an image of the above object into the second artificial intelligence model to obtain the result value; A step of obtaining the accuracy of the first artificial intelligence model based on the above result value and the received result data; and A control method further comprising: a step of configuring update data for updating the classification model of the first artificial intelligence model when the accuracy is below a threshold; 15. In paragraph 14, The above second artificial intelligence model is, Includes a feature matching module for obtaining feature information of an object from an image of the object, The above control method is, A step of obtaining second feature information by inputting an image of the object into the feature matching module; A step of obtaining the similarity between the second feature information and the first feature information; If the above similarity is below the threshold, A control method further comprising: a step of generating the update data for updating the first artificial intelligence model based on the feature matching module;

Images