Information processing device, information processing method, and storage system

The information processing device improves food recognition accuracy in refrigerators by calculating user interest levels and utilizing user-labeled data to refine the recognition model, addressing the low accuracy issue in existing systems.

JP2024163666APending Publication Date: 2024-11-22HITACHI GLOBAL LIFE SOLUTIONS INC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023079479
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing food recognition models in refrigerators suffer from low accuracy, and there is a lack of effective methods to update training data for improving identification accuracy.

Method used

An information processing device that includes an object recognition unit and an interest degree calculation unit, which calculates the interest level of users in specific items based on their handling tendencies, allowing users to label images with high interest, and uses this data to refine the recognition model through supervised learning.

Benefits of technology

Enhances the accuracy of food recognition in refrigerators by leveraging user interest levels to improve the training data, enabling more precise identification of food items and allowing personalized model learning for individual users.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024163666000001_ABST
    Figure 2024163666000001_ABST
Patent Text Reader

Abstract

To attach a correct label to an image of learning data.SOLUTION: An information processing device of the present invention includes: an object recognition unit which recognizes an object contained in a unit image; and an interest degree calculation unit which estimates a tendency when a user handles the object on the basis of the plurality of different unit images, and calculates the interest degree to the objects for each user on the basis of the estimated tendency. Further, the information processing device includes a label requesting unit which request a user having a high interest degree so as to satisfy a prescribed standard to attach label information to the unit image.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a storage system. [Background technology]

[0002] Recently, refrigerators that automatically infer what ingredients are stored inside have become popular. The item storage device of Patent Document 1 manages the features of images of items by linking them to the area in which the features were detected, and if similar features appear repeatedly in the same area, it determines that the features are features of a commonly-stocked item. The refrigerator system of Patent Document 2 pre-stores learned data that has learned the relationship between the image features of ingredients in the refrigerator and the ingredients, and if the classification result is correct in a new classification opportunity, it performs re-learning using the image features related to that classification and the relationship between the ingredients. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6775467 specification [Patent Document 2] Patent No. 7040193 specification Summary of the Invention [Problem to be solved by the invention]

[0004] When using a model such as a neural network to infer the ingredients in a refrigerator, the accuracy of the model becomes an issue. If the accuracy is low, the training data used to train the model must be updated. The article storage device of Patent Document 1 does not assume the use of a model. The refrigerator system of Patent Document 2 does not consider the types of ingredients, and as a result, it lacks a perspective on how to prepare learning data for items with low classification accuracy. [Means for solving the problem]

[0005] The information processing device of the present invention is characterized by comprising an object recognition unit that recognizes objects contained in a unit image, and an interest level calculation unit that estimates a user's tendency to handle the object based on a plurality of different unit images, and calculates a level of interest in the object for each user based on the estimated tendency. Other means will be described in the description of the embodiment of the invention. [Brief description of the drawings]

[0006] [Figure 1] FIG. 2 is a diagram illustrating a configuration of an information processing device. [Diagram 2] 13 is an example of storage facility usage information. [Diagram 3] 11 is an example of user information. [Figure 4] 13 is a flowchart of a procedure for calculating a level of interest. [Diagram 5] 13 is a flowchart of a label request process. [Figure 6] 13 is an example of a label request screen. [Figure 7] 13 is an example of a label setting screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an embodiment of the present invention (referred to as the "present embodiment") will be described in detail with reference to the drawings. The present embodiment is an example in which a recognition model recognizes (infers) the types of ingredients stored in a refrigerator. However, the present invention can also be applied to cases in which objects other than food are stored in a storage unit other than a refrigerator. The present invention can also be applied to cases in which medicines, specimens, etc. are stored in a refrigerator as items other than ingredients.

[0008] (term) An object is a tangible thing that has a shape and whose image can be captured. An ingredient is an object that can be eaten by humans. An item is a common noun that indicates a type of object. A unit image is an image (photograph) that contains a single object. A recognition model is a function that takes a unit image as input and an item as output. A typical recognition model is a neural network (artificial intelligence) that has an input layer, a hidden layer, and an output layer.

[0009] The supervised learning data is a set of combinations of unit images and labels. A label is an item of an object that is attached to a unit image. The user is a user of a repository that stores objects, and is also a user of an information processing device. A requesting user is a user who requests that a correct label be attached to a unit image. A collaborative user is a user who correctly labels unit images. The interest level is the degree to which a user (especially a cooperating user) is interested in an item, and is defined for a combination of a user and an item.

[0010] (Learning and interest in cognitive models) In the recognition model, a propagation parameter is defined for each node in the intermediate layer, which indicates how much information from a node in the intermediate layer is propagated to which node in the next intermediate layer. The propagation parameter of the recognition model can be optimized using supervised learning data. This is generally expressed as "training" the recognition model. A trained recognition model can output the correct item with high accuracy. Suppose that user A has a sufficiently high interest in item B. In this case, the supervised learning data is a unit image in which item B is thought to appear, to which user A has given the label "item B". If the recognition model is trained using this supervised learning data, the accuracy with which the recognition model recognizes item B will increase. In the following, "learning data" means supervised learning data unless otherwise specified.

[0011] (Configuration of information processing device) 1 is a diagram for explaining the configuration of an information processing device 1. The information processing device 1 is a general computer. The information processing device 1 includes a central control device 11, an input device 12 such as a keyboard or a mouse, an output device 13 such as a display, a main memory device 14, an auxiliary memory device 15, and a communication device 16.

[0012] The object recognition unit 21, interest level calculation unit 22, and label request unit 23 in the main memory 14 are programs and are the main units of each process described below. Hereinafter, when "XX unit" is written, it means that the central control device 11 reads each program from the auxiliary memory 15, loads it into the main memory 14, and then executes each process. The auxiliary memory 15 stores repository usage information 31 (described in detail later), user information 32 (described in detail later), learning data 33, and recognition model 34.

[0013] There are multiple kitchens 2 located away from the information processing device 1. In the kitchen 2, there are a refrigerator 6 and a user 4. The refrigerator 6 is equipped with a camera 7 inside or outside. The camera 7 captures unit images of ingredients 8 stored in the refrigerator 6. The user 4 operates a terminal device 5. The terminal device 5 is typically a portable smartphone, but may also be a stationary personal computer. The camera 7 and the terminal device 5 are connected to the information processing device 1 via a wired or wireless network 3. The information processing device 1 and the refrigerator (storage unit) 6 constitute a storage system.

[0014] (Storage facility usage information) 2 is an example of the storage unit usage information 31. In the storage unit usage information 31, in association with the unit image ID stored in the unit image ID column 101, a user ID is stored in a user ID column 102, an image capture time is stored in an image capture time column 103, an item is stored in an item column 104, trend information is stored in a trend information column 105, and an interest level is stored in an interest level column 106.

[0015] The image ID in the image ID column 101 is an identifier for uniquely identifying an image. The user ID in the user ID column 102 is an identifier that uniquely identifies a user. In this embodiment, one user uses one refrigerator 6, so the user ID is also an identifier that uniquely identifies the refrigerator 6. The imaging time in the imaging time column 103 is the year, month, day, hour, minute and second of the time when the camera 7 captured a unit image of the food ingredient 8. The camera 7 captures images of all the food ingredients 8 in the refrigerator 6 every time the user 4 opens or closes the door of the refrigerator 6, or periodically (for example, once every hour). "#" indicates a different value as an abbreviation (same below).

[0016] The item in the item column 104 is the item of the ingredient 8. The item here may be the output of the recognition model 34 in response to the input of a unit image, or may be obtained by other means. An example of other means is a method in which an item stored in a tag attached to the ingredient 8 is read by a tag reader (not shown). When the item is the output of the recognition model 34, it is not necessarily completely correct.

[0017] The trend information in the trend information column 105 includes a storage location (column 105a), a storage period (column 105b), and a storage count (column 105c). The storage location is a three-dimensional coordinate value of the location in the refrigerator 6 where the ingredient 8 is stored (taken in and out). The three-dimensional coordinate system here is, for example, a local coordinate system with a certain point in the refrigerator 6 as the origin. Users tend to store ingredients 8 of interest in a specific storage location (for example, near the chest height) that is easy to take in and out. The specific storage location here is called the "area of ​​interest." "[#,#,#]" indicates that the location is included in the area of ​​interest. The area of ​​interest is defined for each user.

[0018] The storage period is the period during which the same individual food ingredient 8 has been stored in the refrigerator 6. The storage count is the number of times that the same individual food ingredient 8 has been taken in and out of the refrigerator during a given period. The storage position, storage period, and storage count described here are merely examples of trend information. In general, trend information refers to a tendency of a user to handle the object, which is estimated based on a plurality of different unit images.

[0019] The degree of interest in the interest level field 106 is the degree to which the user is interested in an item. The degree of interest is calculated by any method depending on the storage location, storage period, and storage count. For example, the longer the storage location has been in the area of ​​interest, the longer the storage period, and the more times it has been stored, the higher the degree of interest. The interest level here is normalized from 1 to 10. The higher the number, the higher the degree of interest.

[0020] Looking at Figure 2, for example, we can see the following: User U001 stores apples, canned beer, and other items in his refrigerator 6. User U001 stores apples in the region of interest. The storage period and storage frequency of user U001 for apples are sufficiently large. Therefore, user U001's interest level in apples is “10”. If user U001 assigns the label "apple" to a unit image whose item is likely to be "apple," the recognition model 34 trained with the training data that is the combination of that unit image and label is likely to have high accuracy in correctly recognizing apples. Furthermore, it may be possible to recognize apples in more detail, such as "Aomori apples" and "Nagano apples." Similarly, if user U002 assigns the label "natto" to a unit image that is likely to be "natto," the recognition model 34 trained with the training data that is a combination of that unit image and label is likely to have high accuracy in recognizing natto. Furthermore, it may be possible to recognize natto in more detail, such as "crushed natto" or "large grain natto."

[0021] (User information) 3 is an example of the user information 32. In the user information 32, attribute information is stored in an attribute information column 112, items are stored in an item column 113, interest levels are stored in an interest level column 114, and interest items are stored in an interest item column 115 in association with a user ID stored in a user ID column 111.

[0022] The user ID in the user ID column 111 is the same as the user ID in FIG. The attribute information in the attribute information column 112 includes gender (column 112a), age (column 112b), occupation (column 112c), family structure (column 112d), address (column 112e), and words of interest (column 112f). Among these, gender is the gender of the user. Age is the age of the user. Occupation is the occupation of the user. Family structure is the relationship of family members living with the user. Address is the name of the municipality in which the user lives. The words of interest are one or more characteristic words that appeared in documents viewed by the user using his / her terminal device 5 during a specified period of time recently.

[0023] The items in the item column 113 are the same as those in FIG. The interest level in the interest level column 114 is the same as that in FIG. The items of interest in the item of interest column 115 are items that the user is recommended to label to unit images due to the user's high interest in them. Here, if the interest level for an item is equal to or higher than a predetermined threshold (here, "8"), the item becomes an item of interest.

[0024] Looking at Figure 3, for example, we can see the following: User U001 has a high level of interest in all items. Among them, the level of interest in apples and beef is above the threshold. User U001 is a middle-aged female office worker living alone in a city. User U001 may like to eat beef because she has a large disposable income. User U001 may also like to eat sweet apples to relieve stress. User U002 has an interest in natto that is equal to or greater than a threshold. User U002 is an elderly male farmer who lives with his wife in the suburbs. User U002 cultivates soybeans and consumes them for his own consumption, so he may like to eat natto.

[0025] (Interest Level Calculation Process) FIG. 4 is a flowchart of a procedure for calculating the level of interest. In step S201, the interest level calculation unit 22 of the information processing device 1 acquires attribute information. Specifically, first, the interest level calculation unit 22 accepts input of gender, age, occupation, family structure, and address by all users 4 via their own terminal devices 5. Secondly, the interest level calculation unit 22 acquires browser browsing history from the terminal devices 5 used by all users 4 over a specified period in the recent past (e.g., one week), and acquires one or more words of interest based on the browsing history.

[0026] In step S202, the interest level calculation unit 22 creates user information 32 (FIG. 3). Specifically, the interest level calculation unit 22 uses the information received or acquired in step S201 to fill in the user ID column 111 and the attribute information column 112 of the user information 32. At this stage, the item column 113, the interest level column 114, and the item of interest column 115 are blank.

[0027] In step S203, the object recognition unit 21 of the information processing device 1 acquires unit images. Specifically, over a predetermined period, the object recognition unit 21 acquires unit images of ingredients 8 captured by the camera 7 for all users 4. The object recognition unit 21 continues to acquire unit images together with the time each time the door of the refrigerator 6 is opened or closed, or each time a predetermined period elapses, for example, for seven days.

[0028] In step S204, the interest level calculation unit 22 of the information processing device 1 calculates the interest level. Specifically, first, the interest level calculation unit 22 fills in the unit image ID column 101, the user ID column 102, the image capture time column 103, the item column 104, and the trend information column 105 of the storage unit usage information 31 (FIG. 2) using the information acquired in step S203. At this time, the interest level calculation unit 22 may acquire the item (which may not be correct) using an existing recognition model 34. The interest level calculation unit 22 creates and calculates trend information for all users 4 by analyzing the unit images acquired in step S203 in chronological order.

[0029] Secondly, the interest level calculation unit 22 calculates the interest level based on the tendency information using the above-mentioned method, and stores the result in the interest level column 106 of the repository usage information 31.

[0030] In step S205, the interest level calculation unit 22 completes the user information 32 (FIG. 3). Specifically, the interest level calculation unit 22 fills in the item column 113 and the interest level column 114 of the user information 32 using the interest level calculated in "second" of step S204. Then, the interest level calculation unit 22 fills in the interest item column 115 by identifying items whose interest level is equal to or greater than a predetermined threshold as interest items for each user. There may be cases where no interest items are identified for a certain user. Then, the interest level calculation process procedure ends.

[0031] (Label Request Processing Procedure) Fig. 5 is a flowchart of the label request processing procedure. Fig. 6 and Fig. 7 will be referred to as appropriate during the explanation of Fig. 5. For convenience of explanation, it is assumed that a user U101 is dissatisfied with the accuracy of the recognition model 34. The user U101 is a "requesting user."

[0032] In step S301, the label request unit 23 of the information processing device 1 displays a label request screen 51 (FIG. 6). Specifically, the label request unit 23 displays the label request screen 51 on the terminal device 5 of the user U101 in response to an operation of the user U101 on the terminal device 5. The label request unit 23 identifies the unit images 52a to 52d that have had low recognition accuracy in the past, for example, by the following method, and displays these on the label request screen 51. The unit images 52a to 52d are also called "candidate unit images."

[0033] <Method 1> The label request unit 23 sets as candidate unit images those unit images of ingredients 8 stored by the user U101 in the refrigerator 6 that have had low recognition accuracy in the past (the recognition model 34 output an incorrect item). In the learning data when the recognition model 34 is trained, the labels given as items of the candidate unit images may be incorrect. <Method 2> The label request unit 23 sets as a candidate unit image the unit image of the ingredient 8 that the user U101 first puts in the refrigerator 6. The recognition model 34 cannot output the correct item for such a candidate unit image in the first place.

[0034] In step S302, the label request unit 23 accepts a label request. Specifically, the label request unit 23 accepts the selection of one or more “target unit images” from among the candidate unit images 52a to 52d by the user U101 pressing the request buttons 53a to 53d. For convenience of explanation, it is assumed that the user U101 presses the request button 53a. The label request unit 23 determines that the item indicated by the target unit image 52a is an “apple” based on many past recognition results. However, the label request unit 23 may accept the user U101 inputting a character string of an item (target item) for which the user U101 wishes to improve the recognition accuracy. This is because the user may wish to improve the recognition accuracy of a “pear” that is similar in appearance to an “apple”. For convenience of explanation, it is assumed that the user U101 inputs the character string “apple”. It has been confirmed that the target item is an “apple”.

[0035] In step S303, the label request unit 23 identifies a cooperating user. Specifically, the label request unit 23 searches the interest item column 115 of the user information 32 (FIG. 3) using "apple" as a search key, and obtains the user ID of the corresponding record. In this example, the user U001 is the "cooperating user."

[0036] In step S304, the label request unit 23 displays the label setting screen 61 (FIG. 7). Specifically, the label request unit 23 displays the label setting screen 61 on the terminal device 5 of the user U001. The label request unit 23 displays as many unit images 64a to 64d as possible that are thought to show the target item "apple" in the unit image field 62 of the label setting screen 61. The maximum population of unit images displayed here is the unit images of "apple" captured in the refrigerators 6 of all users 4.

[0037] In step S305, the label request unit 23 accepts a label. Specifically, first, the label request unit 23 displays a plurality of clusters 65a to 65d in the classification column 63 of the label setting screen 61. Each of the clusters 65a to 65d has a label input window 66a to 66d. The label request unit 23 accepts the user U001 inputting a character string (apple, Aomori apple, Nagano apple, ●●, . . .) to be a label in each of the label input windows 66a to 66d. Next, the label request unit 23 accepts the copying (drag + paste) of each of the unit images 64a to 64d to one of the clusters 65a to 65d. Furthermore, the label request unit 23 may accept the addition of meta information 67 such as a character string to any of the unit images after copying. Note that the "label information" is a concept including a label and meta information.

[0038] Secondly, the label request unit 23 stores the combination of the unit image and the label as learning data 33. For example, the label request unit 23 stores the combination of the unit image 64a and the label "apple" 66a as the learning data 33 in the auxiliary storage device 15. For example, the label request unit 23 stores the combination of the unit image 64d, the label "Aomori apple" 66b, and the meta information "Bumper harvest this year, especially sweet" 67 as the learning data 33 in the auxiliary storage device 15. Thereafter, the label request process procedure is terminated.

[0039] The object recognition unit 21 uses the learning data 33 thus accumulated to learn the recognition model 34. Then, the object recognition unit 21 uses the learned recognition model 34 to recognize the item of the object included in the unit image. Furthermore, the object recognition unit 21 may learn the recognition model 34 for each requesting user. That is, the object recognition unit 21 stores a user ID that identifies the requesting user in association with the recognition model 34. Then, the object recognition unit 21 learns the recognition model 34 associated with the user U103 with learning data labeled in response to a label request from the user U103, for example.

[0040] (Variation 1) In step S304, the label request unit 23 may cut out a portion in which an object is captured from the unit image, and display the cut-out portion in the unit image field 62. Then, the label request unit 23 may replace the unit image before cutting out with the unit image after cutting out.

[0041] (Variation 2) The label request unit 23 may classify the cooperating users into a plurality of groups by using the attribute information (field 112 in FIG. 3). The following are assumed as examples of classification here. · Create groups of users who live close to each other. · Create groups of users with similar ages, occupations, or family structures. · Create groups of users with similar keywords of interest.

[0042] Then, the label request unit 23 may request users belonging to each of the multiple groups to attach labels to unit images of different items (e.g., apples to group A, beef to group B, natto to group C, ...). Furthermore, the label request unit 23 may request users belonging to each of the multiple groups to attach labels to unit images of the same item (e.g., apples to groups A, B, and C).

[0043] (Variation 3) In step S304, the label request unit 23 may display unit images of multiple items to one cooperating user. At this time, the label request unit 23 displays the unit images in descending order of interest level. For example, if the interest levels are apples > canned beer > natto, the label request unit 23 may display the unit image of apples first in terms of time, or may display it at the top of the screen.

[0044] (Variation 4) In step S304, the label requesting unit 23 may request that a label be attached to the unit image only for an item that is often stored in a predetermined position (for example, ice cream stored in a freezer). The predetermined position may be a region of interest.

[0045] (Variation 5) In step S304, the label request unit 23 may display, for example, a unit image whose item is a "pear" in the unit image column 62 in addition to a unit image whose item is an "apple". At this time, the label request unit 23 determines that these unit images are close to each other in the feature space, regardless of whether they appear to be "apples" or "pears". The feature space is a multidimensional space having, on its axes, each element of the feature vector of the unit image or the feature vector of the character string indicating the item reflected in that unit image.

[0046] (Effects of this embodiment) The effects of the information processing device of this embodiment are as follows. (1) The information processing device enables a user who is highly interested in an object to attach a label to an image of the ingredient. (2) The information processing device can train a model using the labeled images as training data. (3) The information processing device can extract only the portion of the image in which the object is captured and display it to the user. (4) The information processing device enables a user to label images of the same or different objects according to attributes of the user.

[0047] (5) The information processing device can calculate the level of interest of the user based on the position where the user handled the object, etc. (6) The information processing device can display images of objects to the user in order of the degree of interest. (7) The information processing device allows a user to attach a label to an image of an object stored in a given location. (8) The information processing device allows a user to group and label images of objects that have similar characteristics. (9) The information processing device can learn a model for each user.

[0048] The present invention is not limited to the above-described embodiment, but includes various modified examples. For example, the above-described embodiment has been described in detail to easily explain the present invention, and is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0049] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0050] 1. Information processing device 2. Kitchen 3. Network 4. Users 5 Terminal Equipment 6 Refrigerator (storage room) 7. Camera 8 Ingredients (objects) 11 Central control unit 12 Input Devices 13 Output Devices 14 Main memory 15 Auxiliary storage 16 Communication Equipment 21 Object recognition section 22 Interest Level Calculation Unit 23 Label request section 31 Storage facility usage information 32 User Information 33 Training data (supervised training data) 34 Recognition Model 51 Label request screen 61 Label setting screen

Claims

1. an object recognition unit for recognizing an object included in a unit image; Estimating a tendency of a user to handle the object based on the plurality of different unit images; an interest level calculation unit that calculates an interest level in the object for each of the users based on the estimated tendency; To have An information processing device comprising:

2. a label request unit that requests the user whose interest level is high enough to satisfy a predetermined criterion to assign label information to the unit image; 2. The information processing device according to claim 1,

3. The object recognition unit is Using the unit images to which the label information is added as learning data, learning a model in which the unit images are input and the item of the object is output; 3. The information processing apparatus according to claim 2,

4. The label request unit: extracting a portion in which the object is captured from the unit image, and displaying the extracted portion of the object to the user who has a high degree of interest; 4. The information processing apparatus according to claim 3,

5. The label request unit: Classifying the users into a plurality of groups according to attributes of the users; Requesting the users belonging to each of the plurality of groups to attach label information to unit images including the same object, or requesting the users to attach labels to unit images including different objects; 5. The information processing apparatus according to claim 4,

6. The interest level calculation unit Calculating the interest level based on a position where the user handled the object or the number of times the user handled the object; 6. The information processing device according to claim 5,

7. The interest level calculation unit Calculating the interest level for each of the objects; The label request unit: displaying unit images including the object to the users having a high level of interest in the object in descending order of the level of interest; The information processing device according to claim 6 .

8. The label request unit: requesting the user, whose interest level is high enough to satisfy a predetermined criterion, to attach label information to the unit images of the objects stored in a predetermined position in the storage facility; The information processing device according to claim 7 .

9. The label request unit: requesting the user, whose interest level is high enough to satisfy a predetermined criterion, to assign label information to the unit images related to a plurality of objects that are close to each other in a feature space; The information processing device according to claim 8 .

10. The object recognition unit is training said model for each said user; 4. The information processing apparatus according to claim 3,

11. The object recognition unit of the information processing device Recognize objects contained in unit images, The interest level calculation unit of the information processing device Estimating a tendency of a user to handle the object based on the plurality of different unit images; calculating a degree of interest in the object for each of the users based on the estimated tendency; An information processing method comprising:

12. an object recognition unit for recognizing an object included in a unit image; Estimating a tendency of a user to handle the object based on the plurality of different unit images; an interest level calculation unit that calculates an interest level in the object for each of the users based on the estimated tendency; An information processing device comprising: A storage unit for storing the object; A storage system comprising:

Citation Information

Patent Citations

  • Item storage device and item management system

    JP6775467B2

  • Refrigerator System

    JP7040193B2