Method for managing articles, refrigeration device and system for managing articles

Abstract

The present application relates to the technical field of smart home appliances. Disclosed is a method for managing articles, comprising: collecting an image of a target space, and acquiring information of articles stored in the target space; on the basis of the information of the articles, acquiring replacement icons of the articles; and, at an interactive end, displaying the replacement icons of the articles, or displaying the information of the articles while displaying the replacement icons of the articles.

Description

Method for managing items, refrigeration device and system for managing items

[0001] This application is based on and claims priority to Chinese Patent Application No. 202411301384.0, filed on September 18, 2024, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of smart home appliances, for example, to a method for managing items, a refrigeration device and a system for managing items. BACKGROUND

[0003] A refrigerator is a commonly used home appliance for freezing and preserving items. With the development of technology, users have higher requirements for the intelligence of refrigerators, for example, users want to interact with the refrigerator.

[0004] To achieve the interaction between the refrigerator and the user, in the related art, the display screen of the refrigerator displays a real image, and after the user clicks the real image, the display screen displays further information.

[0005] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art:

[0006] Although the related art can achieve the interaction between the refrigerator and the user, since the items in most users' refrigerators are placed in disorder, when the user clicks the real image displayed on the display screen, the problem of mistaken clicking is likely to occur.

[0007] It should be noted that the information disclosed in the above BACKGROUND section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0008] To have a basic understanding of some aspects of the disclosed embodiments, a brief overview is given below. The overview is not an overview in general, nor is it intended to determine key / important constituent elements or delineate the scope of protection of these embodiments, but as a prelude to the detailed description below.

[0009] The embodiments of the present disclosure provide a method for managing items, a refrigeration device and a system for managing items to improve the accuracy of user interaction with images.

[0010] In some embodiments, the method for managing items comprises: collecting an image of a target space, obtaining information of stored items in the target space; obtaining a substitute icon of the item according to the information of the item; displaying the substitute icon of the item on the interaction end, or displaying the substitute icon of the item while displaying the information of the item.

[0011] In some embodiments, the refrigeration device comprises: a cabinet and a cabinet door, the cabinet and / or the cabinet door are provided with a target space for storing articles; an image acquisition device configured to acquire an image of the target space; a first processor configured to analyze the image of the target space to obtain article information; obtain a substitute icon of the article according to the article information; and an interactive terminal configured to display the substitute icon of the article; or in response to a query command of a user, display the substitute icon of the article and information of the article at the same time.

[0012] In some embodiments, the system for managing articles comprises: a refrigeration device comprising an image acquisition device configured to acquire an image of a target space and transmit the image of the target space to a cloud server; the cloud server configured to analyze the image of the target space to obtain article information after receiving the image of the target space; obtain a substitute icon of the article according to the article information; and an interactive terminal of a remote interactive device configured to display the substitute icon of the article; or in response to a query command of a user, display the substitute icon of the article and information of the article at the same time.

[0013] The method for managing articles, the refrigeration device and the system for managing articles provided by the embodiments of the present disclosure can achieve the following technical effects:

[0014] Firstly, the article information is obtained based on the image of the target space, and the substitute icon of the article is obtained according to the article information. Then, the substitute icon of the article is displayed on the interactive terminal, or the substitute icon of the article and the information of the article are displayed at the same time. Compared with the real image, the substitute icon of the article is beautified, more neat, and more conducive to the user to view and / or click the corresponding article, thereby improving the accuracy of the user interaction with the image and enhancing the user experience.

[0015] The foregoing general description and the following description are only exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute a limitation on the embodiments, and elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute a proportional limitation, and wherein:

[0017] FIG. 1 is a schematic diagram of a refrigeration device according to an embodiment of the present disclosure;

[0018] FIG. 2 is a schematic diagram of the interior of a refrigeration device according to an embodiment of the present disclosure;

[0019] FIG. 3 is a schematic diagram of a system for managing articles according to an embodiment of the present disclosure;

[0020] Figure 4 is a schematic diagram of a method for managing articles provided in an embodiment of this disclosure;

[0021] Figure 5 is a schematic diagram of alternative icons for items provided in an embodiment of this disclosure;

[0022] Figure 6 is a schematic diagram of displaying item information on the interactive terminal in response to a user's query command for an alternative icon, according to an embodiment of this disclosure.

[0023] Figure 7 is a schematic diagram of selecting a target item using an alternative icon that encircles the item, provided in an embodiment of this disclosure;

[0024] Figure 8 is a schematic diagram of selecting a target item by clicking on or circling the alternative icons of the items according to an embodiment of this disclosure;

[0025] Figure 9 is a schematic diagram of selecting a target item using an alternative icon of connected items provided in an embodiment of this disclosure;

[0026] Figure 10 is a schematic diagram of a query box provided in an embodiment of this disclosure;

[0027] Figure 11 is a schematic diagram of displaying information of a target item on an interactive terminal according to an embodiment of this disclosure;

[0028] Figure 12 is a schematic diagram of another method for managing articles provided in an embodiment of this disclosure;

[0029] Figure 13 is a schematic diagram of the interactive terminal displaying third prompt information provided in an embodiment of this disclosure;

[0030] Figure 14 is a schematic diagram of the fourth prompt information displayed on the interactive terminal according to an embodiment of this disclosure;

[0031] Figure 15 is a schematic diagram of the interactive terminal displaying the fifth prompt information provided in an embodiment of this disclosure;

[0032] Figure 16 is a schematic diagram of another method for managing articles provided in an embodiment of this disclosure;

[0033] Figure 17 is a schematic diagram of the interaction flow between the refrigeration equipment and the user provided in the embodiments of this disclosure;

[0034] Figure 18-1 is a schematic diagram of a real-time image of an object in a target space provided in an embodiment of this disclosure;

[0035] Figure 18-2 is a schematic diagram showing that the position of the alternative icon of the item provided in the embodiment of this disclosure is consistent with the position of the item in the real-time image, and the alternative icon of the item is proportional to the actual size of the item.

[0036] Figure 19 is an apparatus for managing articles provided in an embodiment of this disclosure.

[0037] Reference signs: 10, refrigeration equipment; 11, cabinet; 12, cabinet door; 13, image acquisition device; 14, drawer; 15, shelf; 20, interactive terminal; 21, display screen; 30, remote interactive device; 40, cloud server; 190, device for managing articles; 191, second processor; 192, memory; 193, communication interface; 194, bus. DETAILED DESCRIPTION

[0038] In order to enable a more detailed understanding of the features and technical content of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, which are only used for reference and do not limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, through multiple details, a sufficient understanding of the disclosed embodiments is provided. However, one or more embodiments can still be implemented without these details. In other cases, in order to simplify the drawings, well-known structures and devices can be simplified.

[0039] Any reference in the specification to any prior art is not, and should not be taken as, an acknowledgment or any form of suggestion that this prior art forms part of the common general knowledge in the field of the application or in any other field, or that this prior art could be practiced or be a prior publication under the applicable jurisdiction.

[0040] The terms "first", "second", and the like in the description of the embodiments of the present disclosure and the claims and the above drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0041] Unless otherwise specified, the term "a plurality of" means two or more.

[0042] In the embodiments of the present disclosure, the character " / " represents a "or" relationship between the objects before and after it. For example, A / B represents: A or B.

[0043] The term "and / or" is a description of the association relationship between objects, which means that there can be three relationships. For example, A and / or B, which means: A or B, or, A and B, three relationships.

[0044] The term "corresponding" can refer to an association relationship or a binding relationship. A and B correspond to each other means that there is an association relationship or a binding relationship between A and B.

[0045] With reference to FIGS. 1 and 2, the present disclosure provides a refrigeration appliance 10, which comprises a cabinet 11, a cabinet door 12, an image acquisition device 13, a first processor, and an interactive terminal 20. The cabinet 11 is internally provided with a storage space for storing food. For example, the cabinet 11 is internally provided with one or more drawers 14, and the internal space of one drawer 14 is a storage space. For another example, the cabinet 11 is internally provided with one or more shelves 15, and the space formed between the shelves 15 and between the shelf 15 and the cabinet 11 is a storage space. The cabinet door 12 is movably arranged on the cabinet 11. The image acquisition device 13 is arranged in the cabinet 11 and is configured to acquire an image of a target space. The target space is a storage space that a user needs to monitor, which can be the internal space of one or more drawers 14, the storage space of the cabinet door 12, or the internal space of the entire refrigeration appliance 10. The user interacts with the refrigeration appliance 10 through the interactive terminal 20.

[0046] Optionally, the image acquisition device 13 comprises a camera or an imaging sensor. The camera comprises one or more of an RGB (Red-Green-Blue) camera, a spectral camera, an infrared camera, and a TOF (Time of Flight) camera.

[0047] Optionally, the target space is the storage space of the drawer 14, and the image acquisition device 13 is arranged at a position capable of acquiring an image of the storage space of the drawer 14, such as the inner top of the cabinet 11, the inner side of the cabinet door 12, the inner side of the cabinet 11, and the top of the drawer 14.

[0048] Optionally, the interactive terminal 20 comprises a display screen 21 arranged on the outer side of the cabinet door 12 of the refrigeration appliance 10 and / or a display screen arranged on the front end panel of the drawer 14. When the display screen is arranged on the outer side of the cabinet door 12, the user can interact with the display screen and view the content displayed on the display screen without opening the cabinet door 12, which is convenient and fast. When the display screen is arranged on the front end panel of the drawer 14, the storage space of the drawer 14 is the target space. In this way, the position of the display screen corresponds to the target space, and the user can view the corresponding target space on the interactive terminal 20, which is more targeted.

[0049] The first processor is configured to receive an image of a target space collected by the image collection device 13, analyze the image of the target space, and obtain information of the article. Then, a substitute icon of the article is generated according to the information of the article. The substitute icon is sent to the interactive terminal 20, and the interactive terminal 20 displays the substitute icon of the article, or displays the information of the article while displaying the substitute icon of the article. When the interactive terminal displays the substitute icon of the article and does not display the information of the article, the first processor is further configured to receive a query command of the substitute icon sent by the user through the interactive terminal 20 or the voice device, generate the information of the article according to the query command, and control the interactive terminal 20 to display the information of the article.

[0050] In combination with FIG. 3, the embodiment of the present disclosure provides a system for managing articles, which comprises a refrigeration device 10 and a remote interactive device 30. Optionally, the refrigeration device 10 comprises an image collection device 13 configured to collect an image of a target space. The remote interactive device 30 comprises an interactive terminal 20. A cloud server 40 is in communication connection with the refrigeration device 10 and the interactive terminal 20. Specifically, the image collection device 13 collects an image of a target space and sends the image of the target space to the cloud server 40. The cloud server 40 analyzes the image of the target space and obtains information of an article. Then, a substitute icon of the article is obtained according to the information of the article, and the substitute icon of the article is sent to the interactive terminal 20. The interactive terminal 20 displays the substitute icon of the article, or displays the information of the article while displaying the substitute icon of the article, so as to interact with the user.

[0051] When the interactive terminal displays the substitute icon of the article and does not display the information of the article, the cloud server 40 is further configured to obtain the information of the article in response to a query command of the substitute icon of the article sent by the user, and send the information of the article to the interactive terminal 20. The interactive terminal 20 displays the information of the article. Optionally, the interactive terminal 20 comprises a smart mobile device and / or a smart home appliance. The smart mobile device comprises a smart phone, a smart home device, a wearable device, a smart mobile device, a virtual reality device, or any combination thereof. In this way, the user can remotely interact with the refrigeration device 10 through the interactive terminal 20, which is more convenient.

[0052] Optionally, the user can send the query command in the form of touch, voice, etc. The touch comprises single click, long press, sliding, circle drawing, etc.

[0053] Optionally, the refrigeration device 10 is a refrigerator or a freezer, etc.

[0054] In combination with FIG. 4, the embodiment of the present disclosure provides a method for managing articles, which comprises:

[0055] S401, collecting an image of a target space and obtaining information of an article stored in the target space.

[0056] S402, obtaining a substitute icon of the article according to the information of the article.

[0057] S403, displaying the substitute icon of the article on the interactive terminal, or displaying the substitute icon of the article and the information of the article at the same time.

[0058] The image acquisition device arranged in the refrigeration equipment cabinet acquires the image of the target space, and then the information of the article is obtained. Optionally, the information of the article includes: name, position and related information. The related information includes: quantity and / or real image and / or freshness and / or storage date and / or shelf life and / or packaging form and / or recipe. The first processor obtains a substitute icon of the article according to the information of the article, and then sends the substitute icon of the article to the interactive terminal. The interactive terminal displays the substitute icon of the article, or displays the substitute icon of the article and the information of the article at the same time. The substitute icon of the article is a non-real and virtual icon, and the name of the article corresponding to the substitute icon is consistent with the name of the article stored in the target space.

[0059] The method for managing articles provided by the embodiments of the present disclosure is used to first obtain the information of the article based on the image of the target space, and then obtain a substitute icon of the article according to the information of the article. Then, the substitute icon of the article is displayed on the interactive terminal, or the substitute icon of the article and the information of the article are displayed at the same time. Compared with the real image, the substitute icon of the article is more beautiful, more clean, and more conducive to the user to view and / or click the corresponding article, thereby improving the accuracy of the user's interaction with the image and improving the user's use experience.

[0060] Optionally, when the substitute icon of the article is displayed on the interactive terminal and the information of the article is not displayed, the information of the article is displayed on the interactive terminal in response to a query command of the user to the substitute icon.

[0061] When the substitute icon of the article is displayed on the interactive terminal and the information of the article is not displayed, the first processor displays the information of the article on the interactive terminal in response to a query command of the user to the substitute icon. Optionally, the user forms the query command by touching the interactive terminal, or forms the query command by sending a voice to the voice terminal. The query command includes a query object, and the query object includes one or more articles corresponding to the substitute icons displayed on the interactive terminal and selected by the user. The first processor sends the information of the article to the interactive terminal to enable the interactive terminal to show the information of the article to the user. In this way, the user interacts with the substitute icon of the article, and the user queries one or more articles corresponding to the substitute icons. The substitute icon is more clean, so the accuracy of the user's interaction with the substitute icon can be improved, the displayed information of the article, i.e., the query result, is more accurate, and the user's experience is improved.

[0062] Optionally, S402, display the substitute icons of the articles on the interactive terminal, including:

[0063] The substitute icons displayed on the interactive terminal are arranged according to a preset rule; wherein,

[0064] The preset rule includes that the positions of the substitute icons of the articles correspond to the actual positions of the articles, and the sizes of the substitute icons of the articles are positively correlated with the real sizes of the articles; or, the substitute icons of the articles are displayed uniformly.

[0065] The substitute icons of the articles arranged according to the preset rule are displayed on the interactive terminal, so as to facilitate the user to view.

[0066] The substitute icons of the articles are shown in Fig. 5. The preset rule includes that the positions of the substitute icons of the articles correspond to the actual positions of the articles, and the sizes of the substitute icons of the articles are positively correlated with the real sizes of the articles. Optionally, the sizes of the substitute icons of the articles are in proportion to the real sizes of the articles. In this way, the positions of the articles in the target space and the relative sizes of the articles are reproduced by the substitute icons, so as to facilitate the user to understand the storage condition of the articles in the target space. Or, the preset rule includes that the substitute icons of the articles are displayed uniformly. In this way, each substitute icon is regularly and uniformly distributed according to the display screen of the interactive terminal, which is more beautiful and facilitates the user to view.

[0067] Optionally, in combination with Fig. 6, the first processor displays the information of the articles on the interactive terminal in response to the query command of the user on the substitute icons, including:

[0068] S601, determine the target articles according to the query command of the user; the target articles are the articles corresponding to one or more of the substitute icons of the displayed articles.

[0069] S602, display the information of the target articles on the interactive terminal.

[0070] The first processor analyzes the query command of the user, and then determines the target articles queried by the user. The target articles are the articles corresponding to one or more of the substitute icons of the articles displayed on the interactive terminal. Then, the information of the target articles is displayed on the interactive terminal. As known from the foregoing, the information of the articles includes the name, the position and the related information, and the related information includes one or more kinds of information, so that the name and the position of the target articles and one or more kinds of information in the related information can be displayed on the interactive terminal. Optionally, when the query command of the user is the touch query / hand-free operation query, if the user clicks / long-presses / slides / circles / links the substitute icons and clicks the query window, the articles corresponding to the substitute icons clicked / long-pressed / slided / circled / linked by the user are determined as the target articles. Optionally, the clicking includes single-clicking, double-clicking and the like, which can be set by the user. When the query command of the user is the voice query, the articles corresponding to the voice of the user are determined as the target articles.

[0071] In the case that the user selects the target item by touch, when the target item is one, the user can select the target item by clicking the alternative icon of the item. When the target item is multiple, if the alternative icons of the items are adjacent, see Fig. 7, the user can enclose the alternative icons together to select the target item. If the alternative icons of the items are not adjacent, see Fig. 8, the user can click or enclose the alternative icons respectively to select the target item, or, see Fig. 9, the user can connect the alternative icons by lines to select the target item.

[0072] Optionally, when the user selects the target item by touch, and then clicks the query button displayed on the interactive terminal, the first processor determines that the query command of the user is to query the quantity and / or real image and / or freshness and / or storage date and / or shelf life and / or packaging form of the target item.

[0073] When the user selects the target item by touch, and then clicks the recipe button displayed on the interactive terminal, the first processor determines that the query command of the user is to query the recipe of the target item.

[0074] Optionally, see Fig. 10, a query box is displayed on the interactive terminal. When the user selects the alternative icon corresponding to the target item, the name of the corresponding item is presented in the query box.

[0075] Optionally, the real image of the target item is displayed on the interactive terminal, including:

[0076] The real image of the target item is displayed on the interactive terminal, and non-target items are not displayed or are blurred; or,

[0077] The real image of the target item is displayed on the interactive terminal, and non-target items are not displayed or are blurred, and the information label of the target item is displayed on one side of the real image of the target item.

[0078] Wherein, the position of the real image corresponds to the real position of the item.

[0079] When the interactive terminal displays the real image of the target object, the real image of the target object is displayed, and the image of the non-target object is not displayed or is blurred. For example, the interactive terminal displays the substitute icons of objects such as tomatoes, cucumbers, carrots, and eggs. When the user selects the target object as a carrot, the closest real image of the carrot in the target space is displayed, and the non-target objects such as tomatoes and cucumbers are not displayed or are blurred, so that the user can quickly lock the target object and understand the real storage state of the target object. Alternatively, the real image of the target object is displayed, and the icon of the non-target object is not displayed or is blurred, and at the same time, the information label corresponding to the target object is displayed on one side of the real image of the target object. For example, referring to FIG. 11, when the user selects the target object as a carrot, the closest real image of the carrot in the target space is displayed, and the non-target objects such as tomatoes and cucumbers are not displayed or are blurred, and at the same time, the virtual information label corresponding to the carrot is displayed on one side of the real image of the carrot. Alternatively, when the interactive terminal displays the target object, the display position of the target object on the interactive terminal corresponds to the real position of the target object in the target space, so that the user can further understand the real storage position of the target object. Alternatively, the information label displays the name, quantity, freshness, storage date, shelf life, and other related information of the target object (such as a carrot), so that the user can understand the information of the target object.

[0080] Alternatively, when the interactive terminal displays the recipe of the target object, if the user selects multiple target objects, the interactive terminal displays the recipe of the multiple target objects. For example, the user selects tomatoes and eggs as target objects, and the interactive terminal displays the method of making scrambled eggs with tomatoes. If the user selects one target object, the interactive terminal displays the recipe containing the target object. For example, the user selects cucumbers as a target object, and the interactive terminal displays the method of making dishes such as cucumber salad, cucumber and egg stir-fry, or cucumber and meat stir-fry. In this way, the user is recommended the recipe related to the target object to meet the user's needs.

[0081] In combination with FIG. 12, the embodiment of the present disclosure provides another method for managing objects, including:

[0082] S401, collecting the image of the target space, and obtaining the information of the stored objects in the target space.

[0083] S402, obtaining the substitute icon of the object according to the information of the object.

[0084] S403, displaying the substitute icon of the object on the interactive terminal, or displaying the substitute icon of the object and displaying the information of the object at the same time.

[0085] S404, after S401 is performed, sending a prompt information according to the change of the information of the object.

[0086] As described above, the information of the item includes name, location and related information. After the first processor acquires the information of the item, the first processor sends prompt information to the user according to the change of the information of the item, so as to remind the user to pay attention to the change of the item. Optionally, the prompt information includes voice prompt information sent by the voice device, text and / or icon prompt information displayed by the display screen, etc. Optionally, the prompt mode of the voice prompt information includes voice broadcast, sound effect, etc. Optionally, the prompt mode of the icon prompt information includes displaying a preset mark, differentiating color, etc.

[0087] Optionally, S404, according to the change of the information of the item, the prompt information is sent, including:

[0088] In the case that the location of the item changes, the first prompt information is sent; and / or,

[0089] In the case that the items in the target space are suitable for common storage conditions, the second prompt information is sent; and / or,

[0090] In the case that the type of the item is not suitable for storage in the target space, the third prompt information is sent; and / or,

[0091] In the case that the freshness of the item changes, the fourth prompt information is sent; and / or,

[0092] In the case that the packaging of the item does not meet the packaging requirements of the target space, the fifth prompt information is sent.

[0093] It is judged whether the location of the item in the target space changes. If the location of the item changes, for example, the item is moved from one drawer to another drawer, or from one shelf to another shelf, or from a drawer to a partition, the first prompt information is sent to prompt the user to change the location of the corresponding item, so as to avoid that the user cannot find the item.

[0094] It is also judged whether various types of items are suitable for simultaneous storage in the target space. If not, the second prompt information is sent. Optionally, when the items can mutually ripen, mutually affect the shelf life, easily mix flavors and / or raw and cooked are mixed, the first processor determines that the corresponding items are not suitable for simultaneous storage in the target space. Optionally, the second prompt information includes marking the alternative icons corresponding to the items. Optionally, the form of the marking is to enclose the corresponding alternative icons with the same color lines. Optionally, the form of the marking is to generate a virtual prompt label on one side of the corresponding alternative icon, and the virtual prompt label displays a text prompt of other items that are not suitable for simultaneous storage with the corresponding item. Optionally, the virtual prompt label is in a preset color, for example, yellow, to give the user a clear prompt.

[0095] The first processor further determines whether various types of items in the target space are suitable for storage in the target space. If not, a third prompt message is sent. For example, referring to FIG. 13, when bananas (not suitable for refrigeration) are stored in the target space, the first processor determines that the bananas are not suitable for storage in the target space, and sends a third prompt message. Optionally, the third prompt message includes marking the corresponding item replacement icon. Optionally, the form of marking is to enclose the corresponding replacement icon with a line of a preset color. Optionally, the form of marking is to generate a virtual prompt label on one side of the corresponding replacement icon, and the virtual prompt label displays a text reminder that the corresponding item is not suitable for storage in the target space and the correct storage method. Optionally, the virtual prompt label is in a preset color, for example, yellow, to give the user a clear prompt.

[0096] In this way, when the user views the replacement icon of the item, the user can intuitively understand whether the item is suitable for storage in the target space at the same time and / or whether the item is suitable for storage in the target space through the marking, to assist the user in more intelligently managing the item.

[0097] The first processor further determines the change in the freshness of the item, and sends a fourth prompt message according to the change in the freshness of the item. Optionally, the fourth prompt message includes marking the corresponding item replacement icon. Optionally, referring to FIG. 14, different freshness corresponds to different color markings. Optionally, the form of marking is to enclose the item. For example, the replacement icon of the item is enclosed with a red line, indicating that the corresponding item has expired. For another example, the replacement icon of the item is enclosed with a yellow line, indicating that the corresponding item is about to expire. In this way, when the user views the replacement icon of the item, the user can intuitively understand the change in the freshness of the item through the marking.

[0098] Optionally, when the form of marking is to enclose the replacement icon, a line of different line type is used to enclose the replacement icon, to distinguish the enclosure marking representing the freshness of the item, the item not being suitable for storage in the same target space, or the item not being suitable for storage in the target storage space. For example, when the enclosure marking represents the freshness of the item, the enclosure line is a curve. When the enclosure marking represents that the item is not suitable for storage in the same target space, the enclosure line is a wavy line. When the enclosure marking represents that the item is not suitable for storage in the target space, the enclosure line is a dashed line. In this way, it is convenient for the user to distinguish.

[0099] The first processor also determines whether the packaging of the item meets the packaging requirement of the target space. If not, a fifth prompt information is sent. Optionally, the fifth prompt information includes marking the replacement icon of the corresponding item. For example, referring to FIG. 15, when it is identified that the fruit is not suitable for bagging, the replacement icon of the fruit is marked. Optionally, the marking is in the form of generating a virtual prompt label on one side of the corresponding replacement icon, and the virtual prompt label displays a text prompt of the corresponding item being unsuitable for bagging and the correct storage method. Optionally, the virtual prompt label is in a preset color, for example, yellow, to give a clear prompt to the user.

[0100] In combination with FIG. 16, the embodiment of the present disclosure provides another method for managing items, including:

[0101] S401, acquiring an image of a target space to obtain information of items stored in the target space.

[0102] S402, obtaining a replacement icon of the item according to the information of the item.

[0103] S403, displaying the replacement icon of the item on an interactive terminal, or displaying the replacement icon of the item while displaying the information of the item.

[0104] S404, after S401 is performed, sending a prompt information according to a change in the information of the item.

[0105] S405, after S401 is performed, determining a target environmental parameter of the target space according to the information of the item.

[0106] S406, controlling the target space to operate according to the target environmental parameter, and displaying the target environmental parameter on the interactive terminal and / or notifying the user of the target environmental parameter through voice.

[0107] After the information of the item is obtained, the target environmental parameter of the target space is determined according to the information of the item. In this way, the environmental parameter suitable for storing the item can be determined according to the name (category) and freshness of the item. Then the target space is controlled to operate according to the target environmental parameter, so that the environmental parameter of the target space reaches the environmental parameter suitable for storing the item. At the same time, the target environmental parameter is displayed on the interactive terminal, and / or the target environmental parameter is notified to the user through voice, so as to facilitate the user to understand the adjustment of the environmental parameter of the target space.

[0108] Optionally, the environmental parameter of the target space includes one or more of temperature, humidity, oxygen concentration, sterilization parameter, etc.

[0109] Optionally, when the target space stores multiple items, the first processor sets priorities according to the quantity, price, freshness, etc. of the items to determine the target environmental parameter. For example, the target environmental parameter of the target space is determined according to the category and freshness of the item with the largest quantity of the same category. For another example, the target environmental parameter of the target space is determined according to the category and freshness of the item with the highest price. The specific priority can be set by the user.

[0110] The interaction process between the refrigeration device and the user provided by the embodiment of the present disclosure will be described in detail below with reference to FIG. 17, taking the refrigeration device with a touch interactive terminal and the image acquisition device collecting the image in the drawer as an example:

[0111] S1701, the user clicks the picture of the drawer displayed on the interactive terminal.

[0112] S1702, a request for viewing the storage state of the drawer is sent. The request for viewing includes the real-time real image of the storage state of the drawer collected by the image acquisition device.

[0113] S1703, in response to the request for viewing, the substitute icon of the item is obtained according to the real-time real image of the storage state.

[0114] S1704, the substitute icon of the item is sent.

[0115] S1705, the substitute icon of the item is uniformly distributed on the display picture of the interactive terminal, as shown in FIG. 5. Alternatively, the position of the substitute icon of the item is consistent with the position of the item in the real-time real image, and the substitute icon of the item is in a scale ratio with the real size of the item, as shown in FIGS. 18-1 and 18-2.

[0116] S1706, in the case that the position of the item changes, the item is about to expire, has expired, is not suitable to be stored in the current drawer, is not suitable to be stored in the current drawer together with other items, is not suitable to be stored in a bag, etc., the substitute icon of the corresponding item is marked, and a virtual prompt label is generated on the side of the corresponding substitute icon of the item.

[0117] S1707, the substitute icon with the mark and the virtual prompt label is sent to the interactive terminal.

[0118] S1708, the user clicks the substitute icon of a certain item displayed on the interactive terminal.

[0119] S1709, a request for querying the information of the item is sent.

[0120] S1710, an item display image is generated, in which the real image of the target item is generated, the non-target item is not displayed or is blurred, and a virtual information label of the target item is generated on the side of the real image of the target item.

[0121] S1711, sending the item display image.

[0122] S1712, the user clicks, encircles or connects the substitute icon of the item on the interactive terminal.

[0123] S1713, sending the query request for querying the recipe of the item.

[0124] S1714, sending the recipe.

[0125] S1715, determining the target environment parameter of the drawer according to the information such as the category, quantity and freshness of the item.

[0126] S1716, the drawer operates according to the target environment parameter.

[0127] S1717, sending the target environment parameter to the interactive terminal.

[0128] The following takes the setting of the image acquisition device in the refrigeration equipment and the image acquisition device acquiring the image in the drawer as an example to explain in detail the remote interaction process between the cloud server, the refrigeration equipment and the user provided by the embodiment of the disclosure:

[0129] The user clicks the picture of the drawer displayed on the interactive terminal.

[0130] The interactive terminal sends a viewing request for viewing the storage state of the drawer to the cloud server. The viewing request includes the real-time real image of the storage state of the drawer acquired by the image acquisition device.

[0131] The cloud server acquires the substitute icon of the item according to the real-time real image of the storage state in response to the viewing request.

[0132] The cloud server sends the substitute icon of the item to the interactive terminal.

[0133] The substitute icons of the items are uniformly distributed on the display picture of the interactive terminal, as shown in FIG. 5. Or the positions of the substitute icons of the items are consistent with the positions of the items in the real-time real image, and the substitute icons of the items are in a proportional size with the real size of the items, as shown in FIGS. 18-1 and 18-2.

[0134] The cloud server marks the substitute icon of the corresponding item when the position of the item changes, the item is about to expire, has expired, is not suitable to be stored in the current drawer, is not suitable to be stored with other items in the current drawer, is not suitable to be stored in a bag, etc., and generates a virtual prompt label on the side of the corresponding substitute icon of the item.

[0135] The first processor of the refrigeration equipment sends the substitute icon with the mark and the virtual prompt label to the interactive terminal.

[0136] The user clicks on a substitute icon of an item displayed on the interactive terminal.

[0137] The interactive terminal sends a query request for the related information of the item to the cloud server.

[0138] The cloud server generates an item display image in which a real image of the target item is displayed, non-target items are not displayed or are blurred, and a virtual information label of the target item is generated on one side of the real image of the target item.

[0139] The cloud server sends the item display image to the interactive terminal.

[0140] The user clicks, encircles, or connects a substitute icon of an item on the interactive terminal.

[0141] The interactive terminal sends a query request for the recipe of the item to the cloud server.

[0142] The cloud server sends the recipe to the interactive terminal.

[0143] The cloud server determines the target environmental parameters of the drawer according to the information such as the type, quantity, and freshness of the item.

[0144] The cloud server sends the target storage environmental parameters to the refrigeration equipment and the interactive terminal.

[0145] The drawer of the refrigeration equipment operates according to the target environmental parameters.

[0146] The following will take the image acquisition device collecting the image of the full space of the refrigeration equipment as an example to explain in detail the interaction process between the refrigeration equipment and the user provided by the embodiments of the present disclosure:

[0147] The user sends a viewing request for viewing the full space of the refrigeration equipment through voice or clicking on the function displayed on the interactive terminal. The full space of the refrigeration equipment includes storage spaces such as shelf storage space, drawer storage space, and box door storage space.

[0148] The interactive terminal displays a substitute image of the full space of the refrigeration equipment. The substitute image of the full space is a non-real and virtual image.

[0149] The user clicks on a certain space in the substitute image of the full space displayed on the interactive terminal.

[0150] The interactive terminal displays an enlarged substitute image of the space clicked by the user, wherein the enlarged substitute image includes a substitute icon of the item stored in the corresponding space, or includes a substitute icon of the item stored in the corresponding space and information of the stored item.

[0151] After that, the user interacts with the substitute icon of the item in the enlarged substitute image of the space, which can refer to the examples listed in the foregoing, and the details are not repeated here.

[0152] In this way, the user can realize remote interaction with the refrigerator through a smart device such as a smart phone or a smart home appliance.

[0153] With reference to FIG. 19, the disclosure provides a device 190 for managing an article, which includes a second processor 191 and a memory 192. Optionally, the device 190 can further include a communication interface 193 and a bus 194. The second processor 191, the communication interface 193, and the memory 192 can communicate with each other through the bus 194. The communication interface 193 can be used for information transmission. The second processor 191 can invoke a logical instruction in the memory 192 to execute the method for managing an article in the above-described embodiments.

[0154] In addition, the logical instruction in the memory 192 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium.

[0155] The memory 192, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, such as program instructions / modules corresponding to the method in the embodiments of the disclosure. The second processor 191 executes the function application and data processing by running the program instructions / modules stored in the memory 192, that is, implements the method for managing an article in the above-described embodiments.

[0156] The memory 192 can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function; the data storage area can store data created during use of the terminal device, and the like. In addition, the memory 192 can include a high-speed random access memory, and can further include a non-volatile memory.

[0157] The disclosure provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are configured to execute the above-described method for managing an article.

[0158] The disclosure provides a computer-readable non-transitory storage medium, which stores program instructions, and the program instructions, when executed, perform the following steps:

[0159] Collect an image of the target space to obtain information of an article stored in the target space;

[0160] Obtain a replacement icon of the article according to the information of the article;

[0161] Display a substitute icon of the item, or display information of the item while displaying the substitute icon of the item.

[0162] The embodiments of the present disclosure provide a computer program, which, when executed by a computer, causes the computer to implement the above method for managing an item.

[0163] The embodiments of the present disclosure provide a computer program product, which comprises computer instructions stored on a computer readable storage medium, and when the program instructions are executed by a computer, the computer implements the above method for managing an item.

[0164] The technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes one or more instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present disclosure. The aforementioned storage medium can be a non-transitory storage medium, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, etc.

[0165] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0166] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0167] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to apparatuses, devices, etc.), can be implemented in other manners. For example, the described apparatus embodiments can be implemented only in a form of a logical function, and can be implemented by using a manner such as software (for example, application program) or the like. In some embodiments, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or indirect coupling between different units, or the coupling or direct coupling or indirect coupling between the displayed or discussed communication connections can be in a form of electrical, mechanical or other forms.

[0168] The flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the system, method and computer program product according to the embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks can occur in an order different from that noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the drawings, the operations or steps corresponding to different blocks can also occur in an order different from that disclosed in the descriptions, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. Each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for managing items, comprising: collecting images of a target space to obtain information of items stored in the target space; obtaining substitute icons of the items according to the information of the items; displaying the substitute icons of the items on an interactive terminal, or displaying the substitute icons of the items while displaying the information of the items.

2. The method of claim 1, wherein, when the substitute icons of the items are displayed on the interactive terminal and the information of the items is not displayed, in response to a query command of a user on the substitute icons, displaying the information of the items on the interactive terminal.

3. The method of claim 1 or 2, wherein, the substitute icons displayed on the interactive terminal are arranged according to preset rules, the preset rules comprising: positions of the substitute icons of the items correspond to actual positions of the items, and sizes of the substitute icons of the items are positively correlated with real sizes of the items; or the substitute icons of the items are displayed uniformly.

4. The method of claim 2 or 3, wherein, in response to the query command of the user on the substitute icons, displaying the information of the items on the interactive terminal, comprising: determining a target item according to the query command of the user, the target item being one or more of the items corresponding to the substitute icons displayed; displaying the information of the target item on the interactive terminal.

5. The method of any one of claims 1 to 4, wherein: the information of the items comprises names, positions and related information; the related information comprises quantities and / or real images and / or freshness and / or storage dates and / or shelf lives and / or packaging forms and / or recipes.

6. The method of claim 5, further comprising: sending prompt information according to changes in the information of the items.

7. The method of claim 6, wherein, sending prompt information according to changes in the information of the items, comprising: in the case of a change in the positions of the items, sending first prompt information; and / or in the case of items in the target space being unsuitable for being stored together, sending second prompt information; and / or in the case of the types of the items being unsuitable for being stored in the target space, sending third prompt information; and / or in the case of changes in the freshness of the items, sending fourth prompt information; and / or in the case of the packaging of the items not meeting packaging requirements of the target space, sending fifth prompt information.

8. The method of any one of claims 1 to 7, further comprising: determining target environmental parameters of the target space according to the information of the items; controlling the target space to operate according to the target environmental parameters, and displaying the target environmental parameters and / or informing the user of the target environmental parameters by voice.

9. An apparatus for managing items, comprising a second processor and a memory, the second processor executing a method for managing items as claimed in any one of claims 1 to 8 stored in the memory.

10. A computer-readable non-transitory storage medium storing program instructions, the program instructions, when executed, performing the following steps: collecting images of a target space to obtain information of items stored in the target space; obtaining substitute icons of the items according to the information of the items; displaying the substitute icons of the items on an interactive terminal, or displaying the substitute icons of the items while displaying the information of the items.

11. A computer program which, when executed by a computer, causes the computer to implement a method for managing items as claimed in any one of claims 1 to 8.

12. A computer program product, comprising computer instructions stored on a computer readable storage medium, which when executed by a computer, cause the computer to implement the method for managing items according to any one of claims 1 to 8.

13. A refrigeration device, comprising: a cabinet and a cabinet door, each of which is provided with a target space for storing items; an image acquisition device configured to acquire an image of the target space; a first processor configured to analyze the image of the target space to obtain information of the items, and obtain a substitute icon of the items according to the information of the items; an interactive terminal configured to display the substitute icon of the items, or in response to a query command of a user, display the substitute icon of the items and the information of the items at the same time.

14. The refrigeration appliance of claim 13, wherein, The image acquisition device comprises a camera or an imaging sensor.

15. The refrigeration device according to claim 13 or 14, wherein: the target space is a storage space of a drawer; the image acquisition device is arranged at an inner top of the cabinet, an inner side of the cabinet door, an inner side of the cabinet, or a top of the drawer, so as to be capable of acquiring an image of the storage space of the drawer.

16. A system for managing items, comprising: a refrigeration device comprising an image acquisition device, which is configured to acquire an image of a target space and transmit the image of the target space to a cloud server; the cloud server, which is configured to analyze the image of the target space to obtain information of the items after receiving the image of the target space, and obtain a substitute icon of the items according to the information of the items; a remote interactive device, an interactive terminal of which is configured to display the substitute icon of the items, or in response to a query command of a user, display the substitute icon of the items and the information of the items at the same time.

17. The system according to claim 16, wherein: the remote interactive device comprises a smart mobile device, and / or a smart home appliance.

Images