Article marking method and device of intelligent kitchen cabinet, intelligent kitchen cabinet and medium

Abstract

The invention provides an article marking method and device for an intelligent kitchen cabinet, the intelligent kitchen cabinet and a medium. The article marking method comprises the steps that position information and image information of a target article placed in the intelligent kitchen cabinet are obtained; the position information is used for indicating a target position of the target article in the intelligent kitchen cabinet; performing image recognition on the image information to obtain article information of the target article; the article information is used for indicating the name, the type and the attribute of the target article; and sending the position information and the article information to a cloud database, so that the cloud database associatively stores the position information and the article information. The target article in the intelligent kitchen cabinet is recognized through the image to obtain the article information, the article information of the target article and the position information capable of indicating the target position where the target article is located are stored in the cloud database in an associated mode, butt joint with the cloud is achieved, the automation degree is high, the recognition accuracy rate is high, and the use experience of a user is good.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a method and device for marking items in a smart kitchen cabinet, a smart kitchen cabinet and a medium. Background Art

[0002] With the rapid development of smart home technology, people's requirements for convenience and intelligence in home life are increasing. As an important part of smart home, the intelligent level of item labeling and management of smart kitchen cabinets is directly related to the user experience. Traditional kitchen cabinets rely entirely on manual memory or paper labels. Users manually label each item, which increases the cost and time cost of use, is inefficient and prone to errors. In addition, some smart kitchen cabinets have poor automatic labeling accuracy and still require users to manually adjust them later. The functions are not perfect, resulting in poor user experience. Summary of the invention

[0003] In view of the problems existing in the above-mentioned prior art, the present invention provides an item marking method and device for a smart kitchen cabinet, a smart kitchen cabinet and a medium; the technical solution is as follows:

[0004] In one aspect, the present invention provides an object marking method for a smart cabinet, which is used to mark a target object placed in the smart cabinet, and the method comprises:

[0005] Acquire location information and image information of a target object placed in a smart kitchen cabinet; the location information is used to indicate a target location of the target object in the smart kitchen cabinet;

[0006] Performing image recognition on the image information to obtain item information of the target item; the item information is used to indicate the name, type and attribute of the target item;

[0007] The location information and the item information are sent to a cloud database, so that the cloud database stores the location information and the item information in association with each other.

[0008] Further, performing image recognition on the image information to obtain the object information of the target object includes:

[0009] Target detection is performed on the image information to obtain an image recognition result; the image recognition result includes an image position mark of the target object in the image information and type information of the target object.

[0010] Furthermore, the target item has an item identification code, and the item identification code is used to indicate the name, type and attribute of the target item; performing image recognition on the image information to obtain the item information of the target item includes:

[0011] Identifying an item identification code of the target item in the image information;

[0012] The object identification code of the target object is matched based on a preset corresponding relationship between the identification code and the object information to obtain the object information of the target object.

[0013] Furthermore, after sending the location information and the item information to the cloud database so that the cloud database stores the location information and the item information in association with each other, the method further includes:

[0014] In response to the search instruction for the target item, the location information corresponding to the target item is determined based on the location information and the corresponding item information stored in the cloud database.

[0015] Furthermore, the method further comprises:

[0016] When the smart kitchen cabinet is detected to be opened, obtaining image update information of each item at each position in the smart kitchen cabinet;

[0017] Acquire the historical image information of each of the positions; the historical image information is the image information of the corresponding position acquired before the smart cabinet is opened;

[0018] For each position, determining a difference result between the image update information and the historical image information;

[0019] If the difference result indicates that the image update information is inconsistent with the historical image information, image recognition is performed on the image update information to obtain updated item information at the location, and the location information of the location and the updated item information are sent to the cloud database, so that the cloud database associates and stores the location information and the updated item information.

[0020] Furthermore, the smart kitchen cabinet includes a plurality of storage spaces, and the method further includes:

[0021] In response to a remote control instruction carrying a storage space identifier, the storage space corresponding to the storage space identifier in the smart kitchen cabinet corresponding to the remote control instruction is controlled to be opened or closed.

[0022] Furthermore, the method further comprises:

[0023] Upon receiving a remote viewing instruction carrying a target item identifier sent by an interactive terminal, controlling the cloud database to send the item information and the location information of the target item corresponding to the target item identifier to the interactive terminal; and / or

[0024] When a remote editing instruction carrying target item editing information is received from the interactive terminal, the item information of the target item is updated according to the target item editing information, updated item information is obtained, and the updated item information and the location information are associated and stored in the cloud database.

[0025] On the other hand, the present invention also provides an item marking device for a smart cabinet, comprising:

[0026] An information acquisition module, used to acquire position information and image information of a target object placed in the smart kitchen cabinet; the position information is used to indicate a target position of the target object in the smart kitchen cabinet;

[0027] A recognition module, used to perform image recognition on the image information to obtain item information of the target item; the item information is used to indicate the name, type and attribute of the target item;

[0028] The sending module is used to send the location information and the item information to a cloud database, so that the cloud database stores the location information and the item information in an associated manner.

[0029] On the other hand, the present invention further provides a smart kitchen cabinet, comprising the item marking device of the smart cabinet as described above.

[0030] On the other hand, the present invention provides a medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the item marking method of the smart cabinet as described in any of the above items.

[0031] The implementation of the present invention has the following beneficial effects:

[0032] 1. The present invention obtains the object information of the target object by image recognition at the target position in the smart kitchen cabinet, and associates the object information of the target object with the position information indicating the target position of the target object and stores them in a cloud database, thereby achieving docking with the cloud, with a high degree of automation, a high recognition accuracy, and a good user experience. In addition, associating the position information and the object information and storing them in the cloud database can also help improve the accuracy of subsequent image recognition, further improve the marking accuracy of the object marking method of the smart kitchen cabinet, and improve the user experience.

[0033] 2. The present invention can also interact with multiple interactive terminals through a cloud database to achieve cross-platform synchronization and sharing functions, allowing users to switch and collaborate between different interactive terminals, greatly improving the flexibility and convenience of the item marking method of the smart kitchen cabinet, and further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings used in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 A logical structure diagram of an item marking method for a smart kitchen cabinet provided by an embodiment of the present invention;

[0036] Figure 2 A logical structure diagram of an image recognition method provided by an embodiment of the present invention;

[0037] Figure 3 A logical structure diagram of another image recognition method provided by an embodiment of the present invention;

[0038] Figure 4 A logical structure diagram of another method for marking items in a smart kitchen cabinet provided by an embodiment of the present invention;

[0039] Figure 5 A logical structure diagram of an interaction method between a cloud database and an interactive terminal provided by an embodiment of the present invention;

[0040] Figure 6 A structural block diagram of an item marking device for a smart kitchen cabinet provided by an embodiment of the present invention;

[0041] Figure 7 The present invention is a hardware structure block diagram of an electronic device for executing an item marking method for a smart kitchen cabinet provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore cannot be understood as limiting the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0043] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention can be implemented in an order other than the following diagrams or the following descriptions. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0044] In view of the problem that traditional kitchen cabinet storage relies on manual memory or paper labels, which is inefficient and prone to errors, the embodiment of the present invention provides an item marking method, device, smart kitchen cabinet and medium for a smart kitchen cabinet. The item marking method for the smart kitchen cabinet is based on the item marking device for the smart kitchen cabinet provided in the embodiment of the present application, and is applied to the smart kitchen cabinet provided in the embodiment of the present invention to mark the target items placed in the smart kitchen cabinet; first, the position information and image information of the target item placed in the smart kitchen cabinet are obtained, and the position information is used to indicate the position of the target item in the smart kitchen cabinet; then, image recognition is performed on the image information to obtain the item information of the target item, and the item information is used to indicate the name, type and attribute of the target item; then, the position information and item information of the target item are sent to a cloud database, so that the cloud database associates and stores the position information and item information. Through the item marking method for the smart kitchen cabinet, the items placed in the smart kitchen cabinet can be automatically identified based on image recognition, and the automatic identification and marking of the items inside the smart cabinet can be realized, and the connection with the cloud database can be realized, the marking accuracy is high, and it is also convenient for remote control, which can effectively improve the convenience of using the smart kitchen cabinet and greatly improve the user experience.

[0045] The following is a detailed description of the method for marking items in the smart kitchen cabinet according to an embodiment of the present invention. Figure 1 , the method comprising:

[0046] S101, obtaining location information and image information of a target object placed in a smart kitchen cabinet.

[0047] Among them, the smart kitchen cabinet includes at least one storage space, that is, the smart kitchen cabinet may include one storage space or multiple storage spaces; in some exemplary embodiments, a single storage space may be a single kitchen cabinet; in some exemplary embodiments, the smart kitchen cabinet includes multiple storage spaces, each storage space is used to store items of the same category, exemplarily, one storage space is used to store food ingredients, another storage space is used to store seasonings, etc.; in other exemplary embodiments, the smart kitchen cabinet includes multiple storage spaces, at least one storage space among the multiple storage spaces is used to store multiple categories of items; in this way, the storage method is flexible and can meet the different storage needs of different users.

[0048] In some exemplary embodiments, each storage space may further include multiple storage layers, and each storage layer may include multiple storage locations; in other exemplary embodiments, each storage space may include multiple storage compartments; in this way, the item marking method of the smart kitchen cabinet can be applied to a variety of smart kitchen cabinets and has a wide range of applications.

[0049] There are multiple items placed in the smart kitchen cabinet, and the target item is one of the multiple items placed in the smart kitchen cabinet. In some embodiments, it can be an item that a user opens the kitchen cabinet and puts in.

[0050] In some exemplary embodiments, the image information may include at least one of a static picture, a dynamic picture, and a video clip, and the image information includes an image of a target object at a target position, so as to improve the accuracy of subsequent image recognition, further improve the accuracy and reliability of the object information and the associated storage in the cloud database, reduce the risk of users manually adjusting the mark later, and improve the user experience.

[0051] The image information can be acquired by at least one image acquisition device. In some exemplary embodiments, the image information is acquired by one image acquisition device to acquire an overall image of the interior of the kitchen cabinet where the target object is located, or to acquire an image corresponding to the storage location or storage compartment where the target object is located. In other exemplary embodiments, the image information is acquired by multiple image acquisition devices to improve the sufficiency of the information of the target object contained in the image information and to improve the accuracy of subsequent image recognition and determination of object information. Exemplarily, the image acquisition device is used to obtain an initial image of the target object by photographing and imaging. The image acquisition device may include a camera or other imaging device. In some specific embodiments, the image acquisition device may be a camera, which is convenient for taking clear images, occupies little space, and is conducive to cost saving.

[0052] The position information is used to indicate the target position of the target object in the smart kitchen cabinet, specifically to indicate the storage space where the target object is located in the smart kitchen cabinet, and the storage location or storage compartment in the storage space; in some exemplary embodiments, the position information can be determined by taking an overall image of the interior of the smart kitchen cabinet and performing image analysis on the overall image; in other exemplary embodiments, an image acquisition device is provided in each storage location or each storage compartment, and the position information can be determined by taking a picture of the position of the corresponding image acquisition device of a single target object.

[0053] In addition, in some exemplary embodiments, an image acquisition device can be used to identify the target object among multiple objects in the kitchen cabinet through motion detection and other methods. When a moving object is detected inside the smart kitchen cabinet, the moving object will be automatically determined as the target object, so that the target object can be marked in a targeted manner. The targeting is good and it avoids re-image acquisition, image recognition and associated storage of all objects in the smart kitchen cabinet, which can greatly save computing power and improve the efficiency of object marking.

[0054] S103: Perform image recognition on the image information to obtain object information of the target object.

[0055] Among them, each item has its own name, type and item attributes and other characteristics. Image recognition analyzes and recognizes image information containing the target item through image processing algorithm to determine the name, type and attribute information of the target item contained in the image information. The item information of the target item is used to indicate the characteristic information such as the name, type and attribute of the target item; for example, through image recognition, it can be identified that the target item put in by the user is black pepper, which belongs to the seasoning category in the smart kitchen cabinet, and the item attributes are spicy and hot.

[0056] S105: Send the location information and the item information to a cloud database, so that the cloud database stores the location information and the item information in an associated manner.

[0057] After determining the item information, the correspondence between the location information and the item information can be determined, and the location information and the item information are sent to the cloud database. After receiving the information, the cloud database will store them in association, so that the item information of the target item and the location information used to indicate the location of the target item are associated, so that subsequent users can perform positioning, search, modification and other operations according to the cloud database during use, saving search time and greatly improving convenience of use.

[0058] In addition, for each item in the smart kitchen cabinet, the above steps S101 to S105 can be repeatedly performed to mark the item, so as to achieve accurate marking of each item in the smart kitchen cabinet, improve the overall accuracy and adequacy of item marking in the smart kitchen cabinet, and enhance the user experience.

[0059] Specifically, Figure 2 As shown, in some exemplary embodiments, the image information is subjected to image recognition to obtain the object information of the target object, that is, step S103 includes:

[0060] S202: Perform target detection on the image information to obtain an image recognition result.

[0061] Target detection is a technology that automatically identifies objects in images or videos and determines their positions and categories through computer algorithms. The main goal is to accurately find the position of objects and identify the categories of objects in a given image or video. In this embodiment, target detection includes the steps of candidate region generation, feature extraction, object classification and bounding box regression. Among them, the candidate region generation step generates a series of candidate regions that may contain target objects through sliding windows, region filling methods, etc.; then, the feature extraction step uses image processing technology to extract the feature representation of each candidate region, which can be extracted through feature extraction methods such as edge detection, color histogram and texture analysis; then, the object classification step inputs the features of each candidate region into a machine learning algorithm for classification, and determines whether each candidate region is a target object or a background. Specifically, the classification can be performed through support vector machines (SVM) and convolutional neural networks (CNN). Finally, the bounding box regression step accurately determines the position of the candidate region classified as a target through the bounding box regression technology, completes the target detection, and obtains the image recognition result, which includes the image position mark of the target object in the image information and the type information of the target object.

[0062] In some exemplary embodiments, target detection can be achieved through Haar feature cascade classifier, HOG feature + SVM classifier, R-CNN (Region-based Convolutional Neural Networks) series of methods and YOLO (You Only Look Once) series of methods, with high detection efficiency and good accuracy.

[0063] Specifically, in some exemplary embodiments, the target item has an item identification code, and the item identification code is used to indicate the name, type and attribute of the target item; Figure 3As shown, in some exemplary embodiments, the image information is subjected to image recognition to obtain the object information of the target object, that is, step S103 includes:

[0064] S301, identifying an item identification code of the target item in the image information.

[0065] S303: Match the item identification code of the target item based on a preset corresponding relationship between the identification code and the item information to obtain the item information of the target item.

[0066] In some exemplary embodiments, the identification code (or item identification code) may be at least one of a barcode and a QR code; in some exemplary embodiments, the smart kitchen cabinet is provided with an identification device, such as a barcode scanner and an RFID reader / writer, and when a target item with an item identification code is placed in the smart kitchen cabinet, the identification device can automatically scan or read the item identification code on the target item, and obtain information such as the name, type, and attributes of the target item indicated by the item identification code, that is, obtain the item information of the target item.

[0067] Among them, the preset correspondence is used to indicate the multiple identification codes of multiple items. Through the preset correspondence, when the target item is known, the multiple item identification codes of the target item can be known based on the preset correspondence; conversely, when a specific item identification code is obtained, the target item corresponding to the item identification code can also be determined based on the preset correspondence, that is, the item information of the target item can be determined, with a high matching degree, which can greatly improve the recognition accuracy.

[0068] Specifically, in some exemplary embodiments, after performing image recognition on the image information to obtain the object information of the target object, that is, after step S103, the method further includes:

[0069] Verification information carrying the image information and the object information is sent to the cloud database, so that the cloud database verifies the image information and the object information in response to the verification information.

[0070] That is, in addition to being able to associate and store the location information and item information of the target item, the cloud database can also be used to verify the accuracy of the image recognition step; in some exemplary embodiments, the preset correspondence between the identification code and the item information can be pre-stored in the cloud database, and the target correspondence between the image information and the item information can also be pre-stored in the cloud storage; after sending the verification information, the cloud database receives the verification information, and in response to the verification information, the cloud database can compare and match the image information and the item information based on at least one of the stored preset correspondence and the pre-stored correspondence, so as to verify the matching degree between the identified item information and the image information, thereby greatly improving the accuracy of image recognition and the accuracy of the entire item marking process, reducing the probability that the user needs to make manual adjustments later due to inaccurate recognition results, and greatly improving the user experience.

[0071] Specifically, after sending the location information and the item information to the cloud database so that the cloud database stores the location information and the item information in association, that is, after step S105, the method further includes:

[0072] In response to the search instruction for the target item, the location information corresponding to the target item is determined based on the location information and the corresponding item information stored in the cloud database.

[0073] In some exemplary embodiments, the smart kitchen cabinet is provided with an interactive interface for obtaining various operation instructions and operation information input by the user; in other exemplary embodiments, the smart kitchen cabinet and the cloud database can also be communicatively connected with multiple interactive terminals, so that the smart kitchen cabinet can receive operation information and operation instructions sent by the cloud database and multiple interactive terminals, and can send operation information and operation instructions to the cloud database and at least one of the multiple interactive terminals.

[0074] Among them, the search instruction of the target item is one of the operation instructions, which can be obtained through the interactive interface or sent to the smart kitchen cabinet by the interactive terminal to trigger the smart kitchen cabinet to perform a search operation. The smart kitchen cabinet sends the search information of the target item to the cloud database. After receiving the search information of the target item, the cloud database associates the item information and location information of the target item, and sends the location information of the target item to the smart kitchen cabinet or the interactive terminal that issued the search instruction to display the location information corresponding to the target item to the user, thereby greatly improving the convenience and efficiency of the user in finding the target item, saving user time, and improving the user experience.

[0075] Specifically, Figure 4 As shown, the method also includes:

[0076] S402, when it is detected that the smart kitchen cabinet is opened, obtaining image update information of each item at each position in the smart kitchen cabinet.

[0077] S404: Obtain historical image information of each of the locations.

[0078] S406: For each position, determine the difference result between the image update information and the historical image information.

[0079] S408, if the difference result indicates that the image update information is inconsistent with the historical image information, perform image recognition on the image update information to obtain updated item information at the location, and send the location information of the location and the updated item information to the cloud database, so that the cloud database associates and stores the location information and the updated item information.

[0080] Among them, when it is detected that the smart cabinet is opened, it means that the positions of some objects in the smart cabinet may have changed. In this case, the image acquisition device obtains the image update information of each object at each position in the smart cabinet again; the historical image information is the image information of the corresponding position obtained before the smart cabinet is opened; for each position, by comparing the image update information with the historical image information at the corresponding position, the difference result between the two can be determined; if the difference result indicates that the image update information is consistent with the historical image information, it means that no operation such as replacement, placement or removal of the object has occurred at the position, and the position information indicating the position and the associated stored object information remain unchanged; if the difference result indicates that the image update information is inconsistent with the historical image information, it means that the object at the position has been replaced, placed in or taken out, then based on the image update information, steps S103 to S105 are repeatedly executed, and image recognition and associated storage are re-performed to update the cloud database, that is, update the associated stored position information and the corresponding updated object information, and the automatic recognition and automatic marking have high accuracy, good flexibility and good real-time performance.

[0081] Specifically, the smart kitchen cabinet includes a plurality of storage spaces (i.e., kitchen cabinets). Further, each kitchen cabinet includes a plurality of storage spaces. In some exemplary embodiments, the method further includes:

[0082] In response to a remote control instruction carrying a storage space identifier, the storage space corresponding to the storage space identifier in the smart kitchen cabinet corresponding to the remote control instruction is controlled to be opened or closed.

[0083] Among them, the smart kitchen cabinet is provided with multiple electric devices, and each storage space or kitchen cabinet is provided with an electric device. The electric device is communicatively connected to the smart kitchen cabinet, and the multiple electric devices can be controlled individually. Each electric device is used to control the opening or closing of the corresponding storage space, thereby realizing intelligent control and remote operation of each storage space in the smart kitchen cabinet; in some exemplary embodiments, the electric device is communicatively connected to the item marking device of the smart kitchen cabinet (or the central controller of the smart kitchen cabinet) via wireless communication.

[0084] In some exemplary embodiments, the remote control instruction is a type of operating instruction, and the storage space identifier it carries is used to indicate the target storage space to be controlled, so as to accurately control the opening or closing of the storage space corresponding to the storage space identifier, thereby improving the control accuracy; wherein, the remote control instruction carrying the storage space identifier can be sent to the smart kitchen cabinet through an interactive interface or an interactive terminal (such as a mobile phone APP), and the smart kitchen cabinet receives the remote control instruction carrying the storage space identifier, and sends a control signal to the storage space corresponding to the storage space identifier, so that the electric device of the corresponding storage space performs an opening or closing operation.

[0085] Furthermore, if each storage space has multiple storage locations or multiple storage compartments, an electric device can be provided on each storage location or each storage compartment to individually control the opening and closing of each storage location or each storage compartment; correspondingly, in response to a remote control instruction carrying a storage location identifier (or storage compartment identifier), the storage location (or storage compartment) corresponding to the storage location identifier (or storage compartment identifier) ​​in the smart kitchen cabinet corresponding to the remote control instruction is controlled to be opened or closed, which has good operational flexibility and good user convenience.

[0086] Specifically, Figure 5 As shown, in some exemplary embodiments, the method further comprises:

[0087] S501, upon receiving a remote viewing instruction carrying a target item identifier sent by an interactive terminal, controlling the cloud database to send the item information and the location information of the target item corresponding to the target item identifier to the interactive terminal; and / or

[0088] S503, upon receiving a remote editing instruction carrying target item editing information sent by the interactive terminal, updating the item information of the target item according to the target item editing information, obtaining updated item information and associating the updated item information with the location information and storing it in the cloud database.

[0089] In some exemplary embodiments, the remote viewing instruction and the remote editing instruction are both types of operation instructions, wherein, similar to the target item search instruction, the target item identifier is used to indicate the target item to be viewed, so as to quickly find the location information and item information of the target item, thereby improving the convenience of user viewing; and the target item editing information carried in the remote editing instruction is information input by the user in the interactive interface or the interactive terminal, and is used to indicate the edited item information and the edited location information of the target item, so that when the smart kitchen cabinet receives the remote editing instruction carrying the target item editing information, the item information of the target item is updated according to the target item editing information, so that the updated item information and location information are associated and stored in the cloud database, thereby improving the accuracy of item marking; further, the location information of the target item can also be corrected, and the updated item information and the corrected location information are associated and stored in the cloud database, thereby improving the accuracy of item marking.

[0090] That is, the item marking method of the smart kitchen cabinet can realize cross-platform synchronization, sharing and collaboration between the interactive interface, multiple interactive terminals and the cloud database. Through a secure data synchronization protocol, the information stored in the cloud database can be updated in real time between different devices. Users can seamlessly synchronize and share the item marking status of the smart kitchen cabinet between different interactive interfaces and different interactive terminals, including the item information of the items at various positions in the smart kitchen cabinet and their associated location information. Whether the user is using the touch screen interactive interface of the smart kitchen cabinet at home or using interactive terminals such as mobile phone APP when going out, the user can view and edit the associated stored location information and item information in real time; in addition, the item marking method of the smart kitchen cabinet can also realize information sharing and collaboration between multiple interactive terminals, support information sharing and collaboration between family members, and allow joint management and use of the smart kitchen cabinet, that is, it can realize cross-platform synchronization and sharing functions, provide users with a more flexible and convenient use experience, and promote cooperation and communication between family members.

[0091] In addition, in some exemplary embodiments, if image recognition fails in step S103, a prompt message will be generated to remind the user that the image recognition has failed. The user can manually mark the target item on the interactive terminal, and send the target item information to the item marking device of the smart kitchen cabinet along with the remote editing instruction, so that the item marking device of the smart kitchen cabinet updates the item information of the target item according to the target item editing information when receiving the remote editing instruction carrying the target item editing information, and obtains the updated item information, so as to facilitate the subsequent storage of the updated item information and location information in the cloud database to avoid marking errors.

[0092] Corresponding to the object marking method of the smart kitchen cabinet provided by the above embodiment of the present invention, the object marking device of the smart kitchen cabinet provided by the embodiment of the present invention can implement the object marking method of the smart kitchen cabinet in the above method embodiment, wherein, Figure 6 As shown, the item marking device of the smart kitchen cabinet may include:

[0093] The information acquisition module 610 is used to acquire the position information and image information of the target object placed in the smart kitchen cabinet; the position information is used to indicate the target position of the target object in the smart kitchen cabinet;

[0094] The recognition module 620 is used to perform image recognition on the image information to obtain the object information of the target object; the object information is used to indicate the name, type and attribute of the target object;

[0095] The sending module 630 is used to send the location information and the item information to a cloud database, so that the cloud database stores the location information and the item information in an associated manner.

[0096] Specifically, the identification module 620 may further include:

[0097] The target detection module is used to perform target detection on the image information to obtain an image recognition result; the image recognition result includes an image position mark of the target object in the image information and type information of the target object.

[0098] Specifically, the identification module 620 may further include:

[0099] An identification code recognition module, used to recognize an item identification code of the target item in the image information;

[0100] The matching module is used to match the object identification code of the target object based on a preset corresponding relationship between the identification code and the object information to obtain the object information of the target object.

[0101] Specifically, the item marking device of the smart kitchen cabinet may also include:

[0102] A search module is used to respond to the search instruction of the target item and determine the location information corresponding to the target item based on the location information and the corresponding item information stored in the cloud database.

[0103] Specifically, the item marking device of the smart kitchen cabinet may also include:

[0104] A first image acquisition module, configured to acquire image update information of each item at each position in the smart kitchen cabinet when detecting that the smart kitchen cabinet is opened;

[0105] A second image acquisition module is used to acquire the historical image information of each of the positions; the historical image information is the image information of the corresponding position acquired before the smart cabinet is opened;

[0106] A comparison module, used for determining the difference between the image update information and the historical image information for each position;

[0107] and an updating module for performing image recognition on the image update information to obtain updated item information at the location if the difference result indicates that the image update information is inconsistent with the historical image information, and for sending the location information of the location and the updated item information to the cloud database so that the cloud database associates and stores the location information and the updated item information.

[0108] Specifically, the item marking device of the smart kitchen cabinet may also include:

[0109] The remote opening and closing module is used to respond to a remote control instruction carrying a storage space identifier and control the opening or closing of the storage space corresponding to the storage space identifier in the smart kitchen cabinet corresponding to the remote control instruction.

[0110] Specifically, the item marking device of the smart kitchen cabinet may also include:

[0111] A shared viewing module, configured to control the cloud database to send the object information and the location information of the target object corresponding to the target object identifier to the interactive terminal upon receiving a remote viewing instruction carrying the target object identifier sent by the interactive terminal; and / or

[0112] A shared editing module is used to update the object information of the target object according to the target object editing information when receiving a remote editing instruction carrying the target object editing information sent by the interactive terminal, obtain the updated object information and associate the updated object information with the location information and store it in the cloud database.

[0113] It should be noted that the item marking device for the smart kitchen cabinet provided in the above embodiment, when realizing its functions, only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the item marking device for the smart kitchen cabinet provided in the above embodiment belongs to the same concept as the method embodiment, and its specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0114] The item marking device of the smart kitchen cabinet includes a processor and a memory, wherein the processor (or CPU (Central Processing Unit)) is a core component, and its function is mainly to interpret memory instructions and process data fed back by various modules of the item marking device of the smart kitchen cabinet; the structure of the processor is roughly divided into an operation logic component and a register component, etc. The operation logic component mainly performs related logical calculations (such as shift operations, logical operations, fixed-point or floating-point arithmetic operations and address operations, etc.), and the register component is used to temporarily store instructions, data and addresses.

[0115] The memory is a memory device that can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, which may include but is not limited to: Windows system (an operating system), Linux (an operating system), etc., and the present invention does not limit this; in addition, it can also store application programs required for functions, for example, at least one instruction suitable for being loaded and executed by the processor is also stored in the storage space of the memory, and these instructions can be one or more computer programs (including program codes); and the data storage area can store data created according to the use of the device, etc.; accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0116] The method embodiments provided in the embodiments of the present application can be executed in electronic devices such as mobile terminals, computer terminals, servers or similar computing devices. Figure 7 1 is a hardware structure block diagram of an electronic device for executing an item marking method for a smart kitchen cabinet provided in an embodiment of the present application. Figure 7As shown, the electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 710 (the processor 710 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 730 for storing data, and one or more storage media 720 (such as one or more mass storage devices) for storing application programs 723 or data 722. Among them, the memory 730 and the storage medium 720 can be short-term storage or permanent storage. The program stored in the storage medium 720 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the central processing unit 710 can be configured to communicate with the storage medium 720 to execute a series of instruction operations in the storage medium 720 on the electronic device 700. The electronic device 700 may also include one or more power supplies 760, one or more wired or wireless network interfaces 750, one or more input and output interfaces 740, and / or one or more operating systems 721, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0117] The input / output interface 740 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 700. In one example, the input / output interface 740 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 740 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0118] It can be understood by those skilled in the art that Figure 7 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 7 More or fewer components as shown, or with Figure 7 Different configurations shown.

[0119] An embodiment of the present invention also provides a smart kitchen cabinet, including the item marking device of the smart kitchen cabinet as described above, which can automatically mark the items placed in the smart kitchen cabinet, and store the item information of the target item and the location information indicating the location of the target item in a cloud database, and can further be verified through the cloud to improve the recognition accuracy; in addition, it can also realize interactive sharing with multiple interactive terminals, thereby improving the flexibility and convenience of using the Zhengnan kitchen cabinet and greatly improving the user experience.

[0120] Specifically, the item marking system of the smart kitchen cabinet also includes multiple information collection devices, which are arranged in the cooking utensils, and the multiple information collection devices are used to collect current operating information in the cooking utensils and current state parameter information of the target cooking ingredients; in some exemplary embodiments, the multiple information collection devices include at least one of a temperature collection device, a humidity collection device, a solubility collection device, a pH value collection device and a conductivity collection device, so as to fully reflect the current state of the cooking utensils and improve the accuracy and reliability of subsequent predictions.

[0121] An embodiment of the present invention further provides a medium, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the above-mentioned method for marking items in a smart kitchen cabinet; optionally, the medium can be located in at least one network server among multiple network servers of a computer network; in addition, the medium can include but is not limited to random access memory (RAM), read-only memory (ROM), non-volatile memory (NVM), U disk, mobile hard disk, disk storage device, flash memory device, other volatile solid-state storage devices and other storage media that can store program codes.

[0122] According to one aspect of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for marking items in a smart kitchen cabinet provided in the above-mentioned various optional implementations.

[0123] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0124] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0125] What is described above are only some embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art should understand that the present invention may be subject to various changes and improvements, and any modifications, equivalent substitutions and improvements made in accordance with the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A method for marking items in a smart cabinet, characterized in that: For marking a target object placed in a smart kitchen cabinet, the method comprises: Acquire location information and image information of a target object placed in a smart kitchen cabinet; the location information is used to indicate a target location of the target object in the smart kitchen cabinet; Performing image recognition on the image information to obtain item information of the target item; the item information is used to indicate the name, type and attribute of the target item; The location information and the item information are sent to a cloud database, so that the cloud database stores the location information and the item information in association with each other.

2. The method for marking items in a smart cabinet according to claim 1, characterized in that: The performing image recognition on the image information to obtain the object information of the target object includes: Target detection is performed on the image information to obtain an image recognition result; the image recognition result includes an image position mark of the target object in the image information and type information of the target object.

3. The method for marking items in a smart cabinet according to claim 1, characterized in that: The target object has an object identification code, and the object identification code is used to indicate the name, type and attribute of the target object; performing image recognition on the image information to obtain the object information of the target object includes: Identifying an item identification code of the target item in the image information; The object identification code of the target object is matched based on a preset corresponding relationship between the identification code and the object information to obtain the object information of the target object.

4. The method for marking items in a smart cabinet according to claim 1, characterized in that: After sending the location information and the item information to the cloud database so that the cloud database stores the location information and the item information in association with each other, the method further includes: In response to the search instruction for the target item, the location information corresponding to the target item is determined based on the location information and the corresponding item information stored in the cloud database.

5. The method for marking items in a smart cabinet according to any one of claims 1 to 4, characterized in that: The method further comprises: When the smart kitchen cabinet is detected to be opened, obtaining image update information of each item at each position in the smart kitchen cabinet; Acquire the historical image information of each of the positions; the historical image information is the image information of the corresponding position acquired before the smart cabinet is opened; For each position, determining a difference result between the image update information and the historical image information; If the difference result indicates that the image update information is inconsistent with the historical image information, image recognition is performed on the image update information to obtain updated item information at the location, and the location information of the location and the updated item information are sent to the cloud database, so that the cloud database associates and stores the location information and the updated item information.

6. The method for marking items in a smart cabinet according to any one of claims 1 to 4, characterized in that: The smart kitchen cabinet includes a plurality of storage spaces, and the method further includes: In response to a remote control instruction carrying a storage space identifier, the storage space corresponding to the storage space identifier in the smart kitchen cabinet corresponding to the remote control instruction is controlled to be opened or closed.

7. The method for marking items in a smart cabinet according to any one of claims 1 to 4, characterized in that: The method further comprises: Upon receiving a remote viewing instruction carrying a target item identifier sent by an interactive terminal, controlling the cloud database to send the item information and the location information of the target item corresponding to the target item identifier to the interactive terminal; and / or When a remote editing instruction carrying target item editing information is received from the interactive terminal, the item information of the target item is updated according to the target item editing information, updated item information is obtained, and the updated item information and the location information are associated and stored in the cloud database.

8. An item marking device for a smart cabinet, characterized in that: include: An information acquisition module, used to acquire location information and image information of a target object placed in the smart kitchen cabinet; The location information is used to indicate the target location of the target object in the smart kitchen cabinet; A recognition module, used to perform image recognition on the image information to obtain item information of the target item; the item information is used to indicate the name, type and attribute of the target item; The sending module is used to send the location information and the item information to a cloud database, so that the cloud database stores the location information and the item information in an associated manner.

9. A smart cabinet, characterized in that: Used for storing articles, comprising an article marking device of a smart cabinet as claimed in claim 8.

10. A medium, characterized in that The medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the item marking method of the smart cabinet as described in any one of claims 1-6.

Images