Article identification method for refrigeration device, storage medium, system and refrigeration device

Abstract

The application provides an object recognition method for a refrigeration device, a storage medium, a system and the refrigeration device. The recognition method comprises: obtaining a target image of an object stored in the refrigeration device; obtaining a target feature according to the target image; matching the target feature with an object category feature in an object category feature library in a refrigeration device database one by one and calculating a similarity; if the similarity is lower than a first similarity value, obtaining object category naming information of the object, establishing a mapping relationship between the object category naming information and the target feature and storing the object category naming information in the object category feature library. The storage medium, the system and the refrigeration device can realize the above-mentioned recognition method. In this way, the refrigeration device in use can recognize new types of objects, and the object recognition range of the refrigeration device is expanded.

Description

Technical Field

[0001] This invention relates to the field of household appliances, and more particularly to a method, storage medium, system, and refrigeration equipment for identifying items in refrigeration equipment. Background Technology

[0002] With technological advancements, users have increasingly higher demands for refrigeration equipment, making intelligent transformation a new research and development direction for refrigeration equipment. Refrigeration equipment, such as refrigerators, is generally equipped with an item recognition system. This system typically includes a camera installed within the refrigeration unit. By recognizing photos of items inside the storage compartment taken by the camera, it identifies the items stored within the refrigeration unit, enabling intelligent management of the items. Current item recognition systems for refrigeration equipment usually require pre-training for specific types of items before leaving the factory so that the refrigeration unit can recognize those items. However, this design has the following drawbacks: users may store a wide variety of items in the refrigeration unit during use, such as newly introduced products. Therefore, it is impossible to ensure that the item type recognition training before leaving the factory covers all categories of items, resulting in the refrigeration unit being unable to recognize and manage certain types of items. Summary of the Invention

[0003] The purpose of this invention is to provide a method, storage medium, system, and refrigeration equipment for identifying items. By acquiring the item category naming information, establishing a mapping relationship between the item category naming information and the target features of the item, and storing it in the item category feature library, the refrigeration equipment can be enabled to identify new types of items after being put into use, thus expanding the item identification range of the refrigeration equipment.

[0004] To achieve the above-mentioned objective, one embodiment of the present invention provides a method for identifying items in a refrigeration device, wherein the identification method includes:

[0005] Acquire target images of items stored inside the refrigeration equipment;

[0006] Obtain target features based on the target image;

[0007] The target features are matched one by one with the item type features in the item type feature library of the refrigeration equipment database, and the similarity is calculated.

[0008] If the similarity values ​​are all lower than the first similarity value, then the item category naming information of the item is obtained, and a mapping relationship is established between the item category naming information and the target feature and stored in the item category feature library.

[0009] As a further improvement to one embodiment of the present invention, it further includes:

[0010] The target features are matched one by one with the features in the non-managed feature library and the similarity is calculated.

[0011] If a similarity value greater than the second similarity value exists, the target feature is stored in the unmanaged feature library, where the second similarity value is greater than or equal to the first similarity value.

[0012] As a further improvement to one embodiment of the present invention, "obtaining the item category naming information" includes:

[0013] Output item category naming suggestions and receive the input item category naming information.

[0014] As a further improvement to one embodiment of the present invention, it further includes:

[0015] If the naming information is not received, the target feature is stored in the non-managed feature library.

[0016] As a further improvement to one embodiment of the present invention, it further includes:

[0017] If there exists a similarity value greater than or equal to the first similarity value, then the maximum similarity value is obtained.

[0018] Determine whether the maximum similarity is greater than the third similarity value, wherein the third similarity value is greater than the first similarity value;

[0019] If so, then delete the target feature data;

[0020] If not, then obtain the item category to which the item category feature corresponding to the maximum similarity belongs, establish a mapping relationship between the item category and the target feature, and store it in the item category feature library.

[0021] As a further improvement to one embodiment of the present invention, the item type feature library includes sub-item type feature libraries that correspond one-to-one with item types, and each sub-item type feature library includes item type features corresponding to the item type.

[0022] "Obtaining the item category to which the item category feature corresponding to the maximum similarity belongs, establishing a mapping relationship between the item category and the target feature, and storing it in the item category feature database" specifically includes:

[0023] Obtain the item category to which the item category feature corresponding to the maximum similarity belongs, establish a mapping relationship between the item category and the target feature, and store the target feature in the sub-item category feature library corresponding to the item category.

[0024] As a further improvement to one embodiment of the present invention, it further includes:

[0025] Obtain the total number of historical target features that match the item type feature corresponding to the maximum similarity and have been stored in the sub-item type feature library;

[0026] If the total number reaches a preset value, the target feature is deleted.

[0027] As a further improvement to one embodiment of the present invention, it further includes:

[0028] Obtain the similarity value range containing the maximum similarity.

[0029] Obtain the total number of historical maximum similarities corresponding to the aforementioned similarity value intervals;

[0030] If the total number reaches the set interval value, the target feature is deleted. The set interval value is inversely proportional to the size of the similarity value in the similarity value interval.

[0031] As a further improvement to one embodiment of the present invention, it further includes:

[0032] Obtain the total number of item category features in the sub-item category feature library. If the total number reaches a preset value, delete the target feature.

[0033] As a further improvement to one embodiment of the present invention, it further includes:

[0034] If there exists a similarity value greater than or equal to the first similarity value, then the maximum similarity value is obtained.

[0035] Determine whether the maximum similarity is lower than the fourth similarity value, wherein the fourth similarity value is greater than the first similarity value;

[0036] If so, obtain the item category to which the item category feature corresponding to the maximum similarity belongs, and output a prompt message confirming whether the target item belongs to that item category;

[0037] If the instruction information obtained indicates that the target does not belong to the item category, then the item category naming information of the item is obtained, and a mapping relationship is established between the target feature and the item category naming information and stored in the item category feature library.

[0038] As a further improvement to one embodiment of the present invention, it further includes: obtaining the cumulative number of unmatched days for each item type in the refrigeration equipment item type database;

[0039] If the cumulative number of unmatched days reaches the first preset number of days, a prompt message will be output asking whether the item type needs to be managed.

[0040] If an instruction is received that the item type needs to be managed, the data for that item type is retained until the second preset day value is reached;

[0041] If an instruction is received indicating that the item type does not need to be managed, then the item type characteristic data for that item type is deleted.

[0042] As a further improvement to one embodiment of the present invention, it further includes:

[0043] The target feature and the item type feature are both N-dimensional feature vectors with the same dimensions, where N dimensions include 512 or 256 dimensions.

[0044] As a further improvement to one embodiment of the present invention, "acquiring a target image of an item stored in the refrigeration device" specifically includes:

[0045] Acquire images captured by the refrigeration equipment;

[0046] The image is input into the detection model, and any structures in the image that are not inherent to the refrigeration equipment are uniformly classified as foreground.

[0047] Obtain a target image of the item stored inside the refrigeration device from the foreground.

[0048] To achieve the above-mentioned objectives, one embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the article identification method for refrigeration equipment described in any of the above embodiments.

[0049] To achieve the above-mentioned objectives, one embodiment of the present invention provides an item identification system, which includes a control module and an item identification module. The item identification system further includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps in the item identification method for refrigeration equipment described in any of the above embodiments.

[0050] To achieve the above-mentioned objectives, one embodiment of the present invention provides a refrigeration device, the refrigeration device including an item identification system, the item identification system including a control module and an item identification module, wherein the item identification system further includes a memory and a processor, the memory storing a computer program that can run on the processor, and when the processor executes the computer program, it implements the steps in the item identification method for the refrigeration device described in any of the above embodiments.

[0051] Compared with the prior art, the present invention obtains the item category naming information, establishes a mapping relationship between the item category naming information and the target features of the item, and stores it in the item category feature database. Its beneficial effect is that it enables the refrigeration equipment after it is put into use to identify new types of items, thus expanding the item recognition range of the refrigeration equipment. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the structure of a refrigeration device according to an embodiment of the present invention;

[0053] Figure 2 yes Figure 1 A schematic diagram of the refrigeration equipment database shown;

[0054] Figure 3 This is a flowchart of an embodiment of the item identification method of the present invention;

[0055] Figure 4 This is a flowchart of another embodiment of the item identification method of the present invention. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0057] Reference Figure 1 In one embodiment of the present invention, a refrigeration device 100 is provided. The refrigeration device 100 may include a housing 1 and a storage compartment 2 disposed within the housing 1. The storage compartment 2 may include a refrigerator compartment and a freezer compartment for storing items 11. The refrigeration device 100 may also include a door 3 for opening and closing the storage compartment 2. The door 3 may be provided with a bottle holder or a door compartment, etc., for storing items 11.

[0058] In this embodiment, the refrigeration device 100 can be a refrigerator. Of course, the refrigeration device 100 in this application can also be a freezer, wine cabinet, commercial display cabinet, etc.

[0059] Reference Figure 1 , Figure 2 , Figure 3 Furthermore, in another embodiment of the present invention, an item identification method that can be used in a refrigeration device 100 is provided, wherein the identification method may include:

[0060] Obtain the target image of item 11 stored in refrigeration equipment 100;

[0061] Target feature 5 is obtained from the target image;

[0062] The target feature 5 is matched one by one with the item type feature 9 in the item type feature library 7 of the refrigeration equipment database 6, and the similarity is calculated.

[0063] If the similarity values ​​are all lower than the first similarity value, then the item category naming information of item 11 is obtained, the item category naming information is mapped to the target feature 5 and stored in the item category feature library 7.

[0064] In this embodiment, the target image may be a segmented image of item 11 containing only one item 11.

[0065] In actual identification, there are usually multiple items 11 stored in the refrigeration device 100. The image recognition module can segment the acquired image containing multiple items 11 as needed, and acquire several target images containing only one item 11. Then, the several target images can be identified separately. This setting can prevent different items 11 from interfering with each other during identification and improve the accuracy of identification.

[0066] In this embodiment, target feature 5 can refer to the feature information of the target obtained from the target image, such as color, texture, edge line, size, etc.

[0067] In this embodiment, the article type feature 9 can refer to the feature information obtained from a certain article 11 that can be used to identify that article 11.

[0068] For example, the type of item 11 could be an apple or a potato, and the item type feature 9 could be feature information obtained from images of apples or potatoes that could be used to identify whether item 11 belongs to apples or potatoes.

[0069] In this embodiment, there may be several item type features 9 of the same item type, and several different item type features 9 may belong to the same item type.

[0070] For example, if the item type is apple, multiple images of apples can be collected from the top, bottom, left, and right sides of the apple. The item type feature 9 of apples can be obtained from these multiple apple images. In this way, several sets of apple item type features 9 that are different from each other can be obtained. Correspondingly, although these several item type features 9 are different, they all belong to apples.

[0071] In this embodiment, the refrigeration equipment database 6 can refer to a collection of various information and data stored in the hard disk of the refrigeration equipment 100. The refrigeration equipment 100 can perform various operations on the data and information in the database, such as calling, comparing, matching, and querying.

[0072] In this embodiment, the item type feature library 7 can refer to a sub-database in the refrigeration equipment database 6 specifically used to store various item type features 9.

[0073] The item type feature library 7 may include several different sets of item type features 9 belonging to the same item type, and may also include several different sets of item type features 9 belonging to different item types.

[0074] The naming information for an item category can refer to the name used to represent a specific item category. For example, the naming information for an item category could be banana, grape, etc.

[0075] In this embodiment, the data information of the item type feature 9 and the data information of the item type to which it belongs are mapped in the refrigeration equipment database 6. That is, the data information of the item type to which it belongs can be found through the data information of the item type feature 9, including the name of the item type and other information. Through the data information of a specific item type, the data information of all item type features 9 under that item type can be found.

[0076] In the actual process of item recognition in the refrigeration equipment 100, the image captured by the image recognition module can be acquired first, and then the image segmentation can be performed to obtain the target image. After that, the acquired target feature 5 is matched one by one with all the item type features 9 in the item type feature library 7, and the similarity between the target feature 5 and the item type features 9 stored in the item type feature library 7 is compared.

[0077] If the similarity between the target feature 5 and all the item category features 9 stored in the item category feature library 7 is lower than the preset first similarity value, then it can be considered that the similarity between the item category features 7 in the existing item category feature library 7 of the refrigeration device 100 and the target feature 5 is not high. The existing item category feature library 7 of the refrigeration device 100 does not contain data information related to the item category to which the target feature 5 belongs. The target feature 5 and its item category belong to new data information for the existing item category feature library 7 of the refrigeration device 100. Therefore, it is necessary to obtain the item category naming information of the item 11, establish a mapping relationship between the item category naming information and the target feature 5, and store both the item category naming information and the target feature 5 in the item category feature library 7 so that the refrigeration device 100 can identify the item 11 that belongs to the same item category as the current target feature 5 when it performs item identification next time.

[0078] In this embodiment, the similarity can refer to the degree of similarity between target feature 5 and a certain item category feature in the item category feature library, and can be represented by a value between 0 and 1.

[0079] In real life, the refrigeration equipment 100 undergoes pre-training for the identification of items 11 before leaving the factory, enabling the refrigeration equipment 100 to intelligently identify items 11 during use and thus manage them. For example, it helps users understand all the information about items 11 already stored in the refrigeration equipment 100, and informs users of the cumulative storage time of items 11.

[0080] However, since users may store a variety of items 11 in the refrigeration equipment 100, such as newly developed food products or products from other countries, it is impossible to ensure that all categories of items 11 are covered during the item recognition training before the refrigeration equipment 100 leaves the factory. As a result, the refrigeration equipment 100 cannot identify and manage certain items 11 during use.

[0081] Therefore, by adopting the design scheme of the present invention, after the refrigeration equipment 100 is put into use, the refrigeration equipment 100 can automatically supplement the refrigeration equipment database 6 with new item type information and item type feature 9 information. This enables the refrigeration equipment 100 to have the ability to identify and train new item types after it is put into use, and the item identification ability of the refrigeration equipment 100 can be continuously updated over time, expanding the item identification range of the refrigeration equipment 100, effectively improving the identification and management effect of the refrigeration equipment 100 on item 11, improving user satisfaction, realizing the self-updating of the refrigeration equipment database 6, realizing the autonomous identification of food ingredients, and recognizing the category of food ingredients does not require pre-shipment training, and the database does not need to be manually updated, thus realizing the self-learning of food ingredients.

[0082] Reference Figure 3 and Figure 4 Furthermore, in another embodiment of the present invention, the article identification method applicable to the refrigeration device 100 may further include:

[0083] The target feature 5 and the item type feature 9 are both N-dimensional feature vectors with the same dimensions, and the N dimensions may include 512 dimensions or 256 dimensions.

[0084] This setup facilitates the matching and similarity calculation of target feature 5 and item category feature 9. Furthermore, both target feature 5 and item category feature 9 employ high-dimensional feature vectors, ensuring that the acquired feature data is more comprehensive and detailed, thereby guaranteeing the accuracy of the matching results.

[0085] Furthermore, in another embodiment of the present invention, the similarity between target feature 5 and item category feature 9 can be calculated using either the COSIN similarity algorithm or the Faiss (Facebook AI Similarity Search) algorithm.

[0086] Furthermore, in another embodiment of the present invention, the first similarity value may be 0.5.

[0087] Reference Figure 4 In further, in another embodiment of the present invention, the article recognition method that can be used in the refrigeration device 100, wherein "acquiring the target image of the article 11 stored in the refrigeration device 100" specifically includes:

[0088] Acquire images captured by the refrigeration equipment 100;

[0089] The image is input into the detection model, and any elements in the image that are not inherent structures of the cooling device 100 are uniformly classified as foreground.

[0090] Obtain a target image of the item 11 stored in the refrigeration device 100 from the foreground.

[0091] In this embodiment, the detection model can be established based on the YOLO series of target detection and recognition algorithms.

[0092] In the actual object recognition process of the refrigeration device 100, images captured by the image recognition module of the refrigeration device 100 can be acquired first, and then the images can be fed into the detection model. This detection model can be a model pre-trained using the refrigeration device 100 for recognition. This detection model can segment and initially classify the image content, classifying parts of the image that belong to the inherent structure of the refrigeration device 100 as background that does not require recognition, and classifying parts of the image that do not belong to the inherent structure of the refrigeration device 100 as foreground that awaits further recognition. After the initial classification of the image content is completed by the detection model, only the foreground needs to be considered, and the target image needs to be obtained from the foreground.

[0093] This configuration reduces the interference of the inherent structure of the cooling device 100 in the image on the acquisition and recognition of the target image, facilitates the acquisition of the target image, simplifies the computation, and improves the efficiency and accuracy of target image acquisition.

[0094] Reference Figure 4 Furthermore, in another embodiment of the present invention, the article identification method applicable to the refrigeration device 100, as described in S7-S8, may further include:

[0095] The target feature 5 is matched one by one with the features in the non-managed feature library and the similarity is calculated.

[0096] If a similarity value greater than the second similarity value exists, the target feature 5 is stored in the unmanaged feature library, where the second similarity value is greater than or equal to the first similarity value.

[0097] In this embodiment, the database of the refrigeration equipment 100 may include a non-managed database 8. The non-managed database 8 may refer to a collection of data information on item type characteristics 9 for which the refrigeration equipment 100 does not need to identify and manage item types.

[0098] In real life, users may not need to manage certain types of items based on their usage habits. A non-managed database 8 can be set up in the refrigeration equipment database 6, and this database can be used specifically to store data on these item types that do not require identification or management.

[0099] During item identification, target feature 5 can be matched with item category feature 9 in the non-managed database 8, and a similarity score can be calculated. If a similarity score is greater than a pre-set second similarity value, it indicates that the item category to which target feature 5 belongs is one that the user does not need to identify or manage, and therefore no further identification operation is required for target feature 5. Additionally, target feature 5 can be stored in the non-managed feature library to update and expand the data information in the non-managed database 8.

[0100] In this embodiment, the second similarity value can be 0.6.

[0101] This configuration allows for identification and management operations based on user needs, making the identification and management of items by the refrigeration equipment 100 more tailored to user requirements and more personalized. It avoids sending information about items 11 that do not require management to users, thus improving the user experience. In addition, it can expand the data information in the non-management database 8, allowing the data information in the non-management database 8 to be continuously updated over time, effectively improving the identification and management effect of the refrigeration equipment 100 on items 11.

[0102] Reference Figure 4 In S10-S11, further, in another embodiment of the present invention, the article identification method that can be used for refrigeration equipment 100, wherein "obtaining the article type naming information of the article 11" may include:

[0103] Output item category naming suggestions and receive the input item category naming information.

[0104] In this embodiment, the refrigeration device 100 may be equipped with an interaction module. The interaction module may be a voice interaction module or a display screen interaction module, etc. It can send output item category naming prompts to the user through the interaction module and receive the item category naming information input by the user as the name of the item category to which the target feature 5 belongs.

[0105] This setup allows for easy and quick access to information about the type of a new item 11 by asking the user. It is highly practical, and users can input the name of the item type according to their own habits, making the item type information stored in the refrigeration equipment database 6 more in line with user habits, meet user needs, and be more personalized.

[0106] Furthermore, in another embodiment of the present invention, the refrigeration device 100 can also automatically generate the name of the item 11 and use it as the item category naming information for the item 11. For example, the refrigeration device 100 can generate Lebel1 as the name of the item category to which the target feature 5 belongs.

[0107] Reference Figure 4 In S12, further, in another embodiment of the present invention, the article identification method that can be used in the refrigeration device 100 may further include:

[0108] If the naming information is not received, the target feature 5 is stored in the non-managed feature library.

[0109] In actual use, if no naming information is received from the user, or if the user is not required to manage item 11, it means that the user does not need to manage the ingredient. The target feature 5 can be directly stored in the non-managed feature library to avoid asking the user again next time.

[0110] This configuration allows for identification and management operations based on user needs, making the identification and management of items by the refrigeration equipment 100 more tailored to user requirements and more personalized. It also avoids repeatedly sending users information about items 11 that do not require management, thus improving the user experience. Furthermore, it allows for the expansion of data in the non-management database 8, enabling the data in the non-management database 8 to be continuously updated over time, effectively improving the identification and management performance of items 11 by the refrigeration equipment 100.

[0111] Reference Figure 4 S14-S17, further, in another embodiment of the present invention, the article identification method that can be used in the refrigeration device 100 may further include:

[0112] If there exists a similarity value greater than or equal to the first similarity value, then the maximum similarity value is obtained.

[0113] Determine whether the maximum similarity is greater than the third similarity value, wherein the third similarity value is greater than the first similarity value;

[0114] If so, then delete the data for target feature 5;

[0115] If not, then obtain the item category to which the item category feature 9 corresponding to the maximum similarity belongs, establish a mapping relationship between the item category and the target feature 5, and store it in the item category feature library 7.

[0116] In this embodiment, the maximum similarity is the largest similarity value among several similarities obtained after comparing the target feature 5 with the item type feature 9 in the item type feature library 7 one by one.

[0117] In this embodiment, the third similarity value can be a value close to 1, such as 0.9.

[0118] In practical use, if the similarity between target feature 5 and a certain item category feature 9 in the item category feature library 7 is greater than the preset third similarity, it indicates that target feature 5 and item category feature 9 belong to the same item category and are very similar. Therefore, storing target feature 5 in the item category feature library 7 is not very valuable. Furthermore, when the refrigeration equipment 100 performs item recognition again, it will increase the number of feature comparisons and calculations when the new target feature 5 is matched one-to-one with the item category feature 9 in the item category feature library 7, reducing recognition efficiency and occupying the storage space of the refrigeration equipment database 6. Therefore, there is no need to store target feature 5 in the item category feature library 7; the data of target feature 5 can be deleted directly.

[0119] If the target feature 5 has a maximum similarity between the first and third similarity values ​​and the item category feature 9 in the item category feature library 7, it means that the target feature 5 is most similar to the item category feature 9 corresponding to the maximum similarity value, and can be considered to belong to the same item category. At the same time, the target feature 5 and the item category feature 9 have certain differences. Storing the target feature 5 into the item category feature library 7 has certain value and can expand and update the data information in the item category feature library 7. Therefore, the operation of storing the target feature 5 into the item category feature library 7 can be performed.

[0120] This configuration can expand and update the data information in the refrigeration equipment database 6 while avoiding storing item type features 9 that are too similar, ensuring the diversity and differences of item 11 features in the refrigeration equipment database 6, avoiding excessive overlap of data information in the item 11 feature library, improving recognition efficiency, reducing computational load, and saving storage space in the refrigeration equipment database 6.

[0121] Reference Figure 4In S17, further, in another embodiment of the present invention, the item identification method that can be used for refrigeration equipment 100 includes an item type feature library 7 that may include a sub-item type feature library 10 that corresponds one-to-one with item types, and the sub-item type feature library 10 may include item type features 9 corresponding to the item types.

[0122] "Obtaining the item category to which the item category feature 9 corresponding to the maximum similarity belongs, establishing a mapping relationship between the item category and the target feature 5, and storing it in the item category feature library 7" may specifically include:

[0123] Obtain the item category to which the item category feature 9 corresponding to the maximum similarity belongs, establish a mapping relationship between the item category and the target feature 5, and store the target feature 5 in the sub-item category feature library 10 corresponding to the item category.

[0124] In this embodiment, the sub-item category feature library 10 can be a sub-database within the item category feature library 7. The item category feature library 7 can contain several sub-item category feature libraries 10, and each sub-item category feature library 10 can correspond to one item category. Each sub-item category feature library 10 can contain several sets of item category features 9 for that item category.

[0125] For example, an apple can correspond to a sub-item category feature library 10, which can store several different item category features 9 related to apples.

[0126] In this embodiment, a mapping relationship can be established between the data information of item types and the sub-item type database. The target feature 5 is stored in the sub-item type database, and the target feature 5 can automatically establish a mapping relationship with the data information of that item type.

[0127] This setup facilitates the management, comparison, matching, and querying of data information within the item category feature library 7, and makes it easier to establish a mapping relationship between the data information of the item category and the item category feature 9.

[0128] Reference Figure 4 S20-S23, further, in another embodiment of the present invention, the article identification method that can be used in the refrigeration device 100 may further include:

[0129] Obtain the total number of historical target features 5 that match the item category feature 9 corresponding to the maximum similarity and have been stored in the sub-item category feature library 10;

[0130] If the total number reaches a preset value, then the target feature 5 is deleted.

[0131] The historical target feature 5 can refer to the target feature 5 that matched the item category feature 9 before the current target feature 5. The item category feature 9 refers to the item category feature 9 that corresponds to the maximum similarity obtained by matching the current target feature 5.

[0132] The maximum historical similarity can refer to the maximum similarity obtained after matching the historical target feature 5 with all the item category features 9 in the item category feature library 7 one by one.

[0133] In practical use, the refrigeration equipment 100 will undergo countless object recognition processes. In order to prevent the data information of the target feature 5 obtained during each recognition from being stored in the refrigeration equipment database 6 without restraint, it is necessary to take certain measures to limit the amount of data information of the target feature 5 stored.

[0134] In this embodiment, a method is adopted to limit the number of target features 5 that match the same item category feature 9 and obtain the maximum similarity. When the total number of target features 5 that match the same item category feature 9 and obtain the maximum similarity reaches a set value, no more target features 5 that match the same item category feature 9 and obtain the maximum similarity will be stored.

[0135] The reason is that the target features 5 that match the same item category feature 9 and achieve the highest similarity are likely to be quite similar to each other. Therefore, storing a certain number of target features 5 that match the same item category feature 9 and achieve the highest similarity is sufficient to meet the requirements. However, storing too many of them would lead to a high degree of overlap in the data of item category features 9 within the sub-item category feature library 10 to which the item category feature 9 belongs, thus occupying storage space in the sub-item category feature library 10.

[0136] This configuration can expand and update the data information in the refrigeration equipment database 6 while avoiding storing item type features 9 that are too similar, ensuring the diversity and differences of item 11 features in the refrigeration equipment database 6, avoiding excessive overlap of data information in the item 11 feature library, improving recognition efficiency, reducing computational load, and saving storage space in the refrigeration equipment database 6.

[0137] Reference Figure 4 S20-S23, further, in another embodiment of the present invention, the article identification method that can be used in the refrigeration device 100 may further include:

[0138] Obtain the similarity value range containing the maximum similarity;

[0139] Obtain the total number of historical maximum similarities corresponding to the aforementioned similarity value intervals;

[0140] If the total number reaches the interval setting value, then the target feature 5 is deleted. The interval setting value is inversely proportional to the size of the similarity value in the similarity value interval.

[0141] In this embodiment, the similarity value interval refers to an interval that covers a certain range of values.

[0142] The numerical range of the similarity value interval can be between the first similarity value interval and the third similarity value interval. Furthermore, the interval between the first and third similarity values ​​can be divided into several similarity value intervals, each corresponding to a specific interval setting value.

[0143] For example, the first similarity value can be 0.5, the third similarity value can be 0.9, and the range between 0.5 and 0.9 can be divided into four similarity value intervals: the first similarity value interval can be [0.5, 0.6), the second similarity value interval can be [0.6, 0.7), the third similarity value interval can be [0.7, 0.8), and the fourth similarity value interval can be [0.8, 0.9].

[0144] Furthermore, the interval setting value can be inversely proportional to the magnitude of the similarity value of the similarity value interval. In this embodiment, the interval setting value of the first similarity value interval [0.5, 0.6) can be 200, the interval setting value of the second similarity value interval [0.6, 0.7) can be 150, the interval setting value of the third similarity value interval [0.7, 0.8) can be 100, and the interval setting value of the fourth similarity value interval [0.8, 0.9) can be 50.

[0145] The reason for this setting is that the greater the historical maximum similarity obtained by matching the same item category feature 9, the higher the similarity between the corresponding historical target feature 5 and the item category feature 9, and the lower the comparison value stored in the database.

[0146] This configuration can expand and update the data information in the refrigeration equipment database 6 while avoiding storing item type features 9 that are too similar, ensuring the diversity and differences of item 11 features in the refrigeration equipment database 6, avoiding excessive overlap of data information in the item 11 feature library, improving recognition efficiency, reducing computational load, and saving storage space in the refrigeration equipment database 6.

[0147] Reference Figure 4 Furthermore, in another embodiment of the present invention, the article identification method applicable to the refrigeration device 100, as described in S18-S19, may further include:

[0148] Obtain the total number of item category features 9 in the sub-item category feature library 10. If the total number reaches a preset value, delete the target feature 5.

[0149] In this embodiment, a method is adopted to limit the total number of item type features 9 stored in the same sub-item type feature library 10. When the total number of item type features 9 stored in the same sub-item type feature library 10 reaches a preset value, no more item type features 9 will be stored in that sub-item type feature library 10.

[0150] In this embodiment, the preset value of the total number of item type features 9 in the sub-item type feature library 10 can be 2000.

[0151] This configuration can expand and update the data information in the refrigeration equipment database 6 while avoiding storing item type features 9 that are too similar, ensuring the diversity and differences of item 11 features in the refrigeration equipment database 6, avoiding excessive overlap of data information in the item 11 feature library, improving recognition efficiency, reducing computational load, and saving storage space in the refrigeration equipment database 6.

[0152] Reference Figure 4 In S18-S23, further, in another embodiment of the present invention, a means of combining limiting the number of target features 5 that match the same item type feature 9 to obtain the maximum similarity with the storage of the same item type feature 9 with limiting the total number of item type features 9 stored in the same sub-item type feature library 10 can be adopted.

[0153] Once the total number of item category features 9 stored in the same sub-item category feature library 10 reaches a preset value, regardless of whether the total number of target features 5 that match the same item category feature 9 with the highest similarity reaches the set value, no further item category features 9 will be stored in the sub-item category feature library 10.

[0154] This configuration can expand and update the data information in the refrigeration equipment database 6 while avoiding storing item type features 9 that are too similar, ensuring the diversity and differences of item 11 features in the refrigeration equipment database 6, avoiding excessive overlap of data information in the item 11 feature library, improving recognition efficiency, reducing computational load, and saving storage space in the refrigeration equipment database 6.

[0155] Furthermore, in another embodiment of the present invention, the article identification method applicable to the refrigeration device 100 may further include:

[0156] If there exists a similarity value greater than or equal to the first similarity value, then the maximum similarity value is obtained.

[0157] Determine whether the maximum similarity is lower than the fourth similarity value, wherein the fourth similarity value is greater than the first similarity value;

[0158] If so, obtain the item category to which the item category feature 9 corresponding to the maximum similarity belongs, and output a prompt message confirming whether the target item 11 belongs to that item category;

[0159] If the instruction information obtained indicates that the target does not belong to the item category, then the item category naming information of the item 11 is obtained, and a mapping relationship is established between the target feature 5 and the item category naming information and stored in the item category feature library 7.

[0160] In the actual process of item recognition, although the maximum similarity obtained by matching target feature 5 with item category feature 9 in item category feature library 7 is greater than the first similarity value, target feature 5 may not belong to the same item category as item category feature 9. For example, cabbage and iceberg lettuce have a high degree of similarity, but they do not belong to the same item category.

[0161] Therefore, if all target features 5 with a similarity greater than or equal to the first similarity value are directly identified as belonging to a certain item category already existing in the refrigeration equipment database 6, it is easy to cause the item category identification of target features 5 that have a maximum similarity greater than the first similarity value but are very close to the first similarity value to be incorrect.

[0162] Therefore, the user can be asked to confirm the type of item to which the target feature 5 belongs through the interaction module, thereby ensuring the accuracy of item identification by the refrigeration equipment 100.

[0163] In this embodiment, the fourth similarity value can be 0.6.

[0164] This setup increases the accuracy of item recognition by the refrigeration device 100 through user interaction, minimizing the possibility of incorrect item recognition. It is simple and convenient to operate.

[0165] Furthermore, in another embodiment of the present invention, the article identification method applicable to the refrigeration device 100 may further include:

[0166] Obtain the cumulative number of unmatched days for each item type in the 100 item type database of the refrigeration equipment;

[0167] If the cumulative number of unmatched days reaches the first preset number of days, a prompt message will be output asking whether the item type needs to be managed.

[0168] If an instruction is received that the item type needs to be managed, the data for that item type is retained until the second preset day value is reached;

[0169] If an instruction is received indicating that the item type does not need to be managed, then delete the item type feature 9 data for that item type.

[0170] In this embodiment, the cumulative number of unmatched days can be the consecutive cumulative number of days when all item category features 9 of a certain item category have not been matched to obtain the maximum similarity.

[0171] If none of the item category features 9 of a certain item category can be matched to obtain the maximum similarity, it means that the item 11 stored in the refrigeration equipment 100 does not contain that item category.

[0172] The cumulative number of unmatched days is the cumulative value of consecutive days. If on a certain day a certain item category feature of the item category matches the target feature with the maximum similarity, then the cumulative number of unmatched days is recalculated.

[0173] In real life, society develops rapidly, and all sorts of items 11 are constantly being introduced. Some items 11 may have existed before, but have since ceased production forever. It's also possible that users will never need some items 11. To prevent data on these items 11 from remaining in the database of the refrigeration equipment 100, it is necessary to take certain measures to clean up the data belonging to these items 11.

[0174] In the actual operation of this embodiment, the cumulative number of unmatched days can be calculated for all item types in the refrigeration equipment database 6. When the cumulative number of unmatched days for a certain item 11 reaches a certain value, the user can be asked through the interactive module whether the item type still needs to be managed. If the user inputs the instruction information that the item type needs to be managed, then the item type can continue to be managed. If the user inputs the instruction information that the item type does not need to be managed, then the item type feature 9 data of the item type can be completely deleted.

[0175] In this embodiment, the first preset number of days can be 180 days, and the second preset number of days can be 180 days.

[0176] In practical use, if an item is not matched with the maximum similarity score for 180 consecutive days, it means that the item may no longer be produced or used by the user. In this case, the user can be asked whether they still need to manage it. If the user answers yes, the data information of the item will be retained for another 180 days. If the user answers no, it can be deleted directly.

[0177] The first preset number of days and the second preset number of days may also be different.

[0178] This setup allows for identification and management operations based on user needs, making the identification and management of refrigeration equipment 100 more tailored to user requirements and more personalized, enhancing the user experience, improving the efficiency of item identification in refrigeration equipment 100, reducing computational load, and saving storage space in the refrigeration equipment database 6. It is convenient, quick, and highly practical.

[0179] Furthermore, in another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the article identification method that can be used in the refrigeration device 100 as described in any of the above embodiments.

[0180] Furthermore, in another embodiment of the present invention, an item identification system is provided. The item identification system may include a control module and an item identification module. The item identification system may also include a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps in the item identification method for use in the refrigeration device 100 described in any of the above embodiments.

[0181] Furthermore, in another embodiment of the present invention, a refrigeration device 100 is provided. The refrigeration device 100 may include an item recognition system. The item recognition system may include a control module and an item recognition module. The item recognition system may also include a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps in the item recognition method for the refrigeration device 100 described in any of the above embodiments.

[0182] Reference Figure 1 In this embodiment, the item recognition module may include a camera 4 mounted on the cabinet 1 or the door 3. There may be multiple cameras 4, which may be positioned at different locations on the cabinet 1 and the door 3. Under the control of the control module, the image recognition module can capture and recognize images of the items 11 stored inside the refrigeration equipment 100, thereby enabling intelligent management of the items 11.

[0183] In summary, the item identification method, storage medium, system, and refrigeration equipment 100 of the present invention, by acquiring the item type naming information of item 11, establishing a mapping relationship between the item type naming information and the target feature 5 of item 11, and storing it in the item type feature library 7, can solve the problem in the prior art that the refrigeration equipment 100 cannot ensure that all categories of items 11 are covered in the item type identification training before leaving the factory, resulting in the refrigeration equipment 100 being unable to identify and manage certain types of items 11.

[0184] By adopting the technical solution in this application, after the refrigeration equipment 100 is put into use, it can automatically supplement the refrigeration equipment database 6 with new item type information and item type feature information 9. This enables the refrigeration equipment 100 to have the ability to identify and train on new item types, allowing its item recognition capability to be continuously updated over time, expanding the item recognition range of the refrigeration equipment 100, effectively improving the identification and management effect of the refrigeration equipment 100 on item 11, and increasing user satisfaction. The self-updating method of the refrigeration equipment database 6 enables autonomous identification of food ingredients, and the identification of food ingredient categories does not require training, and the database does not need to be manually updated, thus achieving self-learning of food ingredients. Furthermore, it can perform identification and management operations according to user needs, making the item identification and management of the refrigeration equipment 100 more responsive to user needs and more personalized, improving the user experience, increasing the efficiency of item identification of the refrigeration equipment 100, reducing computational load, and saving storage space in the refrigeration equipment database 6. It is convenient, quick, and highly practical.

[0185] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0186] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying items in refrigeration equipment, characterized in that, The identification method includes: Acquire target images of items stored inside the refrigeration equipment; Obtain target features based on the target image; The target features are matched one by one with the item type features in the item type feature library of the refrigeration equipment database, and the similarity is calculated. If the similarity values ​​are all lower than the first similarity value, then the item category naming information of the item is obtained, the item category naming information is mapped to the target feature and stored in the item category feature library; If there exists a similarity value greater than or equal to the first similarity value, then the maximum similarity value is obtained. Determine whether the maximum similarity is greater than the third similarity value, wherein the third similarity value is greater than the first similarity value; If so, then delete the target feature data; If not, then obtain the item category to which the item category feature corresponding to the maximum similarity belongs, establish a mapping relationship between the item category and the target feature, and store it in the item category feature library; The item category feature library includes sub-item category feature libraries that correspond one-to-one with item categories. Each sub-item category feature library contains item category features corresponding to the item category. "Obtaining the item category to which the item category feature corresponding to the maximum similarity belongs, establishing a mapping relationship between the item category and the target feature, and storing it in the item category feature database" specifically includes: Obtain the item category to which the item category feature corresponding to the maximum similarity belongs, establish a mapping relationship between the item category and the target feature, and store the target feature in the sub-item category feature library corresponding to the item category.

2. The article identification method for refrigeration equipment as described in claim 1, characterized in that, Also includes: The target features are matched one by one with the features in the non-managed feature library and the similarity is calculated. If a similarity value greater than the second similarity value exists, the target feature is stored in the unmanaged feature library, where the second similarity value is greater than or equal to the first similarity value.

3. The article identification method for refrigeration equipment as described in claim 2, characterized in that, "Obtaining the item category naming information" includes: Output item category naming suggestions and receive the input item category naming information.

4. The article identification method for refrigeration equipment as described in claim 3, characterized in that, Also includes: If the naming information is not received, the target feature is stored in the non-managed feature library.

5. The article identification method for refrigeration equipment as described in claim 1, characterized in that, Also includes: Obtain the total number of historical target features that match the item type feature corresponding to the maximum similarity and have been stored in the sub-item type feature library; If the total number reaches a preset value, the target feature is deleted.

6. The article identification method for refrigeration equipment as described in claim 5, characterized in that, Also includes: Obtain the similarity value range containing the maximum similarity; Obtain the total number of historical maximum similarities corresponding to the aforementioned similarity value intervals; If the total number reaches the set interval value, the target feature is deleted. The set interval value is inversely proportional to the size of the similarity value in the similarity value interval.

7. The article identification method for refrigeration equipment as described in claim 1, characterized in that, Also includes: Obtain the total number of item category features in the sub-item category feature library. If the total number reaches a preset value, delete the target feature.

8. The article identification method for refrigeration equipment as described in claim 1, characterized in that, Also includes: If there exists a similarity value greater than or equal to the first similarity value, then the maximum similarity value is obtained. Determine whether the maximum similarity is lower than the fourth similarity value, wherein the fourth similarity value is greater than the first similarity value; If so, obtain the item category to which the item category feature corresponding to the maximum similarity belongs, and output a prompt message confirming whether the target item belongs to that item category; If the instruction information obtained indicates that the target does not belong to the item category, then the item category naming information of the item is obtained, and a mapping relationship is established between the target feature and the item category naming information and stored in the item category feature library.

9. The article identification method for refrigeration equipment as described in claim 1, characterized in that, Also includes: Obtain the cumulative number of unmatched days for each item category in the refrigeration equipment item category database; If the cumulative number of unmatched days reaches the first preset number of days, a prompt message will be output asking whether the item type needs to be managed. If an instruction is received that the item type needs to be managed, the data for that item type is retained until the second preset day value is reached; If an instruction is received indicating that the item type does not need to be managed, then the item type characteristic data for that item type is deleted.

10. The article identification method for refrigeration equipment as described in claim 1, characterized in that, Also includes: The target feature and the item type feature are both N-dimensional feature vectors with the same dimensions, where N dimensions include 512 or 256 dimensions.

11. The article identification method for refrigeration equipment as described in claim 1, characterized in that, "Acquiring a target image of an item stored inside the refrigeration equipment" specifically includes: Acquire images captured by the refrigeration equipment; The image is input into the detection model, and any structures in the image that are not inherent to the refrigeration equipment are uniformly classified as foreground. Obtain a target image of the item stored inside the refrigeration device from the foreground.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the article identification method for refrigeration equipment according to any one of claims 1-11.

13. An item recognition system, the item recognition system comprising a control module and an item recognition module, characterized in that, The item identification system further includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the item identification method for refrigeration equipment according to any one of claims 1-11.

14. A refrigeration device, the refrigeration device comprising an item recognition system, the item recognition system comprising a control module and an item recognition module, characterized in that, The item identification system further includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the item identification method for refrigeration equipment according to any one of claims 1-11.

Images