Article identification method, storage medium, and refrigeration apparatus

Abstract

The application provides an article identification method, a storage medium and a refrigeration equipment. The article identification method comprises the following steps: turning on a first light source, controlling a camera to capture a first image of an article stored in a storage space; performing target detection on the first image and generating a corresponding identification frame; obtaining a first identification frame and a second identification frame which are mutually overlapped, the second identification frame being located in the first identification frame, and marking a frame selection area of the second identification frame as a to-be-confirmed area; turning off the first light source, turning on a second light source, and controlling the camera to capture a second image of the article stored in the storage space, the installation surfaces of the first light source and the second light source being different; judging whether a target in the to-be-confirmed area produces a projection under irradiation of the second light source according to the first image and the second image; and if not, deleting the second identification frame. In this way, the article pattern printed on the article package can be avoided from being identified as a real existing article, and the accuracy of the article identification result is improved.

Description

Technical Field

[0001] This invention relates to the field of household appliances, and more particularly to an item identification method, storage medium, and refrigeration equipment. Background Technology

[0002] With the advancement of technology, users have increasingly higher requirements for refrigeration equipment, making intelligent transformation a new research and development direction for refrigeration equipment. Refrigeration equipment, such as refrigerators, is often used to store items. In order to intelligently manage the items stored in refrigeration equipment, cameras are often installed inside. Refrigeration equipment can capture images of the storage space through cameras, and then obtain information about the items stored inside by recognizing the images captured by the camera. In existing technologies, item recognition methods generally perform target detection on the images captured by the camera, generating detection boxes that correspond one-to-one with the targets in the image. By recognizing each detection box, all item information contained in the image can be obtained. However, this design has the following drawback: when the outer packaging of an item has a clear image of the item printed on it, it may cause the target detection process to mistakenly identify the image of the item printed on the outer packaging as the item, generating detection boxes and ultimately leading to inaccurate item recognition results. Summary of the Invention

[0003] The purpose of this invention is to provide an item identification method, storage medium, and cooling device. By setting a first light source and a second light source with different mounting surfaces, the existence of a real item in the area to be identified is determined based on the projection of the target in the area to be identified under the light source. This can avoid identifying the item pattern printed on the item packaging as a real item and improve the accuracy of the item identification results.

[0004] To achieve the above-mentioned objective, one embodiment of the present invention provides an item identification method, comprising:

[0005] Turn on the first light source and control the camera to capture the first image of the items stored in the storage space;

[0006] Target detection is performed on the first image, and a corresponding recognition box is generated at the maximum contour of each target, wherein the target has the appearance features of the object to be identified;

[0007] Obtain a first and a second recognition box that overlap with the selected areas, with the second recognition box located within the first recognition box, and mark the selected area of ​​the second recognition box as the area to be confirmed;

[0008] The first light source is turned off, the second light source is turned on, and the camera is controlled to capture a second image of the items stored in the storage space. The mounting surfaces of the first light source and the second light source are opposite.

[0009] judging whether the target in the to-be-confirmed area produces a projection under the irradiation of the second light source according to the first image and the second image;

[0010] If not, the second recognition frame is deleted.

[0011] As a further improvement of an embodiment of the present application, the article recognition method, wherein the step of judging whether the target in the to-be-confirmed area produces a projection under the irradiation of the second light source according to the first image and the second image comprises:

[0012] judging whether there is an illumination area illuminated by the second light source in the to-be-confirmed area according to the first image and the second image;

[0013] If yes, a projection comparison area formed by extending the to-be-confirmed area away from the second light source is obtained;

[0014] judging whether there is a projection area extending from outside the to-be-confirmed area to the to-be-confirmed area in the projection comparison area according to the first image and the second image;

[0015] If yes, it is determined that the target in the to-be-confirmed area produces a projection under the irradiation of the second light source.

[0016] As a further improvement of an embodiment of the present application, the article recognition method, wherein the step of judging whether there is an illumination area illuminated by the second light source in the to-be-confirmed area according to the first image and the second image comprises:

[0017] obtaining a first brightness parameter of each pixel point of the to-be-confirmed area in the first image;

[0018] obtaining a second brightness parameter of each pixel point of the to-be-confirmed area in the second image;

[0019] judging whether there is a first pixel point in the second image, in which the brightness change of the second brightness parameter relative to the first brightness parameter is within a first preset range, and the number of the first pixel points is greater than or equal to a first preset value;

[0020] If yes, it is determined that there is an illumination area illuminated by the second light source in the to-be-confirmed area.

[0021] As a further improvement of an embodiment of the present application, the article recognition method, wherein,

[0022] the step of obtaining a projection comparison area formed by extending the to-be-confirmed area away from the second light source comprises:

[0023] obtaining a first area formed by extending the to-be-confirmed area to the periphery in the second image;

[0024] a first line extending from the center point of the to-be-confirmed region to a direction away from the second light source in the second image is obtained;

[0025] a second line and a third line parallel to the first line and respectively located on two sides of the first line are obtained;

[0026] a region surrounded by the second line, the third line, a boundary line of the first region and a boundary line of the to-be-confirmed region is the projection comparison region.

[0027] As a further improvement of an embodiment of the present application, the article recognition method, wherein,

[0028] the second line passes through a first point which is a boundary point of the to-be-confirmed region farthest from the first line on a side where the second line is located;

[0029] the third line passes through a second point which is a boundary point of the to-be-confirmed region farthest from the first line on a side where the third line is located.

[0030] As a further improvement of an embodiment of the present application, the article recognition method, wherein,

[0031] the step of judging whether there is a projection region extending from outside the to-be-confirmed region to the to-be-confirmed region in the projection comparison region according to the first image and the second image comprises:

[0032] a third brightness parameter of each pixel point of the projection comparison region in the first image is obtained;

[0033] a fourth brightness parameter of each pixel point of the projection comparison region in the second image is obtained;

[0034] it is judged whether there are a plurality of second pixel points in the second image whose fourth brightness parameter has a brightness increase relative to the third brightness parameter within a second preset range;

[0035] if yes, and there are continuous second pixel points extending from outside the to-be-confirmed region to the to-be-confirmed region, it is determined that there is a projection region extending from outside the to-be-confirmed region to the to-be-confirmed region in the projection comparison region.

[0036] As a further improvement of one embodiment of the present invention, in the item recognition method, the first light source is a surface light source, the second light source is a point light source, the storage space includes a bearing surface for bearing items, a top surface opposite to the bearing surface, and a side surface connecting the bearing surface and the top surface, the first light source and the camera are disposed on the top surface or on the same side surface, and the second light source is disposed on the top surface or the side surface where the first light source is not located.

[0037] As a further improvement to one embodiment of the present invention, in the item recognition method, the second light source has a plurality of components, and different second light sources are positioned at different angles relative to the items in the storage space. The method further includes:

[0038] Turn on the second light source one by one and acquire the corresponding second image;

[0039] Each time a second image is acquired, it is determined whether the target in the area to be confirmed is projected under the current second light source illumination based on the first image and the current second image.

[0040] If so, then stop turning on the other second light sources;

[0041] If not, then turn off the current second light source and turn on the next second light source;

[0042] If the target in the area to be confirmed does not produce a projection under the illumination of all the second light sources, then the second recognition frame is deleted.

[0043] As a further improvement to one embodiment of the present invention, the item identification method further includes:

[0044] If the target in the area to be confirmed in the second image casts a projection under the illumination of the second light source;

[0045] Then, the selected area of ​​the first recognition box in the first image is identified and a first recognition result is obtained; the area to be confirmed in the first image is identified and a second recognition result is obtained.

[0046] Determine whether the first identification result and the second identification result belong to different components of the same item;

[0047] If so, delete the second recognition box.

[0048] As a further improvement to one embodiment of the present invention, the item identification method further includes:

[0049] If the first identification result and the second identification result do not belong to different components of the same item, then the first identification result and the second identification result are output respectively.

[0050] As a further improvement of an embodiment of the present application, the article recognition method, wherein, further comprising:

[0051] If the first recognition result is a bottle or a bag, and the second recognition result is a cover, it is determined that the first recognition result and the second recognition result are different component structures of an article, and the second recognition frame is deleted.

[0052] As a further improvement of an embodiment of the present application, the article recognition method, wherein, the first preset range is within 20%, the second preset range is greater than 30%, and the boundary length of the first area is three times the corresponding boundary length of the to-be-confirmed area.

[0053] To achieve the above-mentioned purposes, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to realize the steps of the article recognition method in any of the above-mentioned embodiments.

[0054] To achieve the above-mentioned purposes, an embodiment of the present application provides a refrigeration device, which comprises a storage space and a camera arranged in the storage space, wherein the storage space is provided with a first light source and a second light source, and the refrigeration device comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor realizes the steps of the article recognition method in any of the above-mentioned embodiments when executing the computer program.

[0055] As a further improvement of an embodiment of the present application, the refrigeration device, wherein the camera and the first light source are arranged on the top wall of the storage space, and the second light source has a plurality of second light sources and is arranged on the front side wall, and / or the rear side wall, and / or the left side wall, and / or the right side wall of the storage space, respectively, and is located in the upper middle of the storage space.

[0056] Compared with the prior art, the first light source and the second light source with different installation surfaces are arranged, and whether there is a real article in the to-be-confirmed area is determined according to the projection of the target in the to-be-confirmed area under the light source, which has the beneficial effects that it can avoid identifying the printed article pattern on the article packaging as a real existing article, and improve the accuracy of the article recognition result. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a structural schematic diagram of a refrigeration device of an embodiment of the present application;

[0058] Figure 2 is a structural schematic diagram of a storage space of an embodiment of the present application;

[0059] Figure 3 is a flow chart of an article identification method according to an embodiment of the present application;

[0060] Figure 4 is a schematic view of a first image according to an embodiment of the present application;

[0061] Figure 5 is a schematic view of a second image 12 according to an embodiment of the present application;

[0062] Figure 6 is a schematic view of a third image according to an embodiment of the present application; Figure 4 is a schematic view of a second kind of mutually overlapping bounding boxes;

[0063] Figure 7 is a schematic view of a third kind of mutually overlapping bounding boxes. Figure 4 DETAILED DESCRIPTION The present application will be described in detail below with reference to specific embodiments shown in the drawings. However, these embodiments do not limit the present application, and any changes in structure, method, or function made by those of ordinary skill in the art based on these embodiments are included in the scope of the present application.

[0064] Referring to

[0065] and Figure 1 , in an embodiment of the present application, a refrigeration device 1, such as a refrigerator, a freezer, a commercial display cabinet, or the like, is provided. The refrigeration device 1 can include a storage space 5 and a camera 4 disposed in the storage space 5. The refrigeration device 1 can include a cabinet 2 and a door 3 pivotably disposed on the cabinet 2. The storage space 5 can be used to store articles. The storage space 5 can be formed inside the cabinet 2 or inside the door 3. Figure 2 The storage space 5 can be provided with the camera 4. The camera 4 can be used to capture images of articles stored in the storage space 5. The refrigeration device 1 can obtain article information of articles stored in the storage space 5 by identifying images captured by the camera 4, so as to manage the articles stored in the storage space 5, such as informing a user of which articles are stored in the storage space 5.

[0066] The storage space 5 can be provided with a light source for illuminating articles stored in the storage space 5. The camera 4 can capture images of the articles when the light source illuminates the articles, and transmit the captured images to the refrigeration device 1 so that the refrigeration device 1 identifies the images. The light source can be an LED lamp or the like.

[0067] Referring to

[0068] Figure 2 ​Further, in the embodiment, the storage space 5 can be provided with a first light source 6 and a second light source 7. The first light source 6 and the second light source 7 can be arranged at different positions in the storage space 5 to illuminate the items in the storage space 5 from different angles. The first light source 6 and the second light source 7 can be independently turned on and off, and the camera 4 can take photos when different light sources are turned on, so as to obtain different images of the items under the illumination of different light sources.

[0069] With reference to Figures 2-5 Further, in the embodiment, an item identification method is provided, which can include:

[0070] Turning on the first light source 6, and controlling the camera 4 to take a first image 8 of the items stored in the storage space 5;

[0071] Performing target detection on the first image 8, and generating a corresponding identification frame at the maximum contour of each target, the target having an appearance feature of the item to be identified;

[0072] Obtaining a first identification frame 9 and a second identification frame 10 that overlap with each other, the second identification frame 10 being located in the first identification frame 9, and marking the frame selection area of the second identification frame 10 as a to-be-confirmed area 11;

[0073] Turning off the first light source 6 and turning on the second light source 7, and controlling the camera 4 to take a second image 12 of the items stored in the storage space 5, the mounting surfaces of the first light source 6 and the second light source 7 being different;

[0074] Judging whether a target in the to-be-confirmed area 11 produces a projection under the illumination of the second light source 7 according to the first image 8 and the second image 12;

[0075] If not, deleting the second identification frame 10.

[0076] In the embodiment, the method of performing target detection on an image and generating a corresponding identification frame can refer to an existing image target detection algorithm. The method of identifying the image in the identification frame to obtain the item framed by the identification frame can refer to an existing image classification and identification algorithm.

[0077] The item to be identified can be an item that needs to be identified by the refrigeration equipment 1 according to the image of the item so as to be managed.

[0078] The staff can use the image classification and identification algorithm to identify and train the refrigeration equipment 1, so that the refrigeration equipment 1 can identify the item to be identified from the image. The item to be identified corresponding to refrigeration equipment 1 of different purposes and different types can be different. For example, when the refrigeration equipment 1 is a refrigerator, the item to be identified can include various food materials, such as fruits, vegetables, meat, dairy products, beverages, etc.

[0079] Correspondingly, the refrigeration equipment 1 should detect the target with the appearance feature of the to-be-identified article as the target to be further identified when performing target detection on the image, and does not need to identify the region without the appearance feature of the to-be-identified article as the target even if there is an article. Further, the appearance feature of the to-be-identified article can include the shape, color, texture and the like of the article.

[0080] For example, the image captured by the camera 4 of the refrigerator has an apple and a shelf carrying the apple. Since the apple belongs to the to-be-identified article of the refrigerator, the image thereof has the appearance feature of the apple, so that the apple in the image can be regarded as the target and the identification frame completely covering the image of the apple can be generated at the largest contour of the apple when performing target detection. Since the shelf does not belong to the to-be-identified article of the refrigerator and does not have the appearance feature of the to-be-identified article, it is not regarded as the target when performing target detection.

[0081] Since the outer package of many to-be-identified articles can have a clear image of the article printed thereon. For example, the package of a bagged chicken powder used for making fried chicken can have a fried chicken image printed thereon. When performing target detection on the image containing the bagged chicken powder, the refrigeration equipment 1 not only generates the identification frame of the bagged chicken powder as the target, but also generates the identification frame of the fried chicken image printed on the bagged chicken powder as the target.

[0082] If the refrigeration equipment 1 identifies both of the identification frames, two identification results are obtained, one is the chicken powder and the other is the fried chicken. This can cause the refrigeration equipment 1 to regard the fried chicken which actually does not exist as the real food material existing in the storage space 5 for management, and further cause the refrigeration equipment 1 to store incorrect article information and provide incorrect article information to the user and the like.

[0083] In view of this situation, the projection generated by the article under the irradiation of the light source can be used to distinguish whether the target in the identification frame is the article or the printed image. If the target in the identification frame is only the printed image, the projection cannot be generated obviously, and if the target in the identification frame is the real article, the corresponding projection can be generated under the irradiation of the light source.

[0084] In the embodiment, the first image 8 of the article in the storage space 5 can be obtained by turning on the first light source 6, and all the identification frames corresponding to the to-be-identified articles possibly existing in the storage space 5 can be obtained by performing target detection on the first image 8.

[0085] In order to accurately detect all the to-be-identified articles possibly existing in the first image 8, the first light source 6 preferably uniformly irradiates all the to-be-identified articles in the first image 8, so that the images of all the to-be-identified articles in the first image 8 are as clear as possible, thereby improving the effect of image target detection.

[0086] Referring to Figure 2 and Figure 4 Further, in the embodiment, the first light source 6 can be a surface light source. For example, the entire mounting surface where the first light source 6 is located can be made into a light emitting structure, or a plurality of surface light emitting areas can be arranged on the mounting surface where the first light source 6 is located, so as to uniformly illuminate the to-be-identified articles placed in the storage space 5, and meanwhile, as few projections as possible or almost no projection can exist in the first image 8, so as to avoid the existence of large area light and dark areas in the first image 8 due to the height difference between adjacent to-be-identified articles and the like, and affect the target detection effect.

[0087] After the target detection is completed through the first image 8, since the identification frame corresponding to the printed image on the outer package of the to-be-identified article should be in the identification frame corresponding to the to-be-identified article, the frame selection area of the second identification frame 10 located in the first identification frame 9 can be marked as a to-be-confirmed area 11 by obtaining the first identification frame 9 and the second identification frame 10 which are mutually overlapped.

[0088] Referring to Figure 5 In the embodiment, after the to-be-confirmed area 11 is determined, the second light source 7 can be turned on and the second image 12 can be obtained. The second light source 7 can be located on a different mounting surface from the first light source 6, so that the second light source 7 can illuminate the to-be-confirmed area 11 from a different angle from the first light source 6.

[0089] Further, in the embodiment, the second light source 7 can be a point light source. For example, a plurality of small point light emitting bodies such as bulbs and the like with large light emitting intensity can be arranged on the side of the storage space 5 other than the side where the first light source 6 is located. If the target of the to-be-confirmed area 11 is a real existing article, a projection can be generated under the irradiation of the second light source 7 and the projection can be obviously embodied in the second image 12.

[0090] After the second image 12 is obtained, whether the to-be-confirmed area 11 generates a projection in the second image 12 can be determined according to the characteristics of the article projection. For example, the color of the article projection is generally black and gray, and the projection area of the article is darker than the area irradiated by the light source.

[0091] If it is known from the obtained first image 8 and the second image 12 that the to-be-confirmed area 11 does not generate a projection under the irradiation of the second light source 7, it is indicated that the target in the to-be-confirmed area 11 is a printed article image, and the second identification frame 10 does not need to be identified and can be directly deleted.

[0092] In this way, it can be avoided that the printed article pattern on the article package is identified as a real existing article, and the accuracy of the article identification result is improved.

[0093] Referring to Figure 2 and Figure 4 Further, in another embodiment of the present application, the storage space 5 can include a carrying surface for carrying articles, a top surface arranged opposite to the carrying surface, and a side surface connected between the carrying surface and the top surface, the first light source 6 and the camera 4 can be arranged on the top surface or on the same side surface, and the second light source 7 can be arranged on the top surface or the side surface other than the first light source 6.

[0094] Further, in the present embodiment, the camera 4 and the first light source 6 can be arranged on the top wall of the storage space 5, and the second light source 7 can be arranged on the front side wall, and / or the rear side wall, and / or the left side wall, and / or the right side wall of the storage space 5, and the second light source 7 can be arranged in the upper middle of the storage space 5.

[0095] In this way, the first light source 6 can uniformly irradiate all the articles to be identified in the first image 8, and at the same time, if the target in the to-be-confirmed area 11 is a real article, a projection can be generated under the irradiation of the second light source 7 and the projection can be clearly shown in the second image 12, and in addition, the camera 4 can be prevented from being backlit, thereby ensuring the image shooting effect of the camera 4.

[0096] Further, in the present embodiment, different second light sources 7 are arranged at different angles relative to the articles in the storage space 5, and the control method can include:

[0097] opening the second light sources 7 one by one and obtaining the corresponding second images 12;

[0098] obtaining one second image 12 at a time, and determining whether the target in the to-be-confirmed area 11 generates a projection under the irradiation of the current second light source 7 according to the first image 8 and the current second image 12;

[0099] if yes, stopping opening other second light sources 7;

[0100] if no, closing the current second light source 7 and opening the next second light source 7;

[0101] if the target in the to-be-confirmed area 11 does not generate a projection under the irradiation of all the second light sources 7, deleting the second identification frame 10.

[0102] Since the second light source 7 is fixedly arranged, if the angle and position of the second light source 7 are not appropriate, it can cause a judgment error in the case that the target in the to-be-confirmed area 11 is a real article and the generated projection is not obvious, resulting in the second identification frame 10 being deleted.

[0103] In order to improve the accuracy and reliability of the judgment result, a plurality of second light sources 7 can be arranged at different positions of the storage space 5, and the second light sources 7 are sequentially turned on to determine whether the projection is generated in the to-be-confirmed area 11. If no projection is generated in the to-be-confirmed area 11 under any second light source 7 after all the second light sources 7 are turned on, it is indicated that the to-be-confirmed area 11 is only a printed image of the article, and the second identification frame 10 does not need to be identified, and can be directly deleted.

[0104] In this way, it can be more accurately determined whether the to-be-confirmed area 11 is a printed image of the article, the accuracy and reliability of the judgment result are ensured, misjudgment is avoided, and the accuracy of the article identification result is improved.

[0105] Reference Figure 5 Further, in another embodiment of the present application, the article identification method, wherein "judging whether the target in the to-be-confirmed area 11 generates a projection under the irradiation of the second light source 7 according to the first image 8 and the second image 12" can include:

[0106] judging whether there is an illumination area 13 illuminated by the second light source 7 in the to-be-confirmed area 11 according to the first image 8 and the second image 12;

[0107] If yes, a projection comparison area 14 formed by the to-be-confirmed area 11 extending away from the second light source 7 is obtained;

[0108] judging whether there is a projection area extending from outside the to-be-confirmed area 11 to the to-be-confirmed area 11 in the projection comparison area 14 according to the first image 8 and the second image 12;

[0109] If yes, it is determined that the target in the to-be-confirmed area 11 generates a projection under the irradiation of the second light source 7.

[0110] Since other articles can be placed around the to-be-confirmed area 11, the projection of the articles around the to-be-confirmed area 11 can fall on the to-be-confirmed area 11 or around the to-be-confirmed area 11 under the irradiation of the second light source 7, which causes that the to-be-confirmed area 11 cannot be accurately determined whether the projection is generated, and even the articles around the to-be-confirmed area 11 shield the to-be-confirmed area 11, which causes that the second light source 7 cannot irradiate the to-be-confirmed area 11.

[0111] Thus in the embodiment, it can be determined by the first image 8 and the second image 12 whether there is an illumination area 13 illuminated by the second light source 7 in the to-be-confirmed area 11. The determination method of the illumination area 13 can be determined according to the color, brightness and other characteristics of the to-be-confirmed area 11 under the illumination of the first light source 6, for example, if the to-be-confirmed area 11 is covered by a shadow or not illuminated by the second light source 7, the color of the picture presented in the second image 12 will be gray-black, significantly darkened, and so on compared with the first image 8.

[0112] When it is confirmed that there is an illumination area 13 in the to-be-confirmed area 11, it means that the to-be-confirmed area 11 can be illuminated by the second light source 7, and accordingly, if the target in the to-be-confirmed area is a real existing object, a projection should be generated.

[0113] Since the object projection is on the backlight surface of the object, and the size of the object projection is related to the size of the object, it can be determined whether there is a corresponding projection in a specific range away from the second light source 7 outside the to-be-confirmed area 11, that is, a projection comparison area 14 formed by the to-be-confirmed area 11 extending away from the second light source 7. The projection comparison area 14 can be a region for confirming whether there is a projection of the to-be-confirmed area 11.

[0114] The projection area can refer to a coherent area formed by the projection of the object. Since the object projection area is connected to the object and continuously extends away from the light source, if there is a projection area continuously extending from outside the to-be-confirmed area 11 to the to-be-confirmed area 11 in the projection comparison area 14, it means that the target in the to-be-confirmed area 11 produces a projection under the illumination of the second light source 7.

[0115] In this way, it can be accurately determined whether the target in the to-be-confirmed area 11 produces a projection under the illumination of the second light source 7, avoiding misjudgment caused by surrounding objects blocking and other situations, ensuring the accuracy and reliability of the judgment result, and further improving the accuracy of the object recognition result.

[0116] Reference Figure 4 and Figure 5 Further, in another embodiment of the present application, the object recognition method, wherein the method of "determining whether there is an illumination area 13 illuminated by the second light source 7 in the to-be-confirmed area 11 according to the first image 8 and the second image 12" can include:

[0117] obtaining a first brightness parameter of each pixel point of the to-be-confirmed area 11 in the first image 8;

[0118] obtaining a second brightness parameter of each pixel point of the to-be-confirmed area 11 in the second image 12;

[0119] determining whether there are first pixel points in the second image 12 in which the second brightness parameter has a change in darkness relative to the first brightness parameter within a first preset range, and the number of the first pixel points is greater than or equal to a first preset value;

[0120] If yes, it is determined that there is an illumination area 13 in the to-be-confirmed area 11 that is illuminated by the second light source 7.

[0121] An image is composed of a plurality of pixel points, and the brightness parameter can be a gray value of a pixel point. The gray value can represent a parameter value of a color of a pixel point between the darkest black and the brightest white, and the range is generally between 0 and 255, with white being 255 and black being 0.

[0122] After the first image 8 and the second image 12 are acquired, in order to identify the color of each pixel point, the corresponding RGB parameters of each pixel point are generally automatically generated, the RGB parameters are composed of R (red), G (green), and B (blue), and a color is represented by a linear combination of three color components, and any color is related to the three components.

[0123] The RGB parameters of each pixel point can be converted into a gray value by using an existing gray scale conversion algorithm, thereby facilitating comparison of the brightness of the pixel points.

[0124] If there is an area in the to-be-confirmed area 11 that is illuminated by the second light source 7, the brightness of the area in the second image 12 may be less than the brightness of the area in the first image 8, but the range of the change in darkness is limited. If the change in darkness is too much, it indicates that the area is not illuminated by the second light source 7 or there is a projection.

[0125] The first pixel points in the to-be-confirmed area 11 that are illuminated by the second light source 7 are determined by comparing the changes in the brightness parameters of the corresponding pixel points in the first image 8 and the second image 12. If there are the first pixel points, it indicates that there is an area in the to-be-confirmed area 11 that is illuminated by the second light source 7.

[0126] In addition, in order to ensure that the determination result is more accurate, the number of the first pixel points can be preset to be greater than a first preset value. The first preset value can be determined according to the proportion of the total number of pixel points in the to-be-confirmed area 11, or can be a fixed value, such as 20.

[0127] Further, in the embodiment, the first preset range is within 20%. That is, the increase in darkness of the second brightness parameter of a pixel point in the second image 12 relative to the first brightness parameter of the corresponding pixel point in the first image 8 cannot exceed 20%.

[0128] In this way, whether the lighted area 13 illuminated by the second light source 7 exists in the to-be-confirmed area 11 can be accurately determined according to the comparison of the corresponding pixel parameters in the first image 8 and the second image 12, and the method is simple, efficient and accurate.

[0129] With reference to Figure 5 Further, in another embodiment of the present application, the article identification method, wherein,

[0130] The method of obtaining the projection comparison area 14 formed by the to-be-confirmed area 11 extending away from the second light source 7 can include:

[0131] obtaining a first area 15 in the second image 12 formed by the to-be-confirmed area 11 extending to the periphery;

[0132] obtaining a first line 16 in the second image 12 extending away from the second light source 7 from the center point of the to-be-confirmed area 11;

[0133] obtaining a second line 17 and a third line 18 parallel to the first line 16 and respectively located on the two sides of the first line 16;

[0134] The area surrounded by the second line 17, the third line 18, the boundary line of the first area 15 and the boundary line of the to-be-confirmed area 11 is the projection comparison area 14.

[0135] Further, in this embodiment, the boundary length of the first area 15 can be three times the corresponding boundary length of the to-be-confirmed area 11.

[0136] Further, in another embodiment of the present application, the article identification method, wherein,

[0137] The second line 17 passes through a first point, which is the boundary point of the to-be-confirmed area 11 farthest from the first line 16 on the side where the second line 17 is located;

[0138] The third line 18 passes through a second point, which is the boundary point of the to-be-confirmed area 11 farthest from the first line 16 on the side where the third line 18 is located.

[0139] In this way, the area range of the projection comparison area 14 can be quickly and accurately determined, ensuring that the obtained projection comparison area 14 is an area formed by the to-be-confirmed area 11 extending away from the second light source 7, and avoiding that the area range of the projection comparison area 14 is too large or too small to affect the correct determination of the projection area.

[0140] With reference to Figure 4 and Figure 5Further, in another embodiment of the present application, the article identification method, wherein

[0141] The determining whether the projection area exists in the projection comparison area 14 and extends from the to-be-confirmed area 11 to the to-be-confirmed area 11 according to the first image 8 and the second image 12 can include:

[0142] Obtaining a third brightness parameter of each pixel point of the projection comparison area 14 in the first image 8;

[0143] Obtaining a fourth brightness parameter of each pixel point of the projection comparison area 14 in the second image 12;

[0144] Determining whether there are a plurality of second pixel points in the second image 12, in which a brightness increase amount of the fourth brightness parameter relative to the third brightness parameter is within a second preset range;

[0145] If yes, and there are continuous second pixel points extending from the to-be-confirmed area 11 to the to-be-confirmed area 11, it is determined that the projection area exists in the projection comparison area 14 and extends from the to-be-confirmed area 11 to the to-be-confirmed area 11.

[0146] Further, in the present embodiment, the second preset range can be greater than 30%.

[0147] If the projection area exists in the projection comparison area 14, the brightness of the area in the second image 12 should be obviously increased relative to the brightness in the first image 8.

[0148] Therefore, whether the pixel points with a brightness increase amount greater than a certain range, i.e., the second pixel points, exist in the projection comparison area 14 can be determined by comparing the brightness parameters of the corresponding pixel points of the to-be-confirmed area 11 in the first image 8 and the second image 12. If yes, and there are a plurality of continuous second pixel points extending from the to-be-confirmed area 11 to the to-be-confirmed area 11, it is indicated that the projection area exists in the projection comparison area 14 and extends from the to-be-confirmed area 11 to the to-be-confirmed area 11.

[0149] In this way, whether the projection area of the to-be-confirmed area 11 exists in the projection comparison area 14 can be accurately determined according to the comparison of the corresponding pixel point parameters in the first image 8 and the second image 12. The method is simple, efficient, and highly accurate.

[0150] Referring to Figure 2 , Figure 5 and Figure 6 , further, in another embodiment of the present application, the article identification method, wherein

[0151] If the target in the to-be-confirmed area 11 in the second image 12 generates a projection under the irradiation of the second light source 7;

[0152] the first recognition result and the second recognition result are identified as different component structures of one article, and the second recognition frame 10 is deleted.

[0153] the first recognition result and the second recognition result are identified as different component structures of one article, and the second recognition frame 10 is deleted.

[0154] the first recognition result and the second recognition result are identified as different component structures of one article, and the second recognition frame 10 is deleted.

[0155] Since the shapes of articles are various, one article can be composed of two structures with large appearance difference, which further causes multiple recognition frames to be generated on one article. For example, a wine bottle and a bottle cap, since the bottle cap has a relatively independent article appearance, it can be detected as a single target and a corresponding recognition frame is generated. Moreover, since the bottle cap has a certain volume, it can generate a projection under the irradiation of the second light source 7, and thus can be regarded as an independent article recognition and output a recognition result.

[0156] To avoid the above situation from affecting the accuracy of the final output recognition result, after the two recognition frames are respectively identified, it is determined whether the recognition results belong to different component structures of one article. If yes, the second recognition frame 10 is deleted.

[0157] Further, in the embodiment, the article recognition method, wherein

[0158] If the first recognition result is a bottle or a bag, and the second recognition result is a cap, it is determined that the first recognition result and the second recognition result are different component structures of one article, and the second recognition frame 10 is deleted.

[0159] Referring to Figure 2 , Figure 5 and Figure 7 Further, in the embodiment, the article recognition method, wherein if the first recognition result and the second recognition result do not belong to different component structures of one article, the first recognition result and the second recognition result are respectively output. This situation indicates that the recognition frames that overlap with each other frame two independent articles, and only one article is placed on another article, for example, a beverage and an egg placed on the beverage, and thus the recognition results of the two articles can be respectively output.

[0160] In this way, the accuracy and reliability of the recognition result can be improved, multiple recognition results of one article can be avoided, and when articles are stacked, the recognition results of the stacked articles can be respectively output.

[0161] Further, in another embodiment of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the article identification method in any of the above embodiments.

[0162] Further, in another embodiment of the present application, the refrigeration device 1 can include a memory and a processor, the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the article identification method in any of the above embodiments.

[0163] In summary, the article identification method, the storage medium and the refrigeration device 1 in the present application can determine whether there is a real article in the to-be-confirmed area 11 according to the projection of the target under the light source in the to-be-confirmed area 11 by setting the first light source 6 and the second light source 7 with different installation surfaces, solve the problem that in the prior art, when the outer packaging of the article is printed with a clear article image, the article image printed on the outer packaging of the article is mistakenly regarded as the article to generate a detection frame and recognition when the image is target detected, and finally lead to inaccurate article identification results.

[0164] The technical solutions in the present application can avoid identifying the article image printed on the article packaging as a real existing article, can avoid outputting multiple identification results for the same article, can ensure that when the articles are stacked, the stacked articles can be output with identification results respectively, can improve the accuracy and reliability of the article identification results, and further improve the food management effect of the refrigeration device 1.

[0165] It should be understood that although the present specification is described in terms of embodiments, each embodiment does not contain only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be combined appropriately to form other embodiments that those skilled in the art can understand.

[0166] The series of detailed descriptions listed above are only specific descriptions of the feasible embodiments of the present application, and are not intended to limit the protection scope of the present application, and any equivalent embodiments or changes made without departing from the spirit of the present application should be included in the protection scope of the present application.

Claims

1. An article identification method characterized by comprising: The method comprises: turning on a first light source, and controlling a camera to capture a first image of an object stored in a storage space; performing target detection on the first image, and generating a corresponding recognition box at a maximum contour of each target, the target having an appearance feature of an object to be identified; obtaining a first recognition box and a second recognition box that are mutually overlapped, the second recognition box being located in the first recognition box, and marking a box selection area of the second recognition box as a confirmation area; turning off the first light source, turning on a second light source, and controlling the camera to capture a second image of the object stored in the storage space, the first light source being arranged on a top wall of the storage space, the second light source having a plurality of light sources and being arranged on a front side wall, a rear side wall, a left side wall and / or a right side wall of the storage space, and the second light source being located in an upper middle part of the storage space; judging whether a target in the confirmation area produces a projection under illumination of the second light source according to the first image and the second image; if not, deleting the second recognition box.

2. The article identification method of claim 1, wherein, The method of judging whether a target in the confirmation area produces a projection under illumination of the second light source according to the first image and the second image comprises: judging whether there is an illumination area in the confirmation area that is illuminated by the second light source according to the first image and the second image; if so, obtaining a projection comparison area formed by the confirmation area extending away from the second light source; judging whether there is a projection area in the projection comparison area that extends from outside the confirmation area to the confirmation area according to the first image and the second image; and if so, determining that the target in the confirmation area produces a projection under illumination of the second light source.

3. The article identification method of claim 2, wherein, The method of judging whether there is an illumination area in the confirmation area that is illuminated by the second light source according to the first image and the second image comprises: obtaining a first brightness parameter of each pixel point in the confirmation area in the first image; obtaining a second brightness parameter of each pixel point in the confirmation area in the second image; judging whether there is a first pixel point in the second image in which a brightness change of the second brightness parameter relative to the first brightness parameter is within a first preset range, and the number of the first pixel points is greater than or equal to a first preset value; and if so, determining that there is an illumination area in the confirmation area that is illuminated by the second light source.

4. The object identification method of claim 3, wherein the method of obtaining a projection comparison area formed by the confirmation area extending away from the second light source comprises: obtaining a first area formed by the confirmation area extending to the periphery in the second image; obtaining a first line extending away from the second light source from a center point of the confirmation area in the second image; obtaining a second line and a third line that are parallel to the first line and are respectively located on two sides of the first line; and an area surrounded by the second line, the third line, a boundary line of the first area and a boundary line of the confirmation area is the projection comparison area.

5. The object identification method of claim 4, wherein ​ The second line passes through a first point, which is a boundary point of the to-be-confirmed area farthest from the first line on a side where the second line is located; The third line passes through a second point, which is a boundary point of the to-be-confirmed area farthest from the first line on a side where the third line is located. 6.The article identification method of claim 4, wherein The determining whether the projection area extending from outside the to-be-confirmed area to the to-be-confirmed area exists in the projection comparison area according to the first image and the second image comprises: obtaining a third brightness parameter of each pixel point of the projection comparison area in the first image; obtaining a fourth brightness parameter of each pixel point of the projection comparison area in the second image; determining whether there are a plurality of second pixel points in the second image, the fourth brightness parameter of which increases in brightness relative to the third brightness parameter within a second preset range; if yes, and there are continuous second pixel points extending from outside the to-be-confirmed area to the to-be-confirmed area, it is determined that the projection area extending from outside the to-be-confirmed area to the to-be-confirmed area exists in the projection comparison area.

7. The article identification method of claim 1, wherein, The first light source is a surface light source, the second light source is a point light source, the storage space includes a bearing surface for bearing articles, a top wall opposite to the bearing surface, and a side wall connected between the bearing surface and the top wall, the first light source and the camera are arranged on the top wall, the second light source has a plurality of second light sources and is arranged on the front side wall, the rear side wall, the left side wall, and / or the right side wall of the storage space, and the second light source is located in the upper middle of the storage space.

8. The article identification method of claim 1, wherein, The second light source has a plurality of second light sources, different second light sources are at different angles relative to the articles in the storage space, and the method further comprises: turning on the second light sources one by one and obtaining corresponding second images; each time one second image is obtained, it is determined whether the target in the to-be-confirmed area produces a projection under the irradiation of the current second light source according to the first image and the current second image; if yes, the other second light sources are stopped from being turned on; if no, the current second light source is turned off and the next second light source is turned on; if the target in the to-be-confirmed area does not produce a projection under the irradiation of all the second light sources, the second identification frame is deleted.

9. The article identification method of claim 1, wherein, Further comprising: if the target in the to-be-confirmed area produces a projection under the irradiation of the second light source in the second image; the first identification result and the second identification result are identified as belonging to different component structures of one article; if yes, the second identification frame is deleted. Further comprising:

10. The article identification method of claim 9, wherein, if the first identification result and the second identification result do not belong to different component structures of one article, the first identification result and the second identification result are output respectively. Further comprising:

11. The article identification method of claim 9, wherein, if the first identification result is a bottle or a bag, and the second identification result is a cover, it is determined that the first identification result and the second identification result are different component structures of one article, and the second identification frame is deleted. ​ 12. The article identification method of claim 6, wherein, The first preset range is within 20%, the second preset range is greater than 30%, and the boundary length of the first area is three times the corresponding boundary length of the area to be confirmed.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps in the article identification method of any one of claims 1-12.

14. A refrigeration appliance comprising a storage space, a camera disposed within the storage space, wherein, The storage space is provided with a first light source and a second light source, the refrigeration equipment comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements the steps in the article identification method of any one of claims 1-12 when executing the computer program.

15. The refrigeration appliance of claim 14, wherein, The camera and the first light source are arranged on the top wall of the storage space, the second light source has a plurality of second light sources and is arranged on the front side wall, and / or the rear side wall, and / or the left side wall, and / or the right side wall of the storage space, and the second light source is located in the upper middle of the storage space.

Images