Method and apparatus for detecting fresh produce

Abstract

The application discloses a kind of detection method and device of fresh goods, which comprises the following steps: collecting shelf image, and detecting real fresh basket and price tag on the shelf from the shelf image;Based on the price tag on the shelf, generate virtual fresh basket;According to the position information of the real fresh basket, the position information of the virtual fresh basket is corrected;Based on the real fresh basket and the virtual fresh basket, the fresh goods on the shelf is detected.The application detects the real fresh basket on the shelf from the shelf image, uses virtual fresh basket to assist supplement the possible missed detection, and then detects and divides the fresh goods based on the real fresh basket and the virtual fresh basket, which can improve the detection accuracy of fresh goods, so that the detection algorithm can be better applied in actual scene, has certain detection algorithm fitting degree, improves the algorithm universality, and has the possibility of large-scale use.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for detecting fresh produce. Background Technology

[0002] With the deep penetration and development of smart retail in supermarkets and convenience stores, digital shelves, as a key component of smart retail, primarily function to intelligently detect out-of-stock status, intelligently analyze sales data of shelf products, and intelligently optimize product display. In supermarket settings, standard product shelves can utilize mature target detection algorithms to detect each product, allowing for further display and tracking / analysis of out-of-stock status.

[0003] To date, deep learning algorithms based on massive amounts of data have significant importance and high accuracy in the field of visual algorithms. By collecting as much relevant scene data as possible manually, existing mature image segmentation and detection algorithms can be used to directly detect goods.

[0004] However, compared to standard goods, the goods in the fresh produce area are complex and diverse, and are characterized by scattered placement, flexible stacking, and rapid consumption within a specific time period. It is impossible to effectively use detection algorithms to detect the target goods, resulting in inaccurate overall data. This leads to huge costs incurred in the process of investing a lot of manpower, financial resources, and time, and ultimately has poor robustness. Summary of the Invention

[0005] This invention provides a method and apparatus for detecting fresh produce, thereby addressing the shortcomings of existing technologies in accurately detecting fresh produce.

[0006] This invention provides a method for detecting fresh produce, comprising the following steps:

[0007] Collect images of the shelves and detect the actual fresh produce baskets and price tags on the shelves from the images;

[0008] Based on the price tags on the shelves, a virtual fresh produce basket is generated;

[0009] The position information of the virtual fresh food basket is corrected based on the position information of the real fresh food basket;

[0010] Based on the real fresh produce basket and the virtual fresh produce basket, the fresh produce on the shelf is inspected.

[0011] The present invention also provides a detection device for fresh produce, comprising:

[0012] The data acquisition module is used to acquire images of the shelving.

[0013] The first detection module is used to detect the actual fresh produce baskets and price tags on the shelf from the shelf image;

[0014] The generation module is used to generate virtual fresh produce baskets based on the price tags on the shelf;

[0015] The correction module is used to correct the position information of the virtual fresh food basket based on the position information of the real fresh food basket;

[0016] The second detection module is used to detect fresh produce on the shelf based on the real fresh produce basket and the virtual fresh produce basket.

[0017] This invention detects real fresh produce baskets on shelves from shelf images, uses virtual fresh produce baskets to supplement potential missed detections, and then performs detection and segmentation of fresh produce based on real and virtual fresh produce baskets. This improves the detection accuracy of fresh produce, allows the detection algorithm to be better applied to real-world scenarios, has a certain degree of algorithm fit, enhances the algorithm's universality, and has the potential for large-scale use. Attached Figure Description

[0018] Figure 1 This is a flowchart of a method for detecting fresh goods according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the structure of a fresh produce detection device according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] In fresh produce inspection scenarios, supermarket managers often place fresh produce in baskets or boxes to enhance aesthetics and facilitate management. Therefore, this invention does not directly inspect the goods on the fresh produce shelves. Instead, it first detects each fresh produce basket unit and then segments the goods based on each basket unit. Specifically, each basket or box can be treated as a natural unit to divide the fresh produce shelf, with each unit considered a smaller unit. Then, based on each smaller unit, detection or segmentation algorithms are used to detect the fresh produce within, thereby improving the overall data accuracy. This invention refers to these baskets for holding fresh produce as "fresh produce baskets." When some fresh produce is not categorized in a basket, a virtual smaller unit is generated to supplement it; that is, a virtual fresh produce basket is generated. Furthermore, to distinguish between the two, in some parts of this invention, these baskets or boxes for holding fresh produce are referred to as real fresh produce baskets.

[0022] This invention provides a method for detecting fresh produce, such as... Figure 1 As shown, it includes the following steps:

[0023] Step 101: Acquire shelf images and detect the actual fresh produce baskets and price tags on the shelves from the shelf images.

[0024] In this embodiment, a camera can be used to capture images of the shelf, and the content of the shelf images includes real fresh produce baskets and price tags on the shelf.

[0025] Furthermore, the actual fresh produce baskets on the shelf can be detected from the shelf image using a fresh produce basket detection algorithm, i.e., baskets or boxes used to place fresh produce; the price tags on the shelf can be detected from the shelf image using an electronic price tag detection algorithm; in addition, the edge coordinates of the entire shelf can also be detected from the shelf image.

[0026] Step 102: Generate a virtual fresh produce basket based on the price tags on the shelf.

[0027] Specifically, the center point of the line connecting adjacent price tags in the horizontal direction can be obtained; a virtual fresh produce basket can be generated based on the center point.

[0028] In this embodiment, four center points can be combined into a virtual fresh food basket. The four center points include a first center point, a second center point, a third center point, and a fourth center point. The first center point and the second center point are adjacent in the horizontal direction, the third center point and the fourth center point are adjacent in the horizontal direction, the first center point and the third center point are adjacent in the vertical direction, and the second center point and the fourth center point are adjacent in the vertical direction.

[0029] Alternatively, two center points and the edge of the shelf can be combined to form a virtual fresh produce basket, wherein the two center points include a fifth center point and a sixth center point, and the fifth center point and the sixth center point are adjacent to each other in the horizontal or vertical direction.

[0030] Step 103: Correct the position information of the virtual fresh produce basket based on the position information of the real fresh produce basket.

[0031] Specifically, the first dataset and the second dataset can be iterated in a loop to determine whether the center point of the i-th fresh food basket in the first dataset is in the j-th virtual fresh food basket in the second dataset; the first dataset includes all real fresh food baskets detected from the shelf image, and the second dataset includes all virtual fresh food baskets; i and j are both integers greater than or equal to 1;

[0032] If the center point of the i-th fresh food basket is in the j-th virtual fresh food basket, then the overlapping area is removed from the j-th virtual fresh food basket. The overlapping area is the region that overlaps between the i-th fresh food basket and the j-th virtual fresh food basket.

[0033] If the center point of all fresh produce baskets in the first dataset is not in the j-th virtual fresh produce basket, then the j-th virtual fresh produce basket is added to the first dataset.

[0034] Step 104: Based on the real fresh produce basket and the virtual fresh produce basket, inspect the fresh produce on the shelf.

[0035] Specifically, fresh produce on the shelves can be inspected based on all the fresh produce crates in the first dataset.

[0036] By performing the steps described above in this embodiment, it can be determined that the first dataset contains all real fresh food baskets detected from the shelf images and the generated virtual fresh food baskets. Based on all the fresh food baskets, the goods on the shelf can be detected and segmented one by one, thereby realizing the detection of all goods on the fresh food shelf.

[0037] This invention detects real fresh produce baskets on shelves from shelf images, uses virtual fresh produce baskets to supplement any possible missed detections, and then performs detection and segmentation of fresh produce based on the real and virtual fresh produce baskets. This improves the detection accuracy of fresh produce, allows the detection algorithm to be better applied to real-world scenarios, has a certain degree of algorithm suitability, enhances the algorithm's universality, and has the potential for large-scale use.

[0038] like Figure 2 The diagram shown is a structural schematic of a fresh produce detection device according to an embodiment of the present invention, comprising:

[0039] The acquisition module 210 is used to acquire images of the shelf.

[0040] The first detection module 220 is used to detect the actual fresh food baskets and price tags on the shelf from the shelf image.

[0041] The generation module 230 is used to generate a virtual fresh produce basket based on the price tags on the shelf.

[0042] Specifically, the generation module 230 includes:

[0043] The `get` submodule is used to obtain the center point of the line connecting adjacent price tags in the horizontal direction;

[0044] A generation submodule is used to generate a virtual fresh produce basket based on the center point.

[0045] In this embodiment, the generation submodule is specifically used to combine four center points into a virtual fresh food basket. The four center points include a first center point, a second center point, a third center point, and a fourth center point. The first center point and the second center point are adjacent in the horizontal direction, the third center point and the fourth center point are adjacent in the horizontal direction, the first center point and the third center point are adjacent in the vertical direction, and the second center point and the fourth center point are adjacent in the vertical direction.

[0046] or,

[0047] Two center points and the edge of the shelf are combined to form a virtual fresh food basket. The two center points include a fifth center point and a sixth center point, which are adjacent to each other in the horizontal or vertical direction.

[0048] The correction module 240 is used to correct the position information of the virtual fresh food basket based on the position information of the real fresh food basket.

[0049] Specifically, the correction module 240 is used to iterate through the first dataset and the second dataset, and determine whether the center point of the i-th fresh food basket in the first dataset is in the j-th virtual fresh food basket in the second dataset; the first dataset includes all real fresh food baskets detected from the shelf image, and the second dataset includes all virtual fresh food baskets; i and j are both integers greater than or equal to 1;

[0050] If the center point of the i-th fresh food basket is in the j-th virtual fresh food basket, then the overlapping area is removed from the j-th virtual fresh food basket. The overlapping area is the region that overlaps between the i-th fresh food basket and the j-th virtual fresh food basket.

[0051] If the center point of all fresh produce baskets in the first dataset is not in the j-th virtual fresh produce basket, then the j-th virtual fresh produce basket is added to the first dataset.

[0052] The second detection module 250 is used to detect fresh goods on the shelf based on the real fresh food basket and the virtual fresh food basket.

[0053] Specifically, the second detection module 250 is used to detect fresh produce on the shelf based on all the fresh produce baskets in the first dataset.

[0054] This invention detects real fresh produce baskets on shelves from shelf images, uses virtual fresh produce baskets to supplement any possible missed detections, and then performs detection and segmentation of fresh produce based on the real and virtual fresh produce baskets. This improves the detection accuracy of fresh produce, allows the detection algorithm to be better applied to real-world scenarios, has a certain degree of algorithm suitability, enhances the algorithm's universality, and has the potential for large-scale use.

[0055] The steps in the methods described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0056] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting fresh produce, characterized in that, Includes the following steps: Collect images of the shelves and detect the actual fresh produce baskets and price tags on the shelves from the images; Based on the price tags on the shelves, a virtual fresh produce basket is generated; The position information of the virtual fresh food basket is corrected based on the position information of the real fresh food basket; Based on the real fresh produce basket and the virtual fresh produce basket, the fresh produce on the shelf is inspected; This allows for the detection of all goods on the fresh produce shelf.

2. The method of claim 1, wherein, The process of generating a virtual fresh produce basket based on the price tags on the shelf specifically includes: Find the center point of the line connecting adjacent price tags in the horizontal direction; A virtual fresh produce basket is generated based on the center point.

3. The method of claim 2, wherein, The step of generating a virtual fresh produce basket based on the center point specifically includes: Four center points are combined to form a virtual fresh food basket. The four center points include a first center point, a second center point, a third center point, and a fourth center point. The first center point and the second center point are adjacent in the horizontal direction, the third center point and the fourth center point are adjacent in the horizontal direction, the first center point and the third center point are adjacent in the vertical direction, and the second center point and the fourth center point are adjacent in the vertical direction. or, Two center points and the edge of the shelf are combined to form a virtual fresh food basket. The two center points include a fifth center point and a sixth center point, which are adjacent to each other in the horizontal or vertical direction.

4. The method of claim 1, wherein, The step of correcting the position information of the virtual fresh produce basket based on the position information of the real fresh produce basket specifically includes: Iterate through the first dataset and the second dataset, and determine whether the center point of the i-th fresh food basket in the first dataset is in the j-th virtual fresh food basket in the second dataset; the first dataset includes all real fresh food baskets detected from the shelf image, and the second dataset includes all virtual fresh food baskets; i and j are both integers greater than or equal to 1; If the center point of the i-th fresh food basket is in the j-th virtual fresh food basket, then the overlapping area is removed from the j-th virtual fresh food basket. The overlapping area is the region that overlaps between the i-th fresh food basket and the j-th virtual fresh food basket. If the center point of all fresh produce baskets in the first dataset is not in the j-th virtual fresh produce basket, then the j-th virtual fresh produce basket is added to the first dataset.

5. The method as described in claim 4, characterized in that, The process of detecting fresh produce on the shelf based on the real and virtual fresh produce baskets specifically includes: Based on all the fresh produce crates in the first dataset, the fresh produce on the shelves is inspected.

6. A device for detecting fresh produce, characterized in that include: The data acquisition module is used to acquire images of the shelving. The first detection module is used to detect the actual fresh produce baskets and price tags on the shelf from the shelf image; The generation module is used to generate virtual fresh produce baskets based on the price tags on the shelf; The correction module is used to correct the position information of the virtual fresh food basket based on the position information of the real fresh food basket; The second detection module is used to detect fresh goods on the shelf based on the real fresh food basket and the virtual fresh food basket; thereby realizing the detection of all goods on the fresh food shelf.

7. The apparatus of claim 6, wherein, The generation module includes: The `get` submodule is used to obtain the center point of the line connecting adjacent price tags in the horizontal direction; A generation submodule is used to generate a virtual fresh produce basket based on the center point.

8. The apparatus as claimed in claim 7, characterized in that, The generation submodule is specifically used to combine four center points into a virtual fresh food basket. The four center points include a first center point, a second center point, a third center point, and a fourth center point. The first center point and the second center point are adjacent in the horizontal direction, the third center point and the fourth center point are adjacent in the horizontal direction, the first center point and the third center point are adjacent in the vertical direction, and the second center point and the fourth center point are adjacent in the vertical direction. or, Two center points and the edge of the shelf are combined to form a virtual fresh food basket. The two center points include a fifth center point and a sixth center point, which are adjacent to each other in the horizontal or vertical direction.

9. The apparatus as claimed in claim 6, characterized in that, The correction module is specifically used to iterate through the first dataset and the second dataset, and determine whether the center point of the i-th fresh food basket in the first dataset is in the j-th virtual fresh food basket in the second dataset; the first dataset includes all real fresh food baskets detected from the shelf image, and the second dataset includes all virtual fresh food baskets; i and j are both integers greater than or equal to 1; If the center point of the i-th fresh food basket is in the j-th virtual fresh food basket, then the overlapping area is removed from the j-th virtual fresh food basket. The overlapping area is the region that overlaps between the i-th fresh food basket and the j-th virtual fresh food basket. If the center point of all fresh produce baskets in the first dataset is not in the j-th virtual fresh produce basket, then the j-th virtual fresh produce basket is added to the first dataset.

10. The apparatus as claimed in claim 9, characterized in that, The second detection module is specifically used to detect fresh produce on the shelf based on all the fresh produce crates in the first dataset.

Images