A method and apparatus for segmenting a shelf image

Abstract

The application discloses a method and device for segmenting a shelf image, which is applied to a shelf with multiple guide rails, edges of each guide rail are provided with a plug, and the plug is used for preventing a price tag on the guide rail from sliding out of the guide rail. The method comprises the following steps: acquiring a shelf image, detecting multiple shelf guide rail areas from the shelf image; detecting a plug area based on each shelf guide rail area respectively to obtain coordinates of multiple plug areas on the shelf image; fitting a segmentation line according to the coordinates of the multiple plug areas, and segmenting the shelf image based on the segmentation line. The embodiment of the application can avoid small target detection, thereby accurately realizing segmentation of the shelf image.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to a method and apparatus for segmenting shelf images. Background Technology

[0002] With the rapid development of artificial intelligence, smart retail has also developed rapidly. Intelligent digital shelves are a key component of smart retail; however, the key lies in how to manage them meticulously. Small shelf segmentation is a powerful tool for this refined management.

[0003] Small shelf segmentation management enables precise and intelligent classification and management of goods. It not only helps supermarkets achieve fast, accurate, and efficient management of product displays but also facilitates rapid product search and location, improving customer convenience and shopping experience. In summary, small shelf segmentation is an important area of ​​AI technology application in the supermarket industry, helping supermarkets achieve refined management and optimization, improve operational efficiency and customer satisfaction, and ultimately promote the digitalization, automation, and intelligent development of the supermarket industry.

[0004] However, when segmenting small shelves using computer vision, the complex real-world environment necessitates the use of detection markers to segment the shelves. When the markers are too small, it becomes difficult to guarantee the accuracy of small target detection, which in turn affects the precision of target detection. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for segmenting shelf images to address the shortcomings of insufficient accuracy in target detection in the prior art.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows:

[0007] In a first aspect, a method for segmenting shelf images is provided, applicable to a shelf with multiple guide rails, each guide rail having an edge end provided with a plug for preventing price tags on the guide rails from sliding off the guide rails, the method comprising the following steps:

[0008] Acquire a shelf image and detect multiple shelf guide rail areas from the shelf image;

[0009] Based on the detection of the end blockage area in each of the shelf guide rail areas, the coordinates of multiple end blockage areas on the shelf image are obtained;

[0010] A segmentation line is fitted based on the coordinates of the multiple end cap regions, and the shelf image is segmented based on the segmentation line.

[0011] Secondly, an apparatus for segmenting shelf images is provided, applied to a shelf having multiple guide rails, each guide rail having an edge provided with a plug for preventing price tags on the guide rails from sliding off the guide rails, the apparatus comprising:

[0012] The acquisition module is used to acquire a shelf image and detect multiple shelf guide rail areas from the shelf image;

[0013] The detection module is used to detect the end cap area based on each of the shelf guide rail areas, and obtain the coordinates of multiple end cap areas on the shelf image;

[0014] The segmentation module is used to fit a segmentation line based on the coordinates of the multiple end cap regions, and to segment the shelf image based on the segmentation line.

[0015] Thirdly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method for segmenting shelf images.

[0016] Compared with the prior art, this application has the following advantages:

[0017] This application embodiment detects multiple shelf guide rail areas from the shelf image, and detects end cap areas based on each shelf guide rail area to obtain the coordinates of multiple end cap areas on the shelf image; a segmentation line is fitted based on the coordinates of the detected multiple end cap areas, and the shelf image is segmented based on the segmentation line, which can avoid the detection of small targets and thus accurately achieve the segmentation of the shelf image. Attached Figure Description

[0018] Figure 1 This is a flowchart of a method for segmenting a shelf image provided in an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of a device for segmenting shelf images provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] To address the challenges of segmenting small shelves, this application proposes a method for segmenting small shelves based on shelf guide rail and end cap detection. In real-world scenarios, supermarket shelf guide rails typically have end caps at their edges to prevent price tags from sliding off. The proposed small shelf segmentation method includes: First, attaching reflective strips to the end caps. Second, using target detection algorithms from computer vision and artificial intelligence, a detection model is trained based on the target's image and label information using machine learning methods. This model is then used for target detection, and subsequent target detections all employ this method. The acquired shelf guide rail area is then expanded, and reflective strip (end cap) detection is performed in the expanded target area to obtain the segmentation points for the small shelves (end caps exist at both ends of the shelf guide rail; detecting an end cap determines the area of ​​each small shelf). Finally, the small shelves are segmented using these segmentation points. The method proposed in this application not only avoids overly obvious small shelf segmentation markers but also avoids the accuracy issues associated with small target detection, while also meeting the requirements for segmenting small shelves in dimly lit nighttime scenarios with adequate lighting.

[0022] The following description, in conjunction with the accompanying drawings, details a method for segmenting shelf images provided in this application through specific embodiments and application scenarios.

[0023] like Figure 1 The diagram shown is a flowchart of a method for segmenting shelf images according to an embodiment of this application. The method is applied to a shelf with multiple guide rails, each guide rail having an edge end cap to prevent price tags on the guide rails from sliding off. The method includes the following steps:

[0024] Step 101: Obtain a shelf image and detect multiple shelf guide rail areas from the shelf image.

[0025] Step 102: Detect the end blockage area based on each shelf guide rail area to obtain the coordinates of multiple end blockage areas on the shelf image.

[0026] Specifically, after detecting multiple shelf guide rail areas from the shelf image, each shelf guide rail area can be expanded; correspondingly, end cap areas can be detected in each expanded shelf guide rail area.

[0027] In this embodiment, after detecting the end blockage area based on each shelf guide rail area and obtaining the coordinates of multiple end blockage areas on the shelf image, it can also be determined whether there are two end blockage areas with intersecting areas based on the coordinates of the multiple end blockage areas on the shelf image; if so, one of the two end blockage areas is deduplicated.

[0028] In addition, multiple price tags can be set on the guide rail; accordingly, after detecting the end blockage area based on each shelf guide rail area and obtaining the coordinates of multiple end blockage areas on the shelf image, it can also be determined whether each end blockage area intersects with at least one price tag area among the multiple price tag areas based on the coordinates of the multiple end blockage areas on the shelf image and the coordinates of the multiple price tag areas on the shelf image; if at least one end blockage area intersects with at least one price tag area among the multiple price tag areas, then the at least one end blockage area is determined to be a false detection area.

[0029] In this embodiment, each end cap may be provided with a reflective element; correspondingly, the reflective element can be detected based on each of the shelf guide rail areas, and the coordinates of the detected multiple reflective elements on the shelf image are respectively used as the coordinates of multiple end cap areas on the shelf image.

[0030] Step 103: Fit a dividing line based on the coordinates of the multiple end cap regions, and segment the shelf image based on the dividing line.

[0031] Specifically, the multiple end-blocking areas can be grouped according to their coordinates; based on the coordinates of the multiple end-blocking areas in each group, segmentation lines are fitted respectively, and the shelf image is segmented based on the segmentation lines.

[0032] In this embodiment, grouping the multiple blocking regions according to their coordinates specifically includes: obtaining the difference between the x-axis coordinates of each blocking region; if the difference between the x-axis coordinates of two blocking regions is less than or equal to a preset multiple of the width of one of the blocking regions, then the two blocking regions are determined to be blocking regions in the same group; correspondingly, the mean of the x-axis coordinates of the blocking regions in the same group can be calculated, and the position of the dividing line can be determined based on the mean.

[0033] This application embodiment detects end-block areas within the shelf guide rail area, fits segmentation lines based on the coordinates of multiple detected end-block areas, and segments the shelf image based on the segmentation lines. This avoids the detection of small targets, thereby accurately segmenting the shelf image.

[0034] In this embodiment, a small shelf segmentation method based on shelf rail and end cap detection can be implemented using the shelf rails and end caps inherent in the shelf itself, along with reflective strips (the reflective strips are similar in color to the shelf rails) attached to the end caps. The reflective strips are materials that can reflect light back to the light source along its original path with a certain intensity. The reasons for using reflective strips with a color similar to the rails are as follows: 1. Similar colors allow for small shelf segmentation with almost no alteration to the shelf itself; 2. Reflective strips enable small shelf segmentation under nighttime lighting. After determining the position of the reflective strips (end caps), a small shelf segmentation line can be fitted to complete the segmentation.

[0035] Specifically, to prevent interference points (similar to blockages) in the supermarket image and the accuracy impact caused by directly detecting ultra-small targets such as reflective strips (blockages), the acquired supermarket image is first subjected to shelf rail detection using target detection algorithms from the fields of computer vision and artificial intelligence, including single-stage or two-stage detection methods. Subsequently, the acquired shelf rail detection area is expanded to avoid missing reflective strips (blockages), with the expansion on both sides being 1 / 10 the width of the rail detection frame. Reflective strip (blockage) detection is then performed on the expanded rail detection area using the same method as the shelf rail detection. Since the reflective strips (blockages) detected in the rail detection area have coordinates relative to the rail area, small shelf segmentation is required to determine the absolute coordinates of the reflective strips (blockages) in the original supermarket image. By recording the coordinates of the reflective strips (blockages) relative to the rail area and the coordinates of the rails relative to the original image, the coordinates of the reflective strips (blockages) can be returned to the original image. To avoid the expanded guide rail area repeatedly including plugs from other guide rails, the coordinates of the reflective strips (plugs) returned to the supermarket image are deduplicated by judging whether there are intersecting areas between the coordinates returned to the original image.

[0036] Furthermore, the obtained deduplicated reflective strip (end) coordinates are grouped on the x-axis. The grouping method is as follows: if the difference in the x-axis coordinates of the reflective strip (end) is within the width of three reflective strips (ends), it is determined to be a reflective strip (end) within the same group of dividing lines. Thus, the dividing lines are fitted, and the small shelf is finally divided. The fitting method is the mean of the x-axis coordinates of the coordinates within the same group.

[0037] In addition, when the price tag border is mistakenly detected as a reflective strip, the price tag is detected at the same time as the shelf guide rail. The coordinates after deduplication of the reflective strip (end) are filtered with the coordinates of the price tag. The filtering method is to determine whether there is an intersecting area by the coordinates. If there is, it is determined that the price tag edge is a false detection.

[0038] In this embodiment, an image of the supermarket shelves is acquired using an AI camera or an AI inspection robot. First, a shelf guide rail algorithm detects the shelf guide rail area in the shelf image and expands the detected area. Then, reflective strips (end-ends) are detected in the expanded shelf guide rail area, and the coordinates are returned to the supermarket shelf image acquired by the AI ​​camera or AI inspection robot. Finally, by deduplicating and grouping the coordinates of the reflective strips (end-ends), the segmentation lines of the small shelves are fitted, thus completing the segmentation of the small shelves.

[0039] This invention avoids the accuracy issues associated with detecting extremely small targets by first inspecting the shelf guide rails and then inspecting the reflective strips (endcaps) within them. The dividing lines are established by detecting the coordinates of the reflective strips affixed to the endcaps. Relying on the reflective properties of the strips, small shelf divisions can be accurately performed even under low-light conditions at night. Compared to methods that use clearly visible and easily detectable dividing marks, this invention only affixes reflective strips of a similar color to the guide rails to the endcaps, minimizing any impact on the aesthetics of the supermarket and eliminating the need for staff to manage additional dividing marks.

[0040] like Figure 2 As shown, an apparatus for segmenting shelf images is provided in an embodiment of this application. It is applied to a shelf with multiple guide rails, each guide rail having an edge end cap to prevent price tags on the guide rails from sliding off. The apparatus includes:

[0041] The acquisition module 210 is used to acquire a shelf image and detect multiple shelf guide rail areas from the shelf image.

[0042] The detection module 220 is used to detect the end blockage area based on each of the shelf guide rail areas, and obtain the coordinates of multiple end blockage areas on the shelf image.

[0043] Each of the plugs is provided with a reflective element;

[0044] Accordingly, the detection module 220 is specifically used to detect reflective elements based on each of the shelf guide rail areas, and to use the coordinates of the detected multiple reflective elements on the shelf image as the coordinates of multiple end cap areas on the shelf image.

[0045] The segmentation module 230 is used to fit a segmentation line based on the coordinates of the multiple end cap regions, and to segment the shelf image based on the segmentation line.

[0046] The segmentation module 230 specifically includes:

[0047] The grouping submodule is used to group the multiple plug regions according to their coordinates.

[0048] Specifically, the grouping submodule is used to obtain the difference between the x-axis coordinates of each plug region. If the difference between the x-axis coordinates of two plug regions is less than or equal to a preset multiple of the width of one of the plug regions, then the two plug regions are determined to be plug regions in the same group.

[0049] The fitting submodule is used to fit dividing lines based on the coordinates of multiple end cap regions in each group.

[0050] The fitting submodule is specifically used to calculate the mean x-axis coordinate of the end cap region within the same group, and determine the position of the dividing line based on the mean value.

[0051] The segmentation submodule is used to segment the shelf image based on the segmentation lines.

[0052] In this embodiment, the above-mentioned device further includes:

[0053] An expansion module is used to expand each of the shelf guide rail areas respectively.

[0054] Accordingly, the detection module 220 is specifically used to detect the end blockage area in each expanded shelf guide rail area, and obtain the coordinates of multiple end blockage areas on the shelf image.

[0055] In this embodiment, the above-mentioned device further includes:

[0056] The deduplication module is used to determine whether there are two end-blocking areas with intersecting areas based on the coordinates of the multiple end-blocking areas on the shelf image; if so, the module deduplicates one of the two end-blocking areas.

[0057] In this embodiment, multiple price tags can be provided on the guide rail; correspondingly, the above-mentioned device also includes:

[0058] The judgment module is used to determine whether each blockage area intersects with at least one price tag area among the multiple price tag areas based on the coordinates of the multiple blockage areas and the coordinates of the multiple price tag areas on the shelf image; if at least one blockage area intersects with at least one price tag area among the multiple price tag areas, then the at least one blockage area is determined to be a false detection area.

[0059] This application embodiment detects end-block areas within the shelf guide rail area, fits segmentation lines based on the coordinates of multiple detected end-block areas, and segments the shelf image based on the segmentation lines. This avoids the detection of small targets, thereby accurately segmenting the shelf image.

[0060] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described method embodiment for segmenting shelf images and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0061] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0062] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0063] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of segmenting a shelf image, characterized by, The application is applied to a shelf with multiple guide rails, the edge of each guide rail is provided with a plug for preventing a price tag on the guide rail from sliding out of the guide rail, and the method comprises the following steps: Obtaining a shelf image, detecting multiple shelf guide rail regions from the shelf image; Detecting a plug region based on each shelf guide rail region respectively to obtain the coordinates of multiple plug regions on the shelf image; Fitting a segmentation line according to the coordinates of the multiple plug regions and segmenting the shelf image based on the segmentation line; Each plug is provided with a reflective element, and the guide rail is provided with multiple price tags; after the coordinates of the multiple plug regions on the shelf image are obtained based on the detection of the plug region of each shelf guide rail region respectively, the method further comprises the following steps: According to the coordinates of the multiple plug regions on the shelf image and the coordinates of the multiple price tag regions on the shelf image, it is judged whether each plug region has an intersection region with at least one price tag region in the multiple price tag regions; If at least one plug region in the multiple plug regions has an intersection region with at least one price tag region in the multiple price tag regions, it is determined that the at least one plug region is a false detection region.

2. The method of claim 1, wherein, After the multiple shelf guide rail regions are detected from the shelf image, the method further comprises the following steps: Each shelf guide rail region is expanded respectively; The detection of the plug region based on each shelf guide rail region respectively specifically comprises the following steps: The plug region is detected in each expanded shelf guide rail region respectively.

3. The method of claim 1, wherein, The fitting of the segmentation line according to the coordinates of the multiple plug regions and the segmentation of the shelf image based on the segmentation line specifically comprises the following steps: The multiple plug regions are grouped according to the coordinates of the multiple plug regions; Based on the coordinates of the multiple plug regions in each group, a segmentation line is fitted respectively, and the shelf image is segmented based on the segmentation line.

4. The method of claim 3, wherein, The grouping of the multiple plug regions according to the coordinates of the multiple plug regions specifically comprises the following steps: The difference between the x-axis coordinates of each plug region is obtained, and if the difference between the x-axis coordinates of two plug regions is less than or equal to a preset multiple of the width of one of the plug regions, the two plug regions are determined as plug regions in the same group; The fitting of the segmentation line based on the coordinates of the multiple plug regions in each group specifically comprises the following steps: The mean value of the x-axis coordinates of the plug regions in the same group is calculated, and the position of the segmentation line is determined according to the mean value.

5. The method of claim 2, wherein, After the coordinates of the multiple plug regions on the shelf image are obtained based on the detection of the plug region of each shelf guide rail region respectively, the method further comprises the following steps: According to the coordinates of the multiple plug regions on the shelf image, it is judged whether there are two plug regions with an intersection region; If there are, one of the two plug regions is removed.

6. The method according to claim 1, wherein the detection of the plug region based on each shelf guide rail region respectively to obtain the coordinates of the multiple plug regions on the shelf image specifically comprises the following steps: ​ The device is applied to a shelf with multiple guide rails, edges of each guide rail are provided with a plug, the plug is used to prevent a price tag on the guide rail from sliding out of the guide rail, and the device comprises:

7. An apparatus for segmenting a shelf image, characterized by An acquisition module is configured to acquire a shelf image and detect multiple shelf guide rail regions from the shelf image; A detection module is configured to detect plug regions in each shelf guide rail region to obtain coordinates of the multiple plug regions on the shelf image; A segmentation module is configured to fit a segmentation line according to the coordinates of the multiple plug regions and segment the shelf image based on the segmentation line; Each plug is provided with a reflective element, and the guide rail is provided with multiple price tags; The device further comprises: A judgment module is configured to judge whether each plug region has an intersection region with at least one price tag region according to the coordinates of the multiple plug regions on the shelf image and coordinates of multiple price tag regions on the shelf image; if at least one plug region of the multiple plug regions has an intersection region with at least one price tag region of the multiple price tag regions, it is determined that the at least one plug region is a false detection region. Further comprising:

8. The apparatus of claim 7, wherein, An expansion module is configured to expand each shelf guide rail region respectively; The detection module is specifically configured to detect plug regions in each expanded shelf guide rail region to obtain coordinates of the multiple plug regions on the shelf image.

9. The device of claim 7, wherein: The segmentation module specifically comprises: A grouping sub-module is configured to group the multiple plug regions according to the coordinates of the multiple plug regions; A fitting sub-module is configured to fit a segmentation line based on the coordinates of the multiple plug regions in each group; A segmentation sub-module is configured to segment the shelf image based on the segmentation line.

10. The device of claim 9, wherein: The grouping sub-module is specifically configured to obtain a difference between x-axis coordinates of each plug region, and if the difference between x-axis coordinates of two plug regions is less than or equal to a preset multiple of the width of one of the plug regions, the two plug regions are determined as plug regions in a same group; The fitting sub-module is specifically configured to calculate a mean value of x-axis coordinates of plug regions in a same group and determine a position of a segmentation line according to the mean value. Further comprising:

11. The apparatus of claim 8, wherein, A de-duplication module is configured to judge whether there are two plug regions with an intersection region according to the coordinates of the multiple plug regions on the shelf image; If there are, one of the two plug regions is de-duplicated.

12. The device of claim 7, wherein: The detection module is specifically configured to detect reflective elements based on each shelf guide rail region and take coordinates of the detected multiple reflective elements on the shelf image as coordinates of the multiple plug regions on the shelf image. ​ 13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the method for segmenting the shelf image according to any one of claims 1 to 6.

Images