METHOD FOR IMAGE ANALYSIS OF A STRUCTURE FORMING A SUPPORT FOR PRODUCT STORAGE

Abstract

According to one aspect, a computer-implemented image analysis method is proposed for images of products arranged on a series of stacked shelves. The method comprises: - product detection (PRD) in the image (IMG), - label detection (LBL) in the image (IMG), - definition of zones (PZN) of identical products in the image based on the locations of the detected labels (LBL) in the image (IMG), - association of each product (PRD) with a zone (PZN) of identical products based on its location in the image (IMG). Figure 2 (for the abstract)

Description

Title of the invention: METHOD FOR IMAGE ANALYSIS OF A STRUCTURE FORMING A SUPPORT FOR PRODUCT STORAGE

[0001] The present invention relates to the computer detection of products on a structure forming a support for the storage of these products.

[0002] A shelving unit is a support structure composed of shelves, used to organize and store products, for example in retail stores.

[0003] The analysis of products stored in shelving units makes it possible, in particular, to monitor product stocks in order to prevent stockouts and facilitate replenishment.

[0004] In order to simplify the analysis of these products, computer vision solutions have been developed to detect and / or recognize the stored products.

[0005] Computer vision detection uses cameras and image analysis algorithms to identify the presence and location of products.

[0006] Recognition, on the other hand, aims to specifically identify the types of products.

[0007] In particular, computer vision systems capture images or Video feeds of products on shelves. Initially, detection algorithms identify the presence or absence of products and determine their exact location. Then, recognition algorithms analyze the images to identify specific products, their brands, and their variants. For example, in a supermarket, these systems can detect empty shelf spaces and recognize missing products to send restocking alerts.

[0008] Recognition algorithms can be complex and require significant computing power and memory. For example, convolutional neural networks (also known as CNNs) can be used for product recognition.

[0009] Convolutional neural networks are a class of artificial neural networks particularly effective for image processing. These convolutional neural networks are trained on large sets of images, enabling them to recognize and classify products with high accuracy. These networks analyze the visual characteristics of products, such as shape, color, and patterns, to identify them correctly even under varying lighting and arrangement conditions.

[0010] Current computer vision-based solutions have significant drawbacks. One of the main limitations is the computing power and memory required to process and analyze images, especially in real time.

[0011] However, the number of products to be identified on a shelf may significantly increase the computing load, requiring additional processing and memory resources to maintain system performance.

[0012] However, the limited capacity of certain devices, such as multifunction phones and tablets, hinders the use of known computer vision-based product detection and recognition solutions. These devices often lack the computing power and memory necessary to efficiently execute these complex algorithms.

[0013] There is therefore a need to propose a solution to facilitate the detection and recognition of products arranged on shelves of racking elements.

[0014] According to one aspect, a computer-implemented method for image analysis of a structure forming a support for product storage is proposed, the method comprising: - an implementation of a product detection function in the image, i.e., a detection of product representations in the image (for example, an implementation of an algorithm and / or a product detection model in the image), - an implementation of a label detection function in the image, i.e., a detection of label representations in the image (for example, an implementation of an algorithm and / or a model for label detection in the image), - a determination of areas of identical products in the image from the locations of the labels detected in the image, - an association of each product with an area of ​​identical products based on its location in the image.

[0015] Such a process makes it possible to group detected products into zones of identical products. This allows products in the same group to be considered identical because they are located in the same zone of identical products.

[0016] Such a process therefore makes it possible to group identical products from a simple detection of products and labels, without having to recognize each detected product.

[0017] Such a method also makes it possible to deduce that an area of ​​identical products on a shelf is empty when no product is associated with that area of ​​identical products. This information can then be used to plan product replenishment.

[0018] In an advantageous embodiment, the method further comprises an implementation of a product recognition function (for example, an implementation of an algorithm and / or a product recognition model) for a single product in each defined area of ​​identical products.

[0019] Such a recognition function makes it possible to determine the identity of the product taken as input to this function. The product identity provided by the product recognition function is then applied to all products associated with the identical product area.

[0020] In such a process, the product recognition function is therefore implemented for a single product detected per defined area of ​​identical products.

[0021] Such a method thus avoids repeating the product recognition function for each product associated with the same area of ​​identical products. This reduces the computational, memory, and processing time required to identify products arranged on shelves represented in an image.

[0022] Preferably, the product detection function is configured to define a frame for each detected product, this frame delimiting the detected product.

[0023] Advantageously, associating a product with a zone of identical products includes calculating a ratio between: - an overlap zone between the product frame and the identical product zone, and - a total surface area of ​​the product frame, then an association of the product with this area of ​​identical products according to the calculated ratio (especially if the calculated ratio is greater than or equal to a given threshold).

[0024] Thus, it is possible to associate a product with an area of ​​identical products even if that product is not entirely contained within the area of ​​identical products, for example when the product is slightly offset from an area of ​​identical products.

[0025] In an advantageous embodiment, the determination of identical product zones includes an implementation of an algorithm for identifying longitudinal axes along which extend in the image of the shelves of the support structure, these longitudinal axes being identified from the locations of the labels and / or products detected in the image, each identical product zone being delimited from the identified longitudinal axes and the detected labels.

[0026] It is possible to identify the longitudinal axes along which the shelves extend from the locations of the labels only. Indeed, the labels are placed directly on the shelves.

[0027] Alternatively, it is possible to identify the longitudinal axes along which the shelves extend from the product locations only. In particular, the products arranged on a shelf have a lower edge in contact with the shelf. Thus, the lower edge of these products is close to the longitudinal axes of the shelves.

[0028] Nevertheless, it is advantageous to take into account the location of the labels and products to increase the number of points available for defining the affine functions. The parameters of the affine function associated with a longitudinal axis are the slope of this axis and the ordinate with respect to a corner of the image, for example, the upper left corner of the image.

[0029] Advantageously, the longitudinal axis identification algorithm includes a definition of longitudinal axes from linear regressions performed on the locations of the labels and / or products detected in the image.

[0030] Preferably, the algorithm for identifying the longitudinal axes along which the shelves extend comprises: - a definition of an initial line from a linear regression performed on the location of all detected labels and / or products, then - at least one iteration of defining candidate lines including: A calculation of residuals between the locations of the detected labels and / or products and the initial line or previously defined candidate lines, then a definition of residual groups based on their values, A definition of a candidate line for each group of residuals defined from a linear regression on the locations of the labels and / or products associated with that group of residuals, - then a definition of said longitudinal axes according to a relevance score of the candidate lines, the relevance score being dependent on a sum of the residuals associated with these candidate lines, a penalty on the number of candidate lines defined at each iteration and on the number of outliers, corresponding to label and / or product locations, obtained in relation to the candidate lines defined at each iteration.

[0031] Such a relevance score makes it possible to improve the identification of the longitudinal axes along which the shelves extend in the image.

[0032] Advantageously, linear regressions are performed by implementing an ordinary least squares algorithm.

[0033] According to another aspect, a computer program product is proposed comprising instructions which, when executed by a computer, lead the computer to implement an analysis process as described above.

[0034] According to another aspect, a computer-readable data carrier is proposed, on which a computer program product as described above is recorded.

[0035] According to another aspect, a computer system is proposed comprising: - a memory storing a computer program product as described previously, and - a processing unit configured to run such a computer program product.

[0036] Other advantages and features of the invention will become apparent upon examination of the detailed description of embodiments, which are by no means limiting, and the accompanying drawings in which:

[0037] [Fig.1]

[0038] [Fig.2]

[0039] [Fig.3]

[0040] [Fig.4]

[0041] [Fig.5]

[0042] [Fig.6]

[0043] [Fig.7]

[0044] [Fig.8]

[0045] [Fig.9]

[0046] [Fig. 10]

[0047] [Fig.1 1]

[0048] [Fig. 12] illustrate embodiments and implementations of the invention.

[0049] Figure 1 schematically illustrates one embodiment of a system The SYS computer system comprises a processing unit (PU) and a memory module (MM). The PU is configured to execute a computer program (PRG) stored in the MMM.

[0050] The MEM memory is also configured to store an image (IMG) illustrating a set of shelves on which various products are arranged. The computer program is configured to take this image as input. The image can be acquired by an image or video acquisition device. For example, the acquisition device has a resolution greater than or equal to 400,000 pixels. The computer system (SYS) may include the acquisition device. The computer system (SYS) may be a computer, a multifunction phone, a tablet computer, etc.

[0051] The computer program PRG includes instructions which, when executed by the processing unit, cause it to implement an image analysis method IMG in which products are represented arranged on a set of stacked shelves. An implementation method for such a method is described below in relation to [Fig. 3].

[0052] Figure 2 schematically illustrates an IMG image of a structure serving as a support for product storage, for example a set of shelves of shelving, this IMG image can be used as input for the computer program.

[0053] In this figure, two superimposed shelves Eb E2 of the same shelving unit are shown.

[0054] Each shelf can hold different PRD products. Some products may be identical to other products arranged on the shelves. For example, [Fig. 2] illustrates two identical PRD_A products, one PRD_B product, four PRD_C products, one PRD_D product, and two PRD_E products. Identical products arranged on a shelf are grouped in the same PZN zone of identical products. Identical products on a shelf are associated with the same LBL label. For example, PRD_A products are located in a PZN_A zone of identical products and are associated with the LBL_A label, PRD_B product is located in a PZN_B zone of identical products and is associated with the LBL_B label, etc.

[0055] Each LBL label can be a paper label or an electronic label. Each label can display information about the product associated with it. This information includes, in particular, the name and price of the product. Each label is placed on a longitudinal edge of the shelf supporting the identical product(s) associated with it.

[0056] Each identical product PZN zone is therefore associated with a single label. Advantageously, the LBL label associated with an identical product PZN zone is positioned to define a lateral boundary, in particular a left lateral boundary, of its associated identical product zone. Thus, the lateral boundaries of an identical product PZN zone are defined by the LBL label associated with that product zone and another LBL label associated with the adjacent identical product zone. The lower boundary of an identical product zone is defined by the shelf on which the products in that zone are placed. The upper boundary of an identical product zone is defined by the shelf above the products in that zone, or by the top of the IMG image for products located on the highest shelf.

[0057] Some identical product areas may be empty of products. For example, in [Fig.2], the products associated with the label LBL_F are absent.

[0058] Generally, products in an area of ​​identical products are placed directly on the shelf that supports them. However, some products may be placed on top of other identical products.

[0059] The arrangement of the labels makes it possible to simply delimit the lateral limits of the areas of identical products, and thus makes it possible to simplify the analysis of the image, as described below.

[0060] In what follows, an orthogonal coordinate system RPR is defined in the image. This orthogonal coordinate system RPR has its origin O in the upper left corner of the image. This coordinate system It includes an x-axis for abscissas and a y-axis for ordinates. Other coordinate systems can of course be used.

[0061]

[0062]

[0063] Figure 3 illustrates an example of some steps in an image analysis process depicting products arranged on a set of stacked shelves. As previously mentioned, this analysis process can be implemented by the computer system running the computer program. The process includes a step 20 of implementing a PRD product detection function in the image. This detection function is configured to detect product representations in the image provided as input to the computer program. The PRD product detection algorithm is configured to define a PFRM frame for each detected product. Thus, each detected product is delimited by a PFRM frame. The PFRM frame can be defined from two points corresponding to two opposite corners of the PFRM frame. For example, the frame of a detected product Pj is defined by a point defining the upper left corner of the frame with coordinates (xp, where xp is the x-coordinate of this point and y^J is the y-coordinate of this point). The frame of a detected product Pj is also defined by a point defining the lower right corner of the frame with coordinates (xyS y%J), where x is the x-coordinate of this point and y^j is the y-coordinate of this point.

[0064] Alternatively, the detection function can be configured to detect and delimit each product by shapes other than a PFRM frame and / or by features other than opposite points. For example, each product can be delimited by a quadrilateral or by segmentation.

[0065] The product detection function can implement various product detection algorithms and / or models, including a segmentation algorithm. For example, it is possible to implement one or more of the following detection algorithms or models: "YOLO" (short for "You Only Look Once"), "SAM" (short for "Segment Anything Model"), "ViT" (short for "Vision Transformer"), "RETINA DET", "SSD" (short for "Single Shot Detector"), and "R-CNN" (short for "Region-based Convolutional Neural Network"). Additional processing can be implemented to define rectangular bounding frames from an initial detected shape (e.g., a quadrilateral or a segmentation).

[0066] The method also includes a step 21 of implementing an algorithm for detecting LBL labels in the image. This step can be implemented before,

[0067]

[0068] after or at the same time as step 20 of the implementation of the PDT product detection algorithm. This algorithm is configured to detect label representations in the image provided as input to the computer program. Similar to the product detection algorithm, the LBL label detection algorithm is configured to define a DFRM frame for each detected label. Thus, each detected label is delimited by a DFRM frame. The DFRM frame can be defined from two points corresponding to two opposite corners of the DFRM frame. For example, the frame of a detected Dk label is defined by a point defining the upper-left corner of the frame and having coordinates (x^k, y^k), where x is the x-coordinate of this point and y^k is the y-coordinate of this point. The frame of a The detected Dk label is also defined by a point defining the lower right corner. of the frame and having coordinates (x^k, where x^k is the abscissa of this point and y^k is the ordinate of this point.

[0069] The label detection function can implement various product detection algorithms and / or models, including a segmentation algorithm. For example, it is possible to implement one or more of the following detection algorithms or models: "YOLO" (short for "You Only Look Once"), "SAM" (short for "Segment Anything Model"), "ViT" (short for "Vision Transformer"), "RETINA DET", "SSD" (short for "Single Shot Detector"), and "R-CNN" (short for "Region-based Convolutional Neural Network"). Additional processing can be implemented to define rectangular bounding boxes from an initial detected shape (e.g., a quadrilateral or a segmentation).

[0070] Figure 4 illustrates an example of an FRM frame that can be generated from the detection of a product or a label.

[0071] In addition, representative points for replacements of labels and / or products detected in the image are defined. For example, the representative point for the location of a product is defined at the center of the lower side of the frame delimiting that product (e.g., ((xi+x2) / 2,y2)). The representative point for the location of a label is defined at the center of the upper side of the frame delimiting this label (eg, ((xi+x2 ) / 2,y J). These points are located near the shelves and can therefore indicate their presence.

[0072] Figure 5 illustrates PFRM and DFRM frames that can be obtained by implementing product and label detection algorithms, as well as points mij representative of the locations of the products and labels detected, respectively.

[0073] Returning to [Fig. 2], the process then includes a step 22 of determining areas of identical products in the image. The detected areas of identical products are then delimited by ZFRM frames.

[0074] In order to define an area of ​​identical products, the method includes an implementation 22-1 of an algorithm for identifying longitudinal axes along which the shelves extend in the image. Each shelf in the image is therefore associated with a longitudinal axis.

[0075] These longitudinal axes are defined by developing affine functions representing these longitudinal axes. These affine functions are developed from the locations of the labels and / or products detected in the image.

[0076] In particular, the PLBL points representing the location of the labels and / or the PPRD points representing the location of the products are used to estimate the parameters a;, b; of the affine functions a;x+bi.

[0077]

[0078]

[0079]

[0080] It is advantageous to consider the placement of labels and products to increase the number of points available for defining linear functions. The parameters of the linear function associated with a longitudinal axis are the slope of that axis and the y-coordinate relative to the top-left corner of the image. Products placed on top of other products are not considered when more accurately defining affine functions. Only products positioned directly on a shelf are considered for this purpose. Products placed on top of other products can be identified by defining a contact zone under each product. If this contact zone overlaps the frame of a product located below the product associated with that contact zone, then the product defining the contact zone is considered to be placed on top of another product. An example of a contact zone ZCj associated with a product frame Pj arranged on a product frame Pj+i is illustrated in [Fig.6]. The contact area has a width corresponding to the width of the product frame associated with that contact area. The contact area has a height h. This height h can be defined empirically. This height can be defined according to the following mathematical formula: , / ( \ y1 / D, n \, where F is a coefficient that can be defined from Empirically, nd is the number of labels detected in the image, y^t is the ordinate of the top side of the label frame Dk, and y^t is the ordinate of the side lower of the label frame Dk. When V is equal to 1, then the height h of the contact area is equal to the average height of the labels.

[0081] Alternatively, to determine whether one product is placed on another product, it is also possible to check whether the vertical distance between two frames delimiting these two products is less than a threshold. This threshold corresponds in particular to the height of the contact zone defined previously.

[0082] For example, in [Fig.7], the points of the product Pj+1 and Pj+3 are not taken into account for the development of an affine function a;x+bi.

[0083] An algorithm is implemented to develop the affine functions representing the longitudinal axes along which the shelves extend in the image. This algorithm is configured to determine the number of longitudinal axes to be developed.

[0084] This algorithm includes an initialization step 22-1-1 illustrated in [Fig. 2]. In this step 22-1-1, an initial linear regression is performed on the set of points representing the locations of the labels and / or products placed directly on the shelves. This linear regression can, for example, be performed by executing an ordinary least squares method.

[0085] This initial linear regression allows us to obtain the parameters of an initial DINIT line, illustrated in figures 8 and 9. This initial DINIT line gives an idea of ​​the orientation of the shelves in the image.

[0086] Next, initial residuals are calculated in step 22-1-2. These initial residuals correspond to the vertical distance between the initial line and each representative point of a location of a label or a product placed directly on the shelves.

[0087] The [Fig.9] represents an initial residue r / nW between a point mz and the initial line DINIT.

[0088] Once the initial residuals have been calculated, as illustrated in [Fig. 2], the method includes an implementation 22-1-3 of a clustering algorithm for creating groups of initial residuals based on their values. For example, the clustering algorithm can be chosen from a k-means clustering algorithm, a density-based clustering algorithm (also known as DBSCAN, for density-based spatial clustering of applications with noise), or a hierarchical clustering algorithm. However, it is perfectly possible to use other clustering algorithms.

[0089] The residual partitioning algorithm aims to group the residuals associated with points located at a similar vertical distance from the initial line. These points are generally associated with labels or products on the same shelf.

[0090] As illustrated in [Fig. 2], a linear regression is then applied in step 22-1-4 to each group of residuals so as to obtain an affine function associated with each cluster. Each affine function defines a candidate line. This linear regression can also be performed using the ordinary least squares method.

[0091] For each defined affine function, the algorithm identifies outliers with respect to that affine function. The outliers are identified by analyzing the distance between the line defined by the affine function and the points associated with the residues of the group obtained after defining the affine function.

[0092] Outliers can for example be associated with products located set back from the longitudinal edge of the shelf.

[0093] The resulting affine functions are then evaluated by calculating a relevance score. This relevance score allows the affine functions to be evaluated according to several criteria, notably in order to find a compromise between the quality of the linear regressions, the integration of the points in the construction of the functions, and the overall consistency of the constructed shelves.

[0094] For example, the relevance score is calculated using the following mathematical formula:

[0095] SCR = C + ax noutnc, where a and [3] are predefined coefficients, nout is the number of outliers with respect to the affine function, nc is the number of defined residual groups (also corresponding to the number of candidate lines), and

[0096] £ _yWe yp°ù lrw| is the absolute value of the residue associated with point Z of the affine function w to the residue group w.

[0097] New residuals are then calculated between the points and the set of defined affine functions.

[0098] Steps 22-1-2 to 22-1-4 can be iterated several times to search for new candidate lines that improve the relevance score. Thus, at each iteration, groups of residuals are created from the residuals calculated in the previous iteration, and new candidate lines are generated from these groups of residuals.

[0099] These iterations allow convergence towards a partitioning of the residuals, resulting in an optimal solution in which each defined affine function represents a longitudinal axis along which a shelf extends in the image. Thus, the algorithm generates affine functions that consistently represent the shelves.

[0100] Fig. 10 illustrates longitudinal axes AXn, AX1, AX1+i that can be defined from the location of the products and / or labels.

[0101] Each label is then associated with the affine function associated with the shelf on which the label is placed. This association is made according to the position of the label with respect to the longitudinal axis defined by the affine function.

[0102] Once the affine functions have been developed, the identical product zones can be defined in step 22-2. Each identical product zone is defined from the longitudinal axes representing the shelves and a label associated with a product.

[0103] Preferably, each identical product zone is rectangular. In this way, each identical product zone can be defined by two points. However, it is perfectly possible to define a parallelepiped-shaped identical product zone using four points.

[0104] Each zone of identical products on a shelf is delimited on the left and right by two labels following one another along the length of that same shelf. Each zone of identical products is delimited at the top and bottom by these labels and by the affine function associated with the shelf above.

[0105] Figure 11 illustrates examples of identical product zones Z2, Z4, and Z6 that can be defined. Figure 11 illustrates shelves Ei to E5 and label frames Di to Dn.

[0106] More particularly, a rectangular zone Zk of identical products can be defined by two opposite points.

[0107] For example, a zone Zk of identical products can be defined by a point defining the top left corner of the zone and a point defining the bottom right corner of the zone.

[0108] The point defining the top left corner of the zone Zk of identical products has an abscissa of the xDki of the left side of the label associated with the products in this zone Zk of identical products. This same point has an ordinate of a minimum value between the ordinate C i(xDki) of the affine function f^ associated with the shelf above for the abscissa xDki of the left side of the label associated with the products in this zone Zk of identical products and the ordinate f ,(xl)k+l J of the affine function associated with the shelf above for the abscissa xDk+1i of the left side of the label located to the right of the zone Zk of identical products on the same shelf.

[0109] In other words, the point defining the top left corner of the identical product area has the following coordinates: (xDki, mintf j(xDki), fi4(xDk+1i )))•

[0110] The point defining the bottom right corner of the identical product area has an abscissa of the xDk+1i coordinate of the left side of the label located to the right of the identical product area. This same point has a ordinate that is the maximum value between the yDki ordinate of the top side of the label associated with the products in the product area. identical and the yDk+1i ordinate of the top side of the label located to the right of the identical product area.

[0111] In other words, the point defining the bottom right corner of the identical product area has the following coordinates: (xDk+1i, max(yDki, yDk+1i))

[0112] If no other label is located to the right of the label associated with the products in the identical product area, on the same shelf, then the identical product area is defined by the location of its associated label and by the right side of the image.

[0113] In particular, the point defining the top-left corner of the Zk zone of identical products has an abscissa equal to the xDki coordinate of the left side of the label associated with the products in this Zk zone of identical products. The ordinate of this same point is the minimum value between the ordinate fki(xDki) of the affine function fi associated with the top shelf for the xDki coordinate of the left side of the label associated with the products in this Zk zone of identical products and the ordinate fi(img_wdth) of the affine function associated with the top shelf for the img_wdth coordinate of the right side of the image. The value img_wdth corresponds to the width of the image.

[0114] In other words, the point defining the top left corner of the identical product area has the following coordinates: (xDki, min(fi i(xDki), £4 (img_wdth))).

[0115] The point defining the bottom right corner of the identical product area has an abscissa equal to the abscissa of the right side of the image. This same point has a ordinate equal to the maximum value between the ordinate yDki of the top side of the label associated with the products in the identical product area Zk and the ordinate f;(img_wdth) of the affine function f; associated with the shelf of products in this identical product area for the abscissa img_wdth of the right side of the image.

[0116] In other words, the point defining the bottom right corner of the identical product area has the following coordinates: (img_wdth, max(yDki, f;(img_wdth))).

[0117] Furthermore, according to specific embodiments and for each shelf, if products are located to the left of the leftmost label on the shelf, then these products are disregarded to avoid errors. According to other embodiments, they form a product zone.

[0118] The process then includes a step 23 of associating the detected PRD products with the PZN zones of identical products.

[0119] In this step, each PRD product is associated with a zone in which that product is located. It is possible that a product may only be partially present in a zone of identical products, for example, when the product is located on the left or right side of the zone of identical products. Thus, the overlap zone The relationship between a detected product and an area of ​​identical products is studied in order to associate or not this product with this area of ​​identical products.

[0120] Naturally, there are other ways to define product areas.

[0121] The overlap zone is defined by the common area between the identical product zone and the frame delimiting the detected product.

[0122] Fig. 12 illustrates an overlap zone INTj k between a product Pj and a zone Zk of identical products.

[0123] In particular, the area of ​​the overlap zone between a product Pj and an area of ​​identical products Zk can be calculated using the following mathematical formula:

[0124] SrfZR~(m^n(x2'^k) ' max^fxy^ - jAj)' where is the x-coordinate of the left side of the frame of product Pj, is the x-coordinate of the right side of the frame of product Pj, is the y-coordinate of the top side of the frame of product Pj, is the y-coordinate of the bottom side of the frame of product Pj, is the x-coordinate of the left side of the zone Zk of identical products, is the x-coordinate of the right side of the zone Zk of identical products, is the y-coordinate of the top side of the zone Zk of identical products, is the y-coordinate of the bottom side of the zone Zk of identical products,

[0125] Then a ratio is calculated between the area ^rfZR of the overlap zone is the total surface area SrfpRl^p of the product frame Pj. This ratio is therefore calculated according to the The following mathematical formula:

[0126] RT = Srf J ZÂ $rf FRMPl OR $rfFRMPj - xp^j

[0127] This ratio is then compared to a threshold δ. This threshold can be defined empirically. This threshold is defined to limit the risk that the same product, partially placed in two adjacent zones of identical products, will be associated with both of these identical product zones.

[0128] If the ratio is greater than or equal to this threshold ô, then this means that the surface area of ​​the overlap zone between product Pj and the identical product zone Zk is If the ratio is sufficiently large relative to the total area of ​​frame Pj, then the product is considered to be within the identical product zone. In this case, product Pj is associated with the identical product zone. If the ratio is less than the threshold, then the product is not associated with the identical product zone.

[0129] In order to prevent a product from being assigned to several identical product zones simultaneously, it is possible to assign the product to the identical product zone for which the overlap area is the largest.

[0130] Returning to [Fig. 2], the process then includes a step 24 of implementing a product recognition algorithm. The product recognition algorithm is configured to deliver a product identity from the image of a detected product. The image of a detected product can be extracted from the initial image using the PFRM frame delimiting the detected product. The product recognition algorithm can be a convolutional artificial neural network, in particular trained in a supervised manner from a database of labeled product images.

[0131] Advantageously, when an area of ​​identical products includes several identical products, the product recognition algorithm is implemented for only one detected product. The product identity provided by the product recognition algorithm is then applied to all products associated with the area of ​​identical products.

[0132] The product recognition algorithm is therefore implemented for a single product detected per defined area of ​​identical products. This method avoids repeating the product recognition algorithm for each product associated with the same area of ​​identical products. This reduces the computational, memory, and processing time required to identify products arranged on shelves represented in an image.

[0133] Such a process also makes it possible to deduce that an area of ​​identical products on a shelf is empty when no product is associated with that area of ​​identical products in step 23.

[0134] Of course, the present invention is susceptible to various variations and modifications which will become apparent to those skilled in the art. For example, in the embodiments of the image analysis method described above, only one image is analyzed in order to simplify understanding of the invention. Nevertheless, it is entirely possible to adapt the described embodiments to allow for the analysis of a sequence of images. In this case, for example, it is possible to periodically or non-periodically analyze images from the image sequence in order to detect the products present in those images.

Claims

Demands

1. A computer-implemented method for analyzing the image of a structure forming a support for product storage (PRD), the method comprising: - an implementation of a product detection function (PRD) in the image (IMG), - an implementation of a label detection function (LBL) in the image (IMG), - a determination of areas (PZN) of identical products in the image from the locations of the labels (LBL) detected in the image (IMG), - an association of each product (PRD) with an area (PZN) of identical products according to the location of this product (PRD) in the image (IMG) relative to the determined areas (PZN) of identical products.

2. A method according to claim 1, further comprising an implementation of a product recognition function for a single product (PRD) in each defined identical product zone.

3. A method according to any one of claims 1 or 2, wherein the product detection function (PRD) is configured to define a frame (PFRM) for each detected product, this frame (PFRM) delimiting the detected product (PRD).

4. A method according to any one of claims 1 to 3, wherein the association of a product (PRD) with an area (PZN) of identical products comprises a calculation of a ratio between: - an overlap area between the frame (PFRM) of the product and the area (PZN) of identical products (PZN), and - a total surface area of ​​the frame of the product (PFRM), then an association of the product (PRD) with this area (PZN) of products according to the calculated ratio.

5. A method according to claim 4, wherein the determination of zones (PZN) of identical products comprises an implementation of an algorithm for identifying longitudinal axes (AX1, AX2, AX1+1) along which extend in the image of the shelves of the support structure, these longitudinal axes being identified from the locations of the detected labels and / or products in the image, each area of ​​identical products is delimited from the identified longitudinal axes and the detected labels.

6. A method according to claim 5, wherein the longitudinal axis identification algorithm includes a definition of longitudinal axes from linear regressions performed on the locations of the labels and / or products detected in the image.

7. A method according to any one of claims 5 or 6, wherein the algorithm for identifying longitudinal axes along which the shelves extend comprises: - defining an initial line (DINIT) from a linear regression performed on the location of all detected labels and / or products, then - at least one iteration of candidate line definition comprising: O calculating residuals between the locations of the detected labels and / or products and the initial line or previously defined candidate lines, then O defining groups of residuals based on their values, O defining a candidate line for each group of residuals defined from a linear regression on the locations of the labels and / or products associated with that group of residuals, - then defining said longitudinal axes according to a relevance score of the candidate lines,The relevance score depends on a sum of the residuals associated with these candidate lines, a penalty on the number of candidate lines defined at each iteration, and on the number of outliers, corresponding to label and / or product locations, obtained relative to the candidate lines defined at each iteration.

8. A method according to any one of claims 6 or 7, wherein the linear regressions are performed by implementing an ordinary least squares algorithm.

9. Product computer program comprising instructions which, when executed by a computer, cause the computer to implement an analysis method according to any one of claims 1 Q 2

10. 1 d O. Computer-readable data carrier on which a computer program product (CPP) according to claim 9 is recorded.

11. Computer system comprising: - a memory (MEM) storing a computer program product (PRG) according to claim 9, and - a processing unit (PU) configured to run such a computer program product.

Images