Analysis method, analysis device, and analysis system

Abstract

The analysis method includes: acquiring an image including a plurality of articles; acquiring the uniformity of the plurality of articles on the basis of the image; a step for determining the display status of the plurality of articles on the basis of the uniformity; and outputting the result of the determination.

Description

Analysis method, analysis device, and analysis system Technical Field

[0001] The present invention relates to an analysis method, an analysis device, and an analysis system for analyzing the display conditions of articles in a store shelf or the like. Background Art

[0002] Items stored in store shelves or warehouses are arranged or displayed to meet specific rules to ensure efficient use and safety of the space, or to achieve storage requirements or display purposes. For example, in fresh food cases in supermarkets, products are displayed to meet specific display requirements to improve the appearance of the cases, ensure freshness of fresh food, and minimize losses.

[0003] Furthermore, it is known that the display of merchandise in store shelves affects merchandise sales. For example, Patent Document 1 proposes identifying specific merchandise from a captured image of its display status on a shelf, and generating multiple shelf display configuration candidates representing display statuses effective for the sales of the specific merchandise based on analysis results based on the identified specific merchandise and its sales data.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2021-108209

[0007] Summary of the Invention

[0008] [Problems to be solved by the invention]

[0009] For example, Patent Document 1 describes the need to maintain the display conditions of items in order to maintain sales gains, such as increased sales. Typically, to maintain this, store staff monitor the display conditions and, if any discrepancies are detected, make corrections. However, having staff perform this verification not only incurs labor costs and is inefficient, but also presents objectively inconsistent evaluations due to individual differences in the staff's experience and understanding.

[0010] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide an analysis method, an analysis device, and an analysis system capable of quantitatively analyzing the display conditions of articles.

[0011] [Methods of solving the problem]

[0012] The analysis method according to the present invention comprises the steps of acquiring an image including a plurality of articles; acquiring the orderliness of the plurality of articles based on the image; determining the display condition of the plurality of articles based on the orderliness; and outputting the determination result.

[0013] The analyzing device according to the present invention comprises: an image acquiring unit for acquiring an image including a plurality of objects; a neatness acquiring unit for acquiring the neatness of the plurality of objects based on the image; a determining unit for determining the display condition of the plurality of objects based on the neatness; and an output unit for outputting the determination result.

[0014] The analysis system involved in the present invention comprises: a photographing device for photographing an image including multiple objects; an image acquisition unit for acquiring the image from the photographing device; a neatness acquisition unit for acquiring the neatness of the multiple objects based on the image; a determination unit for determining the display conditions of the multiple objects based on the neatness; and an output unit for outputting the determination result.

[0015] [Effects of the Invention]

[0016] According to the analysis method of the present invention, by acquiring the orderliness of a plurality of articles from an image including the plurality of articles, the display condition of the articles can be quantitatively analyzed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG1 is a schematic configuration diagram of an analysis system according to the first embodiment.

[0018] FIG2 is a control block diagram of the analysis system according to the first embodiment.

[0019] FIG3 is a flowchart showing operations performed by the processing device according to the first embodiment.

[0020] FIG. 4 is an example of an image including a plurality of products P1 captured by the imaging device.

[0021] FIG. 5 is a graph showing the results of texture analysis performed on the image in FIG. 4 .

[0022] FIG. 6 is an example of an image including a plurality of products P2 captured by the imaging device.

[0023] FIG. 7 is a graph showing the results of texture analysis performed on the image in FIG. 6 .

[0024] FIG. 8 is a diagram showing examples of the number of main directions and the number of angles of the main directions.

[0025] FIG. 9 is a diagram showing examples of the number of main directions and the number of angles of the main directions.

[0026] FIG10 is a control block diagram of the analysis system according to the second embodiment.

[0027] FIG11 is a flowchart showing operations performed by the processing device according to the second embodiment.

[0028] FIG. 12 is an example of a reference image including a plurality of products P3 stored in the storage unit according to the second embodiment.

[0029] FIG. 13 is an example of an image including a plurality of products P3 captured by the imaging device.

[0030] FIG. 14 is an example of an image including a plurality of products P3 captured by the imaging device.

[0031] (Explanation of Symbols)

[0032] 1. 1A: Processing device; 2: Camera; 3: Shelf; 11: Image acquisition unit; 12: Image preprocessing unit; 13: Texture detection unit; 14: Regularity acquisition unit; 15: Determination unit; 16: Output unit; 17: Storage unit; 18: Output unit; 100. 100A: Analysis system; 171: Reference image; 172: Comparison model. DETAILED DESCRIPTION

[0033] The following describes an analysis system 100 for implementing an analysis method according to an embodiment of the present invention, with reference to the accompanying drawings. Identical or corresponding components are denoted by the same reference numerals in the various figures, and their descriptions are omitted or simplified as appropriate. Furthermore, the shapes, sizes, and configurations of the components depicted in the various figures may be modified as appropriate within the scope of the present invention.

[0034] Implementation method 1.

[0035] FIG1 is a schematic diagram of an analysis system 100 according to Embodiment 1. The analysis system 100 according to this embodiment is used in retail stores such as supermarkets to analyze the display conditions of multiple products P stored on shelves 3 within the store. Furthermore, “multiple products P” refers to a situation where there are multiple products of the same type. The analysis system 100 includes a processing device 1 and a camera 2. The processing device 1 functions as an analysis device and is a PC or a cloud server equipped with a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) and a memory.

[0036] Camera 2 is a fixed camera mounted on the ceiling or wall of the store. Camera 2 captures images of multiple products P stored on shelves 3. Camera 2 is adjusted at an angle and distance to capture front-facing images of multiple products P. Alternatively, camera 2 can utilize dedicated camera software to capture products P from the most optimal angle and distance possible. The optimal angle and distance take into account the following considerations: capturing the display of products P from the front as closely as possible, ensuring that the entire product P is captured within the image, and ensuring that the area occupied by the product P within the image is not too small compared to surrounding objects or the background.

[0037] The processing device 1 and the imaging device 2 are connected to each other in a wired or wireless manner so as to be communicable. An image captured by the imaging device 2 is transmitted to the processing device 1 .

[0038] FIG2 is a control block diagram of an analysis system 100 according to Embodiment 1. As shown in FIG2 , the processing device 1 includes an image acquisition unit 11, an image preprocessing unit 12, a texture detection unit 13, a regularity acquisition unit 14, a determination unit 15, an output unit 16, and a storage unit 17. The image acquisition unit 11, the image preprocessing unit 12, the texture detection unit 13, the regularity acquisition unit 14, the determination unit 15, and the output unit 16 are functional units implemented by executing a program on the CPU or GPU of the processing device 1. Furthermore, at least one of the functional units may be implemented using a processing circuit such as an ASIC or FPGA. Furthermore, a portion of the functional units of the processing device 1 may be provided in other external devices.

[0039] The image acquisition unit 11 acquires an image of the shelf 3 including a plurality of products P, which is captured by the imaging device 2, and transmits the image to the image pre-processing unit 12. The image acquisition unit 11 acquires the image of the shelf 3 at a preset time interval.

[0040] The image preprocessing unit 12 performs preprocessing to improve the quality of the acquired image. For example, the image preprocessing unit 12 performs edge extraction, structural element extraction, or color space conversion on the acquired image to emphasize image features or information and suppress noise or unnecessary information. The image preprocessing unit 12 specifically performs color space conversion and brightness adjustment, taking into account the effects of lighting, as well as noise filtering. Furthermore, when an image of the shelf 3 is captured by the camera 2, the placement and angle of the products P in the image may differ from the actual display of the products P, depending on the shooting angle or distance of the camera 2. Therefore, the image preprocessing unit 12 performs image processing to adjust the angle so that the image reflects the actual display of the products P. Specifically, when an image is captured from an oblique angle, rectangular objects appear to be distorted into a trapezoidal shape in the image. Therefore, the image preprocessing unit 12 preferably performs processing to restore the trapezoidal shape of the object to a rectangular shape.

[0041] The texture detection unit 13 performs texture analysis on the pre-processed image, and detects multiple elements contained in the image and their distribution directions. Texture refers to a number of basic elements that appear periodically in the image, or the distribution of image information that is attenuated or enhanced according to a certain rule. In the image captured by the camera 2, when multiple products P are neatly displayed, a strong periodicity is exhibited in the image. That is, differences appear in the texture of the image depending on the display standards of the products P. Therefore, in the processing device 1 of this embodiment, texture analysis technology is used to detect the distribution direction of elements from the image of the shelf 3 including multiple products P, and analyze the display status of the products P. The specific processing of the texture detection unit 13 will be described in detail later.

[0042] The neatness acquisition unit 14 analyzes the periodicity of the multiple elements and distribution directions detected by the texture detection unit 13 to obtain the neatness of the products P. The neatness is an indicator of the display condition of the products P and is expressed as a value between 0.0 and 1.0. A neatness closer to 1.0 indicates a more neatly displayed product P, while a neatness closer to 0.0 indicates a more random and disorderly display of the products P. The specific processing of the neatness acquisition unit 14 will be described in detail later.

[0043] The determination unit 15 determines whether the products P are neatly displayed based on the neatness obtained by the neatness acquisition unit 14. If the neatness obtained by the neatness acquisition unit 14 is equal to or greater than a threshold value, the determination unit 15 determines that the products P are neatly displayed. The threshold value is, for example, 0.8, which is preset and stored in the storage unit 17. The determination result determined by the determination unit 15 is output to the output unit 16.

[0044] The output unit 16 outputs the determination result of the determination unit 15 to the store manager and the like. Specifically, when the determination unit 15 determines that the product P is not neatly displayed, the output unit 16 outputs an improvement notice indicating that the display of the product P needs to be improved to the manager and the like as the result of the determination. The improvement notice includes the position information of the shelf 3 on which the product P is displayed and the identification information of the product P. The output unit 16 can display the improvement notice on the display unit of the processing device 1 or send it to an external device. The external device is, for example, a management device of a store in which the analysis system 100 is provided, or an information communication terminal such as a smart phone or tablet owned by the manager. As a result, the manager who receives the improvement notice can understand that the product P is not neatly displayed and improve the display.

[0045] The storage unit 17 is a volatile or non-volatile memory such as RAM, ROM, or flash memory. The storage unit 17 stores programs executed by the CPU of the processing device 1 and parameters such as thresholds used in processing within each component of the processing device 1. The storage unit 17 may also store shelf information and product information. Shelf information includes information such as the location of shelves 3 within the store and the shelf display layout of products P stored on shelves 3. Product information includes identification information such as the type and name of the product.

[0046] FIG3 is a flowchart illustrating the operations performed by processing device 1 according to Embodiment 1. First, an image of shelf 3 containing a plurality of products P is captured by imaging device 2 and acquired by image acquisition unit 11 of processing device 1 ( S1 ). The acquired image is then preprocessed by image preprocessing unit 12 ( S2 ). Next, texture detection unit 13 performs texture analysis on the preprocessed image to detect multiple elements and their distribution directions ( S3 ).

[0047] The texture analysis in the texture detection unit 13 is described. The texture detection unit 13 performs texture analysis on the pre-processed image, and simultaneously detects multiple elements and distribution directions. Specifically, the texture detection unit 13 obtains basic elements and the rules of their spatial distribution by, for example, using the conventional Wavelet transform to perform texture analysis. Furthermore, the frequency components corresponding to the sizes of the basic elements that are periodically distributed in the frequency domain by the sliding window Fourier transform are detected, and their spatial positions are located by the sliding window Fourier transform. In addition, Contourlet transform or Shearlet transform can also be used. In addition to this, the texture detection unit 13 can also use GCLM (Gray-Level Co-occurrence Matrix) to perform texture analysis.

[0048] Texture analysis will be described with reference to Figures 4 to 7. Figure 4 shows an example of an image including multiple products P1 captured by camera 2, and Figure 5 is a graph showing the results of texture analysis performed on the image in Figure 4. Furthermore, Figure 6 shows another example of an image including product P2 captured by camera 2, and Figure 7 is a graph showing the results of texture analysis performed on the image in Figure 6. In Figures 5 and 7, the horizontal axis represents angle (distribution direction) and the vertical axis represents energy. Energy represents periodicity, with greater energy indicating higher periodicity. An angle of 0° corresponds to the horizontal direction.

[0049] As shown in Figure 5, in the image of Figure 4, the energy increases in the directions of angles 0°, 180°, and 360°. In other words, the image of Figure 4 shows a high degree of periodicity in the horizontal direction. On the other hand, as shown in Figure 7, in the image of Figure 6, the energy increases in the directions of angles 0° to 50°, 110° to 240°, and 300° to 360°. In other words, the image of Figure 6 shows periodicity in multiple directions.

[0050] The texture detection unit 13 acquires the following information by performing texture analysis.

[0051] Element group = {1st element, 2nd element, ..., nth element}

[0052] Element = {element ID, position, range (or size), size in the first direction, size in the second direction, ..., size in the mth direction}

[0053] Direction group = {angle of the first direction, angle of the second direction, ..., angle of the mth direction}

[0054] The 1st to nth elements included in the element group are the elements detected by texture analysis. Furthermore, the magnitudes in the 1st to mth directions of each element are the magnitudes of the energy in those directions. Furthermore, the angles in the 1st to mth directions are the angles in the directions that exhibit periodicity, measured counterclockwise relative to the horizontal. The groups of information are not limited to the above and may be arbitrary, as long as the mutual relationships are clear.

[0055] Returning to Figure 3 , the regularity acquisition unit 14 analyzes the information acquired by the texture detection unit 13 to acquire regularity (S4). The processing performed by the regularity acquisition unit 14 will be described. The regularity acquisition unit 14 first extracts target elements from the multiple elements detected during texture analysis. The multiple elements detected during texture analysis also include elements other than the product P, such as the surrounding structure of the product P. Therefore, the regularity acquisition unit 14 extracts elements that include the product P, the target of analysis, from the elements detected during texture analysis.

[0056] The regularity acquisition unit 14 first totals the number of the 1st to nth elements, and rearranges the 1st to nth elements in order of large to small number. The number of the 1st to nth elements is obtained as an intermediate result of texture analysis. Then, the regularity acquisition unit 14 obtains the distribution range of each element for the elements whose number is greater than the threshold. The distribution range is the range occupied by each element on the image. Specifically, when the number of the first element is greater than the threshold, the regularity acquisition unit 14 uses the square or rectangle including the four coordinates of the upper left, lower left, upper right, and lower right among all the coordinates of the multiple first elements as the distribution range of the first element. Similarly, the regularity acquisition unit 14 calculates the distribution range of all elements whose number is greater than the threshold.

[0057] Then, the regularity acquisition unit 14 extracts the elements with suitable distribution range as the target elements. Here, the regularity acquisition unit 14 compares the distribution range of each element with the pre-set target range, and extracts the elements with high similarity as the target elements. Specifically, the regularity acquisition unit 14 compares the size of the area of ​​the target range with the size of the area of ​​the distribution range of each element. The closer the size is, the higher the similarity is judged to be. The target range is the range within the image that includes the product P that becomes the analysis object, and is pre-set by the administrator and stored in the storage unit 17. For example, the target range is the range of the image of the shelf 3 obtained by the image acquisition unit 11. It is inferred that the image obtained by the image acquisition unit 11 contains a large number of target elements, so the elements with a distribution range close to the range of the image become the target elements.

[0058] Furthermore, if the target range is not stored in the storage unit 17, the regularity acquisition unit 14 may estimate whether each element is a target element based on its distribution and area. Since the imaging conditions are limited, the estimation of whether each element is a target element is based on the assumption that the target products P are relatively evenly distributed within the area.

[0059] Specifically, the regularity acquisition unit 14 divides the distribution range of each element into a plurality (t) of cells. The number of cells t is represented by t = w × h. w is the number of cells in the horizontal direction, and h is the number of cells in the vertical direction. w and h are empirically determined. For example, w and h should take into account the size of the element (product) and are preferably at least several times the size of the element.

[0060] The regularity acquisition unit 14 then calculates the number of elements contained in each cell. Furthermore, the regularity acquisition unit 14 calculates the mean of the number of elements in all cells for each element and the variance (difference from the mean) within each cell. The variances of each cell are then sorted in order, with the smallest variance at the top. From the top 1 / 3, the element with the largest area is selected as the target element. The area refers to the size of each element.

[0061] Next, the regularity acquisition unit 14 selects the main direction of the extracted object element. Specifically, the regularity acquisition unit 14 selects the main direction from each direction based on the energy magnitude of the object element detected by the texture detection unit 13. Specifically, the regularity acquisition unit 14 first arranges the first through m-th directions in order of energy magnitude. Then, the energy magnitudes of the ordered directions are summed up, starting from the highest. If the sum of the summed energies up to the R-th direction accounts for 85% or more of the total energy, the direction up to the R-th direction is designated as the main direction. For example, if the total energy of an object element is 100, the energy magnitude of the first direction is 60, and the energy magnitude of the second direction is 25, then the main directions of the object element are both the first and m-th directions. Alternatively, if the total energy of an object element is 100, and the energy magnitude of the first direction is 85, then the main direction of the object element is only the single first direction. The 85% used as the criterion is an example, and the value may be appropriately selected from the range of 65% to 90% based on experience.

[0062] The neatness acquisition unit 14 then calculates the neatness. Products P being neatly displayed means that they are periodically distributed in a certain direction. Furthermore, periodically distributed means that the number of directions in which products P are distributed is small, and there is a certain correlation between these directions. Therefore, the neatness acquisition unit 14 calculates the neatness based on the number of main directions in the target element and the relationship between these main directions.

[0063] In detail, the regularity acquisition unit 14 sets the regularity to 1.0 when the main direction of the object element is one. In addition, the regularity acquisition unit 14 sets the regularity to 0.0 when the main direction of the object element is multiple and the number of main directions is more than 85% of the total number of directions. In addition, 85% used as a judgment standard is an example, and it is appropriately selected based on experience in the range of 65% to 90%. In addition, the regularity acquisition unit 14 calculates the regularity according to the following formula (1) when it does not correspond to any of the above. Here, α is a weight coefficient, which is, for example, 0.1. d n is the number of principal directions, a n is the number of principal direction angles.

[0064] S=1.0-α×d n ×a n …(1)

[0065] FIG8 and FIG9 are diagrams showing examples of the number of main directions and the number of angles of the main directions. In the example shown in FIG8 , directions d1 , d2 , and d3 are main directions, and the number of main directions dn 3. In addition, in the case of FIG8 , the angles formed by the directions d1, d2, and d3 are all 120 degrees, and the number of main direction angles a n On the other hand, in the example shown in FIG9 , directions d1 , d2 , and d3 are main directions, and the number of main directions d n 9, the angles formed by directions d1 and d2, and directions d1 and d3 are 90 degrees, and the angle formed by directions d2 and d3 is 180 degrees, so the number of main direction angles a is n is 2. When these values ​​are substituted into the above formula (1), the example of FIG9 has lower regularity than the example of FIG8.

[0066] Furthermore, examining the regularity of the images shown in Figures 4 and 6, the image in Figure 4 has fewer directions (angles) with high energy, as shown in Figure 5, resulting in fewer main directions and high regularity. On the other hand, the image in Figure 6 has more directions (angles) with high energy, resulting in a greater number of main directions and low regularity.

[0067] Returning to Figure 3 , after obtaining the orderliness, the determination unit 15 determines whether the orderliness is above a threshold value ( S5 ). If the orderliness is above the threshold value ( S5 : “Yes”), it is assumed that the products are neatly displayed, and this process ends. On the other hand, if the orderliness is below the threshold value ( S5 : “No”), it is assumed that the products are not neatly displayed, and the output unit 16 outputs an improvement notice ( S6 ). By regularly repeating this process, changes in the display status of the products P can be automatically detected and notified.

[0068] As described above, the analysis system 100 of this embodiment can quantitatively analyze the display condition of products by obtaining the degree of orderliness from images containing multiple products P. This eliminates the need for store staff to monitor the display condition, reducing labor costs and improving efficiency. Furthermore, it allows for objective standardization of the criteria for determining the display condition.

[0069] Furthermore, in the analysis system 100 of this embodiment, texture analysis is performed to obtain not only the degree of neatness but also the direction of the periodicity of the item display. Therefore, the output unit 16 may output the principal direction obtained by the neatness acquisition unit 14 as the result of the determination to a store manager, etc. This allows the store manager to compare the principal direction received from the output unit 16 with the reference direction and easily determine in which direction the display is disordered. As a result, it is possible to easily provide instructions to store staff on how to improve the display. Furthermore, the reference direction is the principal direction that achieves the highest degree of neatness, and this is pre-determined in the work standards, etc. Alternatively, the reference direction may be pre-stored in the storage unit 17 in the processing device 1. In this case, the determination unit 15 may obtain the deviation between the principal direction obtained by the neatness acquisition unit 14 and the reference direction, and output this as a direction for display improvement from the output unit 16. Furthermore, the output unit 16 may output the neatness obtained by the neatness acquisition unit 14 as the result of the determination.

[0070] Implementation method 2.

[0071] Implementation 2 will be described. Techniques for detecting image texture include two types: distribution features that are intuitively related to observation results (intuitive methods) and abstract features that are not intuitively related to observation results (abstract methods). Intuitive methods analyze using a method similar to (or close to) human perception, and the texture analysis technique used in Implementation 1 corresponds to this method. In contrast, Implementation 2 uses a deep learning model as an abstract method to compare with reference image 171 to determine uniformity.

[0072] FIG10 is a control block diagram of an analysis system 100A according to Embodiment 2. As shown in FIG10 , the analysis system 100A according to Embodiment 2 includes a processing device 1A and an imaging device 2. The structure and function of the imaging device 2 are the same as those in Embodiment 1. As shown in FIG10 , the processing device 1A includes an image acquisition unit 11, an image preprocessing unit 12, a regularity acquisition unit 14, a determination unit 15, an output unit 16, and a storage unit 17. The image acquisition unit 11, the image preprocessing unit 12, the regularity acquisition unit 14, the determination unit 15, and the output unit 16 are functional units implemented by executing a program on the CPU or GPU of the processing device 1. In addition, at least one of the functional units may be implemented using a processing circuit such as an ASIC or FPGA.

[0073] The functions of the image acquisition unit 11, image preprocessing unit 12, determination unit 15, and output unit 16 are the same as those in Embodiment 1. Furthermore, in this embodiment, the image preprocessing unit 12 performs noise filtering as preprocessing, and performs image resizing when the size of the image acquired by the image acquisition unit 11 does not conform to the input specifications of a comparison model 172 described later.

[0074] The storage unit 17 is a volatile or nonvolatile memory such as RAM, ROM, or flash memory. The storage unit 17 stores programs executed by the CPU of the processing device 1, parameters such as thresholds used in processing within each component of the processing device 1, shelf information, and product information. In this embodiment, the storage unit 17 also stores a reference image 171 and a comparison model 172.

[0075] Reference image 171 is an image of neatly displayed goods, captured in advance by camera 2 for each type of goods and stored in storage unit 17. Comparison model 172 is a deep learning model that uses a convolutional neural network (CNN) to output the distance between reference image 171 and the image (target image) preprocessed by image preprocessing unit 12. Comparison model 172 performs convolution and pooling on reference image 171 and the target image to extract feature vectors. Furthermore, the distance between the feature vectors of reference image 171 and the target image is calculated. Furthermore, as a model that outputs the distance between reference image 171 and the target image, various algorithms, in addition to CNN, can be used.

[0076] The regularity acquisition unit 14 inputs the reference image 171 and the object image preprocessed by the image preprocessing unit 12 into the comparison model 172 stored in the storage unit 17, and obtains the distance between the reference image 171 and the object image as an output. Moreover, the distance between the obtained reference image 171 and the object image is used as the regularity. The distance output by the comparison model 172 is a numerical value between 0 and 1. When the distance is 0, the reference image 171 and the object image are completely consistent, and when the distance is 1, the reference image 171 and the object image are completely different. That is, in this embodiment, the closer the regularity (distance) is to 0, the more neatly the products P are displayed, and the closer the regularity (distance) is to 1, the more randomly and disorderly the products P are displayed.

[0077] FIG11 is a flowchart illustrating the operations performed by the processing device 1A according to Embodiment 2. First, an image of a shelf 3 containing a plurality of products P is captured by the imaging device 2 and acquired by the image acquisition unit 11 of the processing device 1 ( S11 ). The acquired image is then preprocessed by the image preprocessing unit 12 ( S12 ). Next, the regularity acquisition unit 14 acquires regularity using the comparison model 172 ( S13 ).

[0078] The acquisition of the neatness involved in Embodiment 2 will be described with reference to Figures 12 to 14. Figure 12 is an example of a reference image 171 including a plurality of products P3 stored in the storage unit 17 according to Embodiment 2. Figures 13 and 14 are examples of images including a plurality of products P3 captured by the camera 2. When the reference image 171 of Figure 12 and the image of Figure 13 are input into the comparison model 172, the distance output from the comparison model 172 is, for example, 0.06. Furthermore, when the reference image 171 of Figure 12 and the image of Figure 14 are input into the comparison model 172, the distance output from the comparison model 172 is, for example, 0.15. That is, the neatness of the products P3 at the time when the image of Figure 13 was acquired was 0.06, which is higher than the neatness (0.15) of the products P3 at the time when the image of Figure 14 was acquired.

[0079] Returning to Figure 11 , after obtaining the neatness, the determination unit 15 determines whether the neatness is below a threshold ( S14 ). The threshold in this case is, for example, 0.1. If the neatness is below the threshold ( S14 : “Yes”), it is inferred that the products are neatly displayed, and this process ends. On the other hand, if the neatness is greater than the threshold ( S14 : “No”), it is inferred that the products are not neatly displayed, and an improvement notice is output via the output unit 16 ( S15 ). By regularly repeating this process, changes in the display status of the products P can be automatically detected and notified.

[0080] As described above, the analysis system 100A of this embodiment can also quantitatively analyze the display condition of products by obtaining the degree of orderliness from images containing multiple products P. This eliminates the need for store staff to monitor the display condition, reducing labor costs and improving efficiency. Furthermore, it allows for objective standardization of the criteria for determining the display condition.

[0081] The above is a description of an embodiment, but the above embodiments are subject to modification and combination. For example, the above embodiment analyzes the display status of products P from an image of a shelf 3 in a retail store, but the present invention is not limited to this embodiment and can be applied to methods for analyzing the display status of items from an image containing multiple items. Furthermore, the above embodiment analyzes the display status of products P on a shelf 3 in a store, but the present invention is not limited to this embodiment and can also analyze the display status of multiple items in locations other than the shelf 3.

[0082] Furthermore, the imaging device 2 is not limited to a fixed camera installed in a store; it may also be a mobile terminal such as a mobile phone or smartphone equipped with a camera, or a portable camera. In this case, a store employee or the like operates the imaging device 2 to capture images of a plurality of products P. In this case, operating rules may be set to enable images of the products P to be captured at an ideal angle and distance.

[0083] Furthermore, the texture detection unit 13 is not limited to performing texture analysis to determine energy in each direction, and may also determine other characteristics such as entropy.

[0084] Alternatively, the texture detection unit 13 may detect distribution directions after detecting elements. In this case, the texture detection unit 13 first detects multiple basic elements from the image. Element detection can be achieved using object detection technology using deep learning models such as SSD (Single Shot Multibox Detector), image segmentation, or object segmentation technology. After detecting elements, the texture detection unit 13 analyzes the position of each detected element to determine its distribution direction.

[0085] In this case, the texture detection unit 13 acquires the following information.

[0086] Element group = {1st element, 2nd element, ..., nth element}

[0087] Element = {element ID, position, range (or size), number in the first direction, number in the second direction, ..., number in the mth direction}

[0088] Direction group = {angle of the first direction, angle of the second direction, ..., angle of the mth direction}

[0089] The 1st to nth elements included in the element group are elements detected by object detection technology. Furthermore, the number of elements in the 1st to mth directions is the number of elements in each of the 1st to mth directions. Furthermore, the angles in the 1st to mth directions are counterclockwise angles relative to the horizontal. The information groups are not limited to the above and may be arbitrary, as long as the mutual relationship is clear.

[0090] In this case, the main direction is selected in the regularity acquisition unit 14 based on the number of elements in each direction. Specifically, the regularity acquisition unit 14 first arranges the 1st direction to the mth direction in order of the number of elements from most to least. Then, the number of elements in each direction with the order added is added up in order from the highest. Then, when the total number of elements added up to the Rth direction is more than 85% of the total number of elements, the direction up to the Rth is set as the main direction. For example, when the total number of elements is 100, the number of elements in the first direction as the first is 60, and the number of elements in the mth direction as the second is 25, the main directions of the element are the first direction and the mth direction. In addition, for example, when the total number of elements is 100 and the number of elements in the first direction as the first is 85, the main direction of the element is only one first direction.

[0091] Furthermore, in the above embodiment, the determination unit 15 determines the display condition of the products P based on neatness, but the present invention is not limited to this. The determination unit 15 may also determine whether the products P are neatly displayed based on the number of principal directions and the relationship between the principal directions. Specifically, the determination unit 15 may determine that the products P are neatly displayed if the number of principal directions is three or fewer and the angles between the principal directions are less than a threshold angle (e.g., 45 degrees), or if the number of principal directions is three or more but the angles between the principal directions are approximately the same and the ratio of the number of principal directions to the total number of directions is less than 50%.

[0092] Furthermore, while the above embodiment describes an image containing multiple products of a single type, the analysis system 100 can also analyze the display conditions of each product in an image containing multiple products of multiple types. In this case, the image preprocessing unit 12 or the like can separate the image into multiple regions based on the type of product and determine the degree of neatness for each region.

[0093] In addition, if the image quality of the acquired image is not affected by the subsequent processing, the pre-processing by the image pre-processing unit 12 may be omitted.

Claims

1. An analytical method comprising: The step of obtaining an image including a plurality of items; A step of obtaining the neatness of a plurality of the objects according to the image; A step of determining the display status of the plurality of items according to the neatness; and The step of outputting the result of the determination.

2. The analysis method according to claim 1, wherein The steps of obtaining the regularity include: performing texture analysis on the image to detect a plurality of elements contained in the image and distribution directions of the plurality of elements; and A step of analyzing the detected plurality of elements and the distribution directions to obtain the regularity.

3. The analysis method according to claim 2, wherein The steps of obtaining the regularity include: A step of extracting a target element from the plurality of elements; and A step of obtaining the regularity based on the number of main directions of the target elements and the relationship between the main directions.

4. The analysis method according to claim 3, wherein In the output step, As a result of the determination, the main direction of the target element or a deviation of the main direction from a direction serving as a reference for the arrangement of the target elements is output.

5. The analysis method according to claim 1, wherein In the step of obtaining the degree of orderliness, the degree of orderliness is obtained using a deep learning model that compares a reference image of a display state of a plurality of the articles with the obtained image.

6. An analysis device comprising: an image acquisition unit that acquires an image including a plurality of objects; an orderly degree obtaining unit for obtaining orderly degrees of the plurality of articles based on the image; a determination unit that determines a display condition of the plurality of articles based on the orderliness; and The output unit outputs the result of the determination.

7. An analysis system comprising: a photographing device for photographing an image including a plurality of objects; an image acquisition unit that acquires the image from the imaging device; an orderly degree obtaining unit for obtaining orderly degrees of the plurality of articles based on the image; a determination unit, configured to determine a display condition of the plurality of articles based on the orderliness; as well as The output unit outputs the result of the determination.

Images