Detection system

Abstract

To provide a detection system that can further reliably sense products and is easier to install.SOLUTION: A detection system comprises a server for holding pattern data for identifying types of products handled in a store, and a detection device for identifying the products by being communicably connected to the server. The detection device includes: a sensor unit provided along a placement surface of a shelf, which is installed such that a plurality of products can be placed on the placement surface; and an information processing unit for comparing output of the sensor unit with the pattern data to identify types of products and the number of products for each type. The sensor unit has a planar optical sensor arranged to face a plurality of products across the placement surface. The planar optical sensor has a resolution capable of identifying a shape of one side of the placement surface for each of the plurality of products placed on the placement surface, and characters, identifiers, and patterns applied to the one side. The pattern data includes information indicating the shape of the one side and the characters, identifiers, and patterns applied to the one side.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to detection systems. [Background technology]

[0002] A method for managing the movement of products on shelves by combining an imaging device and a weight sensor is known (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-189691 Summary of the Invention [Problem to be solved by the invention]

[0004] The method described in Patent Document 1 has several operational constraints necessary for detecting the movement of products, making it difficult to install and operate a device to realize this method. Specifically, these constraints include setting the position and angle of view of the imaging device so that the image is not obstructed by customers carrying products, and ensuring that all products are within the imaging range of the imaging device. Furthermore, considering the improvement in the accuracy of product identification using weight sensors, additional constraints are added, such as not placing multiple types of products with the same weight on the same weight sensor, eliminating individual weight differences for products treated as the same type of product, and providing individual weight sensors for each location where products that cannot be distinguished by weight are placed, making operation even more difficult.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a detection system that can sense products more reliably and is easier to install. [Means for solving the problem]

[0006] A detection system according to one aspect of the present disclosure comprises a server that stores pattern data for identifying the types of products handled in a store, and a detection device that is capable of communicating with the server and identifies the products, wherein the detection device comprises a sensor unit that is provided along the loading surface of a shelf on which a plurality of products can be placed, and an information processing unit that compares the output of the sensor unit with the pattern data to identify the type of product and the number of each type of product, wherein the sensor unit has a planar optical sensor that is positioned facing the plurality of products across the loading surface, and the planar optical sensor has a resolution that enables it to identify the shape of one surface of each of the plurality of products placed on the loading surface, the surface facing the loading surface, and the characters, identifiers, and patterns applied to the one surface, and the pattern data includes information indicating the shape of the one surface, and the characters, identifiers, and patterns applied to the one surface. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing the main components of the detection system. [Figure 2] FIG. 2 is a schematic overhead view of a store in which the first and second gates of the detection system are provided. [Figure 3] FIG. 3 is a schematic diagram showing an example of the configuration of a shelf. [Figure 4] FIG. 4 is a schematic layer diagram showing the main configuration of the shelf board with sensing function. [Figure 5] FIG. 5 is a block diagram showing a configuration related to the operation of the planar optical sensor and a functional configuration related to information processing performed based on light detected by the planar optical sensor. [Figure 6] FIG. 6 is a schematic diagram showing an example of a shelf board with sensing functions and the state of the products on the shelf board before and after the number of products placed thereon changes. [Figure 7] FIG. 7 is a schematic diagram showing a method for identifying a product. [Figure 8] FIG. 8 is a schematic diagram showing an example in which the arrangement of some or all of the products in the sensing area does not match before and after the change. [Figure 9]FIG. 9 is a diagram showing the configuration of the second DB. [Figure 10] FIG. 10 is a schematic diagram showing an example of the contents of product data. [Figure 11] FIG. 11 is a schematic diagram showing an example of the contents of customer data. [Figure 12] FIG. 12 is a diagram showing the configuration of the first DB. [Figure 13] FIG. 13 is a schematic diagram showing an example of the contents of product inventory data. [Figure 14] FIG. 14 is a schematic diagram showing an example of the contents of store-specific customer data. [Figure 15] FIG. 15 is a schematic diagram showing an example of the contents of purchaser data. [Figure 16] FIG. 16 is a schematic diagram showing an example in which the position of an item placed on a shelf is moved by a customer. [Figure 17] FIG. 17 is a diagram showing an example of the contents of product inventory data after the "first pattern" occurs. [Figure 18] FIG. 18 is a diagram showing an example of the contents of the store-specific customer data after the "first pattern" occurs. [Figure 19] FIG. 19 is a diagram showing an example of the contents of the purchaser data after the "first pattern" occurs. [Figure 20] FIG. 20 is a diagram showing an example of the contents of the product inventory data after the "second pattern" occurs. [Figure 21] FIG. 21 is a schematic diagram showing an example of the contents of purchaser data when the customer-specific information in the purchaser data is a customer-specific number. [Figure 22] FIG. 22 is a flowchart showing the process flow when the payment process is automatically performed when leaving the store. [Figure 23] FIG. 23 is a flowchart showing the flow of the store entry process. [Figure 24] FIG. 24 is a flowchart showing the flow of the product movement confirmation process. [Figure 25] FIG. 25 is a flowchart showing the flow of the first store exit process. [Figure 26] FIG. 26 is a flowchart showing the flow of processing when a customer uses a terminal to independently manage products that the customer takes off the shelves. [Figure 27] FIG. 27 is a flowchart showing the flow of the second store exit process. [Figure 28] FIG. 28 is a flowchart showing the process flow when it is assumed that a customer uses a self-checkout to complete payment before entering the second gate. [Figure 29] FIG. 29 is a flowchart showing the flow of the third store exit process. [Figure 30] FIG. 30 is a schematic diagram showing an example of a precautionary measure for providing a surface on which a stably placed product in the sensing area AA can be placed stably, when the stably placed product does not have a surface on which the stably placed product can be placed in the sensing area AA. [Figure 31] FIG. 31 is a schematic diagram showing a vending machine stocked with products. [Figure 32] FIG. 32 is a schematic diagram showing a locker containing merchandise. DETAILED DESCRIPTION OF THE INVENTION

[0008] Each embodiment of the present disclosure will be described below with reference to the drawings. The disclosure is merely an example, and appropriate modifications that a person skilled in the art can easily conceive while maintaining the gist of the invention are naturally included within the scope of the present disclosure. Furthermore, to clarify the explanation, the drawings may show the width, thickness, shape, etc. of each part schematically compared to the actual embodiment, but these are merely examples and are not intended to limit the interpretation of the present disclosure. Furthermore, in this specification and each drawing, elements similar to those described above with reference to the previous drawings will be designated by the same reference numerals, and detailed descriptions may be omitted as appropriate.

[0009] 1 is a block diagram showing the main components of a detection system 1. The detection system 1 includes a detection device 10, a first gate 40, a second gate 50, a server 60, and a terminal 90.

[0010] FIG. 2 is a schematic overhead view of a store ST in which a first gate 40 and a second gate 50 of the detection system 1 are installed. The first gate 40 is installed at the entrance for customers HC to enter the store ST. In FIG. 2, customers HC1, HC2, HC3, and HC4 are shown as specific examples of customers HC. Hereinafter, the term "customer HC" refers to a person who could become a customer of the store ST, such as customers HC1, HC2, HC3, and HC4. As shown in FIGS. 1 and 2, the first gate 40 includes a first reading unit 42, an entry detection unit 43, and an opening / closing mechanism 44.

[0011] The first reading unit 42 reads identification information held by the terminal 90 of the customer HC. The identification information is held by an identification information holding unit 99 of the terminal 90. The specific implementation of the identification information holding unit 99 corresponds to that of the first reading unit 42. For example, if the identification information holding unit 99 is assumed to be an RF tag used in RFID (Radio Frequency Identification), the first reading unit 42 is an RF tag reader. This description is merely one example of a combination of the first reading unit 42 and the identification information holding unit 99, and is not limited to this. For example, the first reading unit 42 may be configured to be capable of acquiring identification information through non-contact communication of multiple standards.

[0012] The entry detection unit 43 detects the entry of a customer HC into the first gate 40. In FIG. 2 , the entry detection unit 43 is configured to capture an image of the first gate 40 and its vicinity and detect the customer HC captured in the captured image. However, the specific configuration of the entry detection unit 43 is not limited to this. For example, the entry detection unit 43 may be configured to determine that a customer HC has entered the first gate 40 when it detects the blocking of light such as infrared light on the customer HC's path through the first gate 40. The entry detection unit 43 activates the operation of the first reading unit 42 and the opening / closing mechanism 44 in response to the detection of the customer HC. In other words, if the entry detection unit 43 does not detect a customer HC for a predetermined active time or longer, the first reading unit 42 and the opening / closing mechanism 44 enter a sleep state. This reduces power consumption of the first gate 40.

[0013] The opening / closing mechanism 44 is provided on the entrance path of the customer HC at the first gate 40. The opening / closing mechanism 44 operates to restrict the entry of a customer HC who is not authorized to enter the store ST. While a so-called flap door type opening / closing mechanism is shown in FIG. 2, this is not limiting. For example, the opening / closing mechanism 44 may be a turnstile type, a retractable type, or any other type of gate.

[0014] The second gate 50 is provided at an exit for customers HC to exit the store ST. As shown in FIGS. 1 and 2, the second gate 50 includes a second reading unit 52, an entry detection unit 53, and an opening / closing mechanism 54. The second reading unit 52 has the same configuration as the first reading unit 42. The entry detection unit 53 has the same configuration as the entry detection unit 43. The opening / closing mechanism 54 has the same configuration as the opening / closing mechanism 54.

[0015] 1, the first gate 40 has a communication unit 41. The second gate 50 has a communication unit 51. The communication units 41 and 51 function as a NIC (Network Interface Controller) for communicating with the detection device 10 via the communication network NW.

[0016] A customer HC who enters the store ST via the first gate 40 and exits the store ST via the second gate 50 may purchase merchandise prepared within the store ST. Information processing by the detection system 1 is used to manage merchandise in the store ST. Specifically, merchandise is placed on shelves SH such as shelves SH1 and SH2 in FIG. 2. Hereinafter, when a shelf SH is referred to, it refers to a shelf such as shelves SH1 and SH2. Shelves SH may also be provided, for example, in a free area FSA within the store ST.

[0017] Fig. 3 is a schematic diagram showing an example of the configuration of a shelf SH. In Figs. 2 and 3, a front side FR and a back side BA are shown to indicate the orientation of the shelf SH. A customer HC can take out products placed on the shelf SH from the front side FR. A first support portion FR1 of the shelf SH is provided on the back side BA.

[0018] The first support portion FR1 is a frame-wall-like member installed on the second support portion FR2. The first support portion FR1 supports a sensing-function-equipped shelf board SHB, such as sensing-function-equipped shelf boards SHB1, SHB2, SHB3, and SHB4, on the rear side BA. Hereinafter, when the term sensing-function-equipped shelf board SHB is used, it refers to a shelf-like configuration on which products can be placed, such as sensing-function-equipped shelf boards SHB1, SHB2, SHB3, and SHB4. The second support portion FR2 is a base-like member that is provided from the front side FR to the rear side BA and supports the first support portion FR1.

[0019] If the floor surface of the store ST on which the second support portion FR2 is placed is considered to be a horizontal plane, the sensing shelf board SHB is inclined downward from the rear side BA toward the front side FR relative to the horizontal plane. A first direction Dx, a second direction Dy, and a third direction Dz are defined as orientations of the surface of the inclined sensing shelf board SHB. The first direction Dx and the second direction Dy are two directions along the surface of the inclined sensing shelf board SHB, and are perpendicular to the first direction Dx and the second direction Dy. The third direction Dz is a direction perpendicular to the surface of the inclined sensing shelf board SHB. The tip of the arrow indicating the third direction Dz indicates the side (upper side) of the sensing shelf board SHB on which products are placed. In Figure 3, the product placed on shelf board SHB2 with sensing function is product PAC1, the product placed on shelf board SHB3 with sensing function is product PAC2, and the product placed on shelf board SHB4 with sensing function is product PAC3. Hereinafter, when product PAC is written, it refers to products such as product PAC1, PAC2, and PAC3.

[0020] FIG. 4 is a schematic layer diagram showing the main components of the sensing shelf board SHB. The sensing shelf board SHB includes a light source 7, a light guide plate 2, an end member 79, a member 89, a planar optical sensor 81, a shelf board BOA, and a support member SUP. The light source 7 is a light-emitting element that emits light. Specifically, the light source 7 is, for example, an LED (Light Emitting Diode), but is not limited to this and may have other configurations that emit light in response to power supply. The light guide plate 2 is a plate-shaped optical member that is translucent and light-scattering. The plate surface of the light guide plate 2 is aligned with the first direction Dx and the second direction Dy. The light guide plate 2 guides light from the light source 7, which is located on one side of the light guide plate 2 in the first direction Dx, toward the product PAC. The light guide plate 2 also transmits light from the product PAC and guides it toward the member 89. The end member 79 is located on the opposite side of the light guide plate 2 from the light source 7. The end member 79 is an optical member whose light reflectance is adjusted so that the light from the light source 7 is more uniform across the entire surface of the light guide plate 2. The light source 7 and the end member 79 are disposed on either side of the light guide plate 2, with the light guide plate 2 sandwiched between them.

[0021] The member 89 is an optical member interposed between the light guide plate 2 and the planar optical sensor 81. The member 89 limits the traveling direction of light from the light guide plate 2 to the planar optical sensor 81. The traveling direction of light limited by the member 89 is determined by the direction of holes formed in the member 89, which is made of a light-blocking material. For example, the member 89 in this embodiment is a light-blocking material having a plurality of fine holes penetrating along the third direction Dz. Light emitted from the light source 7 is guided to the product PAC by the light guide plate 2. The light is reflected by the surface of the product PAC facing the light guide plate 2, passes through the light guide plate 2, and reaches the member 89. A portion of the light that reaches the member 89 is blocked by the member 89, and the remaining portion passes through the holes in the member 89 and reaches the planar optical sensor 81. Although a gap is present between the light guide plate 2 and the member 89 in FIG. 4, this gap is not necessary.

[0022] The planar optical sensor 81 is a planar light detection unit in which a plurality of light detection elements are arranged along a Dx-Dy plane. The Dx-Dy plane is a plane extending along the first direction Dx and the second direction Dy.

[0023] FIG. 5 is a block diagram showing the configuration related to the operation of the planar optical sensor 81 and the functional configuration related to information processing performed based on light detected by the planar optical sensor 81. The planar optical sensor 81 includes a plurality of photodetection elements arranged in a matrix, such as the photodetection element 812 shown in FIG. 5. Each of the photodetection elements is, for example, a photodiode formed on a substrate 811, but other photodetection elements may be used. The sensing area AA indicates the area in the planar optical sensor 81 where a plurality of photodetection elements, such as the photodetection element 812, are arranged. The planar optical sensor 81 that detects light in the sensing area AA functions as a so-called planar optical sensor. The sensing area AA of the planar optical sensor 81 faces the component 89. As described with reference to FIG. 4, the light that reaches the planar optical sensor 81 is light reflected by the surface of the product PAC facing the light guide plate 2. In other words, the planar optical sensor 81 detects light to detect a product PAC placed within the sensing area AA of the shelf board SHB with sensing function.

[0024] The substrate 811 is the substrate of the planar optical sensor 81. Furthermore, gate line driving circuits 814A and 814B, a signal line driving circuit 815A, and an ROIC 816 are mounted on the substrate 811. The gate line driving circuits 814A and 814B are gate drivers that provide driving signals to the gates of the photodetector elements 812. The gate line driving circuits 814A and 814B are connected to the photodetector elements 812 via gate lines (not shown). The signal line driving circuit 815A is a source driver that transmits the output from the photodetector elements 812 to the ROIC 816. The signal line driving circuit 815A is connected to the photodetector elements 812 via signal lines (not shown). When a driving signal is provided from the gate driver, the photodetector elements 812 output a signal corresponding to the degree of light detection to the signal line. The output is output to the detection device 10 via the signal line driving circuit 815A and the ROIC 816. The ROIC 816 is a so-called readout integrated circuit. The ROIC 816 controls the operation of the gate driver and the source driver to read out a signal indicating the degree of light detection by the photodetection element 812. The ROIC 816 also controls the operation of the gate driver and the source driver related to the reset operation of the photodetection element 812.

[0025] 4 is a plate-shaped member. The support member SUP supports the light source 7, the member 89, and the planar optical sensor 81 on the upper surface side of the shelf board BOA. The support member SUP has, for example, a frame-like shape that surrounds the sensing area AA when viewed from a plan view, but may also be a member that covers two sides of the sensing area AA that face each other in the first direction Dx.

[0026] The sensing shelf boards SHB1, SHB2, SHB3, and SHB4 shown in FIG. 3 are each a sensing shelf board SHB. Therefore, it can be said that a shelf SH is provided with multiple sensing shelf boards SHB. Furthermore, the light source 7, light guide plate 2, and planar optical sensor 81 shown in FIG. 4 constitute the sensor unit 11 of the detection device 10 shown in FIG. 1. Therefore, it can be said that one sensing shelf board SHB has one sensor unit 11. In other words, it can be said that the shelf SH has multiple sensor units 11. In FIG. 1, the detection device 10 includes one block of the sensor unit 11, but this description does not limit the detection device 10 to one sensor unit 11. Multiple sensor units 11 may be provided, as in the case of shelf SH. Furthermore, by installing multiple shelves SH, such as shelves SH1 and SH2, even more sensor units 11 may be included in the configuration of the detection device 10.

[0027] 4, an intervening portion 129 is further provided between the light guide plate 2 and the product PAC. The intervening portion 129 is, for example, a light-transmitting protective film for the light guide plate 2, but a weight sensor 125, which will be described later, may also be provided at the position of the intervening portion 129. In the embodiment, the surface of the intervening portion 129 facing the product PAC functions as the placement surface of the shelf board SHB with sensing function for placing the product PAC, i.e., as the placement surface of the shelf.

[0028] As described above, the light source 7, the light guide plate 2, and the planar optical sensor 81 shown in Fig. 4 constitute the sensor unit 11 of the detection device 10 shown in Fig. 1. The detection device 10 shown in Fig. 1 includes the sensor unit 11, a trigger generation unit 12, and an information processing unit 20. The trigger generation unit 12 generates an output that functions as a trigger to operate the sensor unit 11. The trigger generation unit 12 will be described in detail later.

[0029] 1, information processing unit 20 includes calculation unit 21, communication unit 22, input unit 23, output unit 24, memory unit 25, etc. Calculation unit 21 has a calculation circuit that functions as a CPU (Central Processing Unit), and reads out software programs such as information processing program 26 stored in memory unit 25 and data referenced in the execution of the software programs (hereinafter, referred to as programs, etc.) and executes and processes them.

[0030] The communication unit 22 functions as a NIC for communicating with the first gate 40, the second gate 50, the server 60, and the terminal 90 via the communication network NW. The input unit 23 has one or more input devices, such as a keyboard, a mouse, a touch panel, and the like, and receives input operations from the administrator of the information processing unit 20. The output unit 24 has one or more output devices, such as a display, a speaker, a printer, and the like, and performs output according to the processing content of the calculation unit 21.

[0031] The storage unit 25 has a hard disk drive (HDD), a solid state drive (SSD), and other storage devices, and stores various programs, etc. In Fig. 1, an information processing program 26 and a first DB 27 are illustrated as programs, etc., stored in the storage unit 25. Note that "DB" in the first DB 27 and a second DB (described later) stands for database.

[0032] 5 functions by the calculation unit 21 executing the information processing program 26 stored in the storage unit 25. The processing unit 30 functions as a trigger detection unit 31, a timing control unit 32, an image analysis unit 33, a determination unit 34, a DB processing unit 35, a settlement processing unit 36, and an entry / exit processing unit 37. When performing the functions of the DB processing unit 35, the settlement processing unit 36, and the entry / exit processing unit 37, the processing unit 30 refers to the first DB 27.

[0033] As shown in FIG. 5 , the information processing unit 20 further includes a first operation control circuit 28 and a second operation control circuit 29. The first operation control circuit 28 and the second operation control circuit 29 are circuits that operate under the control of the timing control unit 32. The first operation control circuit 28 outputs a command to the ROIC 816 to operate the planar optical sensor 81. The second operation control circuit 29 outputs a command to the light source drive circuit 74 to operate the light source 7. The light source drive circuit 74 is a circuit that controls the lighting of the light source 7. For example, if the light source 7 is an LED, the light source drive circuit 74 is an LED driver. The light source 7 is turned on or off under the control of the light source drive circuit 74.

[0034] When the sensor unit 11 operates, the light source 7 is turned on and the planar optical sensor 81 detects light. An output according to the degree of light detection by the planar optical sensor 81 is then output to the information processing unit 20 via the signal line driving circuit 815A and the ROIC 816. Hereinafter, unless otherwise specified, the term "sensing" refers to the operation of the sensor unit 11 that obtains such an output.

[0035] As described above, information processing by the detection system 1 (see FIG. 1) is used to manage products in the store ST (see FIG. 2). Specifically, the operation of the sensor unit 11 is involved in detecting products such as product PAC on a shelf board SHB (see FIG. 4) with a sensing function provided on a shelf SH (see FIG. 3). Product detection by the operation of the sensor unit 11 will be described below with reference to FIGS. 6 and 8.

[0036] 6 is a schematic diagram showing an example of the state of the sensing shelf SHB and the products on the sensing shelf SHB before and after a change in the number of products placed thereon. In the example shown in FIG. 6, the products placed within the sensing area AA of the sensing shelf SHB are shown as product P1 and product P2.

[0037] Both product P1 and product P2 are a form of product PAC. Product P1 and product P2 are both packaged in rectangular parallelepiped packages, but have different dimensions. Specifically, when product P1 and product P2 are placed so that their longest sides are aligned in the first direction Dx, the length P1x of product P1 in the first direction Dx is longer than the length P2x of product P2 in the first direction Dx. In this case, the lengths of the two sides of product P1 other than the longest side are also longer than the lengths of the sides of product P2. Figure 6 shows that the length P1y of product P1 in the second direction Dy is longer than the length P2y of product P2 in the second direction Dy.

[0038] 6, the length AAx of the sensing area AA in the first direction Dx and the length AAy of the sensing area AA in the second direction Dy are significantly larger than the lengths P1x, P1y, P2x, and P2y. That is, the sensing area AA of the planar optical sensor 81 of the embodiment has a sufficient area to accommodate multiple products PAC.

[0039] In the "Before Change" column of Figure 6, the "State" column indicates that three products P1 and five products P2 are placed within the sensing area AA. As described with reference to Figure 4, products PAC such as products P1 and P2 placed within the sensing area AA of the shelf board SHB with sensing function can be detected by detecting light using the planar optical sensor 81. Therefore, when sensing is performed in this "state" "before change," an output corresponding to the state in which three products P1 and five products P2 are placed within the sensing area AA is generated from the sensor unit 11, as shown in the "Sensing Result" column of the "Before Change" column.

[0040] Meanwhile, in the "After Change" column of FIG. 6, the "State" column indicates that three products P1 and four products P2 are placed within the sensing area AA. When sensing is performed in this "state" of "After Change," as shown in the "Sensing Result" column of the "After Change" column, an output corresponding to the state in which three products P1 and four products P2 are placed within the sensing area AA is generated from the sensor unit 11. Here, the "change" in the expressions "Before Change" and "After Change" refers to a change in the state of the products placed on the shelf board SHB with sensing function (e.g., a decrease in the number). This change in the state of the products appears as a change in the sensing result, making it possible for the processing unit 30 to detect it.

[0041] In the "after change" state of FIG. 6, the number of products P2 has decreased by one compared to the "before change" state. This is also reflected in the output of the sensor unit 11 shown in the "sensing result" column. Therefore, when the output corresponding to the "sensing result" "before change" is compared with the output corresponding to the "sensing result" "after change," a difference corresponding to the decrease in the number of products P2 is generated. In the "difference" column of FIG. 6, the portion corresponding to the product P2 that was present within the sensing area AA "before change" but is no longer present within the sensing area AA "after change" is shown as a dashed rectangle SP2.

[0042] 6 shows an example in which the dashed rectangle SP2 can be extracted as a simple image difference, but the positions of multiple products PAC such as products P1 and P2 placed in the sensing area AA may change. Therefore, in the embodiment, a mechanism is provided to determine the type of product PAC and the number of each type of product PAC regardless of the positions of the multiple products PAC, and to compare and determine the results before and after the change.

[0043] Fig. 7 is a schematic diagram showing a method for identifying a product PAC. Product PP, product PQ, and product PR shown in Fig. 7 are different types of product PAC. When multiple types of product PAC are mixed in the sensing area AA, methods for identifying various types of product PAC include the shape and dimensions of one side of the product PAC facing the sensing area AA, and the image pattern of one side of the product PAC facing the sensing area AA.

[0044] For example, there is a method for identifying a product PAC based on the shape and dimensions of a surface of the product PAC facing the sensing area AA in a planar view. Product PACs with different shapes in a planar view can be identified as different types of product PACs. Even if multiple types of product PACs have the same shape, the type of product PAC can be identified based on the dimension of the longest side of the shape in a planar view and the dimension in a direction perpendicular to the dimension of the side. In the "Example" in the "Shape and Dimensions" column of FIG. 7, the length Py of the product PP is the dimension of the side, and the length Px of the product PP is the dimension in a direction perpendicular to the dimension of the side. The products P1 and P2 described with reference to FIG. 6 both have rectangular shapes in a planar view, but are identified based on the difference between the length P1y and the length P2y, which correspond to the dimension of the side, and the difference between the length P1x and the length P2x, which correspond to the dimension in a direction perpendicular to the dimension of the side.

[0045] There is also a method for identifying a product PAC based on the image pattern of one side of the product PAC facing the sensing area AA. Product PACs with different image patterns can be identified as different types of product PAC. In the "Example" of FIG. 7, an example in which the image pattern of product PQ is characterized by the character string "BCD" is shown as "in-package image pattern 1." Also, an example in which the image pattern of product PR is characterized by a row of rectangles resembling a one-dimensional barcode is shown as "in-package image pattern 2." Products PQ and PR are identified by differences in image patterns, such as the difference in features between "in-package image pattern 1" and "in-package image pattern 2."

[0046] In identifying product PACs, the identification process may be completed if the product can be identified using at least one of the features of shape, size, and image pattern. Therefore, products that can be identified using just one of these features may be identified at that point. Furthermore, if multiple features can be combined to improve the accuracy of identification, they may be combined.

[0047] As described with reference to FIG. 7 , the planar optical sensor 81 has a resolution capable of identifying the shape of the surface facing the planar optical sensor 81 of each of the multiple products PAC placed on the sensing shelf SHB, as well as the characters, identifiers, and patterns on that surface. This resolution is achieved by the multiple photodetector elements provided in the planar optical sensor 81. Specifically, this resolution is achieved by the photodetector elements 812 arranged in a matrix, as described with reference to FIG. 5 . The identifier referred to here refers to, for example, a barcode such as the "in-package image pattern 2." Other identifiers not shown in FIG. 7 , such as two-dimensional barcodes, are also included in the term "identifier." Even patterns that are not intended to function as identifiers can be used as image patterns to identify products.

[0048] In this embodiment, first, an image pattern of the surface of the product PAC facing the sensing area AA is extracted by comparing it with a blank image pattern in which nothing is placed in the sensing area AA. Then, based on the shape, dimensions, and features contained in the extracted image pattern, the type of product PAC that each image pattern represents is identified. By performing this identification, even if product PACs of the same type are placed in the sensing area AA but in different orientations, product PACs of the same type can be identified as the same type and each type of product PAC can be counted. Furthermore, different product PACs can be identified as different types and each type of product PAC can be counted individually.

[0049] 8 is a schematic diagram showing an example in which the arrangement of some or all of the product PACs in the sensing area AA does not match before and after the change. By identifying the product PACs with reference to FIG. 7, even if the arrangement of some or all of the product PACs in the sensing area AA does not match before and after the change, it is possible to determine the type of product PAC and the number of each type of product PAC and then detect the change in the product PACs placed in the sensing area AA.

[0050] The number of products P1 and the number of products P2 in the sensing area AA shown in the "sensing result" "before change" in FIG. 8 are the same as the "sensing result" "before change" described with reference to FIG. 6. In the "first example" of the "sensing result" "after change" in FIG. 8, the number of products P1 and the number of products P2 in the sensing area AA have not changed from the "sensing result" "before change", but the arrangement of products P1 and P2 has changed. However, by identifying products PAC described with reference to FIG. 7, in the embodiment, it can be detected that the number of products P1 and the number of products P2 in the sensing area AA have not changed, even if the arrangement of products P1 and P2 has changed.

[0051] Furthermore, in the "sensing results" "after change" in Figure 8, "Example 2," "Example 3," and "Example 4," not only has the arrangement of product P1 and product P2 changed, but at least one of the number of products P1 and the number of products P2 has also changed. Specifically, in "Example 2," the number of products P1 has decreased by one compared to "before change." In "Example 3," the number of products P1 and the number of products P2 have each decreased by one compared to "before change." In "Example 4," the number of products P1 has increased by one compared to "before change." In these examples as well, the change in number can be detected by identifying product PAC, as described with reference to Figure 7.

[0052] The identification of the type and number of product PACs as described with reference to Figures 6 to 8 is performed by the image analysis unit 33 of the processing unit 30 shown in Figure 5. Then, the determination unit 34 compares the sensing results before and after the change, and detects a change in the type and number of product PACs placed in the sensing area AA.

[0053] Of the sensing results before and after the change, the sensing result after the change is acquired when the trigger detection unit 31 of the processing unit 30 detects a trigger output from the trigger generation unit 12. In FIG. 1, the trigger generation unit 12 is exemplified by an imaging unit 121 and a weight sensor 125.

[0054] As shown in FIG. 2, the imaging unit 121 is an imaging device that captures moving images and is installed to capture a product movement detection range CA, which is the range on the front side FR of a shelf SH, such as shelves SH1 and SH2. The imaging unit 121 has an imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and outputs moving image data by periodically outputting light detected by the image sensor as image data. In the example shown in FIG. 2, a customer HC2 enters the product movement detection range CA on the front side FR of shelf SH1. When the imaging unit 121 captures an image of any product PAC within the product movement detection range CA, the captured image data output from the imaging unit 121 is treated as a trigger. More specifically, the trigger detection unit 31 shown in FIG. 5 performs pattern matching between the captured image data output from the imaging unit 121 and pre-prepared image data of the product PAC, and treats the captured image data as a trigger when it is determined that the captured image data includes image data of the product PAC.

[0055] When the trigger detection unit 31 shown in FIG. 5 detects a trigger, the trigger detection unit 31 operates the timing control unit 32. Under the control of the trigger detection unit 31, the timing control unit 32 controls the operation timing of the first operation control circuit 28 and the second operation control circuit 29. When the timing control unit 32 operates, the light source 7 is turned on under the control of the second operation control circuit 29, and the planar optical sensor 81 functions as a planar optical sensor and senses the sensing area AA under the control of the first operation control circuit 28. The sensing data obtained as a result of this sensing is output to the image analysis unit 33. The image analysis unit 33 identifies the type and number of commodity PACs as described with reference to FIGS. 6 to 8. Then, the determination unit 34 compares the sensing results before and after the change and detects a change in the type and number of commodity PACs placed in the sensing area AA.

[0056] 2 is an object (obstacle) that prevents customers HC from entering the side of the shelf SH. An additional imaging unit 121 may be provided to add a product movement detection range CA to the range where the no-entry area DS is located.

[0057] Although the case where the output of the imaging unit 121 is treated as the trigger has been described above, the output of the weight sensor 125 may also be treated as the trigger. When the weight sensor 125 is employed, the weight sensor 125 is provided on the shelf SH described with reference to FIG. 3. The weight sensor 125 is provided, for example, between the second support portion FR2 and the floor of the store ST, and detects changes in the weight of the shelf SH. When a product PAC placed on the shelf SH is taken out or a product PAC is added to the shelf SH, the weight of the shelf SH changes. This change in weight is detected by the weight sensor 125. The weight sensor 125 generates an output indicating that a weight change has been detected. This output is treated as a trigger by the trigger detection unit 31. Therefore, the same processing flow occurs as when image data captured by the imaging unit 121 is treated as the trigger.

[0058] When the determination unit 34 determines that there has been a change in the type and number of products PAC placed within the sensing area AA, the DB processing unit 35 shown in Fig. 5 accesses the first DB 27. The contents of the data managed in the first DB 27 and the second DB 66 of the server 60 will be described below with reference to Figs. 9 to 14.

[0059] FIG. 9 is a diagram showing the configuration of the second DB 66. The second DB 66 includes product data 661 and customer data 662. The server 60 functioning as the second DB 66 is an information processing device that is capable of communicating with the detection device 10 via the communication network NW. The calculation unit 61 has the same configuration as the calculation unit 21. The communication unit 62 has the same configuration as the communication unit 22. The input unit 63 has the same configuration as the input unit 23. The output unit 64 has the same configuration as the output unit 24. The memory unit 65 has the same configuration as the memory unit 25, but the contents stored therein differ from those of the memory unit 25. The memory unit 65 stores programs and the like for functioning as the second DB 66.

[0060] 10 is a schematic diagram showing an example of the contents of the product data 661. The product data 661 is data in a table format having columns such as product identification information, product type, optical identification pattern, and optical identification pattern reference path. Data on products handled by multiple stores ST that share the server 60 is registered in the product data 661. One record (row) of the product data 661 is set for one product.

[0061] Each product identification information field (box) is set with a unique character string that identifies the product. For example, "product 131" and "product 251" are unique character strings that each identify a different product. The same applies to other character strings set in product identification information fields.

[0062] The product type field contains information indicating the type of product. For example, "type A1," "type B1," "type C1," and "type E1" indicate that they are different types of products. Furthermore, "type A1" is set in the product type field of a record in which "product 131," "product 132," or "product 133" is set in the product identification information field. This indicates that "product 131," "product 132," and "product 133" are the same type of product.

[0063] The optical identification pattern field contains a character string indicating the name given to the optical identification pattern corresponding to the type of product PAC. The optical identification pattern reference path field contains a file path in which optical identification pattern data corresponding to the name set in the optical identification pattern field is stored. The file path also refers to a storage area in the storage unit 65 of the server 60. That is, the data (optical identification pattern) indicated by the file path is stored in the storage unit 65. The data of the "image pattern of a certain type of product PAC" contained in the sensing result and the data of the "certain type" of optical identification pattern are determined to match in terms of pattern matching. As a specific example, assume that a "type A1" product PAC is sensed within the sensing area AA. In this case, the image pattern contained in the sensing result and the optical identification pattern data obtained by referencing the optical identification pattern path of the record in the product data 661 shown in FIG. 10 where the product type is "type A1" are determined to match in product identification by the image analysis unit 33.

[0064] 11 is a schematic diagram showing an example of the contents of customer data 662. The customer data 662 is data in a table format with columns for customer unique number, name, and payment information. One record of customer data 662 is set for one customer HC.

[0065] The customer unique number field contains a unique number assigned to each customer HC. In other words, the number (customer unique number) set in the customer unique number field is different for each customer HC. In the embodiment, the customer unique number corresponds to the identification information stored in the identification information storage unit 99. The name field contains a character string indicating the name of the customer HC. The payment information field contains information indicating the service used by the customer HC to settle (pay) for the product.

[0066] Fig. 12 is a diagram showing the configuration of the first DB 27. The first DB 27 includes product inventory data 271 and store-specific customer data 272. The second DB 66 described with reference to Figs. 9 to 11 includes data referenced by the detection devices 10 of multiple store STs that share the server 60. In contrast, the first DB 27 includes data necessary for managing products handled by the detection device 10 of a single store ST.

[0067] 13 is a schematic diagram showing an example of the contents of the product inventory data 271. The product inventory data 271 is data in a table format having columns for product identification information, location, and price. One record (row) of the product inventory data 271 is set for one product.

[0068] A unique character string identifying each product is set in the product identification information field. If the character string set in the product identification information field of the product data 661 described with reference to Fig. 10 is the same as the character string set in the product identification information field of the product inventory data 271 shown in Fig. 13, the product PAC managed by the record of the product data 661 is the same as the product PAC managed by the record of the product inventory data 271.

[0069] The location field contains information indicating where each product is currently located. For example, a product PAC in a record with "Purchaser 120" set in the location field indicates that it is held by a customer HC managed by "Purchaser 120." Furthermore, a product PAC in a record with "Shelf 130" set in the location field indicates that it is placed on a shelf SH managed by "Shelf 130." Of the character strings set in the location field, the character string indicating a specific customer HC corresponds to the character string set in the in-store information field of customer data 662, which will be described later.

[0070] The price field contains a numerical value that indicates the price of each product. For example, if the price field of a record contains "220", the amount required at the time of payment for a product PAC will be 220 yen. The currency unit of the numerical value can be changed as needed.

[0071] 14 is a schematic diagram showing an example of the contents of the store-specific customer data 272. The store-specific customer data 272 is data in a table format having columns for in-store information, a customer unique number, a total amount, and a payment flag. One record of the store-specific customer data 272 is set for one customer HC.

[0072] The in-store information field contains a unique character string assigned to each customer HC and managed within a single store ST. The customer unique number field contains a unique number assigned to each customer HC. If the number set in the customer unique number field of the product data 661 described with reference to FIG. 10 is the same as the number set in the customer unique number field of the product inventory data 271 shown in FIG. 13, the customer HC managed by the record of the product data 661 is the same as the customer HC managed by the record of the product inventory data 271. In other words, the customer data 662 is a DB for managing the "in-store information" character string, which is exclusive to each store ST, by assigning it to the customer HC.

[0073] The total amount field contains a value indicating the amount required for payment of the product PAC held by the customer HC managed in each record of the customer data 662. When the character string set in the location field of a record of the product data 661 indicates the customer HC, there is a correlation between the price in the record of the product data 661 and the total amount in the record of the customer data 662. For example, in the example shown in FIG. 13, there are three records in which the character string set in the location field is "Purchaser 120." The numerical values ​​in the price field of these three records are 220, 990, and 440, respectively. The sum of these three numerical values ​​is 1650. Here, in the example shown in FIG. 14, the numerical value set in the total amount field of the record in which the character string set in the store information field is "Purchaser 120" is 1650. Therefore, the numerical value set in the total amount field of a certain record corresponds to the sum of the prices of the products held by the customer HC managed in that record.

[0074] The payment flag field contains a flag value that indicates whether payment for the product held by the customer HC has been completed. The flag value is managed using two values, for example, 0 or 1. A flag value of 0 indicates that payment is incomplete. A flag value of 1 indicates that payment has been completed. Note that the contents indicated by each flag value may be reversed, as long as payment for the product can be managed using the flag value.

[0075] The above has described the information managed in the second DB 66 and the first DB 27, but other information may also be used in the information processing by the detection system 1. For example, data (purchaser data) generated by the arithmetic circuit 91 of the terminal 90 individually owned and used by each customer HC executing and processing the information processing program 96 stored in the arithmetic unit 95 may also be used.

[0076] 15 is a schematic diagram showing an example of the contents of purchaser data. The purchaser data is data in a table format having columns for a customer unique number and car contents information.

[0077] The customer unique number field contains a number unique to the customer HC that owns the terminal 90 in which the purchaser data is stored. If the number set in the customer unique number field of the product data 661 described with reference to Figure 10, the number set in the customer unique number field of the product inventory data 271 described with reference to Figure 13, and the number set in the customer unique number field of the purchaser data shown in Figure 15 are the same, the customer HC managed by these records is the same.

[0078] The cart contents information field contains a character string indicating a product held by the customer HC who owns the terminal 90 in which the purchaser data is stored. If the character string set in the product identification information of the product inventory data 271 described with reference to FIG. 13 is the same as the character string set in the cart contents information field of the purchaser data shown in FIG. 15, the product PACs managed by these records are the same. Furthermore, the information on the product PACs managed by the records of the purchaser data corresponds to the information set in the location field of the product PACs managed by the records of the product inventory data 271 described with reference to FIG. 13. For example, in the example shown in FIG. 13, there are three records in which the character string set in the location field is "Purchaser 120." The character strings "Product 131," "Product 251," and "Product 382" are set in the product identification information fields included in these three records, respectively. These character strings correspond to the character strings set in the cart contents information fields of the three records of the purchaser data shown in FIG. 15. Furthermore, the number set in the customer unique number field of the purchaser data shown in FIG. 15 is "9123456780." 14, the store-specific customer data 272 has a record in which the customer unique number field is set to "9123456780," and the store-specific customer data 272 contains a record in which the customer unique number field is set to "9123456780." In this way, the purchaser data is correlated with the product inventory data 271 and the store-specific customer data 272.

[0079] The terminal 90 is a compact information processing device that the customer HC can carry around, such as a smartphone. The arithmetic circuit 91 is composed of a circuit with arithmetic processing capabilities, such as an SoC (System on a Chip). The communication unit 92 functions similarly to the communication unit 22. While the arithmetic circuit 91 and the communication unit 92 are illustrated separately in FIG. 1, the part functioning as the communication unit 92 may be packaged together in the arithmetic circuit 91 or may be configured as an independent circuit. The input unit 93 and the output unit 94 are configured, for example, like a touch panel display, by integrating a display output function corresponding to the processing content of the arithmetic circuit 91 and an input acceptance function that allows touch operations corresponding to the display output, but the specific configuration may be modified as appropriate. The arithmetic unit 95 is functionally similar to the memory unit 25, but is notable in that it is generally configured using flash memory. The identification information storage unit 99 is as described above.

[0080] Hereinafter, a case where the position of a product PAC placed on a shelf SH is moved by a customer HC will be described with reference to FIGS.

[0081] FIG. 16 is a schematic diagram showing an example in which the position of a product PAC placed on a shelf SH is moved by a customer HC. The "first pattern" shown in FIG. 16 shows a case in which the customer HC takes the product 141 off the shelf 130 and holds it. The basket BAS is a hand-held basket used by the customer HC to hold the product PAC. The product 141 is a product PAC for which the character string set in the product identification information field in the product inventory data 271 described with reference to FIG. 13 is "product 141." The shelf 130 is a shelf board SHB with a sensing function provided on a shelf SH for which the information set in the position field in the product inventory data 271 is "shelf 130." The customer HC shown in FIG. 16 is a customer HC for which the character string set in the in-store information field is "purchaser 120."

[0082] The "second pattern" shown in Figure 16 shows a case where the product 141 held by the customer HC in the "first pattern" is placed on the shelf 140. The shelf 140 is a shelf board SHB with a sensing function provided on the shelf SH whose information set in the position field in the product inventory data 271 is "shelf 140."

[0083] Below, information updates when the "first pattern" and "second pattern" shown in FIG. 16 occur will be described with reference to FIGS. 17 to 20. First, information updates when the "first pattern" occurs will be described with reference to FIGS. 17 to 19. It is assumed that the second DB 66 before the "first pattern" occurs has the contents shown in FIGS. 13 and 14. It is also assumed that the purchaser data before the "first pattern" occurs has the contents shown in FIG. 15.

[0084] Figure 17 is a diagram showing an example of the contents of product inventory data 271 after the "first pattern" has occurred. In product inventory data 271 before the "first pattern" has occurred, the "position" of a record whose product identification information is "product 141" was "shelf 130", as shown in Figure 13. In contrast, in product inventory data 271 after the "first pattern" has occurred, the "position" of a record whose product identification information is "product 141" has become "purchaser 120", as shown in Figure 17.

[0085] FIG. 18 is a diagram showing an example of the contents of the store-specific customer data 272 after the occurrence of the "first pattern." When the "first pattern" occurs, the customer HC holds the product 141 with the value "350" set in the price field, and the value 350 is added to the total amount field in the record of that customer HC. Specifically, in the store-specific customer data 272 before the occurrence of the "first pattern," the "total amount" of that record was "1650," as shown in FIG. 14. In contrast, in the store-specific customer data 272 after the occurrence of the "first pattern," the "total amount" of that record is "2000," as shown in FIG. 18.

[0086] FIG. 19 is a diagram showing an example of the contents of purchaser data after the occurrence of the "first pattern." The number of records in the purchaser data before the occurrence of the "first pattern" was three, as shown in FIG. 15, and the character strings set in the cart contents information fields were "product 131," "product 251," and "product 382," respectively. In contrast, the number of records in the purchaser data after the occurrence of the "first pattern" is four, as shown in FIG. 19, and the character strings set in the cart contents information fields include "product 141" in addition to "product 131," "product 251," and "product 382" from before the occurrence of the "first pattern."

[0087] Next, the update of information when a "second pattern" occurs after the above-mentioned "first pattern" will be described with reference to FIG.

[0088] Figure 20 is a diagram showing an example of the contents of product inventory data 271 after the occurrence of the "second pattern." In the product inventory data 271 before the occurrence of the "second pattern," the "position" of the record whose product identification information is "product 141" was "purchaser 120," as shown in Figure 17. In contrast, in the product inventory data 271 after the occurrence of the "second pattern," the "position" of the record whose product identification information is "product 141" is "shelf 140," as shown in Figure 17.

[0089] The contents of the store-specific customer data 272 after the "second pattern" occurs will be the same as the contents shown in Figure 14. The contents of the purchaser data after the "second pattern" occurs will be the same as the contents shown in Figure 15.

[0090] The processing related to updating the various pieces of information included in the first DB 27, as described with reference to Figures 17, 18, and 20, is performed by the DB processing unit 35 shown in Figure 5. Furthermore, in the "first pattern" and "second pattern" described with reference to Figure 16, an increase or decrease occurs in the number of products PAC placed on the sensing shelf boards SHB of the shelf SH, i.e., the number of products PAC detected within the sensing area AA, and therefore the above-mentioned sensing and processing occurs by the trigger detection unit 31, timing control unit 32, image analysis unit 33, and determination unit 34. Therefore, it can be said that the update of the first DB 27 by the DB processing unit 35 reflects the result of the determination by the determination unit 34.

[0091] Identifying who moved which item may be performed, for example, by the trigger generation unit 12, which captures an image of the customer HC when the customer HC enters the item movement detection range CA shown in FIG. 2 and removes the item 141 from the shelf SH. Specifically, the customer HC may be identified based on an image of the customer HC captured within the image capture range of the image capture unit 121 and an image of the customer HC captured by the entry detection unit 43 when the customer HC enters the ST. Furthermore, the operation of the first gate 40 and the second gate 50 may be controlled to limit the number of customers HC allowed to enter the store ST at one time, thereby limiting the number of customers HC allowed to move items PAC within the store ST to one. Furthermore, the customer HC may independently update their purchaser data using the terminal 90 to reflect the addition of the item 141 to the basket BAS, as shown in FIG. 19.

[0092] When a customer HC takes a product PAC out of the store ST, i.e., when the customer HC purchases a product PAC, a payment process should be performed. The payment process may use a payment system (not shown), such as a payment process using electronic money of a predetermined standard, or another payment process method predetermined within the store ST. Regardless of the payment process, the payment process that can be used when taking a product PAC out of the store ST is predetermined. Furthermore, a system is configured that allows the payment processing unit 36 ​​of the processing unit 30 to check whether the payment process has been completed via the communication network NW. When information indicating that payment for the product PAC held by the customer HC has been completed via the communication network NW is received, the payment processing unit 36 ​​performs an update process to change the payment flag of the record of the customer HC in the store-specific customer data 272 from 0 to 1. Furthermore, a customer HC whose total amount in the store-specific customer data 272 is not 0 and whose payment flag is set to 1, or whose total amount in the store-specific customer data 272 is 0, can be subject to the opening operation of the opening / closing mechanism 54 and can exit the store ST by having the second reading unit 52 of the second gate 50 shown in FIGS. 1 and 2 read the identification information storage unit 99 of the terminal 90. In other words, a customer HC whose total amount in the store-specific customer data 272 is not 0 and whose payment flag is set to 0 is not subject to the opening operation of the opening / closing mechanism 54 and cannot exit the store ST. The opening operation of the opening / closing mechanism 54 is controlled by the entrance / exit processing unit 37 of the processing unit 30 shown in FIG. 5. The entrance / exit processing unit 37 also controls the operation of the opening / closing mechanism 44 in conjunction with the operation of the first gate 40. The first gate 40 and the second gate 50 are connected to the detection device 10 via the communication network NW, and the opening / closing mechanisms 44 and 54 are operated under the control of the entrance / exit processing unit 37.

[0093] In this embodiment, the customer HC uses the type of payment system or payment method indicated by the payment information set in the record of the customer HC in the customer data 662. For example, the customer HC with the customer unique number "9123456780" shown in Figure 11 makes payments using a payment system called "XX Pay."

[0094] 2 is for enabling wireless communication within the store ST to be connected to the communication network NW. When communication between the terminal 90 and the detection device 10 is necessary, it is assumed that this is performed via the communication device NWS. Therefore, the customer HC can use payment via wireless communication even within the store ST. Of course, payment may also be performed using public wireless communication (for example, communication using a mobile communication system such as LTE (Long Term Evolution)) provided by the terminal 90.

[0095] Furthermore, the information unique to the customer HC in the purchaser data described with reference to FIGS. 15 and 19 corresponds to the in-store information in the store-specific customer data 272, but may be based on a customer unique number.

[0096] Figure 21 is a schematic diagram showing an example of the contents of purchaser data when the information unique to the customer HC in the purchaser data is a customer unique number. In the purchaser data shown in Figure 21, the customer unique number in the purchaser data described with reference to Figures 15 and 19 has been replaced with content corresponding to the in-store information in store-specific customer data 272. As described above, except for the points noted above, the purchaser data shown in Figure 21 is the same as the purchaser data described with reference to Figures 15 and 19.

[0097] Next, the flow of processing involved in managing merchandise in the store ST will be described with reference to the flowcharts shown in FIGS.

[0098] 22 is a flowchart showing the flow of processing when the payment processing is automatically performed when leaving the store ST. First, processing is performed when entering the store (step S1).

[0099] 23 is a flowchart showing the flow of the store entry process. When the customer HC enters the store ST through the first gate 40, that is, when the customer HC enters the first gate 40 (step S11), the first reading unit 42 acquires identification information (step S12). The process of step S12 is performed when the customer HC brings the terminal 90 close to the first reading unit 42. The entrance / exit processing unit 37 refers to the product data 661 through communication via the communication network NW, and verifies whether there is a record in the customer data 662 of the second DB 66 in which a customer unique number corresponding to the identification information acquired in the process of step S12 is set (step S13).

[0100] If it is determined that the customer data 662 of the second DB 66 contains a record in which a customer unique number corresponding to the identification information acquired in the processing of step S12 is set (step S14; Yes), the entrance / exit processing unit 37 acquires the record as customer data for the customer HC about to enter the store ST (step S15). The entrance / exit processing unit 37 sets a new record in the store-specific customer data 272 and sets the customer unique number of the record to the customer unique number corresponding to the identification information acquired in the processing of step S12. The entrance / exit processing unit 37 generates the new record as store customer data for the customer HC about to enter the store ST (step S16). Note that, in the processing of step S16, store information is automatically set based on predetermined rules. The initial values ​​of the total amount and the settlement flag are "0." The entrance / exit processing unit 37 also outputs information that functions as a command to operate the opening / closing mechanism 44 of the first gate 40 so that the customer HC about to enter the store ST can enter the store ST. This information is transmitted to the first gate 40 via the communication network NW. In response to the input of this information, the first gate 40 operates the opening / closing mechanism 44 to open the first gate 40 (step S17). Note that the order of the processing of step S16 and the processing of step S17 may be random, and may be performed in parallel, for example.

[0101] If it is determined that the customer data 662 of the second DB 66 does not contain a record in which a customer unique number corresponding to the identification information acquired in the processing of step S12 is set (step S14; No), a first exception processing is performed (step S18). The first exception processing is performed when the customer data 662 of the second DB 66 does not contain a record in which a customer unique number is set, that is, when the customer is deemed to be a new customer HC who has never used the store ST. In the first exception processing, a new record corresponding to the new customer HC is registered in the customer data 662, and various processing steps are performed so that the same processing steps as those of steps S15 to S17 can be applied to the new customer HC.

[0102] After the store entry process described with reference to Fig. 23, sensing is performed as shown in Fig. 22 (step S2). That is, when a new customer HC enters the store ST, the sensor unit 11 operates to identify the type and number of products PAC placed on the sensing shelf board SHB of the shelf SH. The sensing result obtained by the processing of step S2 functions as the "before change" sensing result described with reference to Figs. 6 and 8.

[0103] After the processing of step S2, if a trigger is generated (step S3; Yes), a product movement confirmation process is performed (step S4). The generation of a trigger in the processing of step S3 refers to an output by the above-mentioned trigger generation unit 12. For example, the output of captured image data by the imaging unit 121 and the output accompanying the detection of a change in weight by the weight sensor 125 correspond to the output by the trigger generation unit 12.

[0104] FIG. 24 is a flowchart showing the flow of the product movement confirmation process. Once it is confirmed that a trigger has been generated in the process of step S3, sensing is performed (step S21). The sensing result obtained in the process of step S21 functions as the "post-change" sensing result described with reference to FIGS. 6 and 8. The image analysis unit 33 analyzes the sensing result obtained in the process of step S2 and the sensing result obtained in the process of step S21, respectively, and identifies the type of product PAC and the number of each type of product PAC included in each sensing result. The determination unit 34 determines whether there has been an increase or decrease in the number of product PACs in the sensing area AA based on the analysis result by the image analysis unit 33 (step S22). Specifically, if the type of product PAC and the number of each type of product PAC match between the sensing result obtained in the process of step S2 and the sensing result obtained in the process of step S21, it is determined that there has been no increase or decrease in the number of product PACs in the sensing area AA. In other words, if the types of product PACs and the number of each type of product PAC do not match between the sensing results obtained in the processing of step S2 and the sensing results obtained in the processing of step S21, it is determined that there has been an increase or decrease in the number of product PACs in the sensing area AA.

[0105] If it is determined in the processing of step S23 that there is an increase or decrease in the number of products PAC in the sensing area AA (step S23; Yes), the product inventory data 271 and the store-specific customer data 272 are updated (step S24). To give a specific example, as shown in the above-mentioned FIGS. 17 and 18, the product inventory data 271 and the store-specific customer data 272 are updated when a customer HC takes a product PAC off the shelf SH and holds it. The specific content of this update corresponds to the specific content of the increase or decrease in the number of products PAC that has occurred in the sensing area AA. If it is determined in the processing of step S23 that there is no increase or decrease in the number of products PAC in the sensing area AA (step S23; No), the processing of step S24 is not performed.

[0106] Note that the product movement confirmation process (step S4) is not performed unless a trigger is generated in the process of step S3 shown in Fig. 22 (step S3; No). Note that unless a trigger is generated (step S3; No), the standby process of the processing unit 30 continues in a state where it is possible to respond to the process that would be performed if a trigger were generated (step S3; Yes).

[0107] If a customer HC enters the second gate 50 (step S5; Yes) after the product movement confirmation process described with reference to Fig. 24 or while a trigger is not generated (step S3; No), the first store exit process is performed (step S6). Note that unless a customer HC enters the second gate 50 (step S5; No), after the process of step S4, the process returns to step S2. In other words, the sensing result obtained in the process of step S2 after the process of step S4 occurs functions as the latest "before change" sensing result.

[0108] FIG. 25 is a flowchart showing the flow of the first store exit process. First, the second reading unit 52 acquires identification information (step S31). The process of step S31 is performed when the customer HC brings the terminal 90 close to the second reading unit 52. The entrance / exit processing unit 37 acquires, as store customer data, a record of the store-specific customer data 272 corresponding to the identification information acquired in the process of step S31, and also acquires, as store inventory data, a record of the product inventory data 271 corresponding to the identification information (step S32). The payment processing unit 36 ​​performs payment processing based on the data acquired by the entrance / exit processing unit 37 in the process of step S32 (step S33). As described above, it is assumed that this payment processing is predetermined. It is also assumed that a system is established that allows the payment processing unit 36 ​​of the processing unit 30 to check the completion of this payment processing via the communication network NW. When information indicating that payment for the product PAC held by the customer HC has been completed through the payment process is received via the communication network NW, the processing of step S33 is completed, and the payment flag of the record in the store-specific customer data 272 corresponding to the customer HC is updated to 1. After the processing of step S33, the entrance / exit processing unit 37 outputs information that functions as a command to operate the opening / closing mechanism 54 of the second gate 50 so that the customer HC who is about to leave the store ST can exit the store ST. This information is transmitted to the second gate 50 via the communication network NW. In response to the input of this information, the second gate 50 operates the opening / closing mechanism 54 to open the second gate 50 (step S34).

[0109] The process flow when the payment process is automatically performed when leaving the store ST has been described above with reference to Figures 22 to 25. Next, the process flow when the customer HC uses the terminal 90 to independently manage the products PAC that the customer HC takes out from the shelf SH will be described.

[0110] 26 is a flowchart showing the process flow when the customer HC independently manages the products PAC that the customer HC takes out from the shelf SH using the terminal 90. Even in this case, the process flow relating to the process of step S1 and the process of step S2 is the same as the process flow described with reference to FIG.

[0111] When the customer HC uses the terminal 90 to independently manage the products PAC that the customer HC takes out from the shelf SH, for example, the customer HC performs product registration processing (step S41) before the processing of step S3 described with reference to FIG. 22. The processing of step S41 is processing in which the type and number of products PAC that the customer HC has taken out from the shelf SH are registered in the terminal 90. By the processing of step S41, the purchaser data described with reference to FIG. 15 etc. is generated in the terminal 90, and information corresponding to the taken-out products PAC is registered in each record of the purchaser data. Note that in FIG. 26, the processing of step S3, step S4, and step S5 described with reference to FIG. 22 occur after step S41, but the order of the processing of step S3 and step S41 and the processing of step S3 is not specified.

[0112] As shown in the relationship between the processing of step S2 shown in FIG. 26 and the processing of step S21 (see FIG. 24) included in the processing of step S4, the sensor unit (sensor unit 11) operates before and after the trigger is generated. Here, if the component that generates the trigger is the imaging unit 121, the sensor unit (sensor unit 11) operates before and after the imaging unit 121 captures an image of the movement of products between the customer HC and the shelf SH in the product movement detection range CA. Here, as described with reference to FIGS. 6 and 8, it is assumed that the number of products has decreased after sensing compared to before sensing. In this case, as described with reference to FIGS. 16 to 19, products corresponding to the decreased number of products are treated as being held by the customer HC. Therefore, in an embodiment, the information processing unit 20 compares the sensing results from the sensor unit 11 before and after the movement of goods between the customer HC and the shelf SH is imaged by the imaging unit 121, and determines whether the type of goods PAC and the number of goods PAC of each type in the sensing area AA have changed before and after the image capture, and if the number of goods PAC in the sensing area AA has decreased, it treats the goods PAC corresponding to the decrease in the goods PAC placed in the sensing area AA as being held by the customer HC.

[0113] Furthermore, if the trigger generating component is the weight sensor 125, the sensor unit (sensor unit 11) operates before and after the imaging unit 121 detects a change in weight. In the embodiment, the information processing unit 20 compares the sensing results of the sensor unit 11 before and after the imaging unit 121 detects a change in weight, and determines whether the type of product PAC and the number of each type of product PAC in the sensing area AA have changed before and after the detection.

[0114] When the customer HC uses the terminal 90 to independently manage the product PAC that the customer HC takes out from the shelf SH, if the customer HC enters the second gate 50 during the processing of step S5 (step S5; Yes), the second store exit processing is performed (step S42).

[0115] 27 is a flowchart showing the flow of the second store exit process. First, the second reading unit 52 acquires identification information and purchaser data (step S51). The process of step S51 is performed when the customer HC brings the terminal 90 close to the second reading unit 52. The process of step S51 is notable compared to the process of step S31 described with reference to FIG. 25 in that the entrance / exit processing unit 37 acquires not only identification information but also purchaser data from the terminal 90. After the process of step S51, the process of step S32 described with reference to FIG. 25 is performed.

[0116] After the process of step S32, the settlement processing unit 36 ​​performs a process of checking whether the purchaser data acquired in the process of step S51 is consistent with the product inventory data 271 of the first DB 27 (step S52). For example, as in the product inventory data 271 described with reference to Fig. 13, the positions of three records whose product identification information is "product 131," "product 251," and "product 382" are "purchaser 120," as in the product inventory data described with reference to Fig. 15, and the cart contents information is the three items "product 131," "product 251," and "product 382," and the user of the purchaser data is customer HC assigned with "purchaser 120," it is determined that the purchaser data acquired in the process of step S52 is consistent with the product inventory data 271 of the first DB 27 (step S52; Yes). In other words, if at least one of such inconsistencies in the type and number of products or the identification of the customer HC occurs, it is determined that the purchaser data acquired in the processing of step S52 is inconsistent with the product inventory data 271 of the first DB27 (step S52; No).

[0117] The processing of step S52 can be said to be a process of determining whether the information indicated in the product inventory data 271 of the first DB27, i.e., the type and number of product PACs treated as being held by the customer HC, matches the purchaser data, i.e., the type and number of products that the customer intends to purchase, as indicated by the data managed on the customer HC's terminal 90, at the time of payment.

[0118] If it is determined that the purchaser data acquired in the processing of step S52 matches the product inventory data 271 of the first DB 27 (step S52; Yes), the processing of step S33 and the processing of step S34 described with reference to Fig. 25 are sequentially performed. On the other hand, if it is determined that the purchaser data acquired in the processing of step S52 does not match the product inventory data 271 of the first DB 27 (step S52; No), a second exception processing is performed (step S53). The second exception processing is a processing for confirming and correcting the cause of the inconsistency between the purchaser data acquired in the processing of step S52 and the product inventory data 271 of the first DB 27, and is performed manually, for example, through communication between the staff of the store ST and the customer HC.

[0119] Except for the points noted above, the processing flow when a customer HC uses terminal 90 to independently manage the product PAC that the customer HC takes from the shelf SH is the same as the processing flow when the payment processing is performed automatically upon exiting the store ST.

[0120] 26 and 27, the processing flow when the customer HC uses the terminal 90 to independently manage the product PAC that the customer HC takes out from the shelf SH has been described. Next, the processing flow when the customer HC uses the self-checkout to complete payment before entering the second gate 50 will be described.

[0121] 28 is a flowchart showing the processing flow when it is assumed that the customer HC will use the self-checkout register to complete payment before entering the second gate 50. When it is assumed that the customer HC will use the self-checkout register to complete payment before entering the second gate 50, it is assumed that the payment processing by the self-checkout register is performed between the processing of step S4 and the processing of step S5 in the processing flow when the payment processing described with reference to FIG. 22 is automatically performed upon exiting the store ST (step S61).

[0122] The process of step S61 is a process in which the customer HC uses a self-checkout to voluntarily complete payment before entering the second gate 50. In the embodiment, as shown in FIG. 2, a self-checkout SCH is provided in the store ST so that the customer HC can perform the process of step S61.

[0123] In addition, if it is assumed that the customer HC will use a self-checkout to complete payment before entering the second gate 50, the processing of step S6 in the processing flow when the payment processing described with reference to Figure 25 is performed automatically when exiting the store ST is replaced with a third exit processing (step S60).

[0124] 29 is a flowchart showing the flow of the third store exit process. In the third store exit process, first, the process of step S31 and the process of step S32 described with reference to FIG.

[0125] After processing step S32, the payment processing unit 36 ​​checks whether the customer HC who entered the second gate 50 in the processing of step S5 has completed payment (step S71). Specifically, a check is made to see whether the payment flag of the store-specific customer data 272 obtained in the processing of step S32 is 1. If the payment flag of the store-specific customer data 272 is 1, it is determined that payment has been completed (step S71; Yes), and the processing of step S34 described with reference to FIG. 25 is performed. On the other hand, if the payment flag of the store-specific customer data 272 is 0, it is determined that payment has not been completed (step S71; No), and the processing proceeds to third exception processing (step S72). The third exception processing is a process performed to have the customer HC complete payment, and is performed manually, for example, through communication between the staff of the store ST and the customer HC.

[0126] Except for the points noted above, the processing flow when it is assumed that customer HC uses a self-checkout to complete payment before entering second gate 50 is the same as the processing flow when payment processing is performed automatically upon exiting store ST.

[0127] 4 and other examples are shaped so that the product PAC has a surface that can be placed stably in the sensing area AA, but depending on the shape of the product, there may be cases where the product does not have a surface that can be placed stably in the sensing area AA. For such products, prior measures may be taken to provide a surface that can be placed stably in the sensing area AA.

[0128] Fig. 30 is a schematic diagram showing an example of a precautionary measure for providing a surface on which a product having a shape that does not allow it to be placed stably in the sensing area AA with a surface on which it can be placed stably in the sensing area AA. Fig. 30 shows product SF1 shown as a "first shape example," product SF2 shown as a "second shape example," and product SF3 shown as a "third shape example" as examples of products having a shape that does not allow it to be placed stably in the sensing area AA.

[0129] Product SF1 is a product with a cylindrical shape, like a can of canned food, with the circumferential edge of the bottom of the cylinder protruding beyond the inside of the circumference, so that only the circumferential edge touches the surface on which it is placed. Product SF2 is a product that does not have a flat surface on which the product can be placed stably, or even if it does have a flat surface, the flat surface is too small compared to the overall size of the product, making it unsuitable for stable placement. Product SF2 also includes products of the same type that have minor differences in their exact shape. Product SF3 is a spherical product.

[0130] Whether it is product SF1, product SF2, or product SF3, packaging using package P11 that can hold the product inside or open-type package P12 that can hold at least the bottom of the product provides the product with a surface that can be placed stably in sensing area AA. The bottom surface of package P11 or package P12 becomes a surface that can be placed stably in sensing area AA. In other words, a product packaged in package P11 or package P12 can be handled in the same way as product PAC described above.

[0131] In the above embodiment, it is assumed that products are placed on the shelf board SHB with sensing function of the shelf SH described with reference to Figures 3 and 4, but the shelf board SHB with sensing function is not the only one on which the sensor unit 11 can be provided. Below, other forms in which the sensor unit 11 can be provided will be described with reference to Figures 31 and 32.

[0132] Fig. 31 is a schematic diagram showing a vending machine 910 storing a product SFP. In the vending machine 910 shown in Fig. 31, the product SFP is automatically discharged to an outlet according to the amount inserted, using the vending machine's functions. Here, by providing a sensor unit 11 on the bottom surface of the space in which the product SFP is stored inside the vending machine 910, movement of the product SFP can be automatically detected.

[0133] Fig. 32 is a schematic diagram showing a locker 920 containing an SFQ product. The locker 920 shown in Fig. 32 functions as a paid locker, allowing the frame of the locker containing the SFQ product to be opened in accordance with the amount of money deposited. By providing a sensor unit 11 on the bottom surface of each locker frame, movement of the SFQ product can be automatically detected.

[0134] As described above, according to the embodiment, the detection system 1 includes a server (server 60) that stores pattern data (optical identification patterns) for identifying the types of products handled in a store, and a detection device (detection device 10) that is communicable with the server and identifies the products. The detection device includes a sensor unit (sensor unit 11) that is provided along the mounting surface of a shelf on which multiple products (product PACs) can be placed, and an information processing unit (information processing unit 20) that compares the output of the sensor unit with the pattern data to identify the type of product and the number of each type of product. The sensor unit includes a planar optical sensor (planar optical sensor 81) that is positioned facing the multiple products across the mounting surface. The planar optical sensor has a resolution that enables it to identify the shape of one surface of each of the multiple products placed on the mounting surface, as well as the characters, identifiers, and patterns applied to that surface. The pattern data includes information indicating the shape of the one surface and the characters, identifiers, and patterns applied to that surface (see FIG. 7).

[0135] This allows sensing of products (product PAC) on the placement surface with the simple arrangement condition of providing the planar optical sensor of the sensor unit (planar optical sensor 81) below the placement surface of the shelf. Therefore, it is possible to sense products (product PAC) on the placement surface without being restricted by the imaging range or angle of view of the imaging device. In this way, according to the embodiment, it is possible to provide a detection system (detection system 1) that can sense products more reliably and is easier to install.

[0136] Furthermore, the planar optical sensor (planar optical sensor 81) is less restricted by optical conditions, such as the focal length relative to the product (product PAC), than an imaging device. In particular, when multiple shelves are provided on one shelf SH, as shown in FIG. 3, the fewer such restrictions significantly contribute to the ease of installation of the detection system. For example, in the case of the top shelf (sensing-equipped shelf SHB1) of the shelf SH shown in FIG. 3, there are no particular restrictions on installing the imaging device 900 to capture an image of that shelf. On the other hand, if an imaging device is installed on the underside of the sensing-equipped shelf SHB1 shown in FIG. 3 to capture an image of product PAC1 placed on the sensing-equipped shelf SHB2 for product management, it is necessary to prepare an imaging device that can capture all of the products PAC1 on the sensing-equipped shelf SHB2 within its imaging range, with a focal length of D3, which is the distance D1 between the sensing-equipped shelf SHB1 and the sensing-equipped shelf SHB2 minus the height D2 of the product PAC1. Furthermore, if an imaging device is provided on the underside of the sensing shelf board SHB2 shown in Fig. 3 to capture an image of product PAC2 placed on the sensing shelf board SHB3 for product management, it is necessary to provide an imaging device that can capture all of the products PAC2 on the sensing shelf board SHB3 within its imaging range, with a focal length of D6, which is the distance D4 between the sensing shelf board SHB2 and the sensing shelf board SHB3 minus the height D5 of product PAC2. Furthermore, if an imaging device is provided on the underside of the sensing shelf board SHB3 shown in Fig. 3 to capture an image of product PAC3 placed on the sensing shelf board SHB4 for product management, it is necessary to provide an imaging device that can capture all of the products PAC3 on the sensing shelf board SHB4 within its imaging range, with a focal length of D9, which is the distance D7 between the sensing shelf board SHB3 and the sensing shelf board SHB4 minus the height D8 of product PAC3. Furthermore, the heights of the product PACs placed on the sensing function shelf board SHB2, the sensing function shelf board SHB3, and the sensing function shelf board SHB4 are not necessarily uniform, and some product PACs may block the angle of view of other product PACs.In contrast, such technical difficulties related to focal length and problems with the angle of view do not substantially occur with a planar optical sensor such as the planar optical sensor 81. Therefore, according to the embodiment, it is possible to provide a detection system (detection system 1) that can more reliably sense the product and is easier to install.

[0137] Furthermore, the sensor unit (sensor unit 11) has a light guide plate (light guide plate 2) arranged to face a plurality of products (product PAC) placed on a shelf surface on which the products can be placed, across the surface, and a light source (light source 7) that irradiates light from the side of the light guide plate, and the planar optical sensor (planar optical sensor 81) is arranged to face the plurality of products across the light guide plate, so that the light sources used when the planar optical sensor senses the plurality of products can be concentrated on the opposite side of the surface from the plurality of products, i.e., the side on which the planar optical sensor is provided. This makes it possible to provide a detection system (detection system 1) that can sense the products more reliably and is easier to install.

[0138] The detection device (detection device 10) also includes an imaging unit (imaging unit 121) that captures moving images, the imaging range of the imaging unit covers a path of movement of merchandise (product movement detection range CA) between a customer (customer HC) of the store (store ST) and a shelf (shelf SH), a sensor unit (sensor unit 11) operates before and after the image of the movement of merchandise between the customer and the shelf is captured by the imaging unit, and an information processing unit (information processing unit 20) compares the sensing results of the sensor unit before and after the image of the movement of merchandise between the customer and the shelf is captured by the imaging unit, and determines the type of merchandise placed on the loading surface of the shelf on which a plurality of merchandise (products PAC) can be placed and the number of each type of merchandise. It is determined whether there has been a change before and after the image capture unit captures an image of the movement of products between the customer and the shelf, and if the number of products placed on the placement surface has decreased compared to before the image capture unit captures the movement of products between the customer and the shelf, the number of products corresponding to the decrease in the number of products placed on the placement surface is considered to be held by the customer (see Figures 6, 8, 16 to 19, 24, and 26), thereby enabling operation control of the sensor unit using the image capture unit to be realized, and the sensor unit can be operated only before and after times when there is a high possibility of product movement, without having to operate the sensor unit all the time.In addition, the movement of products from the shelf to the customer can be traced.

[0139] Furthermore, the detection device (detection device 10) is equipped with a weight sensor (weight sensor 125) that detects changes in the weight of the shelf (shelf SH), and the sensor unit (sensor unit 11) operates before and after the weight sensor detects a change in weight, and the information processing unit (information processing unit 20) compares the sensing results of the sensor unit before and after the weight sensor detects a change in weight to determine whether the type of product and the number of each type of product placed on the loading surface of the shelf, which is designed to allow multiple products (products PAC) to be placed on it, have changed before and after the detection.This makes it possible to realize operation control of the sensor unit using the weight sensor, and the sensor unit can be operated only before and after times when there is a high possibility that products are moving, without having to operate the sensor unit all the time.

[0140] The server (server 60) also stores customer information (customer data 662) corresponding to the identification information of the customer's (customer HC's) terminal (terminal 90) and price information (product data 661) indicating the prices of various products, and the customer information includes information indicating the service the customer uses when making a payment. When making a payment using the service, it is determined whether the types and number of products held by the customer match the types and number of products the customer plans to purchase, as indicated by the data managed by the terminal (see FIGS. 15, 19, etc.) (see the processing of step S52 in FIG. 27). This further improves the accuracy of tracing the movement of products.

[0141] The imaging unit 121 may be provided for each shelf board SHB with a sensing function. In this case, the weight sensor 125 is provided, for example, between the shelf board BOA and the support member SUP, but this is not limitative and the location of the weight sensor 125 may be changed as appropriate. The key is that the weight sensor 125 should be able to detect the movement of the product PAC within the sensing area AA as a change in weight.

[0142] Furthermore, images of the product PAC included in the captured images within the product movement detection range CA by the imaging unit 121 may be further used by pattern matching to identify the type and number of the product PAC, and may be compared with the results of identifying the type and number of the product PAC by the sensor unit 11 to further improve accuracy. Also, weight detection of the product PAC by the weight sensor 125 may be further used to identify the moved product PAC. In this case, a "product weight" column is added to the product data 661, and a numerical value indicating the weight of each type of product is set in the field of this column.

[0143] Furthermore, other effects and advantages brought about by the aspects described in this embodiment that are clear from the description in this specification or that can be appropriately thought of by a person skilled in the art are naturally understood to be brought about by the present disclosure. [Explanation of symbols]

[0144] 1. Detection System 2 Light guide plate 7 light source 10. Detection Device 81 Planar optical sensor 90 terminals 99 Identification information storage unit 120 Imaging unit 125 Weight Sensor CA Product movement detection range PAC products SH shelf SHB Shelf with Sensing Function ST Store

Claims

1. A detection system comprising: a server that stores pattern data for identifying types of merchandise handled in a store; and a detection device that is communicable with the server and identifies the merchandise, The detection device includes: a sensor unit provided along a placement surface of a shelf on which a plurality of products can be placed; an information processing unit that compares the output of the sensor unit with the pattern data to identify the type of product and the number of each type of product; the sensor unit has a planar optical sensor arranged to face the plurality of products across the placement surface, the planar optical sensor has a resolution capable of identifying the shape of one surface of each of the plurality of commodities placed on the placement surface, the one surface being adjacent to the placement surface, and characters, identifiers, and patterns applied to the one surface; The pattern data includes information indicating the shape of the one surface and characters, identifiers, and patterns applied to the one surface. Detection system.

2. The sensor unit a light guide plate disposed opposite the plurality of products placed on the placement surface across the placement surface; a light source that irradiates light from a side of the light guide plate, the planar optical sensor is disposed opposite the plurality of products across the light guide plate; The detection system of claim 1 .

3. the detection device includes an imaging unit that captures a moving image; an imaging range of the image captured by the imaging unit covers a path of movement of merchandise between customers in the store and the shelves; the sensor unit operates before and after the image capturing unit captures an image of a movement of a commodity between a customer of the store and the shelf, The information processing unit compares the sensing results from the sensor unit before and after the image capturing unit captures an image of the movement of the products between the customer of the store and the shelf to determine whether the type of products and the number of each type of product placed on the placement surface have changed between before and after the image capturing unit captures the image of the movement of the products between the customer of the store and the shelf, and if the number of products placed on the placement surface has decreased after the image capturing unit captures an image of the movement of the products between the customer of the store and the shelf compared to before the image capturing unit captures the image of the movement of the products between the customer of the store and the shelf, it treats the number of products corresponding to the decrease in the number of products placed on the placement surface as being held by the customer.

3. A detection system according to claim 1 or 2.

4. the detection device includes a weight sensor that detects a change in weight of the shelf; The sensor unit operates before and after the weight sensor detects a change in weight, the information processing unit compares the sensing results of the sensor unit before and after the weight sensor detects a change in weight to determine whether the types of products and the number of each type of product placed on the placement surface have changed between before and after the detection.

3. A detection system according to claim 1 or 2.

5. The server further stores customer information corresponding to identification information of the terminal held by the customer and price information indicating the prices of various products; The customer information includes information indicating a service that the customer uses when making a payment, When making a payment using the service, it is determined whether the type and number of products that are treated by the information processing unit as being held by the customer match the type and number of products that the customer intends to purchase, as indicated by the data managed by the terminal. The detection system of claim 3 .

Images