Information processing device and program

Abstract

To provide an information processing device and a program that can restrain a processing load required for image analysis performed to find an inappropriate action of an operator in sales data processing.SOLUTION: In an information processing device for analyzing a picked up image, a camera picks up an image of an area where an operator of a sales data processing device that processes information to be used for a merchandise transaction is seen and outputs image data, and the information processing device includes an analysis unit, a notification unit, and an adjustment unit. The analysis unit analyzes a first area set at a center part of the image data to output information about a movement of the operator or a transition of the position of an item of merchandise. The notification unit outputs information for notifying a salesclerk of non-registration when it can be determined that an item of merchandise has arrived at a post-registration storage place from a pre-registration storage place of the sales data processing device without passing through a position for accepting a registration operation. The adjustment unit adjusts a boundary position between the first area and a periphery in the image data based on the information output from the analysis unit.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device and a program. [Background technology]

[0002] Conventionally, there are known self-service POS (Point of Sales) terminals in which a shopper operates the terminal to register products and perform payment processing, and semi-self-service POS terminals in which a shopper operates the terminal to perform payment processing after product registration. Some of these POS terminals are equipped with a camera that captures an image of the operator, and by analyzing the image output by the camera, it is possible to detect operational errors or fraudulent behavior by the operator. Here, Patent Document 1 discloses a technology for extracting a person's movement line within a specified area from the output of the camera.

[0003] In analyzing captured images, the larger the image data, the greater the load on the processing unit that performs the analysis. To minimize the load on the processing unit, it is desirable to minimize the image data. However, when detecting improper conduct by an operator in sales data processing, no measures have been taken to minimize the image data used for analysis, and the current practice is to analyze the entire captured image output by the camera. Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to provide an information processing device and a program that can reduce the processing load required for image analysis performed to detect inappropriate behavior by an operator in sales data processing. [Means for solving the problem]

[0005] In an embodiment, an information processing device analyzes an image captured by a camera, the camera capturing an image of a range in which an operator of a sales data processing device that processes information used in product transactions is captured and outputting image data. The information processing device includes an analysis unit, a notification unit, and an adjustment unit. The analysis unit analyzes a first range set in the center of the image data and outputs information related to the operator's actions or the transition of the product's position. The notification unit outputs information to notify non-registration when it can be determined, based on the information output by the analysis unit, that the product has moved from a pre-registration location in the sales data processing device to a post-registration location without passing through a location where a registration operation is accepted. The adjustment unit adjusts the position of the boundary between the first range and the surroundings in the image data based on the information output by the analysis unit. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a perspective view showing an example of the appearance of a self-service POS terminal according to an embodiment as seen from the operator's side. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the self-checkout POS terminal according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of a server according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a functional configuration of the self-checkout POS terminal and the server according to the embodiment. [Figure 5] FIG. 5 is an example of an image captured by the camera according to the embodiment. [Figure 6] FIG. 6 is an example of an image captured by the camera according to the embodiment. [Figure 7] FIG. 7 is a flowchart illustrating an example of processing executed by the server according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] (First embodiment) An embodiment will be described with reference to the drawings. Fig. 1 is a perspective view showing an example of the appearance of a self-service POS terminal 10 according to the embodiment as seen from the operator's side. The self-service POS terminal 10 is an example of a sales data processing device that processes information used in merchandise transactions. The self-service POS terminal 10 is a self-service POS (Point Of Sales) terminal in which a shopper acts as an operator to perform processes related to merchandise registration and payment.

[0008] The self-service POS terminal 10 has a main body 11, a basket stand 12, and a bagging stand 13. The main body 11 is provided between the basket stand 12 and the bagging stand 13. The basket stand 12 is a storage stand for placing baskets containing items to be transacted. The bagging stand 13 is a storage stand on which an open shopping bag can be hung. Registered items are placed in the shopping bag. In other words, registered items are placed on the bagging stand 13.

[0009] The bagging table 13 has a base 131, a bag hook 132, and a temporary storage table 133. The bag hook 132 and the temporary storage table 133 are supported by struts 134 attached to the base 131 and are provided above the base 131. The bag hook 132 is a hook for hanging a bag into which a registered product will be placed. The base 131 is a table that supports the bottom of the bag and on which products that are larger than the bag and unsuitable for being placed in a bag are placed, for example. The temporary storage table 133 is a table on which products that require care when bagging because they are fragile, for example, are temporarily placed.

[0010] The self-service POS terminal 10 further includes a weighing unit 14, a deposit / withdrawal unit 15, a scanner 16, a touch panel display 17, an indicator light 18, a card reader 19, a printer 20, and a camera 21.

[0011] The weighing unit 14 is provided on the bagging table 13. The weighing unit 14 weighs the weight of items such as commodities placed on the bagging table 13 based on the output of a built-in weight sensor. For example, the weighing unit 14 weighs the total weight of the bags set on the bagging table 13 and the commodities placed in the bags. The weight sensor is provided, for example, below the base 131 to support the base 131 and output a signal according to the load on the base 131.

[0012] The deposit / withdrawal unit 15 is provided, for example, in the vertical center of the main body 11. The deposit / withdrawal unit 15 accepts coins and bills inserted by shoppers and dispenses change, etc. Note that, for example, if the self-service POS terminal 10 is configured as a cashless terminal that does not handle cash, the deposit / withdrawal unit 15 may not be provided.

[0013] The scanner 16 is provided, for example, on the top of the main body 11. The scanner 16 has a reading window 161. The scanner 16 reads product information from a code symbol, such as a radio tag or barcode, attached to a product held over the reading window 161. The radio tag is an electronic tag such as an RF (Radio Frequency) tag. The radio tag has an IC (Integrated Circuit) chip that stores information and an antenna. The radio tag transmits tag information stored in the IC chip in response to radio waves received from a radio tag reader. For example, the radio tag stores product information about the product to which the radio tag is attached. The product information includes, for example, a product code that can identify the product.

[0014] The scanner 16 may also have a camera configured to capture an image of a commodity held over the reading window 161, for example. The camera of the scanner 16 includes an imaging element such as a CMOS (Complementary Metal-Oxide Semiconductor) or a CCD (Charge Coupled Device). The reading window 161 is capable of transmitting light captured by the camera of the scanner 16. Image data of the commodity captured by the camera of the scanner 16 is subjected to object recognition processing by an image processing device. As a result, the commodity included in the image data is identified and registered. The image processing device is an external device communicably connected to the self-service POS terminal 10 via, for example, a communication I / F (interface). The object recognition processing may be executed by the self-service POS terminal 10.

[0015] The touch panel display 17 is provided, for example, above the main body 11. The touch panel display 17 displays various information for the operator on the screen and accepts operations by the shopper.

[0016] The indicator light 18 is provided, for example, at the rear of the main body 11. The indicator light 18 is a light that notifies a store clerk that an abnormality has occurred in the self-service POS terminal 10.

[0017] The card reader 19 is provided, for example, on the side of the scanner 16 in the main body 11. The card reader 19 reads information stored on a card such as a credit card.

[0018] The printer 20 is provided, for example, inside the housing of the main body 11. The printer 20 is a printing device that prints receipts showing transaction details and coupons for product discounts and the like on receipt paper, etc. The printer 20 has a receipt discharge unit on the front of the main body 11. The receipt discharge unit is an outlet that discharges receipts and the like issued by the printer 20 to the outside of the self-service POS terminal 10. The receipt discharge unit is provided, for example, between the deposit / withdrawal unit 15 and the scanner 16 in the main body 11.

[0019] The camera 21 is configured to capture an image of the area in which the operator of the self-checkout POS terminal 10 is captured. The camera 21 includes an imaging element such as a CMOS or a CCD. Image data captured and output by the camera 21 is subjected to analysis processing by an analysis unit 311 (described later).

[0020] 2 is a diagram showing an example of a hardware configuration of the self-service POS terminal 10 according to the embodiment. The self-service POS terminal 10 further includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage unit 104, and a communication I / F (interface) 105.

[0021] The weighing unit 14, deposit / withdrawal unit 15, scanner 16, touch panel display 17, indicator light 18, card reader 19, printer 20, camera 21, CPU 101, ROM 102, RAM 103, memory unit 104 and communication I / F 105 are interconnected via a system bus 109 such as a data bus or address bus.

[0022] The storage unit 104 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The storage unit 104 stores a control program, a product master, etc. The control program is an operating system and a program for implementing various functions of the self-checkout POS terminal 10.

[0023] The product master stores product information (product information) in association with a product code, which is identification information that can identify the product. The product information includes, for example, the product name, price, and product weight. When the object recognition process is performed in the self-service POS terminal 10, information on matching features may be associated with the product master as product information. The matching features are information for matching that indicate the appearance characteristics of the product. The matching features are parameterized appearance characteristics such as the standard shape, surface color, pattern, and unevenness of the product.

[0024] The CPU 101 is a processor. The ROM 102 is a storage medium that stores various programs and data. The RAM 103 is a storage medium that temporarily stores various programs and data. The CPU 101 uses the RAM 103 as a work area to execute various programs stored in the ROM 102 or the storage unit 104, etc., thereby realizing the control unit 110 (see FIG. 4, described later). The control unit 110 comprehensively controls each unit of the self-service POS terminal 10.

[0025] The processor of the control unit 110 is not limited to the CPU 101, and other processors such as a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), or FPGA (Field Programmable Gate Array) may also be used.

[0026] The communication I / F 105 is an interface for the self-service POS terminal 10 to communicate with an external device via a network. An example of the external device is the server 300. The server 300 is an example of an information processing device that analyzes the image captured by the camera 21.

[0027] Another external device is, for example, an information processing device (hereinafter referred to as a management device) operated by an administrator such as a store clerk who manages the self-service POS terminal 10. The management device is, for example, a PC (Personal Computer), a tablet terminal, a smartphone, etc. The management device is connected to, for example, multiple self-service POS terminals 10. Note that any one of the multiple self-service POS terminals 10 connected to each other via a network can also be used as the management device. The management device notifies the store clerk of fraud detection based on notification information from the self-service POS terminal 10.

[0028] Still another external device may be an image processing device that performs image processing such as object recognition processing.

[0029] 3 is a diagram showing an example of the hardware configuration of a server 300 according to an embodiment. The server 300 includes a CPU 301, a ROM 302, a RAM 303, a storage unit 304, and a communication I / F 305. These units are connected to each other via a system bus 309 such as a data bus or an address bus.

[0030] The storage unit 304 is a storage device such as an HDD, SSD, flash memory, etc. The storage unit 304 stores control programs, etc. The control programs are programs for implementing an operating system and various functions of the server 300.

[0031] The CPU 301 is a processor. The ROM 302 is a storage medium that stores various programs and data. The RAM 303 is a storage medium that temporarily stores various programs and data. The CPU 301 uses the RAM 303 as a work area to execute various programs stored in the ROM 302 or the storage unit 304, etc., thereby realizing the control unit 310 (see FIG. 4, described later). The control unit 310 performs overall control of each unit of the server 300.

[0032] The processor of the control unit 310 is not limited to the CPU 301, and other processors such as a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), or FPGA (Field Programmable Gate Array) may also be used.

[0033] FIG. 4 is a diagram illustrating an example of the functional configuration of the self-checkout POS terminal 10 and the server 300 according to the embodiment.

[0034] The control unit 110 of the self-service POS terminal 10 functions as various functional units such as a reading unit 111, a registration unit 112, an operation unit 113, a settlement unit 114, an imaging unit 115, and a notification unit 116. The control unit 310 of the server 300 functions as an analysis unit 311, a notification unit 312, an adjustment unit 313, and the like.

[0035] In addition, in this embodiment, some or all of the functions realized by the control unit 310 of the server 300 may be functional units realized by the control unit 110 of the self-service POS terminal 10, may be realized in cooperation with the self-service POS terminal 10, or may be realized by a hardware configuration such as a dedicated circuit installed in the self-service POS terminal 10.

[0036] The reading unit 111 of the self-service POS terminal 10 is one of the input units that accepts input of information about the product to be transacted, and reads and outputs a product code from a code symbol such as a wireless tag or barcode attached to the product via the scanner 16. The reading unit 111 may also identify the product by object recognition based on image data captured by the camera of the scanner 16.

[0037] The registration unit 112 acquires information (product information) from the product master based on the product code output by the reading unit 111, and registers the acquired product information. Here, registration means storing the acquired product information as sales data in a predetermined storage area (for example, provided in the storage unit 104). Note that the registration unit 112 may register a product specified by an operation input to the touch panel display 17, regardless of the product information output by the reading unit 111.

[0038] The operation unit 113 is one of the input units that accepts input of information on the product to be traded, and accepts operations by the operator. More specifically, the operation unit 113 outputs a signal corresponding to the operation input received by the touch panel display 17.

[0039] The settlement unit 114 performs processing (settlement processing) related to the settlement of the products registered by the registration unit 112. For example, the settlement unit 114 calculates the total price of the products registered by the registration unit 112. The settlement unit 114 also calculates the change by subtracting the total amount from the amount inserted into the deposit / withdrawal unit 15. The settlement unit 114 then causes the deposit / withdrawal unit 15 to dispense the change.

[0040] The imaging unit 115 captures an image of the range in which the operator of the self-service POS terminal 10 is captured by the camera 21 and outputs the image. More specifically, the imaging unit 115 controls the camera 21 and outputs (transmits) the captured image output by the camera 21 to the server 300 or the like. The range in which the operator of the self-service POS terminal 10 is captured is a range that includes the position where the operator is expected to be located, which is approximately opposite the operation unit 113 and in front of the self-service POS terminal 10.

[0041] The analysis unit 311 of the server 300 executes a process (analysis process) of analyzing the image captured by the camera 21. The analysis unit 311 outputs information related to the operation of the operator by analyzing a first range 511 in the image data output by the camera 21. Here, Fig. 5 shows an example of an image 510 captured by the camera 21 according to the embodiment.

[0042] The captured image 510 is divided into a plurality of regions, and includes a first range 511, a second range 512, and a third range 513. The first range 511 is set in the center of the captured image 510. The initial value of this first range 511 is set by a serviceman (engineer) who installs the self-service POS terminal 10. The serviceman sets the first range 511 in the captured image 510 by referring to the output of the camera 21. The initial setting is stored in, for example, the storage unit 304.

[0043] The second range 512 and the third range 513 are the remaining portions of the captured image 510 excluding the first range 511, and are ranges that the service technician has determined not to be necessary for image analysis. The third range 513 is a range where image analysis is not performed.

[0044] The second range 512 is a part of the image data and is adjacent to the first range 511 via a boundary 514. In this embodiment, the second range 512 reflects shoppers (waiting people) who appear to be waiting their turn to use the self-service POS terminal 10.

[0045] The above-mentioned "information related to the operator's movements" output by the analysis unit 311 is obtained, for example, by detecting movements using a skeleton estimation technique, and specific examples include the following first to third pieces of information.

[0046] The first information is information indicating detection of non-registration. The analysis unit 311 performs behavior detection by skeletal estimation using images included in the first range 511 among the captured images 510. If the analysis unit 311 can determine from the estimated movement path of the shopper's skeleton that the product has reached the post-registration location (bagging table 13) from the pre-registration location (basket table 12) in the self-service POS terminal 10 without passing through the registration operation acceptance position (near the front of the scanner 16), the analysis unit 311 determines that non-registration has been detected and outputs first information indicating that fact.

[0047] The second information is the number of times the estimated skeleton crosses the boundary 514 between the first range 511 and its surroundings (second range 512). The analysis unit 311 counts the number of times the estimated skeleton of the shopper crosses the boundary 514 and records the number of times in a predetermined area of ​​the storage unit 304.

[0048] The third information is obtained by measuring the duration of time during which the estimated skeleton does not go beyond a virtual line 516 set inside the first range 511, and recording the time in a predetermined area or the like in the storage unit 304. The line 516 is set at a position a predetermined distance away from the boundary 514.

[0049] The notification unit 312 outputs information for notifying a store clerk or the like of non-registration based on the first information output by the analysis unit 311. More specifically, when the analysis unit 311 outputs information indicating non-registration detection, the notification unit 312 receives this information and transmits a notification to the self-service POS terminal 10 or the management device.

[0050] The adjustment unit 313 adjusts the position of the boundary 514 of the first range 511 based on the second and third information output by the analysis unit 311. Here, Fig. 6 is an example of an image 520 captured by the camera 21 according to the embodiment. Note that the above "adjustment" is intended to change the state to a preferable state in accordance with a predetermined standard.

[0051] For example, when the output (second information) of the analysis unit 311 determines that the operator's arm or hand during movement frequently crosses the boundary 514 of the first range 511, the adjustment unit 313 makes an adjustment to widen the first range 511.

[0052] Specifically, the adjustment unit 313 moves the boundary 514 to a predetermined position 515. The position 515 is located in a second range 512 adjacent to the first range 511, and is a predetermined distance away from the boundary 514.

[0053] In addition, in the above determination, the adjustment unit 313 determines that the occurrence frequency of an event in which the skeleton estimated by the analysis unit 311 intersects with the boundary 514 exceeds a predetermined threshold, for example. The frequency is the number of times an event occurs within a predetermined time period.

[0054] Furthermore, when the output (third information) of the analysis unit 311 determines that the arm or hand of the operator during movement will continue not to exceed the above-mentioned line 516, the adjustment unit 313 makes an adjustment to narrow the first range 511 to the line 516 (moves the boundary 514 to the position of the line 516).

[0055] In addition, in the above judgment, the adjustment unit 313 determines that "the above state continues" if, for example, the duration of the state in which the third information, i.e., the skeleton estimated by the analysis unit 311, and the above line 516 do not intersect exceeds a predetermined threshold.

[0056] By making such an adjustment, the first range 511 can be narrowed as much as possible to the extent that the skeleton of the shopper who is the operator of the self-checkout POS terminal 10 does not cross the boundary 514. This also makes it possible to maintain a state in which the skeleton estimated by the analysis unit 311 does not go outside the first range 511, while reducing the load caused by image analysis.

[0057] Here, the analysis unit 311 further analyzes the second range 512 to output fourth information indicating whether or not there are people waiting at the self-service POS terminal 10 and the number of people. This process is executed when the control unit 310 has sufficient spare capacity, and is omitted when the capacity is tight.

[0058] Returning to FIG. 4, the notification unit 116 receives a notification of non-registration detection from the server 300, and activates the indicator light 18 or transmits a notification to the management device.

[0059] The flow of processing according to the embodiment will be described below with reference to Fig. 7. Fig. 7 is a flowchart showing an example of processing executed by the server 300 according to the embodiment.

[0060] When the control unit 310 receives an image captured by the camera 21 (step S1), the analysis unit 311 performs image analysis and obtains first to fourth pieces of information (step S2).

[0061] Next, the control unit 310 performs counting based on the second and third information (step S3).

[0062] Next, the control unit 310 determines whether the position of the boundary 514 is inappropriate based on the counting result of step S3 (step S4). If it is inappropriate (Yes in step S4), the control unit 310 moves the boundary 514 to an appropriate position (step S5).

[0063] If the position of boundary 514 is appropriate in step S4 (No in step S4), control unit 310 skips step S5 and advances the process to step S6.

[0064] The processes of steps S4 and S5 are executed when the control unit 310 has sufficient spare capacity, and may be omitted and postponed when the control unit 310 is under pressure.

[0065] Next, the control unit 310 makes a determination in step S6 based on the fourth information. In step S6, the control unit 310 determines whether the number of people waiting at the self-service POS terminal 10 exceeds a threshold. In step S6, if the number of people waiting exceeds the threshold (Yes in step S6), the control unit 310 handles the waiting people (step S7). Examples of handling the waiting people include providing voice or text guidance about available terminals and calling a support staff member.

[0066] If the number of waiting users does not exceed the threshold in step S6 (No in step S6), control unit 310 skips step S7 and proceeds to step S8.

[0067] The processes of steps S6 to S7 are executed when the control unit 310 has sufficient spare capacity, and are skipped when the capacity is tight. The processes of steps S6 to S7 have a lower priority than the processes of steps S4 to S5. Therefore, when the control unit 310 has insufficient spare capacity, the processes of steps S6 to S7 are skipped, and when there is no spare capacity, the processes of steps S4 to S5 are skipped. In other words, the processes of steps S4 to S5 are executed with priority over the processes of steps S6 to S7.

[0068] Next, if the first information, that is, information indicating non-registration detection, is output, the control unit 310 determines that non-registration is suspected (Yes in step S8), notifies this (step S9), and ends this process.

[0069] Furthermore, in step S8, if the first information is not output, control unit 310 determines that there is no suspicion of non-registration (No in step S8), and returns the process to step S1.

[0070] As described above, according to this embodiment, the behavior detection process for detecting inappropriate behavior by an operator in sales data processing is executed only within the first range 511, so the load on the processing unit (control unit 310) that performs image analysis can be reduced compared to the conventional case in which the entire captured image 510 was analyzed.

[0071] Furthermore, according to this embodiment, the position of the boundary 514 of the first range 511 can be optimized, so that the behavior detection process can be performed just right.

[0072] Furthermore, according to this embodiment, the setting of the first range 511 is not reset for each transaction, so that the setting can be optimized for each self-checkout POS terminal 10 through continuous use.

[0073] In the above embodiment, the server 300 has been described as an example of an information processing device that analyzes the image captured by the camera 21, but in practice, the self-service POS terminal 10 may function as the information processing device. More specifically, the respective functional units (analysis unit 311, notification unit 312, adjustment unit 313) of the server 300 may be provided in the self-service POS terminal 10.

[0074] In the above embodiment, the processing of steps S4 to S5 is performed immediately after step S3, but this is not a limitation. The processing of steps S4 to S5 may not be performed here, but may be performed as a daily process. For example, the processing of steps S4 to S5 may be performed when the device is started up before opening.

[0075] Furthermore, in the above embodiment, the first range 511 is located in the center of the captured image 510, and the adjustment unit 313 adjusts the position of the boundary 514 between the first range 511 and other areas, but this does not deny that the boundary 514 may be located even partially at the edge of the captured image 510. In other words, in practice, the boundary 514 of the first range 511 may include the edge of the captured image 510.

[0076] The technology according to the above embodiment can also be applied to a semi-self-service POS terminal (an example of a checkout device) in which the shopper operates the payment process after product registration. The registration process prior to the payment process here is performed, for example, by a store clerk operating a regular POS terminal. The registration process may also be performed by the shopper themselves. For example, the shopper registers products by operating a POS application on their smartphone or by operating a cart POS (a terminal attached to a cart provided in the store).

[0077] The above-described embodiment can be modified as needed by partially changing the configuration or functions of each of the above-described devices. Therefore, several modifications of the above-described embodiment will be described below as other embodiments. The following mainly focuses on differences from the above-described embodiment, and the same reference numerals will be used for common features with the content already described, and detailed description will be omitted. The modifications described below may be implemented individually or in appropriate combination.

[0078] (Second embodiment) The analysis unit 311 in the above embodiment outputs information related to the operator's actions as the first to third information by analyzing the first range 511, but the analysis unit 311 in the present embodiment outputs information related to the transition of the product's position. The information related to the transition of the product's position is obtained by repeatedly performing image recognition on the moving product in the captured image 510 to track the product's position.

[0079] The first information is information indicating detection of non-registration. The analysis unit 311 obtains the movement path of the recognized product by performing image recognition processing on the first range 511 in the captured image 510. From this movement path, it is determined whether the product has reached the post-registration place (bagging table 13) from the pre-registration place (basket stand 12) in the self-service POS terminal 10 without passing through the registration operation acceptance position (near the front of the scanner 16). If it can be determined that the product has not passed through the registration operation acceptance position on the way from the basket stand 12 to the bagging table 13, the analysis unit 311 determines that non-registration has been detected, and outputs first information indicating that fact.

[0080] The second information is the number of times that the product recognized by the image recognition process crosses the boundary 514 in the direction going out of the first range 511. The analysis unit 311 counts the number of times and records it in a predetermined area of ​​the storage unit 304.

[0081] The third information is obtained by measuring the duration during which the product recognized by the image recognition process does not leave the virtual line 516 set inside the first range 511, and recording the duration in a specified area of ​​the memory unit 304, etc.

[0082] Next, when the output of the analysis unit 311 determines that the frequency of an event in which a moving product goes outside the boundary 514 is high, the adjustment unit 313 of this embodiment performs an adjustment to widen the first range 511. Specifically, the adjustment unit 313 moves the boundary 514 to a predetermined position 515. The frequency is the number of times the event occurs within a predetermined time period, and when this frequency exceeds a predetermined threshold, it is determined that the frequency is "high."

[0083] Furthermore, if the output of the analysis unit 311 determines that the moving product will continue to remain within the line 516 set inside the boundary 514, the adjustment unit 313 makes an adjustment to narrow the first range 511 to the line 516 (to move the boundary 514 to the position of the line 516).

[0084] In the above determination, the adjustment unit 313 determines that "the above state continues" when, for example, the duration of the state in which the product does not leave the line 516 exceeds a predetermined threshold.

[0085] In this way, the information processing device of the second embodiment can achieve the same effects as the previous embodiment.

[0086] The programs executed by each device in the above-described embodiments are provided in advance in a ROM, etc. The programs executed by each device in the above-described embodiments may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a digital versatile disk (DVD).

[0087] Furthermore, the programs executed by each device in the above-described embodiments may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the programs executed by each device in the above-described embodiments may be provided or distributed via a network such as the Internet.

[0088] The program executed by the information processing device of this embodiment has a modular configuration including the above-mentioned units (analysis unit 311, notification unit 312, and adjustment unit 313). A CPU (processor) loads the above-mentioned units onto a main storage device by reading and executing the program from the storage device. As a result, the analysis unit 311, notification unit 312, and adjustment unit 313 are generated on the main storage device.

[0089] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0090] 10... Self-service POS terminals, 11...Main body, 12...Basket stand, 13...Bagging table, 131...base, 132...bag hook, 133...temporary stand, 134...support, 14...Measuring part, 15... Deposit and withdrawal section, 16...scanner, 161...reading window, 17...Touch panel display, 18...Indicator light, 19...Card reader, 20...printer, 21...camera, 101...CPU, 102...ROM, 103...RAM, 104...storage unit, 105...communication I / F, 109...system bus, 110...control unit, 111...reading unit, 112...registration unit, 113...operation unit, 114: Settlement unit, 115: Imaging unit, 116: Notification unit, 300...server, 301...CPU, 302...ROM, 303...RAM, 304...storage unit, 305...communication I / F, 309...system bus, 310...control unit, 311...analysis unit, 312...notification unit, 313...adjustment unit, 510, 520...Captured images, 511...first range, 512...second range, 513...third range, 514...boundary. [Prior art documents] [Patent documents]

[0091] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-048229

Claims

1. In an information processing device that analyzes an image captured by a camera, the camera captures an image of a range in which an operator of a sales data processing device that processes information used in product transactions is captured, and outputs image data; The information processing device includes: an analysis unit that outputs information relating to the movement of the operator or the change in the position of the commodity by analyzing a first range set in the center of the image data; a notification unit that outputs information to notify non-registration when it can be determined based on the information output by the analysis unit that the product has reached a post-registration location in the sales data processing device from a location before registration without passing through a location where a registration operation is accepted; an adjustment unit that adjusts the position of the boundary between the first range and its surroundings in the image data based on the information output by the analysis unit; An information processing device comprising:

2. the analysis unit detects the behavior of the operator as the analysis; The adjustment unit performs adjustment to widen the first range when it is determined from the output of the analysis unit that the arm or hand of the operator during movement frequently crosses the boundary. The information processing device according to claim 1 .

3. the analysis unit detects the behavior of the operator as the analysis; When it is determined from the output of the analysis unit that the arm or hand of the operator during movement continues to be within a line set inside the boundary, the adjustment unit adjusts the first range to narrow the first range to the line. The information processing device according to claim 1 .

4. The analysis unit performs image recognition as the analysis to grasp the transition of the position of the product, The adjustment unit performs an adjustment to widen the first range when it is determined from the output of the analysis unit that the product during movement frequently goes outside the boundary. The information processing device according to claim 1 .

5. The analysis unit performs image recognition as the analysis to grasp the transition of the position of the product, When it is determined from the output of the analysis unit that the moving product continues to remain within a line set inside the boundary, the adjustment unit adjusts the first range to narrow the first range to the line. The information processing device according to claim 1 .

6. The analysis unit analyzes a second range that is part of the image data and is adjacent to the first range via the boundary, and outputs information indicating whether or not there are people waiting to use the sales data processing device, or the number of people. The information processing device according to claim 1 .

7. A computer included in an information processing device that analyzes images captured by a camera, an analysis unit that uses image data output by the camera of the sales data processing device that processes information used in commodity transactions, capturing an image of a range in which an operator is captured, and outputs information relating to the movement of the operator or the transition of the position of the commodity by analyzing a first range set in the center of the image data; a notification unit that outputs information to notify non-registration when it can be determined based on the information output by the analysis unit that the product has reached a post-registration location in the sales data processing device from a location before registration without passing through a location where a registration operation is accepted; an adjustment unit that adjusts the position of the boundary between the first range and its surroundings in the image data based on the information output by the analysis unit; A program to function as a

Images