Store operation support system and store operation support method

Abstract

To provide a shop management supporting system and a method for supporting shop management which allow a manager to easily know about situations of work in a shop.SOLUTION: A management device 6 in a shop management support system includes: an acquisition unit 631 for acquiring a picture from an imaging device for imaging a shop; a setting unit 634 for acquiring multiple tasks showing work to be done in a shop and setting each task in a completion state when a predetermined completion condition is satisfied; specification means for specifying a position of a picture on a time axis for the completion time when a task is set in a completion state when the task is set in the completion state; and a display control unit 633 for causing a manager terminal used by the manager of the shop to selectively display a task that has been set in a completion state and also causing the manager terminal to display a still image or a picture of the situation of work shown by the task on the basis of the specified position on the time axis of the picture for the task when the manager terminal has selected one of the displayed tasks.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a store operation support system and a store operation support method. [Background Art]

[0002] In stores such as convenience stores, various operations such as checking product shelves, replenishing products, and cleaning are carried out by employees. In order to improve store operations, it is preferable for store managers to be able to grasp the situation of such operations.

[0003] Patent Document 1 describes a store employee work management system that displays the work schedule of a store and the video of the interior of the store captured by a camera. In the store employee work management system of Patent Document 1, when a specific time period in the work schedule is selected, the video of the selected time period is displayed. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-141563 [Summary of the Invention] [Problems to be Solved by the Invention]

[0005] In such a management system, it is desired that managers can more efficiently grasp the situation of store operations.

[0006] The present invention has been made to solve the above-described problems, and an object thereof is to provide a store operation support system and a store operation support method that enable managers to easily grasp the situation of store operations. [Means for Solving the Problems]

[0007] The store operation support system according to the present invention is characterized by comprising: acquisition means for acquiring video from a camera that films the store; setting means for acquiring a plurality of tasks indicating work to be performed in the store and setting each task to a completed state when predetermined completion conditions are met; identification means for identifying the position on the time axis of the video corresponding to the completion time when each task is set to a completed state; and display control means for displaying the tasks set to a completed state in a selectable manner on an administrator terminal used by the store manager, and when any of the displayed tasks are selected on the administrator terminal, displaying video or still images of the work indicated by the task on the administrator terminal based on the position on the time axis of the video identified for that task.

[0008] Furthermore, in the store operation support system according to the present invention, it is preferable that the display control means causes the administrator terminal to display a video display area and a task display area on the same screen, and when any of the tasks displayed in the task display area is selected by the administrator terminal, the display control means causes the administrator terminal to display a video or still image of the work situation indicated by the task in the video display area, based on the position on the time axis of the video identified for that task.

[0009] Furthermore, in the store operation support system according to the present invention, it is preferable that the display control means further displays live video showing the store's status in real time in the video display area, and when one of the tasks displayed in the task display area is selected, it displays a video or still image of the work situation indicated by the selected task in the video display area instead of the live video.

[0010] Furthermore, in the store operation support system according to the present invention, it is preferable that the display control means displays a list of acquired tasks on the administrator terminal, in which only tasks that have been set to a completed state can be selected.

[0011] Furthermore, in the store operation support system according to the present invention, it is preferable that the display control means further displays a seek bar on the administrator terminal that indicates the time when the video displayed on the administrator terminal was taken, and also displays the position on the seek bar at the time when the video showing the status of the work indicated by the task set to a completed state was taken.

[0012] Furthermore, in the store operation support system according to the present invention, it is preferable that the identification means estimates the time at which the work indicated by a task was performed based on the completion time and type of task of the task set to a completed state, identifies the position on the time axis of the video corresponding to the work time, and the display control means, when any of the tasks displayed on the administrator terminal is selected, displays on the administrator terminal a video or still image of the position on the time axis identified for the task as a video or still image of the situation of the work indicated by the selected task.

[0013] Furthermore, in the store operation support system according to the present invention, it is preferable that the display control means, when any of the tasks displayed on the administrator terminal is selected, displays a video or still image of the work situation shown by the selected task on the administrator terminal, based on the time axis position identified for the task and the type of task selected.

[0014] The store operation support method according to the present invention is a store operation method executed by a store operation support system, and is characterized by including: an acquisition step of acquiring video from a camera that films the store; a setting step of acquiring a plurality of tasks indicating work to be performed in the store and setting each acquired task to a completed state when predetermined completion conditions are met; an identification step of identifying the position on the time axis of the video corresponding to the time when the task was set to a completed state when the task was set to a completed state; a first display control step of displaying the tasks set to a completed state in a selectable manner on an administrator terminal used by the store manager; and a second display control step of displaying video or still images of the work indicated by the task on the administrator terminal based on the position on the time axis of the video identified for the task when any of the displayed tasks are selected on the administrator terminal. [Effects of the Invention]

[0015] The store operation support system and store operation support method according to the present invention enable managers to easily grasp the status of operations at a store. [Brief explanation of the drawing]

[0016] [Figure 1] This is a functional block diagram of Store Operation Support System 1. [Figure 2] This is a functional block diagram of the control device 6. [Figure 3] This diagram shows the data structure of video table 611. [Figure 4] This diagram shows the data structure of task table 612. [Figure 5] This diagram shows the data structure of type table 613. [Figure 6] This figure shows the task display screen 100. [Figure 7] (A) is a diagram showing the task addition screen 200, and (B) is a diagram showing the report addition screen 300. [Figure 8] This figure shows the live video display screen 400. [Figure 9]It is a diagram showing the recorded video display screen 500. [Figure 10] It is a flowchart showing the flow of the task manual generation process. [Figure 11] It is a flowchart showing the flow of the task display process. [Figure 12] It is a flowchart showing the flow of the completion state setting process. [Figure 13] It is a flowchart showing the flow of the automatic determination process. [Figure 14] It is a flowchart showing the flow of the notification process. [Figure 15] It is a flowchart showing the flow of the video display process. [Figure 16] It is a flowchart showing the flow of the video display process. [Figure 17] It is a functional block diagram of the management device 16. [Figure 18] It is a diagram showing the data structure of the task table 1612. [Figure 19] It is a diagram showing the data structure of the type table 1613.

Embodiments for Carrying out the Invention

[0017] Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to those embodiments, but extends to the invention described in the claims and its equivalents.

[0036] Figure 1 is a functional block diagram of a store operation support system 1 according to an embodiment. The store operation support system 1 is a system that supports the operation of a store T by enabling employees to appropriately perform the tasks that should be performed at the store, and also enables a manager at a management facility K to easily grasp the status of operations at the store. Store T is a convenience store, clothing store or other retail store, commercial facility, etc. When a manager manages multiple stores T, the management facility K is, for example, a management center located remotely from the stores T. When a manager manages only one store T, the management facility K may be the back room of the store T, etc. The store operation support system 1 has a camera 2, a sensor 3 and a store terminal 4 placed in the store T, and a manager terminal 5 and a management device 6 placed in the management facility K. The camera 2, sensor 3, store terminal 4, manager terminal 5 and management device 6 are connected to each other so as to be able to communicate with each other via a network N.

[0019] Camera 2 is a device for photographing the store, such as a surveillance camera. Camera 2 is installed inside or around the store and photographs the store's situation, generating video, which is a moving image containing multiple frame images. Camera 2 transmits the generated video to the management device 6.

[0020] Sensor 3 is a device for generating various information indicating the store's conditions, such as an ultrasonic sensor, an infrared sensor, or a weight sensor. Here, the store's conditions refer to the factors that generate tasks indicating the work that should be performed in the store. For example, it is used to determine whether the product display conditions in the sales area are appropriate, such as the inventory status, display conditions, and temperature control status of the products, or whether the hygiene conditions in the store are appropriate, such as the cleanliness of the store's floor and the condition of the trash cans.

[0021] An ultrasonic sensor, for example, is placed at a predetermined height inside a trash can in a store. By emitting ultrasonic waves horizontally and detecting the reflected waves, it generates information indicating whether or not trash has accumulated up to a predetermined height. This information can be used to indicate whether or not the trash can is full.

[0022] An infrared sensor is installed, for example, in front of or above a product shelf. By emitting infrared light in the direction of the shelf and detecting the reflected waves, it generates information indicating the distance to the products displayed on the shelf. This information can be used to indicate whether the products on the shelf are properly positioned forward.

[0023] A weight sensor, for example, is placed on the shelf surface of a product display shelf and generates information indicating the weight of the items placed on the shelf. This information can be used to indicate whether or not the shelves are properly replenished with products.

[0024] Sensor 3 is not limited to the examples described above, and may be any sensor capable of generating information indicating the store's status, such as a temperature sensor, humidity sensor, motion sensor, or sound sensor. Sensor 3 transmits the information indicating the store's status, as described above, to the management device 6. The video generated by the camera 2 is also used as information indicating the store's status. Hereafter, the video generated by the camera 2 and the information generated by sensor 3 may be collectively referred to as sensor information.

[0025] Store terminal 4 is a terminal used by employees of store T, and is, for example, a mobile phone, smartphone, or tablet carried by an employee. Store terminal 4 may also be a device such as a PC (Personal Computer) installed in store T. Store terminal 4 displays the tasks to be performed at the store to employees and accepts operational input from employees regarding those tasks. Store terminal 4 has a store terminal storage unit 41, a store terminal communication unit 42, a store terminal display unit 43, a store terminal operation unit 44, and a store terminal processing unit 45.

[0026] The store terminal storage unit 41 is configured for storing programs or data, and for example, it includes a semiconductor memory device. The store terminal storage unit 41 stores operating system programs, driver programs, application programs, data, etc., used for processing by the processing unit. Programs are installed from computer-readable and non-temporary portable storage media such as CD (Compact Disc)-ROM (Read Only Memory) or DVD (Digital Versatile Disc)-ROM. Alternatively, programs may be downloaded and installed as package files from storage areas on the internet, such as a web server. Such programs are installed in the management device 6 and executed by the management device 6, and the store terminal 4 may receive an operable display screen and transmit operation results to the management device 6 as appropriate through communication with the management device 6 via a display operation application such as a web browser.

[0027] The store terminal communication unit 42 is configured to enable communication with other devices and includes a communication interface circuit. The communication interface circuit is a communication interface circuit such as a mobile phone network (carrier network), wired LAN (Local Area Network), or wireless LAN provided by a telecommunications carrier. The store terminal communication unit 42 receives data from other devices and supplies it to the processing unit, and also transmits data supplied from the processing unit to other devices.

[0028] The store terminal display unit 43 is configured to display images and includes, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The store terminal display unit 43 displays images based on display data supplied from the processing unit.

[0029] The store terminal operation unit 44 is configured to receive user input and includes, for example, a keypad, keyboard, or mouse. The store terminal operation unit 44 may also include a touch panel. The store terminal operation unit 44 generates signals corresponding to user input and supplies them to the processing unit.

[0030] The store terminal processing unit 45 is a device that comprehensively controls the operation of the device and comprises one or more processors and their peripheral circuits. The store terminal processing unit 45 may include, for example, a CPU (Central Processing Unit). The store terminal processing unit 45 may also include a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. The store terminal processing unit 45 controls the operation of each part based on stored programs and data supplied from each part, and also executes various processes.

[0031] The administrator terminal 5 is a terminal used by the administrator of store T, and is, for example, a smartphone or tablet carried by the administrator. The administrator terminal 5 may also be a device such as a PC installed in the management facility K. The administrator terminal 5 displays video footage of the work status to the administrator. The administrator terminal 5 has an administrator terminal storage unit 51, an administrator terminal communication unit 52, an administrator terminal display unit 53, an administrator terminal operation unit 54, and an administrator terminal processing unit 55. The configurations of the administrator terminal storage unit 51, administrator terminal communication unit 52, administrator terminal display unit 53, administrator terminal operation unit 54, and administrator terminal processing unit 55 are the same as those of the store terminal storage unit 41, store terminal communication unit 42, store terminal display unit 43, store terminal operation unit 44, and store terminal processing unit 45, respectively, so their explanation is omitted.

[0032] The management device 6 is a device that manages the tasks to be performed at the store, and is, for example, a PC or server installed in the management facility K. The functions of the management device 6 may also be implemented through cloud computing.

[0033] Figure 2 is a functional block diagram of the management device 6. The management device 6 includes a storage unit 61, a communication unit 62, and a processing unit 63.

[0034] The storage unit 61 is configured for storing programs or data, and for example, includes a semiconductor memory device. The storage unit 61 stores operating system programs, driver programs, application programs, data, etc., used for processing by the processing unit 63. Programs are installed into the storage unit 61 from a computer-readable, non-temporary portable storage medium such as a CD-ROM or DVD-ROM. Note that the storage unit 61 is just one example of a storage means.

[0035] The memory unit 61 stores the video table 611, task table 612, and type table 613, which will be described later, as data.

[0036] The communication unit 62 is configured to enable the management device 6 to communicate with other devices and includes a communication interface circuit. The communication interface circuit is, for example, a wired LAN or a wireless LAN. The communication unit 62 receives data from other devices and supplies it to the processing unit 63, and also transmits data supplied from the processing unit 63 to other devices.

[0037] The processing unit 63 is a device that comprehensively controls the operation of the management device 6 and comprises one or more processors and their peripheral circuits. For example, the processing unit 63 comprises a CPU. The processing unit 63 may also comprise a GPU, DSP, LSI, ASIC, FPGA, etc. The processing unit 63 controls the operation of each unit and executes various processes based on the program stored in the storage unit 61 and the data supplied from each unit.

[0038] The processing unit 63 includes an acquisition unit 631, a generation unit 632, a display control unit 633, a setting unit 634, a specification unit 635, a calculation unit 636, and a notification unit 637. The acquisition unit 631, generation unit 632, display control unit 633, setting unit 634, specification unit 635, calculation unit 636, and notification unit 637 are examples of acquisition means, generation means, display control means, setting means, specification means, estimation means, and notification means, respectively. Each of these units is a functional module realized by a program executed by the processing unit 63. Each of these units may be implemented in the management device 6 as firmware.

[0039] Figure 3 shows the data structure of the video table 611 stored in the memory unit 61. The video table 611 is a table that manages information about video acquired from the shooting device 2, and stores video ID, file name, shooting location, shooting device, start time, end time, etc., in relation to each other.

[0040] The video ID is information used to identify each of the multiple videos that are moving images. The file name is a string that indicates the file name of each video. The file name may include paths, etc. The shooting location is the shooting location of each video when the shooting device 2 is installed in multiple locations within the store. The shooting device is information that identifies the shooting device 2 that shot the corresponding video. The start time and end time are the start and end times of each video.

[0041] As will be described later, the camera 2 transmits live video showing the situation inside the store in real time to the management device 6. The management device 6 sequentially stores the frame images included in the acquired live video in the storage unit 61. When the management device 6 has stored frame images corresponding to a predetermined time (30 minutes in the example shown in Figure 3), it combines them into a single video file and stores it in the video table 611, associating it with the information described above. Note that the method of generating video is not limited to this example; the camera 2 may also transmit video files at predetermined intervals in addition to the live video.

[0042] Figure 4 shows the data structure of the task table 612 stored in the memory unit 61. The task table 612 is a table that manages information about tasks that should be performed in the store, and stores the task ID, work content, type, generation method, completion conditions, sensor information, deadline, priority, status, completion time, etc., in an interrelated manner.

[0043] The task ID is information used to identify each of multiple tasks. The work content is a string indicating the content of the work performed by each task. This string is displayed on the store terminal display unit 43 and the administrator terminal display unit 53, allowing employees and administrators to understand the tasks that should be performed at the store. The type is information indicating the classification of the work performed by each task, and is set based on the type table 613 described later.

[0044] The generation method indicates whether each task was generated by a manual generation method based on user input (meaning employee or administrator) or by an automated generation method not based on user input.

[0045] Each task is generated by a manual generation method based on user input. For example, if an administrator determines that there is work to be done at a store, they can generate a task by inputting information into the administrator terminal 5. Similarly, if an employee determines that there is work to be done at a store, they can generate a task by inputting information into the store terminal 4. In these cases, the task is associated with information indicating that it was generated by the manual generation method.

[0046] Each task can also be generated automatically based on sensor information, without relying on user input. For example, if a weight sensor installed on a shelf generates information indicating that the shelves are not properly restocked, a task is generated indicating that the shelves should be restocked. If the video footage generated by the camera 2 detects dirt on the store floor, a task is generated indicating that the floor should be cleaned. If an ultrasonic sensor installed on a trash can generates information indicating that the trash can is full, a task is generated indicating that the trash can should be inspected. In these cases, the task is associated with information indicating that it was generated by an automatic generation method.

[0047] Completion conditions indicate when each task is set to a completed state, based on either manual completion conditions (based on user input) or automatic completion conditions (not based on user input). Sensor information indicates which sensor information is used to determine when a task associated with an automatic completion condition is set to a completed state.

[0048] Tasks with associated manual completion conditions are set to a completed state based on the employee's input to the store terminal 4. After performing the work corresponding to each task, the employee makes an input to the store terminal 4 indicating that the work is complete. Tasks with associated manual completion conditions may also be set to a completed state based on the administrator's input to the administrator terminal 5.

[0049] Tasks associated with automatic completion conditions are set to a completed state when sensor information meets the predetermined conditions, without requiring any user input from an employee. In the example shown in Figure 4, if the information generated by the weight sensor installed on "shelf B" indicates that products have been replenished for a predetermined period of time or longer, the task "replenish shelf B," i.e., the task of replenishing products on shelf B, is set to a completed state. Similarly, if the information detected from the image generated by the camera's imaging device 2 indicates that the store floor is clean for a predetermined period of time or longer, the task "clean the floor," i.e., the task of cleaning the store floor, is set to a completed state. If the information generated by the ultrasonic sensor installed on "trash can A" indicates that the trash can is not full for a predetermined period of time or longer, the task "inspect trash can A," i.e., the task of inspecting the condition of trash can A, is set to a completed state.

[0050] Each task is generated either manually based on user input or automatically without user input. On the other hand, depending on whether a task was generated manually or automatically, the manual or automatic completion conditions used to set it to a completed state differ. For example, a task generated manually is set to a completed state using manual completion conditions, while a task generated automatically is set to a completed state using automatic completion conditions.

[0051] This ensures that whether or not tasks to be performed in a store are completed is properly managed. In other words, by setting tasks to a completed state based on automatic completion conditions, employees who performed the work no longer need to input data, thus preventing tasks that indicate completed work from being left unset as completed. On the other hand, if a task that could also be generated by the automatic generation method is generated by the manual generation method, it is likely that an abnormal situation has occurred in the store that requires that work. In this case, it is preferable that the completion status of the work be determined based on user input rather than sensor information.

[0052] For example, if a shelf is stocked with the wrong product, and the weight of the product on the shelf is normal, a task indicating the task of replenishing the product will not be generated based on the information from the weight sensor installed on the shelf. In this case, a user who notices this will generate a task to replenish the product on the shelf using a manual generation method. In such cases, it is not possible to determine whether the wrong product has been removed from the shelf and the correct product has been replenished based on the information from the weight sensor installed on the shelf. Therefore, a manual completion condition is associated with this task, and the task is set to a completed state based on the operational input of the employee who replenished the correct product. This ensures that whether or not the work that should be done in the store is completed is properly managed.

[0053] Furthermore, tasks will not be generated by the automatic generation method if they occur when the floor is not dirty but wet and slippery, or when the trash can is not full but the entrance is blocked, making it difficult to put in trash. In these cases, it is preferable that the completion of the task is determined based on user input rather than sensor information.

[0054] The deadline is information indicating the time when the work indicated by each task should be performed. The priority is information indicating the priority of each task. In the example shown in Figure 4, the priority is set in three levels: "high," "medium," and "low," but this is not the only example; it may be set in two or four or more levels, or it may be set by a numerical value. Priority is used to determine what work employees should do, and as described later, it is also used to output notifications prompting the execution of the work indicated by the task. The deadline and priority of tasks generated by the manual generation method are specified by the user who generated the task. The deadline and priority of tasks generated by the automatic generation method are set based on the type of task. The status is information indicating whether the work indicated by the task has been completed or not. The completion time is the time when the work was set to the completed state. For tasks associated with manual completion conditions, the completion time may also be the time when the employee entered an operation indicating that the work was completed.

[0055] Figure 5 shows the data structure of the type table 613 stored in the memory unit 61. The type table 613 is a table that manages information about the type of task, and stores the type, manually generated keywords, automatic generation method, automatic completion conditions, deadline, priority, work time, etc., in an interrelated manner.

[0056] The "Type" field indicates the classification of the work each task represents and is pre-set according to the tasks to be performed in the store. Manually generated keywords are phrases used to generate each type of task using the manual generation method. In the example shown in Figure 5, "Replenish, Stock" is set as the manual generation keyword for the "Product Replenishment" type task. When a user generating a task specifies a string containing the phrases "Replenish" or "Stock" as the task's work content, the generated task is set to the "Product Replenishment" type. If the work content does not include any of the manually generated keywords, the generated task is set to the "Other" type.

[0057] The automatic generation method is information that indicates the sensor information used when a task is generated by the automatic generation method. In the example shown in Figure 5, "product shelf sensor" is set as the automatic generation method for the "product replenishment" type task, indicating that the "product replenishment" type task is generated based on information generated by a weight sensor installed on the product shelf. "Video" is set as the automatic generation method for the "cleaning" type task, indicating that the "cleaning" task is generated based on video generated by the camera 2. "Trash can inspection" is set as the automatic generation method for the "trash can inspection" type task, indicating that the "trash can inspection" task is generated based on information generated by an ultrasonic sensor installed on the trash can. No automatic generation method is set for the "product shelf inspection," "poster replacement," and "other" type tasks, indicating that these types of tasks are not generated by the automatic generation method.

[0058] Automatic completion conditions are information that indicates sensor information used when a task is set to a completed state based on the automatic completion conditions. In the example shown in Figure 5, the automatic completion condition for a task of the type "product replenishment" is set to "product shelf sensor," and it is shown that the task of the type "product replenishment" is set to a completed state when the information generated by the weight sensor installed on the product shelf satisfies the predetermined completion conditions. Completion conditions are set in advance for each piece of sensor information. For example, the completion condition for the information generated by the weight sensor installed on the product shelf is that the weight of the products placed on the product shelf remains below a predetermined weight for a predetermined period of time or longer.

[0059] For tasks that can be determined to require action based on sensor information, an automatic generation method is set. Similarly, for tasks that can be determined to be completed based on sensor information, an automatic completion condition is set. The types of tasks that can be determined to require action based on sensor information, and the types of tasks that can be determined to be completed based on sensor information, are specified in advance by users such as administrators.

[0060] As described above, tasks generated by the manual generation method are associated with manual completion conditions, and tasks generated by the automatic generation method are associated with automatic completion conditions. Therefore, automatic completion conditions are not set for types for which an automatic generation method is not set, i.e., types that can only be generated by the manual generation method.

[0061] Note that the sensor information set as the automatic generation method and the sensor information set as the automatic completion condition may be different from each other. For example, "shelf sensor" may be set as the automatic generation method for a task of the type "product replenishment," and "camera" may be set as the automatic completion condition. In this case, a task of the type "product replenishment" is generated when the information generated by the weight sensor installed on the product shelf indicates that the weight of the products placed on the product shelf for a predetermined time or longer is less than or equal to a predetermined weight. Then, when the information detected from the image of the product shelf acquired from the camera 2 indicates that a predetermined amount or more of products have been placed on the product shelf for a predetermined time or longer, the task of the type "product replenishment" is set to the completed state.

[0062] The deadline is the time by which the work indicated by the generated task should be performed, when the task is generated by the automatic generation method. The priority is the priority set for the generated task when the task is generated by the automatic generation method. In the example shown in Figure 5, the "Replenish Product" type is associated with a deadline of "15 minutes" and a priority of "High". Therefore, when a task of the "Replenish Product" type is generated by the automatic generation method, the deadline is set to 15 minutes after the time of generation, and the priority is set to "High".

[0063] The work time is a method for calculating the time at which the work indicated by the task is performed and at which the administrator is expected to review the video. As described later, the administrator terminal 5 displays video footage of the work status indicated by the task, which is set to a completed state. At that time, the administrator terminal 5 uses the work time to display video footage of the work status indicated by the task from the video footage captured by the camera 2. Generally, the type of video footage that the administrator wants to review as the status of the work varies depending on the type of work, so the work time is set to different values ​​based on the completion time at which the work is set to a completed state and the type of task.

[0064] For example, for the "shelf inspection" task, the manager would want to see footage of the employee patrolling the store and properly inspecting the shelves, rather than footage of the shelves after the inspection—that is, footage of the work in progress. Therefore, the work time for the "shelf inspection" type is set to a time before the completion time. On the other hand, for the "replenishing stock" task, the manager would want to see footage of the stocked shelves, rather than footage of the employee replenishing stock—that is, footage of the time when the work is completed. Therefore, the work time for the "replenishing stock" type is set to the completion time. In this way, the work time is set in advance according to what kind of footage the manager wants to see regarding the status of each type of work.

[0065] The following describes the various screens displayed on the store terminal display unit 43 and the administrator terminal display unit 53. These screens are displayed based on display data supplied to the store terminal display unit 43 or the administrator terminal display unit 53 from the store terminal processing unit 45 or the administrator terminal processing unit 55.

[0066] Figure 6 shows the task display screen 100 displayed on the store terminal display unit 43. The task display screen 100 is displayed based on the display data of the task display screen 100 transmitted from the management device 6 to the store terminal 4. The task display screen 100 includes a task list 110, a complete button 120, an add task button 130, and an add report button 140.

[0067] Task List 110 is an object that displays a list of information about the generated tasks. Task List 110 displays information about the tasks, including whether the task is completed or not, the task content, the due date, and the priority. In the example shown in Figure 6, whether the task is completed or not is indicated by whether or not there is a checkmark on the left side of each row in Task List 110. The priority is displayed on the right side of each row in Task List 110; if it says "immediately," the priority is "high," if it says "when you have time," the priority is "low," and if there is no indication, the priority is "medium."

[0068] The "Complete" button 120 is an object used to set a task to a completed state. When an employee operates the store terminal control unit 44 and selects the "Complete" button 120, the store terminal 4 sends a task completion instruction to the management device 6.

[0069] The "Complete" button 120 is displayed for tasks in the task list 110 that are not set to a completed state and do not have an automatic completion condition set. Therefore, in the example shown in Figure 6, the "Complete" button 120 is not displayed for the tasks "Inspect shelf A" and "Replenish shelf B," which are set to a completed state. Also, the "Complete" button 120 is not displayed for the tasks "Clean the floor" and "Inspect trash can A," which have an automatic completion condition set. Only the "Complete" button 120 for the task "Inspect trash can B," which is neither set to a completed state nor has an automatic completion condition set, is displayed. The "Complete" button 120 may also be displayed in response to the selection of a task on the task list.

[0070] The task addition button 130 is an object for creating a new task. When an employee operates the store terminal control unit 44 and selects the task addition button 130, the task addition screen 200, described later, is displayed. The report addition button 140 is an object for creating a new report. When an employee operates the store terminal control unit 44 and selects the report addition button 140, the report addition screen 300, described later, is displayed.

[0071] Figure 7(A) shows the task addition screen 200 displayed on the store terminal display unit 43. The task addition screen 200 is displayed based on the display data of the task addition screen 200 transmitted from the management device 6 to the store terminal 4. The task addition screen 200 includes a text box 210, an add button 220, and a cancel button 230.

[0072] The text box 210 is an object for employees to input work details, deadlines, and priority by operating the store terminal control unit 44. The text box 210 may also allow input of the task type. The add button 220 is an object for generating a task. When an employee operates the store terminal control unit 44 and selects the add button 220, the store terminal 4 sends a task generation instruction to the management device 6, including the work details, deadlines, and priority entered in the text box 210. The cancel button 230 is an object for canceling task generation. When an employee operates the store terminal control unit 44 and selects the cancel button 230, the task display screen 100 is displayed.

[0073] Figure 7(B) shows the report addition screen 300 displayed on the store terminal display unit 43. The report addition screen 300 includes a text box 310, an add button 320, and a cancel button 330.

[0074] The text box 310 is an object for employees to input report content and time by operating the store terminal control unit 44. The add button 320 is an object for generating a report. When an employee operates the store terminal control unit 44 and selects the add button 320, the store terminal 4 sends a report generation instruction to the management device 6, which includes the report content and time entered in the text box 310. The cancel button 330 is an object for canceling report generation. When an employee operates the store terminal control unit 44 and selects the cancel button 330, the task display screen 100 is displayed.

[0075] Reports are generated by employees when they have something to communicate to their managers. As described later, the manager terminal 5 displays a video showing the store's situation at the time the selected report was made, by selecting the report. Therefore, employees only need to input the time the incident occurred and a brief description of the report, reducing the burden of reporting on employees. In addition, managers can gain a more concrete understanding of the content of employee reports through the video. Note that inputting the time may be omitted. In this case, the time the report was made may be considered the time the incident occurred.

[0076] Figure 8 shows the live video display screen 400 displayed on the administrator terminal display unit 53. The live video display screen 400 is a screen for displaying live video showing the store's status in real time, and is displayed based on the display data of the live video display screen 400 transmitted from the management device 6 to the administrator terminal 5. The live video display screen 400 includes a switching tab 410, a time display 420, live video 430, a completed task list 440, and an add button 450. The live video 430 is an example of the video display area, and the completed task list 440 is an example of the task display area.

[0077] The switching tab 410 is an object for switching between the live video display screen 400 and the recorded video display screen 500, which will be described later. When the administrator operates the administrator terminal control unit 54 and selects "Recorded Video" on the switching tab 410, the recorded video display screen 500 is displayed instead of the live video display screen 400. The time display 420 is an object that displays a string indicating the current time.

[0078] The live video 430 is video footage acquired from the camera 2 showing the store's status in real time. In the example shown in Figure 8, video footage acquired from six camera 2s is displayed in real time. The completed task list 440 is an object that displays a selectable list of information about only the tasks that have been set to a completed state from among the generated tasks. The add button 450 is an object for creating a new task. When the administrator operates the administrator terminal operation unit 54 and selects the add button 450, a screen similar to the task addition screen 200 described above is displayed. The administrator can create a task by inputting operations on the displayed screen.

[0079] Figure 9 shows the recorded video display screen 500 displayed on the administrator terminal display unit 53. The recorded video display screen 500 is a screen for displaying recorded video showing past conditions of the store, and is displayed based on the display data of the recorded video display screen 500 transmitted from the management device 6 to the administrator terminal 5. The recorded video display screen 500 includes a switching tab 510, a video information display 520, recorded video 530, a seek bar 540, completed task information 550, a completed task list 560, and an add button 570. The recorded video 530 is an example of the video display area, and the completed task information 550 and completed task list 560 are examples of the task display areas.

[0080] The switching tab 510 is an object for switching between the recorded video display screen 500 and the live video display screen 400. When the administrator operates the administrator terminal control unit 54 and selects "Live Video" on the switching tab 510, the live video display screen 400 is displayed instead of the recorded video display screen 500. The video information display 520 is an object for displaying information about the displayed recorded video. In the example shown in Figure 9, the video ID, start time, and end time of the video are shown as information about the recorded video.

[0081] The recorded video 530 is video footage stored in the management device 6 showing past conditions of the store. In the example shown in Figure 8, the videos captured by the six camera devices 2 and stored in the management device 6 are displayed. The seek bar 540 is an object that indicates the playback position of the displayed recorded video 530. The seek bar 540 indicates the time when the frame image displayed as the recorded video 530 was captured by its position on the recorded video display screen 500. The playback position of the recorded video can be changed by the administrator operating the seek bar 540 using the administrator terminal operation unit 54. Note that the recorded video 530 is displayed as a moving image, but it may also be displayed as a still image.

[0082] The completed task information 550 is an object that selectively displays a list of information about only the tasks that have been set to a completed state from among the generated tasks. The completed task information 550 displays the position on the seek bar 540 of the work time for each task in an identifiable manner. In the example shown in Figure 9, the task "Replenish shelf B" is located below the current playback position on the seek bar 540. That is, it indicates that the video of the work time for the task "Replenish shelf B" is currently being displayed. To the right of the task "Replenish shelf B" is the task "Inspect shelf A". That is, it indicates that the video of the work time for "Inspect shelf A" will be displayed after the video of the work time for "Replenish shelf B". When the seek bar 540 is manipulated and the playback position is moved to the position of the task "Inspect shelf A", the video of the work time for "Inspect shelf A" will be displayed.

[0083] Furthermore, the completed task information 550 displays a list of selectable reports generated based on employee input. In the example shown in Figure 9, the report concerning "suspicious person" is displayed to the right of the task "Inspection of shelf A". This indicates that the video of the time of the "suspicious person" report is displayed after the video of the time of the "Inspection of shelf A" task. This allows managers to easily and accurately grasp the store situation by reviewing employee reports along with the video footage from the time of the report.

[0084] The completed task list 560 is a list that displays information about only the tasks that have been set to a completed state from among the generated tasks, and this information can be selected. The add button 570 is an object for creating a new task. When an administrator operates the administrator terminal control unit 54 and selects the add button 570, a screen similar to the task addition screen 200 described above is displayed.

[0085] If any task is selected from the completed task list 440 on the live video display screen 400, the recorded video display screen 500 is displayed instead of the live video display screen 400. At this time, the recorded video display screen 500 is displayed so that the playback position indicated by the seek bar 540 corresponds to the work time of the selected task. The recorded video 530 then displays the video following the frame image of the work time of the selected task. For example, if the task "Replenish shelf B" is selected from the completed task list 440 on the live video display screen 400 shown in Figure 8, the recorded video display screen 500 will be displayed with the playback position of the seek bar 540 set to the work time of "Replenish shelf B," as shown in Figure 9. The recorded video 530 will also display the video following the frame image of the work time of "Replenish shelf B." This makes it easy for administrators to grasp the status of completed tasks.

[0086] If any of the tasks displayed in the completed task list 560 on the recorded video display screen 500 are selected, the playback position indicated by the seek bar 540 will be the work time of the selected task. In addition, the recorded video 530 will display the video following the frame image at the work time of the selected task.

[0087] Furthermore, if any of the tasks displayed in the completed task information 550 are selected, the playback position indicated by the seek bar 540 will be the work time of the selected task. In addition, the recorded video 530 will display the video following the frame image at the work time of the selected task. Similarly, if any of the reports displayed in the completed task information 550 are selected, the playback position indicated by the seek bar 540 will be the time of the selected report, and the recorded video 530 will display the video following the frame image at the time of the selected report.

[0088] In the above explanation, the completed task information 550 on the recorded video display screen 500 is shown as a selectable task; however, the tasks displayed in the completed task information 550 do not necessarily have to be selectable. In this case as well, since the completed task information 550 indicates the position of each task's work time on the seek bar 540, the administrator can easily grasp the status of work in the store by setting the playback position of the seek bar 540 to the position where each task is displayed (in the example shown in Figure 9, the position of the flag mark).

[0089] Furthermore, some of the completed task list 440 on the live video display screen 400, and some of the completed task information 550 and completed task list 560 on the recorded video display screen 500, do not need to be displayed. Instead of the completed task lists 440 and 560, a list containing tasks that are not set to a completed state may be displayed.

[0090] The following describes the processing flow performed by the management device 6. The management device 6 performs manual task generation processing, task display processing, completion status setting processing, automatic determination processing, notification processing, and video display processing. These processes are realized by the processing unit 63 cooperating with each component of the management device 6 based on the program stored in the storage unit 61. Furthermore, these processes are executed in parallel.

[0091] Figure 10 is a flowchart showing the flow of the manual task generation process. The manual task generation process is executed when a store terminal 4 or administrator terminal 5 sends a task generation instruction to the management device 6. Store terminal 4 sends a task generation instruction via store terminal communication unit 42 when the add button 220 on the task addition screen 200 is selected. Administrator terminal 5 sends a task generation instruction via administrator terminal communication unit 52 based on the operation input to the task addition screen that is displayed when the add button 450 on the live video display screen 400 or the add button 570 on the recorded video display screen 500 is selected.

[0092] First, the acquisition unit 631 acquires a task generation instruction via the communication unit 62 (step S11). The task generation instruction includes the work content, deadline, and priority based on the employee's operation input to the store terminal operation unit 44 or the administrator's operation input to the administrator terminal operation unit 54.

[0093] Next, the generation unit 632 identifies the type of task based on the content of the generation instruction (step S12). The generation unit 632 obtains the work content included in the generation instruction. The generation unit 632 refers to the type table 613 and identifies the type of task based on the manually generated keywords included in the obtained work content.

[0094] Next, the generation unit 632 generates a task of the specified type (step S13). The generation unit 632 stores the task in the task table 612, setting the specified type, information indicating that it was generated by the manual generation method, information indicating that it will be set to a completed state based on the manual completion conditions, and the work content, deadline, and priority included in the generation instruction. This completes the manual task generation process.

[0095] Figure 11 is a flowchart showing the flow of the task display process. The task display process is executed when the store terminal 4 sends a request to the management device 6 to display the task display screen 100. In response to an operation to the store terminal operation unit 44 to execute an application program for displaying the task display screen 100, the store terminal 4 sends a request to display the task display screen 100 via the store terminal communication unit 42.

[0096] First, the acquisition unit 631 acquires a display request for the task display screen 100 via the communication unit 62 (step S21).

[0097] Next, the display control unit 633 generates display data for the task display screen 100 (step S22). The display control unit 633 obtains information about the tasks to be displayed in the task list 110 of the task display screen 100 from the task table 612. The display control unit 633 refers to the task table 612 and decides whether or not to display each completion button 120 based on whether or not manual completion conditions are set for each task. The display control unit 633 generates display data for the task display screen 100 that displays each object such as the task list 110 and the completion button 120, based on a pre-set layout.

[0098] Next, the display control unit 633 transmits the display data of the task display screen 100 to the store terminal 4 via the communication unit 62, thereby displaying the task display screen 100 on the store terminal 4 (step S23). This completes the task display process.

[0099] Figure 12 is a flowchart showing the process of setting the completion status. The process of setting the completion status is executed when the store terminal 4 sends a task completion instruction to the management device 6. In response to the operation input to the store terminal operation unit 44 in selecting the completion button 120 on the task display screen 100, the store terminal 4 sends a completion instruction for the task for which the completion button 120 was selected via the store terminal communication unit 42.

[0100] First, the acquisition unit 631 acquires a task completion instruction via the communication unit 62 (step S31). The task completion instruction includes the task ID of the task for which the completion button 120 was selected.

[0101] Next, the setting unit 634 sets the task to a completed state based on the completion instruction (step S32). The setting unit 634 sets the task indicated by the task ID included in the completion instruction to a completed state in the task table 612. The setting unit 634 stores the current time as the task completion time in the task table 612. The setting unit 634 may also store the time when the completion button 120 is selected or the time when the task completion instruction is sent as the task completion time.

[0102] Next, the identification unit 635 calculates the work time based on the completion time and type of the task set to the completed state (step S33). The identification unit 635 obtains the type of task set to the completed state from the task table 612. The identification unit 635 obtains the method for calculating the work time corresponding to the obtained type from the type table 613. The identification unit 635 calculates the work time from the task completion time based on the obtained calculation method.

[0103] Next, the identification unit 635 identifies an image of the work time from among the images stored in the storage unit 61 (step S34). The identification unit 635 refers to the image table 611 and identifies the image containing the frame image of the work time. The identification unit 635 identifies the frame image of the work time from among the identified images. The frame image of the work time is identified, for example, based on a timestamp attached to the frame image included in the image. The identification unit 635 associates information with the identified frame image indicating that it is an image that captures the work status of a task set to the completed state. In this way, the identification unit 635 identifies the position on the time axis of the image corresponding to the work time. This completes the completion state setting process.

[0104] Figure 13 is a flowchart showing the flow of the automatic judgment process. The automatic judgment process is executed by the management device 6 at predetermined intervals.

[0105] First, the acquisition unit 631 acquires sensor information via the communication unit 62 (step S41). The acquisition unit 631 acquires video footage of the store's situation captured by the camera 2 as sensor information via the communication unit 62. The acquisition unit 631 also acquires various types of information indicating the store's situation generated by the sensor 3 from the sensor 3 as sensor information via the communication unit 62. The acquisition unit 631 stores the acquired sensor information in the storage unit 61.

[0106] Next, the generation unit 632 determines whether the generation conditions for generating a task by the automatic generation method are met (step S42). Based on the sensor information stored in the storage unit 61, the generation unit 632 determines whether the various generation conditions stored in the type table 613 are met.

[0107] For example, if information generated by a weight sensor installed on a product shelf indicates that the weight of the products placed on the shelf remains below a threshold for a predetermined period of time or longer, the generation unit 632 determines that the generation conditions for the "product shelf sensor" have been met. Similarly, if information generated by an ultrasonic sensor installed on a trash can indicates that the contents of the trash can remain above a threshold for a predetermined period of time or longer, the generation unit 632 determines that the generation conditions for the "trash can sensor" have been met.

[0108] Furthermore, if the video generated by the camera 2 continuously shows that there is dirt on the store floor for a predetermined period of time or longer, the generation unit 632 determines that the generation conditions related to "video" have been met. For example, the generation unit 632 applies a method such as background subtraction to the frame images included in the video generated by the camera 2 to identify difference regions with different pixel values ​​from a pre-set background image. The generation unit 632 determines whether or not the difference region contains dirt on the floor by extracting feature quantities from the identified difference region. The generation unit 632 determines whether or not the video continuously shows that there is dirt on the store floor for a predetermined period of time or longer by applying this process to multiple images with consecutive shooting times.

[0109] If any task generation conditions are met (step S42-Yes), the generation unit 632 generates a task of the type for which the generation conditions are met (step S43). Based on the sensor information for which the generation conditions are met, the generation unit 632 sets the work content of the task to be generated. For example, if the sensor information for which the generation conditions are met is a weight sensor installed on "product shelf B" and the type of task to be generated is "product replenishment", the generation unit 632 sets the work content to "replenish product shelf B" based on predetermined rules. The generation unit 632 refers to the type table 613 to set the deadline and priority of the task to be generated. The generation unit 632 also sets the content of the automatic completion conditions for the task to be generated based on the sensor information for which the generation conditions are met. For example, if the sensor information for which the generation conditions are met is the weight sensor on "product shelf B", the generation unit 632 sets the automatic completion condition as the completion condition for the task to be generated. The generation unit 632 also sets that the task to be generated is set to a completed state based on the sensor information generated by the weight sensor on "product shelf B". The generation unit 632 stores tasks associated with the set work content, completion conditions, sensor information, deadline, and priority in the task table 612.

[0110] If none of the automatic generation conditions for any task are met (step S42-No), the automatic determination process proceeds to step S44.

[0111] Next, the setting unit 634 determines whether the automatic completion conditions for the task are met (step S44). Based on the sensor information stored in the storage unit 61, the setting unit 634 determines whether the completion conditions for each task stored in the task table 612 are met.

[0112] If it is determined that the automatic completion conditions for a task are met (step S44-Yes), the setting unit 634 sets the task for which the automatic completion conditions are met to a completed state (step S45). The identification unit 635 then calculates the work time of the task set to a completed state (step S46) and identifies the image corresponding to the work time from the images stored in the storage unit 61 (step S47). The processing in steps S45-S47 is the same as the processing in steps S32-S34 of the completion state setting process. This completes the automatic determination process.

[0113] Figure 14 is a flowchart showing the flow of the notification process. The notification process is performed by the management device 6 at predetermined intervals.

[0114] First, the calculation unit 636 determines whether or not there are tasks stored in the task table 612 that are not set to a completed state (step S51). If there are no tasks stored that are not set to a completed state (step S51-No), the notification process ends.

[0115] If there are tasks stored that are not set to a completed state (step S51-Yes), the calculation unit 636 estimates the store's busyness by calculating it (step S52). For example, the calculation unit 636 calculates the busyness based on the number of employees in the store, the number of customers in the store, and the number of tasks that are not set to a completed state.

[0116] The number of employees in a store is calculated, for example, based on the number of employees for each time period, which is stored in the memory unit 61 beforehand, and the current time. The number of customers in a store is calculated, for example, based on images of the store generated by the camera 2. In this case, human regions are extracted from multiple frame images of the images generated by the camera 2 based on feature quantities. Then, the number of people in the store is calculated by associating the same person between multiple frame images using known tracking techniques, etc. The number of customers in the store is calculated by subtracting the number of employees from the calculated number. The number of tasks that have not been set to a completed state is calculated based on the task table 612.

[0117] The calculation unit 636 calculates the level of busyness based on whether the number of employees, the number of customers, and time information (time of day, day, season, etc.) of the store meet predetermined conditions. For example, the calculation unit 636 calculates that the store is busy if one or more of the following conditions are met: there is one employee, there are 10 or more customers, and it is daytime. The calculation unit 636 calculates that the store is busy if it is not calculated to be busy, and one or more of the following conditions are met: there are two staff members, there are three or more customers, and it is nighttime. The calculation unit 636 calculates that the store is busy if it is neither busy nor moderate.

[0118] The methods for calculating busyness are not limited to the examples described above. For example, busyness may be calculated based on the status of tasks that are not set to completed (e.g., priority). Also, busyness may be calculated based on the number of customers waiting to pay, instead of, or in addition to, the number of customers in the store. Furthermore, busyness may be calculated based on weather information. In addition, time periods may be used, such as times when employees change shifts, times when specific tasks are performed (e.g., product deliveries), or times when sales or other events are held.

[0119] Furthermore, the level of busyness may be calculated using only one or two of the following: the number of employees, the number of customers, and time information. Alternatively, the level of busyness may be calculated by comprehensively considering the number of employees, the number of customers, and time information. For example, if there is one employee, 10 or more customers, and it is daytime, each will be assigned 3 points; if there are two staff members, 3 to 9 customers, and it is nighttime, each will be assigned 2 points; and if there are three or more staff members, 2 or fewer customers, and it is morning, each will be assigned 1 point. Then, if the total points are 7 points or more, the level of busyness is calculated as high; if it is 5 to 6 points, the level of busyness is calculated as moderate; and if it is 4 points or less, the level of busyness is calculated as low.

[0120] Next, the notification unit 637 determines whether the conditions for notifying the employee have been met (step S53). The conditions for notifying the employee are based on the store's busyness level. For example, the condition for notifying the employee is that the busyness level is below a predetermined value.

[0121] If the conditions for notifying the employee are not met (step S53-No), proceed to step S55.

[0122] If the conditions for notifying an employee are met (step S53-Yes), the notification unit 637 outputs a notification to the store terminal 4 prompting it to perform the task indicated by the task (step S54). For example, the notification unit 637 sends a notification instruction via the communication unit 62 to a notification server (not shown) containing a message containing the details of tasks that are not set to a completed state. Upon receiving the notification instruction, the notification server sends a notification to the store terminal 4 containing a message prompting it to perform the task indicated by a task whose elapsed time from the deadline is longer than a predetermined time. Upon receiving the notification, the store terminal 4 displays a pop-up message prompting it to perform the task. The message prompting it to perform the task may be, for example, "Please replenish shelf B." The notification server may be a mail server, a push notification server, etc. The notification server may also send notifications to mobile terminals carried by employees, in addition to the store terminal 4.

[0123] Next, the notification unit 637 determines whether the conditions for notifying the administrator have been met (step S55). The conditions for notifying the administrator are, for example, that even after a predetermined time has elapsed since the notification was output to the store terminal 4, the task indicated by the notification has not been set to a completed state.

[0124] If the conditions for notifying the administrator are not met (step S55-No), the notification process ends.

[0125] If the conditions for notifying the administrator are met (step S55-Yes), the notification unit 637 outputs a notification to the administrator terminal 5 prompting it to perform the task indicated by the task (step S54). For example, the notification unit 637 sends a message to the administrator terminal 5 via the notification server prompting it to perform the task indicated by a task that is not set to a completed state. This completes the notification process.

[0126] Figures 15 and 16 are flowcharts illustrating an example of the video display processing flow. The video display processing is performed when the administrator terminal 5 sends a request to display the live video display screen 400. In response to an operation input to the administrator terminal operation unit 54 that causes an application program to display the live video display screen 400 to be executed, the administrator terminal 5 sends a request to display the live video display screen 400 via the administrator terminal communication unit 52.

[0127] Referring to Figure 15, first, the acquisition unit 631 acquires a display request for the live video display screen 400 via the communication unit 62 (step S61).

[0128] Next, the display control unit 633 generates display data for the live video display screen 400 (step S62). The display control unit 633 obtains information from the task table 612 about tasks that are set to a completed state and will be displayed in the completed task list 440 of the live video display screen 400. Based on a pre-configured layout, the display control unit 633 generates display data for the live video display screen 400 that displays each object, including the completed task list 440.

[0129] Next, the display control unit 633 transmits the display data of the live video display screen 400 to the administrator terminal 5 via the communication unit 62, thereby displaying the live video display screen 400 on the administrator terminal 5 (step S63).

[0130] Next, the display control unit 633 transmits the latest frame image from the video stored in the storage unit 61 to the administrator terminal 5 via the communication unit 62 (step S64). The transmitted image is displayed on the administrator terminal 5 as the live video 430 on the live video display screen 400.

[0131] Next, the display control unit 633 determines whether or not a task selection instruction has been received from the administrator terminal 5 (step S65). The administrator terminal 5 sends a task selection instruction via the administrator terminal communication unit 52 if any of the tasks displayed in the completed task list 440 on the live video display screen 400 are selected. The task selection instruction includes the task ID of the selected task.

[0132] If a task selection instruction is received (step S65-Yes), the display control unit 633 sets the frame image associated with information indicating that the status of the selected task has been captured as the playback point of the recorded video (step S65a). The video display process proceeds to step S71 to display the recorded video display screen 500.

[0133] If no task selection instruction has been received (step S65-No), the display control unit 633 determines whether or not a switching instruction has been received from the administrator terminal 5 (step S66). The administrator terminal 5 sends a switching instruction via the administrator terminal communication unit 52 when the switching tab 410 of the live video display screen 400 is selected.

[0134] If a switching instruction is received (step S66-Yes), the video display process proceeds to step S71 to display the recorded video display screen 500.

[0135] If no switching instruction has been received (step S66-No), the display control unit 633 determines whether or not an termination instruction has been received from the administrator terminal 5 (step S67). The administrator terminal 5 transmits a termination instruction via the administrator terminal communication unit 52 when an operation input to terminate the display of the live video display screen 400 is made to the administrator terminal operation unit 54.

[0136] If a termination instruction is received (step S67-Yes), the video display process terminates.

[0137] If no termination instruction is received (step S67-No), the process returns to step S64, and the display control unit 633 transmits the latest frame image from the video stored in the storage unit 61 back to the administrator terminal 5. In other words, the display control unit 633 sequentially transmits the latest frame images acquired from the camera 2 and stored in the storage unit 61 to the administrator terminal 5 until a task selection instruction, switching instruction, or termination instruction is received. This allows the administrator terminal 5 to display the store status in real time.

[0138] Referring to Figure 16, when a task selection instruction or switching instruction is received, the display control unit 633 generates display data for the recorded video display screen 500 (step S71). If a playback point for the recorded video is set, the display control unit 633 sets the seek bar 540 of the recorded video display screen 500 to indicate the set playback point. If no playback point for the recorded video is set, the display control unit 633 sets the playback point of the seek bar 540 to indicate the start of a predetermined video (for example, the video of the most recently generated video file). The display control unit 633 also obtains information from the task table 612 regarding tasks that are set to a completed state and will be displayed in the completed task list 560 of the recorded video display screen 500. Based on a pre-set layout, the display control unit 633 generates display data for the recorded video display screen 500 that displays each object, including the seek bar 540 and the completed task list 560.

[0139] Next, the display control unit 633 transmits the display data of the recorded video display screen 500 to the administrator terminal 5 via the communication unit 62, thereby displaying the recorded video display screen 500 on the administrator terminal 5 (step S72).

[0140] Next, the display control unit 633 transmits the frame image following the current playback point from the video stored in the storage unit 61 to the administrator terminal 5 via the communication unit 62 (step S73). The transmitted image is displayed on the administrator terminal 5 as the recorded video 530 on the recorded video display screen 500.

[0141] Next, the display control unit 633 determines whether or not a task selection instruction has been received from the administrator terminal 5 (step S74). The administrator terminal 5 sends a task selection instruction via the administrator terminal communication unit 52 if any of the tasks displayed in the completed task list 560 on the recorded video display screen 500 are selected. The task selection instruction includes the task ID of the selected task.

[0142] If a task selection instruction is received (step S74-Yes), the display control unit 633 sets the playback location to the work time of the selected task (step S74a). The display control unit 633 sets the playback location to a frame image associated with information indicating that the work status of the selected task has been captured. Then, proceeding to step S73, the display control unit 633 transmits the frame image from the video stored in the storage unit 61 that follows the set playback location to the administrator terminal 5. As a result, the video capturing the work status of the selected task is displayed on the administrator terminal 5.

[0143] If no task selection instruction has been received (step S74-No), the display control unit 633 determines whether or not a seek bar operation instruction has been received from the administrator terminal 5 (step S75). The administrator terminal 5 transmits a seek bar operation instruction via the administrator terminal communication unit 52 when the seek bar on the recorded video display screen is operated and the playback position is changed. The seek bar operation instruction includes information indicating the changed playback position.

[0144] If a seek bar operation instruction is received (step S75-Yes), the display control unit 633 identifies the frame image at the time specified by the seek bar operation based on the timestamp, and sets the identified frame image as the playback location of the recorded video (step S75a). Then, proceeding to step S73, the display control unit 633 sends the frame image from the video stored in the storage unit 61 that follows the set playback location to the administrator terminal 5. As a result, the video of the store's situation at the time corresponding to the specified playback location is displayed on the administrator terminal 5.

[0145] If no seek bar operation instruction is received (step S75-No), the display control unit 633 determines whether or not a switching instruction has been received (step S76). The administrator terminal 5 sends a switching instruction via the administrator terminal communication unit 52 when the switching tab 510 of the recorded video display screen 500 is selected.

[0146] If a switching instruction is received (step S76-Yes), the video display process proceeds to step S62 to display the live video display screen 400.

[0147] If no switching instruction has been received (step S76-No), the display control unit 633 determines whether or not a termination instruction has been received (step S77). The administrator terminal 5 transmits a termination instruction via the administrator terminal communication unit 52 when an operation input to terminate the display of the live video display screen 400 is received from the administrator terminal operation unit 54.

[0148] If a termination instruction is received (step S77-Yes), the video display process terminates.

[0149] If no termination instruction is received (step S77-No), the process returns to step S73, and the display control unit 633 sends the frame image from the video stored in the storage unit 61 that follows the current playback position back to the administrator terminal 5. In other words, the display control unit 633 sequentially sends the frame images acquired from the camera 2 and stored in the storage unit 61 to the administrator terminal 5 until a task selection instruction, seek bar operation instruction, switching instruction, or termination instruction is received. As a result, the video of the store's situation is displayed on the administrator terminal 5.

[0150] As explained above, the management device 6 uses different manual and automatic completion conditions to set a task to a completed state, depending on whether the task was generated by a manual or automatic generation method. This allows the management device 6 to properly manage whether the work to be performed at the store has been completed.

[0151] Furthermore, the management device 6 calculates the store's busyness level and, based on that level, outputs notifications prompting the execution of tasks that have not been set to a completed state. This enables the management device 6 to appropriately encourage the execution of tasks that should be performed at the store while taking into account the burden on employees.

[0152] Preferably, the management device 6 calculates the level of busyness based on the number of customers, the number of employees, and time information of the store. As a result, the level of busyness is appropriately calculated based on various factors of the store, enabling the management device 6 to appropriately encourage the execution of tasks that should be performed at the store while considering the burden on employees from a more multifaceted perspective.

[0153] Furthermore, the management device 6 outputs a notification to the store terminal 4, and if the task indicated in the notification is not set to a completed state even after a predetermined time has elapsed since the notification was output, it outputs another notification to the administrator terminal 5. As a result, the administrator is notified only when the work that should be performed at the store is not carried out within the predetermined time, so the management device 6 can appropriately encourage the execution of the work that should be performed at the store while reducing the burden on the administrator.

[0154] Furthermore, the management device 6 displays tasks set to a completed state on the administrator terminal 5, and when any of the displayed tasks are selected, it displays a frame image of the work status indicated by the selected task on the administrator terminal 5. This allows the management device 6 to easily grasp the status of work at the store.

[0155] Preferably, the management device 6 calculates the work time based on the task completion time and the type of task, identifies a frame image of the work time, and displays the identified frame image when any task is selected on the administrator terminal 5. This allows the management device 6 to easily grasp the status of the work appropriately according to the type of work.

[0156] Preferably, the management device 6 displays a completed task list on the administrator terminal 5, which shows only tasks set to a completed state that can be selected. This allows only tasks set to a completed state, i.e., tasks for which video footage of the work status has been identified, to be selected, thus enabling the administrator to easily grasp the status of work at the store through video.

[0157] Preferably, the management device 6 displays a seek bar on the administrator terminal 5 indicating the playback position of the video displayed on the administrator terminal 5, and also displays the position of the task's work time on the seek bar in an identifiable manner. This allows the management device 6 to enable the administrator to intuitively grasp the status of work in the store by operating the seek bar.

[0158] Furthermore, the management device 6 displays live video on the administrator terminal 5, and when any task is selected on the administrator terminal 5, it displays a frame image of the work time of the selected task instead of the live video. This allows the management device 6 to understand the store's situation in real time and to further understand the work status.

[0159] Various modifications of the control device 6, as described below, may be applied.

[0160] In the above description, the setting unit 634 uses either manual completion conditions or automatic completion conditions to set a task to a completed state, but the example is not limited to this. The setting unit 634 may also set a task generated by the automatic generation method to a completed state when both manual completion conditions and automatic completion conditions are met.

[0161] In this case, in step S43 of the automatic determination process, the generation unit 632 sets both manual completion conditions and automatic completion conditions as completion conditions for the generated task. Then, in step S44, the setting unit 634 determines whether the automatic completion conditions and manual completion conditions are met, respectively.

[0162] If both the automatic completion condition and the manual completion condition are met, the setting unit 634 sets the task to a completed state, calculates the task's work time, and identifies an image of the work time (steps S45-S47). If the manual completion condition is met, it means that the acquisition unit 631 has already acquired the task completion instruction. If neither the automatic completion condition nor the manual completion condition is met, the automatic determination process ends. If only one of the automatic completion condition or the manual completion condition is met, the setting unit 634 stores information indicating that one of them has been met in association with the task, and the automatic determination process ends.

[0163] Furthermore, in step S32 of the completion status setting process, the setting unit 634 determines whether or not information indicating that the automatic completion conditions have been met is associated with the task that has received a completion instruction. The setting unit 634 sets the task to the completed state only if information indicating that the automatic completion conditions have been met is associated with it.

[0164] In this way, tasks are set to a completed state based on both employee input and sensor information, allowing for more accurate management of whether or not tasks that should be performed in the store have been completed.

[0165] Furthermore, if it is determined in step S44 that the automatic completion condition is met but the manual completion condition is not, the notification unit 637 may output a notification to the store terminal 4 or administrator terminal 5 prompting the user to input an operation. This prevents employees from neglecting to input an operation indicating that they have completed the work, which would result in inadequate management of whether or not the work has been completed.

[0166] In the above description, the generation unit 632 generates tasks based on sensor information as an automatic generation method in step S42 of the automatic determination process, but it is not limited to this example. The generation unit 632 may also generate tasks at predetermined intervals based on a schedule as an automatic generation method.

[0167] In this case, in step S42, the generation unit 632 generates a task of a predetermined type if the current time is a predetermined time. The generation unit 632 sets an automatic completion condition as the completion condition for the generated task if an automatic completion condition is associated with the type of task to be generated in the type table 613, and sets a manual completion condition as the completion condition for the generated task if an automatic completion condition is not associated with the type of task to be generated.

[0168] Tasks generated at predetermined intervals often relate to routine operations. In this case, unlike when tasks are generated manually, it is possible to appropriately determine whether the work has been completed based on sensor information. Therefore, as an automatic generation method, by associating automatic completion conditions with tasks generated at predetermined intervals, it is possible to appropriately determine whether the work has been completed.

[0169] In the above description, in step S12 of the task generation process, the generation unit 632 identifies the type of task based on manually generated keywords included in the work content entered by the user, but the example is not limited to this. The generation unit 632 may also apply known morphological and semantic analysis to the work content string to identify the type of task to be generated from among a plurality of pre-set types that is most relevant to the work content string. Alternatively, the generation unit 632 may identify the type of task to be generated by applying a trained model generated by machine learning to the work content string.

[0170] In the above explanation, in steps S53-S54 of the notification processing, the notification unit 637 outputs a notification prompting the execution of tasks indicated by tasks that have not been set to a completed state based on the store's busyness level. However, the example is not limited to this. The notification unit 637 may also output notifications prompting the execution of tasks indicated by each task, based on the status of each task that has not been set to a completed state.

[0171] The task status may be, for example, the elapsed time since the task's due date. In this case, in step S54, the notification unit 637 outputs a notification only for tasks whose elapsed time since the due date is equal to or greater than a predetermined time. The task status may also be the elapsed time since the task was created. In this case, in step S54, the notification unit 637 outputs a notification only for tasks whose elapsed time since the task was created is equal to or greater than a predetermined time. The task status may also be the number of tasks that are not set to a completed state. In this case, in step S54, the notification unit 637 outputs a notification if the number of tasks that are not set to a completed state is equal to or greater than a predetermined number. The task status may also be the task's priority. In this case, in step S54, the notification unit 637 outputs a notification only for tasks whose priority is equal to or greater than a predetermined value. Furthermore, the task status may be both the elapsed time since the due date or the elapsed time since the task was created and the priority. In this case, in step S54, the notification unit 637 outputs a notification only for tasks whose elapsed time since the due date or the elapsed time since the task was created is equal to or greater than a predetermined time, which differs depending on the priority.

[0172] In the above description, the notification unit 637 outputs a notification to the store terminal 4 in steps S53-S54 of the notification processing and outputs a notification to the administrator terminal 5 in steps S55-S56, but the example is not limited to this. The notification unit 637 may output a notification to both the store terminal 4 and the administrator terminal 5 simultaneously when conditions common to both employees and administrators are met.

[0173] In the above description, the identification unit 635 identifies the frame image of the work time in steps S33-S34 of the completion status setting process and steps S46-S47 of the automatic determination process, but it is not limited to this example. The identification unit 635 may identify the frame image of the task completion time without calculating the work time. In this case, steps S33 and S46 are not executed, and the identification unit 635 associates information indicating that the identified frame image is an image of the completion time of a task set to the completion status with the identified frame image in steps S34 and S46.

[0174] Furthermore, in steps S65a and S74a of the video display processing, the display control unit 633 displays an image of the work status indicated by the selected task, based on the frame image which is the image of the completion time and the type of the selected task. For example, the display control unit 633 obtains the method for calculating the work time associated with the type of the selected task from the type table 613. The display control unit 633 obtains the timestamp of the frame image associated with information indicating that it is an image of the completion time of the selected task. Based on the obtained timestamp and the method for calculating the work time, the display control unit 633 sets the frame image with a timestamp corresponding to the work time of the selected task to the playback location. In this way, the manager can grasp the status of the store in real time and further grasp the status of the work.

[0175] The display control unit 633 may select and play back only the video from the start to the completion of the work. In this case, the start time of the work may be set, for example, to a predetermined time prior to the completion of the work according to the task, or it may be set based on the captured video. The start time of the work may also be set by an employee or manager performing a work start operation on the store terminal 4 at the start of the work. This prevents the manager from unnecessarily continuing to check the video after the completion of the work.

[0176] The display control unit 633 may play back the video of the work at a faster playback speed (for example, 1.5 times faster) if the task requires a certain amount of time or more (for example, 10 minutes or more). This allows the manager to efficiently check the status of the work.

[0177] Figure 17 is a functional block diagram of the management device 16 according to another embodiment. The management device 16 has the same configuration as the management device 6, except for the differences described below. The management device 16 differs from the management device 6 in that it has a storage unit 161 instead of the storage unit 61 and the processing unit 63.

[0178] The storage unit 161 differs from the storage unit 61 in that it stores task table 1612 and type table 1613 instead of task table 612 and type table 613.

[0179] Figure 18 shows the data structure of the task table 1612, and Figure 19 shows the data structure of the type table 1613. The task table 1612 differs from the task table 612 in that it does not store the generation method and completion conditions. The type table 1613 also differs from the type table 1613 in that it stores information (manual / automatic) indicating whether manual or automatic completion conditions are used. In other words, in the management device 16, whether manual or automatic completion conditions are used to set a task to a completed state depends on the type of task.

[0180] The following describes the differences between the processes performed by the management device 16 and those performed by the management device 6.

[0181] In step S13 of the task generation process, the generation unit 632 refers to the type table 1613 to determine whether manual completion conditions or automatic completion conditions are used for the type of task identified in step S12. If automatic completion conditions are used, the generation unit 632 identifies the sensor information to be used to determine the automatic completion conditions based on the work content included in the generation instruction and the identified type. For example, if the work content includes the string "product shelf B" and the type is "product replenishment", the weight sensor installed on product shelf B will be used to determine the automatic completion conditions. The generation unit 632 stores the task in the task table 1612, associating information indicating whether the task is set to a completed state based on manual or automatic completion conditions, the sensor information used to determine the automatic completion conditions, the work content included in the generation instruction, the deadline, and the priority. This generates a task of the identified type.

[0182] Furthermore, in step S43 of the automatic determination process, the generation unit 632 sets the work content and type of the task being generated based on the sensor information for which the generation conditions are met. The generation unit 632 refers to the type table 1613 to determine whether manual completion conditions or automatic completion conditions are used for the set type of task. If automatic completion conditions are used, the generation unit 632 sets the sensor information for which the generation conditions are met as the sensor information used to determine the automatic completion conditions of the task being generated. The generation unit 632 refers to the type table 1613 to set the deadline and priority of the task being generated. The generation unit 632 stores the task associated with the set work content, completion conditions, sensor information, deadline, and priority in the task table 1612.

[0183] As explained above, the management device 16 stores multiple types of tasks and, depending on the type of task, uses either manual or automatic completion conditions to set the task to a completed state. This enables the management device 16 to appropriately manage whether or not the work to be performed at the store has been completed.

[0184] Furthermore, the management device 16 may use different manual completion conditions and automatic completion conditions to set a task to a completed state, depending on the type of task and whether the task was generated by a manual or automatic generation method. In this case, the type table 1613 stores information indicating whether to use manual completion conditions, automatic completion conditions, or completion conditions based on the generation method, instead of information indicating manual / automatic.

[0185] In this case, in step S13 of the generation process, the generation unit 632 refers to the type table 1613 to determine whether manual completion conditions, automatic completion conditions, or completion conditions based on the generation method are used for the type of task identified in step S12. If the type table 1613 contains information indicating that manual completion conditions are used, or information indicating that completion conditions based on the generation method are used, the generation unit 632 determines that manual completion conditions are used. If the type table 1613 contains information indicating that automatic completion conditions are used, the generation unit 632 determines that automatic completion conditions are used. If automatic completion conditions are used, the generation unit 632 identifies sensor information to be used to determine the automatic completion conditions based on the work content included in the generation instruction and the identified type. The generation unit 632 stores the task in the task table 1612, associating information indicating whether the task is set to a completed state based on manual or automatic completion conditions, sensor information used to determine the automatic completion conditions, the work content included in the generation instruction, the deadline, and the priority.

[0186] Furthermore, in step S43 of the automatic determination process, the generation unit 632 sets the work content and type of the task being generated based on the sensor information that satisfies the generation conditions. The generation unit 632 refers to the type table 1613 and determines whether manual completion conditions, automatic completion conditions, or completion conditions based on the generation method are used for the set type of task. If the type table 1613 contains information indicating that manual completion conditions are used, the generation unit 632 determines that manual completion conditions are used. If the type table 1613 contains information indicating that automatic completion conditions are used, or information indicating that completion conditions based on the generation method are used, the generation unit 632 determines that automatic completion conditions are used. If automatic completion conditions are used, the generation unit 632 identifies the sensor information used to determine the automatic completion conditions based on the work content included in the generation instruction and the identified type. The generation unit 632 stores the task in the task table 1612 by associating information indicating whether the task is set to a completed state based on manual completion conditions or automatic completion conditions, sensor information used to determine the automatic completion conditions, the work content included in the generation instruction, the deadline, and the priority.

[0187] In this way, the management device 16 can more appropriately manage whether or not the tasks to be performed in the store have been completed. That is, depending on the type of task, there may be cases where different completion conditions should be used depending on the type of task. For example, for tasks such as "replenishing products" and "cleaning," automatic completion conditions are usually set because it is possible to determine when the work has been completed based on sensor information. However, for "replenishing products" tasks generated by the manual generation method when the wrong products have been placed on the shelves, or for "cleaning" tasks generated by the manual generation method when the floor is not dirty but wet and slippery, it may not be possible to appropriately determine when the work has been completed based on sensor information. By setting completion conditions considering the task generation method in addition to the type of task, the completion of the work can be appropriately determined even in such cases, thus enabling more appropriate management of whether or not the tasks to be performed in the store have been completed.

[0188] In addition, in the management device 16, similar to the management device 6, the setting unit 634 may be set to the completed state when both the manual completion condition and the automatic completion condition are met. In this case, the type table 1613 is pre-set with information indicating whether one or both of the manual completion condition and the automatic completion condition are used, depending on the type of task. The notification unit 637 may output a notification to the store terminal 4 if the automatic completion condition is met but the manual completion condition is not met.

[0189] Those skilled in the art will understand that various changes, substitutions, and modifications can be made without departing from the spirit and scope of the present invention. For example, the embodiments and modifications described above may be combined as appropriate within the scope of the invention. [Explanation of Symbols]

[0190] 1. Store operation support system 6 Management device 631 Acquisition Department 632 Generation part 633 Display Control Unit 634 Setting Department 635 Special Department 636 Calculation Department 637 Notification Department

Claims

1. A means for acquiring video footage from a camera that films a store, A setting means that acquires multiple tasks indicating the work to be performed at the aforementioned store, and sets each task to a completed state when predetermined completion conditions are met, When each task is set to a completed state, the identification means identifies the position on the time axis of the video corresponding to the completion time when the task was set to a completed state, A display control means that allows the administrator terminal used by the store manager to display tasks set to the completed state in a selectable format, and when any of the displayed tasks is selected on the administrator terminal, displays on the administrator terminal a video or still image of the work status shown by the selected task, based on the time axis position of the identified video for the selected task, the task set to the completed state. A store operation support system characterized by having the following features.

2. The display control means causes the administrator terminal to display the video display area and the task display area on the same screen, and when any of the tasks displayed in the task display area is selected by the administrator terminal, it displays a video or still image of the work situation indicated by the task in the video display area, based on the time axis position of the video identified for that task. The store operation support system according to claim 1.

3. The display control means further displays live video showing the store's status in real time in the video display area, and when one of the tasks displayed in the task display area is selected, it replaces the live video with a video or still image of the work performed by the selected task in the video display area. The store operation support system according to claim 2.

4. The display control means causes the administrator terminal to display a list of the acquired tasks, in which only the tasks set to the completed state can be selected. A store operation support system according to any one of claims 1 to 3.

5. The display control means further displays a seek bar on the administrator terminal that indicates the time when the video displayed on the administrator terminal was taken, and displays the position on the seek bar at the time when the video showing the status of the work indicated by the task set to the completed state was taken, so as to be identifiable. A store operation support system according to any one of claims 1 to 4.

6. The identifying means estimates the time at which the work indicated by the task was performed based on the completion time of the task set to the completed state and the type of the task, and identifies the position on the time axis of the video corresponding to that work time. The display control means, when any of the tasks displayed on the administrator terminal is selected, causes the administrator terminal to display the video or still image at the specified time axis position for that task as a video or still image capturing the status of the work indicated by the selected task. A store operation support system according to any one of claims 1 to 5.

7. The display control means, when one of the tasks displayed on the administrator terminal is selected, causes the administrator terminal to display a video or still image of the work situation represented by the selected task, based on the identified position on the time axis and the type of the selected task. A store operation support system according to any one of claims 1 to 5.

8. A store operation support method executed by a store operation support system, The acquisition step involves obtaining video footage from a camera that films the store, A setting step which involves acquiring multiple tasks that indicate the work to be performed at the store, and setting each acquired task to a completed state when predetermined completion conditions are met, When the task is set to a completed state, the identification step involves identifying the position on the time axis of the video corresponding to the time when the task was set to a completed state, A first display control step in which the manager of the store makes selectable tasks set to the completed state on the manager terminal used by the manager of the store, A second display control step, in which, when any of the displayed tasks is selected on the administrator terminal, the administrator terminal is shown a video or still image of the work status indicated by the selected task, which is set to the completed state, based on the time axis position of the identified video for the selected task; A method for supporting store operations, characterized by including the following:

Images