Guidance control device, information processing system, information processing method, program

Abstract

To provide a guidance control device that identifies a prescribed action to be performed by a subject and guides the subject to the prescribed action without using a display device.SOLUTION: A guidance control device controls an image projection with a projection device. The guidance control device generates a subject-specific image to be projected onto a first projection range corresponding to a first position where the subject performs a predetermined action and generates a guide image to be projected onto a second projection range corresponding to a second position visible to the subject to guide the subject to the predetermined action. Thereby, the guidance control device controls the projection device to project a projection image including the subject-specific image and the guide image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a guidance control device, an information processing system, an information processing method, and a program.

Background Art

[0002] A technology is known that displays guidance information on a screen to prompt a worker to perform a predetermined work at a work site such as a product distribution center. For example, in a distribution center, in order to sort products individually for each delivery destination, a technology is used that displays guidance information for notifying workers of which product and how many products are to be delivered to which delivery destination on a predetermined monitor such as a digital display device. Patent Document 1 discloses an article sorting facility capable of displaying information related to article sorting work (work instruction information) without using a display. Specifically, in picking facilities where articles ordered by users are collected into collection containers allocated with user orders and delivered, work instruction information is displayed on a head-mounted display (HMD) system using augmented reality (AR).

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, in the technology for displaying guidance information for workers, it is necessary to present guidance information so that the worker can perform the work accurately without errors without using a display device when performing the work. At work sites such as product distribution centers, a plurality of workers perform work simultaneously at close positions, the positional relationship between work objects (products) and work locations (sorting containers) is complicated, and there is a possibility that work errors may occur. In addition, having a worker wear a head-mounted display imposes a burden on the worker.

[0005] The present invention was made to solve the above-mentioned problems and aims to provide an information processing device and guidance control device, an information processing system, an information processing method, and a program for presenting guidance information such as work instructions. [Means for solving the problem]

[0006] A first aspect of the present invention is a guidance control device that can communicate with a projection device. The guidance control device includes a projection control unit that generates a person identification image to be projected onto a first projection range corresponding to a first position where the person performs a predetermined action, and generates a guidance image to be projected onto a second projection range corresponding to a second position visible to the person to guide the person to perform the predetermined action, and controls the projection device to project a projection image that includes the person identification image and the guidance image.

[0007] A second aspect of the present invention is a guidance control system comprising a projection device and a guidance control device that can communicate with the projection device. The guidance control device generates a person identification image to be projected onto a first projection range corresponding to a first position where the person performs a predetermined action, and generates a guidance image to be projected onto a second projection range corresponding to a second position visible to the person to guide the person to perform the predetermined action, and controls the projection device to project a projection image including the person identification image and the guidance image.

[0008] A third aspect of the present invention is a guidance control method. The guidance control method generates a subject identification image to be projected onto a first projection range corresponding to a first position in which the subject performs a predetermined action, generates a guidance image to be projected onto a second projection range corresponding to a second position visible to the subject to guide the subject to the predetermined action, and projects a projection image including the subject identification image and the guidance image.

[0009] A fourth aspect of the present invention is a storage medium that stores a program causing a computer to execute a predetermined processing process. The predetermined processing process comprises: a process for generating a subject identification image to be projected onto a first projection range corresponding to a first position where the subject performs a predetermined action; a process for generating a guidance image to be projected onto a second projection range corresponding to a second position visible to the subject to guide the subject to the predetermined action; and a process for projecting a projection image including the subject identification image and the guidance image. [Effects of the Invention]

[0010] According to the present invention, a predetermined action to be performed by a subject can be identified and the subject can be guided to perform the predetermined action without using a display device. Furthermore, the subject can perform the predetermined action accurately and without errors. [Brief explanation of the drawing]

[0011] [Figure 1] This is a perspective view showing the overall configuration of a guidance and control system according to one embodiment of the present invention. [Figure 2] This is a plan view showing the work situation at a work site to which a guidance control system according to one embodiment of the present invention is applied. [Figure 3] This is a hardware configuration diagram of a guidance control device according to one embodiment of the present invention. [Figure 4] This is a functional block diagram of a guidance control device according to one embodiment of the present invention. [Figure 5] This flowchart shows the processing procedure of a terminal applied to a guidance control device according to one embodiment of the present invention. [Figure 6] This is a flowchart of the image projection process by a guidance control device according to one embodiment of the present invention. [Figure 7] Examples of various data tables used in image projection processing are shown. [Figure 8] An example of a projected image generated by a guidance control device according to one embodiment of the present invention is shown. [Figure 9] This flowchart shows the process for generating projected images, including guidance images for workers. [Figure 10] This flowchart shows the sorting work determination process by a guidance control device according to one embodiment of the present invention. [Figure 11] This is a block diagram showing the minimum configuration of a guidance control device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0012] The guidance control device and guidance control method according to the present invention will be described with reference to the attached drawings. Figure 1 is a perspective view showing the overall configuration of a guidance control system 100 to which a guidance control device 1 according to one embodiment of the present invention is applied. The guidance control system 100 comprises a guidance control device 1, a projector (projection device) 2, a camera (shooting device) 3, and a terminal (management device) 5. The guidance control device 1 is communicated with the projector 2, the camera 3, and the terminal 5. The projector 2 is installed above the worker and projects an image downwards. In other words, the projector 2 projects the image output from the guidance control device 1 into a predetermined projection range. The camera 3 is installed above the worker and captures an image within a predetermined shooting range below, and transmits the captured image to the guidance control device 1. The terminal 5 is used by the worker. When the work object is a product, the guidance control system 100 is installed in a distribution center that delivers products to stores that serve as product delivery destinations.

[0013] For the sake of explanation, Figure 1 shows that one projector 2 and one camera 3 are connected to the guidance control device 1 via communication. However, in reality, the guidance control device 1 is connected to multiple projectors 2 and multiple cameras 3 via communication. In this case, the projectors 2 project images to different projection ranges. The cameras 3 capture images that appear in different shooting ranges. Note that the projection device is not limited to a projector; any device capable of projecting a predetermined image to a predetermined location is acceptable. Similarly, the capturing device is not limited to a camera; any device capable of capturing images within a predetermined shooting range with good resolution is acceptable. Furthermore, the management device is not limited to a terminal; any device capable of providing predetermined instructions and information input to the guidance control device 1 is acceptable. In this embodiment, the predetermined actions performed by the worker (subject) are described as sorting goods destined for stores, but other items besides those sold in stores (for example, parts necessary for assembling industrial products) may also be handled.

[0014] At the distribution center, workers sequentially place items from a product basket 41 on a cart 4 into delivery baskets 51 corresponding to predetermined delivery destinations. At this time, the guidance control device 1 controls the projector 2 so that a target identification image is projected onto the first projection area of ​​the aisle floor in front of the delivery basket 51 for the desired delivery destination, in order to identify the worker who should place the items into that delivery basket 51. The guidance control device 1 controls the system so that a guidance image 420 is projected onto a guidance plate 42 (second projection area) attached to the cart 4 when the worker pushing the cart 4 that has stopped in front of the delivery basket 51 for the desired delivery destination is the worker who should place the items into that delivery basket 51. The guidance plate 42 is a plate-shaped structure onto which a guidance image 420 for sorting work is projected for the worker. In this embodiment, as an example, the guidance plate 42 is attached to the top of the cart 4.

[0015] Figure 2 is a plan view showing a work situation in a distribution center. Figure 2 is a plan view of the work situation of a plurality of workers viewed from above. In Figure 2, on both sides of the passage 6, a plurality of delivery baskets 51 for different delivery destinations are arranged for each delivery destination area. A first projection area 61 is located on the floor surface of the passage 6 near the delivery basket 51. A subject specifying image 610 for specifying a worker is projected onto the first projection area 61. For example, one of a plurality of preset images (subject specifying images 610) such as a plurality of commodities and articles (e.g., chicks, grapes, etc.) is assigned to a worker. The guidance control device 1 specifies, among the subject specifying images 610 assigned to workers, a worker who is to sort commodities to be delivered to a desired delivery destination. The guidance control device 1 specifies the subject specifying image 610 assigned to the worker, and controls the projector 2 such that the subject specifying image 610 is projected onto the first projection area 61 in front of the delivery basket 51 for the delivery destination. The guidance control device 1 controls the projector 2 such that a guidance image 420 is projected onto a guidance plate 42 provided on a cart 4.

[0016] The guidance control device 1 generates a projection image including the subject specifying images 610 corresponding to a plurality of delivery destinations and the guidance image 420, and controls the projector 2 installed in a delivery destination area corresponding to the delivery basket 51 corresponding to these images to project the projection image.

[0017] Figure 3 is a hardware configuration diagram of the guidance control device 1. The guidance control device 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an HDD (Hard Disk Drive) 104, an interface 105, and a communication module 106. Note that the HDD 103 may alternatively be an SDD (Solid State Drive).

[0018] Figure 4 is a functional block diagram of the guidance control device 1. The guidance control device 1 implements the functional units 10 to 15 by having the CPU 101 read and execute a guidance control program that is pre-stored in a memory unit such as ROM 102. Specifically, the guidance control device 1 comprises a control unit 10, a registration reception unit 11, a projection control unit 12, a worker recognition unit 13, an action management unit 14, and a management information update unit 15. The control unit 10 controls the registration reception unit 11, the projection control unit 12, the worker recognition unit 13, the action management unit 14, and the management information update unit 15.

[0019] The registration reception unit 11 accepts registration of the correspondence between a target person (worker) and information indicating a prescribed action (sorting work) to be performed by that person. Information indicating a prescribed action includes, for example, the type of product to be sorted and the quantity of products to be placed in the product basket 41 of the cart 4. The projection control unit 12 generates a projection image including a target person identification image 610 and a guidance image 420, and outputs the projection image to the projector 2. As a result, the projector 2 projects the projection image onto a predetermined area near the delivery basket 51. The worker recognition unit 13 recognizes the worker based on the image captured by the camera 3. The action management unit 14 recognizes the sorting work performed by the worker based on the image captured by the camera 3. The management information update unit 15 updates the sorting management table, which stores management information related to the work, according to whether the work has been completed or not. The guidance control device 1 is connected to the database 40 which records the sorting management table.

[0020] Next, we will explain the processing procedure of terminal 5, which is used by workers when they start their work. Figure 5 is a flowchart of the processing procedure of terminal 5 (steps S101 to S103). As an example, a worker places one or more of a single type of product into a product basket 41 on a cart 4, and then sequentially places them into delivery baskets 51 located on the floor of the distribution center to perform product sorting. First, the worker registers the products they will sort using terminal 5. Specifically, a registration image is displayed on the monitor of terminal 5. The registration image displays buttons for selecting the worker's name, selecting the product, and selecting the quantity of products to be transported at one time on cart 4. As an example, the monitor of terminal 5 is a display equipped with a touch panel. On the monitor of terminal 5, the worker selects their own name, identification information (ID), and other worker identification information, selects the product, and selects the transport operation for the products to be transported at one time on cart 4. For example, after selecting their own name and ID, the worker selects "eggs" from among several different products and chooses "100" eggs to be transported at once in cart 4. Terminal 5 acquires the worker's selected information (worker identification information, product identification information, transport quantity, etc.) (S101). Based on the worker's selected information, Terminal 5 generates a confirmation screen showing the product name "eggs," transport quantity "100," a person identification image 610 (for example, an image of a "chick") stored in association with the worker identification information, and a confirmation button, and displays it on the monitor (S102). The "chick" in the person identification image 610 is an image that reminds the worker of the object of their work (for example, the product "eggs"). By viewing the person identification image 610, the worker can recognize that the image displayed on the monitor of Terminal 5 represents their own work. The worker checks the information displayed on the confirmation screen, and if the product name and quantity to be transported for the sorting work to be performed by the worker match the information registered in advance, the worker recognizes the person identification image 610 and operates the confirmation button. When terminal 5 detects the operation of the confirmation button, it outputs a registration request to the guidance control device 1 including the worker ID corresponding to the worker's name, the product ID of the product "eggs", the quantity of eggs to be transported, and the person image ID indicating the person identification image 610 corresponding to the worker ID (S103). The worker puts 100 eggs into the product basket 41 and places it on the cart 4.The worker pushes cart 4 and moves along the aisle lined with delivery baskets 51 on both sides to sort the product "eggs".

[0021] Next, we will explain the processing of the image projected by the guidance control device 1 under the control of the projector 2 (image projection processing). Figure 6 is a flowchart of the image projection processing (S201~S208). Figure 7 shows examples of various data tables used in the image projection processing. First, the registration reception unit 11 of the guidance control device 1 receives a registration request from the terminal 5 (S201). The registration reception unit 11 links the worker ID, the product ID of the product to be sorted, the quantity to be transported, and the person's image ID and writes them to the sorting management table stored in the database 40 (S202). As a result, information about the products that the worker will sort is recorded in the database 40.

[0022] In Figure 7, the product delivery table stores, linked to the identification information of the delivery destination store (store ID), the identification information of the product the store wishes to deliver (product ID), the quantity of the product the store wishes to deliver, the quantity of products remaining until sorting is completed (remaining sorting quantity), and a flag indicating whether sorting is complete. The projection management table stores the identification information of the delivery destination store (store ID), the coordinates of the first projection range 61 indicating the projection position of the projection image corresponding to the delivery basket 51 addressed to that store, and the identification information of the projector 2 that projects the image to those coordinates (projector ID). The projection control table stores the worker ID, product ID, target person identification image ID, and store ID. The guidance control device 1 generates the above data tables and stores them in a storage unit such as RAM 103.

[0023] The projection control unit 12 of the guidance control device 1 reads the recorded information from the sorting management table. The projection control unit 12 also reads the stored information from the product delivery table. When a new registration request is issued, the projection control unit 12 obtains the product ID and transport quantity of the product to be sorted as included in the registration request. The projection control unit 12 identifies one or more store IDs on the product delivery table that are not "0" for the remaining sorting quantity associated with the product ID (S203). These store IDs correspond to the stores to which the product related to the product ID included in the registration request will be delivered. The projection control unit 12 also generates projection control information that includes the worker ID, product ID, and target identification image ID included in the registration request, as well as one or more store IDs (S204).

[0024] Figure 8 shows an example of a projected image projected by a projector 2 controlled by a guidance control device 1. The projected image includes a storefront image projected onto a first projection range 61 located in front of one or more delivery baskets 51 destined for stores, and a guidance image 420 projected onto the guidance plate 42 of the cart 4. The storefront image includes one or more target identification images 610. The projection control unit 12 generates projection control information and generates a projected image as shown in Figure 8 based on the projection control information. If the guidance plate 42 of the cart 4 is not included in the projection range of the projected image, the guidance image 420 is not included in the projected image generated by the projection control unit 12. As described above, since the projected image includes storefront images projected onto a first projection range 61 located in front of multiple delivery baskets 51 destined for stores, in this embodiment, the projected image is stored in association with the store ID corresponding to the multiple storefront images. The projection control unit 12 stores coordinates indicating the range of the storefront image for each store included in the projected image, for each store ID.

[0025] The projection control unit 12 reads one projection control information recorded in the projection control table and obtains the store ID included in that projection control information (S205). The projection control unit 12 obtains the projection image associated with the store ID from a storage unit such as RAM 103. The projection control unit 12 also obtains the coordinates of the projection range of the storefront image corresponding to the store ID. The projection control unit 12 obtains the subject identification image 610 from the database 40. The projection control unit 12 generates a projection image including the subject identification image 610 corresponding to the coordinates of the projection range of the storefront image identified by the store ID (S206). The projection control unit 12 stores the projection image.

[0026] The projection control unit 12 identifies the projector 2 to project the image based on the projector ID recorded in the projection management table, which is linked to the store ID. Based on the projector ID, the projection control unit 12 obtains the network address of the projector 2 to which the projected image will be sent. The projection control unit 12 sends the projected image to the projector 2 identified by the network address (S207). The projector 2 projects the image received from the projection control unit 12 onto a predetermined projection area on the aisle in front of the delivery cart 51.

[0027] The projection control unit 12 determines whether the projection control information includes other store IDs (S208). If the projection control information includes other store IDs, it generates or updates the projection image linked to the other store ID and performs projection control according to the above process. The projection control unit 12 may also determine whether the next projection control information has been generated. If the next projection control information has been generated, the projection control unit 12 repeats projection control to generate or update the projection image based on the next projection control information. The projection control unit 12 terminates projection control when it terminates the image projection process, such as when the power to the guidance control device 1 is turned off.

[0028] As described above, the guidance control device 1 generates a storefront image containing one or more target identification images 610, and projects multiple projected images containing the storefront images onto a first projection area 61 on the aisle. When a worker sees their own target identification image 610 displayed in the storefront image, they recognize that they need to sort goods into a delivery basket 51 destined for a specific store. For example, if a worker registers "eggs" as the target product for sorting using the terminal 5, they can move the cart 4 along the aisle, stop near the first projection area 61 where their own target identification image 610 is projected, and place a predetermined number of eggs into the delivery basket 51 corresponding to that first projection area 61. This allows workers to determine which store to sort simply by looking at the target identification image 610 projected on the floor, improving the efficiency of sorting work and reducing sorting errors.

[0029] When a worker stops the cart 4 in front of the first projection area 61 on which their own person identification image 610 is projected, a guidance image 420 is projected onto the guidance plate 42, which includes the name of the product to be sorted and the quantity of the product to be placed in the shopping basket 51.

[0030] Next, the process of generating a projected image including a guidance image 420 for the worker will be described. Figure 9 is a flowchart of the process of generating a projected image including a guidance image by the guidance control device 1 (steps S300 to S308). The projection control information described above is generated immediately after the worker completes the registration of the products to be sorted. On the other hand, the guidance image 420 is generated by the guidance control device 1 when the worker has detected the worker after product registration is complete, when the worker moves the cart 4 to perform the sorting work, positions themselves in front of the store basket 51 corresponding to the store to which the products will be delivered, and enters the shooting range of the camera 3.

[0031] The worker pushes cart 4 into the aisle and moves while checking whether their own person identification image 610 is projected onto the first projection area 61 in front of the shopping basket 51. If the worker's own person identification image 610 is projected onto the first projection area 61 on the aisle floor, the worker stops cart 4 at or near the cart placement position corresponding to that first projection area 61. Camera 3 takes images within a predetermined shooting range at predetermined time intervals (for example, 1 second) and transmits the captured images to the guidance control device 1. Note that camera 3 may transmit moving images instead of still images to the guidance control device 1. The guidance control device 1 receives the captured images or moving images from camera 3.

[0032] The operator recognition unit 13 of the guidance control device 1 acquires the captured image (or video) received from the camera 3 (S300). For example, the shooting range of the camera 3 includes multiple shopping carts 51 and the aisle area in front of those shopping carts 51. The shooting range of the camera 3 and the projection range of the projector 2 may or may not overlap. The guidance control device 1 stores in the database 40 the correspondence between the coordinates of the image captured in the shooting range of the camera 3 and the coordinates and store IDs of the projected image projected into the projection range of the projector 2. Therefore, when updating the projected image, the guidance control device 1 can recognize which store's store ID corresponds to which projected image to update based on the coordinates of the captured image.

[0033] The worker recognition unit 13 detects whether the image captured by camera 3 contains a worker identification code (S301). The worker identification code is printed on, for example, a cart 4 or an information plate 42 that can be photographed by camera 3. Alternatively, the worker identification code may be printed in a location that can be photographed by camera 3, such as on the work clothes worn by the worker. The worker identification code may be a two-dimensional code or a combination of alphanumeric characters. When the worker recognition unit 13 detects a worker identification code in the captured image, it retrieves a worker ID that has been stored in advance and is associated with that worker identification code from the database 40. The worker recognition unit 13 also retrieves the coordinates of the worker identification code in the captured image. The worker recognition unit 13 retrieves the store ID corresponding to the coordinates of the worker identification code in the captured image from the database 40. For example, the worker recognition unit 13 sends a request to the projection control unit 12 to generate a guidance image, which includes the worker ID, the store ID corresponding to the coordinates of the worker identification code in the captured image, and the camera ID indicating the camera 3 that transmitted the captured image used to acquire the identification information (S302). The worker recognition unit 13 also sends a request to the activity management unit 14 to confirm an activity, which includes the worker ID, the product ID recorded in the sorting management table linked to the worker ID, the store ID obtained in the above process, and the camera ID (S303).

[0034] The projection control unit 12 acquires the image captured by camera 3 corresponding to the camera ID included in the guidance image generation request. The projection control unit 12 detects the coordinates of the guidance plate 42 shown in the image captured by camera 3. The projection control unit 12 acquires the store ID from the guidance image generation request. The projection control unit 12 also identifies the projector 2 corresponding to the coordinates of the guidance plate 42 shown in the image captured by camera 3. The projection control unit 12 converts the coordinates of the guidance plate 42 in the captured image to the coordinates of the guidance plate 42 in the projected image and determines the position of the guidance image 420 to be placed in the projected image (S304).

[0035] The projection control unit 12 obtains the worker ID and store ID from the guidance image generation request. The projection control unit 12 also obtains the product ID, which is recorded in the sorting management table and is associated with the worker ID. The product ID is information indicating the product that the worker should place in the store basket 51.

[0036] The projection control unit 12 retrieves the remaining sorting quantity of products recorded in the product delivery table, linked to the product ID and store ID obtained from the guidance image generation request. The remaining sorting quantity refers to the number of products that need to be sorted for the store indicated by the store ID.

[0037] The projection control unit 12 retrieves the transport quantity recorded in the sorting management table, which is linked to the worker ID and product ID. The transport quantity is the quantity of the product indicated by the product ID that the worker indicated by the worker ID is currently carrying in the cart 4.

[0038] The projection control unit 12 determines the remaining sorting quantity to be the worker's sorting quantity if the remaining sorting quantity is 1 or greater and the transported quantity is greater than or equal to the remaining sorting quantity. The projection control unit 12 determines the transported quantity to be the worker's sorting quantity if the remaining sorting quantity is 1 or greater and the transported quantity is less than the remaining sorting quantity. If the sorting quantity is 1 or greater, the projection control unit 12 generates action information describing the worker ID, store ID, product ID, and sorting quantity (S305). The projection control unit 12 stores the action information in a memory unit such as RAM 103. The action information indicates what sorting work the worker should perform. If the remaining sorting quantity is "0", the projection control unit 12 determines that the sorting quantity is zero and terminates the sorting work.

[0039] The projection control unit 12 generates a guide image 420 that shows the product name corresponding to the product ID, the product image corresponding to that product ID, and the sorting quantity for the worker (S306). The projection control unit 12 then generates or updates a projected image with the guide image 420 placed in a predetermined position (S307). The projection control unit 12 obtains the network address of the projector 2 corresponding to the coordinates of the guide plate 42 and transmits the projected image to that network address (S308). The projector 2 projects the projected image received from the projection control unit 12 onto the first projection range 61 and the guide plate 42. The guide control device 1 executes the above steps S300 to S308 in parallel for images captured by multiple cameras 3.

[0040] Through the process described above, the guidance image 420 is projected onto the guidance plate 42 installed on the cart 4. The guidance control device 1 determines the position of the guidance image 420 to be placed within the projected image according to the coordinates of the guidance plate 42. Therefore, if the position of the guidance plate 42 is included in the shooting range of the camera 3, and the coordinates within the projection range of the projector 2 corresponding to the coordinates of the guidance plate 42 in the captured image can be identified, the guidance control device 1 can generate a projected image that is aligned so that the guidance image 420 is projected onto the position of the guidance plate 42.

[0041] The worker visually checks the product name, product image, and sorting quantity included in the guidance image 420 projected onto the guidance plate 42 to confirm that there is no discrepancy between the type and quantity of goods the worker is carrying and the information. The worker then takes the sorting quantity of goods from the goods basket 41 and places them into the store basket 51. This completes the worker's sorting of goods destined for the designated store. After that, the worker moves the cart 4 to a position where the person identification image 610 representing the worker is projected onto the aisle on the floor, and continues sorting goods again until all the goods in the goods basket 41 are gone.

[0042] By displaying the guidance image 420 on the guidance plate 42, workers can accurately recognize the work content. Furthermore, workers can perform the work quickly and accurately by following the guidance image 420. Note that the guidance image 420 does not need to be projected onto the guidance plate 42; it may also be projected onto other floor areas or the delivery basket 51.

[0043] The guidance control device 1 needs to determine whether a worker has performed sorting work and, if sorting work has been performed, update the information in the sorting management table and the product delivery table. The worker recognition unit 13 may also determine whether the position of the worker corresponding to the person identification image 610 projected onto the first projection range 61 of the aisle floor surface matches the position of the store that the worker is requesting delivery of the goods being transported (i.e., the position of the store basket 51). In this case, the worker recognition unit 13 obtains the product ID recorded in the sorting management table, linked to the worker ID corresponding to the worker identification code in the image captured by the camera 3. The worker recognition unit 13 also obtains the product ID recorded in the product delivery table, linked to the store ID of the store corresponding to the coordinates of the worker identification code in the image captured by the camera 3. If the above two product IDs match, it can be determined that the worker's position matches the position where the worker will perform sorting work to send the goods being transported to the store that is requesting delivery. The projection control unit 12 may control the projector 2 to project a guidance image 420 prompting the worker to perform sorting work onto the guidance plate 42 (second projection range) if it determines that the position of the worker corresponding to the person identification image 610 projected in the first projection range 61 coincides with the position where sorting work is performed.

[0044] Next, we will explain the process by which the guidance control device 1 determines whether the worker has performed the sorting task (sorting task determination process). Figure 10 is a flowchart of the sorting task determination process (steps S401 to S414). The action management unit 14 of the guidance control device 1 acquires an action confirmation request sent from the worker recognition unit 13 (S401). The action management unit 14 also acquires the image captured by camera 3 corresponding to the camera ID included in the action confirmation request, in the same manner as the worker recognition unit 13 (S402). The action management unit 14 detects the skeletal information of the worker captured in the image captured by camera 3 (S403). As mentioned above, camera 3 captures the worker from above. Since the captured image includes the worker's head and arms, the direction of arm extension when the worker's head and arms are extended can be detected by image analysis. Based on the skeletal information, the action management unit 14 calculates the length of the worker's arms and the direction of arm extension as vectors in the captured image. Note that known techniques can be used to calculate vectors related to the size and direction of movement of objects. The action management unit 14 acquires the coordinates of the delivery basket 51 addressed to the store corresponding to the store ID recorded in the action confirmation information from the database 40. The action management unit 14 determines whether the tip of the worker's arm has entered the coordinate range of the delivery basket 51 based on a vector indicating the direction of extension of the worker's arm and the coordinates of the delivery basket 51 (S404). That is, the action management unit 14 determines whether the tip of the arm (or the worker's hand), determined by the vector of the direction of extension of the worker's arm and the length of the worker's arm, has entered the coordinate range of the delivery basket 51 corresponding to the store ID recorded in the action confirmation request. The action management unit 14 determines that the tip of the worker's arm has entered the coordinate range of the delivery basket 51 corresponding to the store ID recorded in the action confirmation request and determines that the sorting work is complete (S405). The action management unit 14 determines the completion of work based on whether a part of the worker's body has entered a predetermined range of the position related to a predetermined action, but the above determination is one aspect of the determination process of the action management unit 14. If the result of step S404 is "NO", the action management unit 14 determines whether the tip of the worker's arm has entered the coordinate range of another delivery basket 51 based on the extension vector above the worker and the coordinates of the delivery basket 51 (S406). In other words, the action management unit 14 determines whether the tip of the arm, determined by the aforementioned vector and the length of the worker's arm, has entered the coordinate range of a delivery basket 51 destined for a store other than the delivery basket 51 corresponding to the store ID included in the action confirmation request.If the result of the judgment in step S406 is "NO", the processing flow returns to step S401, and the action management unit 14 does not determine that the sorting work is complete.

[0045] If the result of step S404 is "YES", the action management unit 14 determines that the sorting work is complete (S405) and determines whether the action information, including the worker ID, store ID, and product ID included in the action confirmation request, is stored in a storage unit such as RAM 103. The action information is the information generated when the projection control unit 12 generates the guidance image 420, as explained in Figure 9. If the action information, including the worker ID, store ID, and product ID, is stored in the storage unit, the action management unit 14 obtains the sorting quantity included in the action information. If the action management unit 14 determines that the sorting work is complete, it confirms the sorting quantity of the goods by the worker (S407). The action management unit 14 reads the transport quantity recorded in the sorting management table linked to the worker ID and product ID, and subtracts the value of the sorting quantity from the transport quantity (S408). Furthermore, the work management unit 14 reads the remaining sorting quantity recorded in the product delivery table, which is linked to the store ID and product ID, and subtracts the sorting quantity included in the activity information from the remaining sorting quantity (S409). As a result, when an worker completes the sorting work, the remaining sorting quantity of products to be delivered to the store is updated.

[0046] If the result of step S406 is "YES", the action management unit 14 determines that the worker's armpit has entered the coordinate range of a delivery basket 51 destined for a store other than the delivery basket 51 corresponding to the store ID in the action confirmation request, and determines that there is an error in the worker's sorting work. In this case, the action management unit 14 generates a warning image (or warning information) (S410). The action management unit 14 sends a warning image projection request to the projection control unit 12, which includes the warning image and the worker ID and store ID included in the action confirmation request. If the action management unit 14 is unable to determine that the worker's armpit has entered the coordinate range of a delivery basket 51 corresponding to the store ID included in the action confirmation request for a predetermined period of time or longer, it may display a warning image to the worker indicating that the sorting work is not complete on a display device installed on the wall of the delivery center or elsewhere.

[0047] The projection control unit 12 identifies the coordinate range of the guide plate 42 on the cart 4 being pushed by the worker corresponding to the worker ID, based on the store ID and worker ID included in the warning image projection request, from the image captured by camera 3. As explained in Figure 9, the projection control unit 12 identifies the coordinate range of the guide plate 42 that is visible in the image captured by camera 3. The projection control unit 12 converts the coordinates of the guide plate 42 in the captured image to the coordinates of the guide plate 42 in the projected image and determines the position of the warning image to be placed in the projected image (S411). Then, the projection control unit 12 generates a projected image with the warning image (warning information) placed at the above position (S412). The projection control unit 12 obtains the network address of the projector 2 corresponding to the coordinates of the guide plate 42 and transmits the projected image to that network address (S413). As a result, the projector 2 projects the projected image into a predetermined projection range. After that, the action management unit 14 determines whether to terminate the above process depending on whether it has received other action confirmation requests (S414). If other action confirmation requests have been received, the processing flow returns to step S401 and the above steps are repeated.

[0048] As described in steps S406, S410-S413, if an operator mistakes a store basket 51 into which goods to be sorted should be placed, a warning image (warning information) can be displayed to the operator. This improves the accuracy of the sorting process.

[0049] The guidance control device 1 may calculate the total delivery volume for each store based on the sum of the delivery quantities of products destined for stores corresponding to the store IDs recorded in the product delivery table, and then calculate the size of the space along the aisle where the store baskets 51 will be placed based on that total delivery volume. In other words, workers may place the delivery baskets 51 on the floor of the delivery center based on the size of the space for placing the store baskets 51 for each store, according to the total delivery volume calculated by the guidance control device 1. This allows the delivery baskets 51 to be appropriately placed along the aisle according to the total delivery volume for each store.

[0050] Figure 11 is a block diagram showing the minimum configuration of the guidance control device 1. The guidance control device 1 must include at least a projection control unit 12. The projection control unit 12 controls the projection device (projector 2) to project a person identification image 610 indicating a person (worker) onto a first projection range 61 related to the position where the person will perform a predetermined action, and to project a guidance image 420 that guides the person to perform a predetermined action onto a second projection range related to the position where the person will perform a predetermined action.

[0051] In the above description, the configuration and function of the guidance control device 1 were described in a situation where the guidance control system 100 is installed in a distribution center and workers are performing product sorting work. However, the guidance control device 1 is not limited to distribution centers and may be used for other purposes. For example, it may generate a guidance image 420 that can be seen by customers in a store, and project an image including the guidance image 420 into a projection range near the position where the customer takes an item from a shelf. The guidance image 420 may be, for example, an advertisement for a specific product.

[0052] The guidance control device 1 has an internal computer system, and the above-described processing steps are stored as a computer program on a computer-readable recording medium. That is, the above-described processing steps are executed when the computer reads the computer program from the recording medium and executes it. Here, a computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, the computer program may be distributed to a computer via a communication line, and the computer may execute the computer program. The above-described computer program may implement only a part of the functions of the guidance control device 1 described above. Furthermore, the above-described computer program may be a differential file (differential program) that can implement the above-described functions in combination with a pre-installed program already stored in the computer system.

[0053] Finally, the present invention is not limited to the embodiment described above, but also includes various modifications and design changes that do not depart from the scope of the invention as defined in the attached claims.

[0054] Although several embodiments of the present invention have been described, these embodiments are illustrative and do not limit the scope of the invention. These embodiments may be modified in various ways without departing from the spirit of the invention.

[0055] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0056] (Note 1) A guidance control device that can communicate with a projection device, A guidance control device comprising a projection control unit that generates a subject identification image to be projected onto a first projection range corresponding to a first position in which the subject performs a predetermined action, generates a guidance image to be projected onto a second projection range corresponding to a second position in which the subject can see, to guide the subject to the predetermined action, and controls the projection device to project a projection image including the subject identification image and the guidance image.

[0057] (Note 2) The guidance control device according to Appendix 1, further comprising a subject recognition unit that recognizes subject identification information related to the subject identification image, wherein the projection control unit controls the projection device to project the guidance image onto the second projection range in accordance with the subject identification information.

[0058] (Note 3) An action management unit that determines whether the aforementioned person has completed the aforementioned prescribed action, A management information update unit updates management information related to the prescribed action in accordance with the determination result of the aforementioned action management unit, The guidance control device described in Appendix 2 further comprises the following:

[0059] (Note 4) The action management unit detects when a part of the subject's body enters a predetermined range corresponding to the first position where the predetermined action is performed, and determines whether the predetermined action has been completed, as described in Appendix 3.

[0060] (Note 5) The guidance control device according to Appendix 3, wherein the action management unit generates a warning image when the action of the subject differs from the predetermined action, and the projection control unit controls the projection device to project the projection image including the warning image.

[0061] (Note 6) The guidance control device described in Appendix 3 further comprises a registration reception unit for receiving registration of the correspondence between the aforementioned subject and the aforementioned prescribed action.

[0062] (Note 7) The guidance control device according to Appendix 6, wherein the projection control unit identifies a predetermined action corresponding to the identification information of the subject recognized by the subject recognition unit in accordance with the correspondence relationship, and controls the projection device to project the guidance image that provides guidance for the identified predetermined action into the second projection range.

[0063] (Note 8) The guidance control device according to Appendix 1, wherein the predetermined action is the sorting of goods, and the projection control unit includes the delivery destination of the sorted goods, the type of goods, and the quantity of goods in the guidance image and projects them into the second projection range.

[0064] (Note 9) The guidance control device according to Appendix 2, wherein the subject recognition unit determines whether the current position of the subject corresponding to the subject identification image projected in the first projection range coincides with the first position where the predetermined action is to be performed, and the projection control unit projects the guidance image prompting the subject to perform the predetermined action into the second projection range when the current position of the subject coincides with the first position.

[0065] (Note 10) Projection device and The projection device is equipped with a guidance control device that can communicate with the projection device, The guidance control device generates a person identification image to be projected onto a first projection range corresponding to a first position where the person performs a predetermined action, and generates a guidance image to be projected onto a second projection range corresponding to a second position visible to the person to guide the person to perform the predetermined action, and controls the projection device to project a projection image including the person identification image and the guidance image.

[0066] (Note 11) A subject identification image is generated to be projected onto the first projection range corresponding to the first position where the subject performs a prescribed action. A guidance image is generated that is projected onto a second projection range corresponding to a second position visible to the subject, to guide the subject on the predetermined action. A guidance control method for projecting a projection image that includes the aforementioned target identification image and the aforementioned guidance image.

[0067] (Note 12) For computers, A process for generating a subject identification image to be projected onto a first projection range corresponding to the first position where the subject performs a predetermined action, A process for generating a guidance image that is projected onto a second projection range corresponding to a second position visible to the subject to guide the subject on the predetermined action, A storage medium storing a program that causes a process to perform the following: projecting a projection image including the aforementioned person identification image and the aforementioned guidance image.

[0068] This application claims priority based on Japanese Patent Application No. 2018-65938, filed in Japan on March 29, 2018, and the contents of that application are incorporated herein by reference. [Industrial applicability]

[0069] The present invention involves installing a guidance control device in a product center to provide guidance to workers performing product sorting tasks, but is not limited to this. For example, the guidance control device can be installed in a store to provide guidance information to customers when they purchase or pay for products. Furthermore, the guidance control device may be installed in public facilities such as train stations and airports to provide guidance information to passengers and users. [Explanation of symbols]

[0070] 1. Guidance control device 2. Projector (projection device) 3. Camera (filming device) 4 carts 5. Terminal (Management device) 10 Control Unit 11. Registration Department 12 Projection Control Unit 13. Worker Recognition Unit 14 Conduct Management Department 15 Management information update department 40 databases 41 Shopping Cart 42 Information Plate 51 store baskets 420 Guide Images 610 Target Identification Image

Claims

1. A guidance control device that can communicate with a projection device, A projection control unit generates a first projection image to be projected onto a first projection range corresponding to a first position where a worker performs a predetermined action, and generates a second projection image to be projected onto a second projection range corresponding to a second position visible to the worker to guide the worker through the predetermined action, and controls the projection device to project the first projection image and the second projection image. The system includes an action management unit that recognizes the actions of the worker and generates a warning image if the worker's actions differ from the predetermined actions, The projection control unit projects the projection image including the warning image, and controls the projection device to project the second projection image into the second projection range when the position of the worker corresponding to the first projection image projected into the first projection range coincides with the first position. The first projection image is an image assigned to each worker. Guidance control device.

2. The second projection image is projected onto the second position corresponding to the container for work, based on the worker's work content, and includes information indicating work instructions. The guidance control device according to claim 1.

3. The aforementioned work includes the first and second tasks, The projection device is controlled to project onto a position corresponding to the container for the first operation, and then project onto a position corresponding to the container for the second operation. The guidance control device according to claim 2.

4. The projection control unit controls the projection device to project near the container. The guidance control device according to claim 2 or 3.

5. The container is capable of holding goods or articles. The guidance control device according to any one of claims 2 to 4.

6. The projection control unit controls the projection device to project information regarding the quantity of goods or articles to be processed onto the container. The guidance control device according to claim 4.

7. An acquisition unit that acquires captured images of the worker's actions, The system includes a determination unit that determines, based on the captured image, whether or not the worker has completed the work on the container according to the work content. The guidance control device according to any one of claims 2 to 5.

8. The system includes a notification unit that notifies the worker if it is determined that the worker has not completed the work based on the work content for the container. The guidance control device according to claim 7.

9. The projection control unit controls the projection device to project warning information onto the container if it is determined that the worker has not completed the work based on the work content. The guidance control device according to claim 7.

10. The projection control unit controls the projection device to project the warning information at a predetermined position near the container. The guidance control device according to claim 9.

11. The determination unit determines whether the work performed by the worker differs from the work based on the work content, The projection control unit controls the projection device to project warning information when it is determined that the work performed by the worker differs from the work based on the work content. The guidance control device according to any one of claims 7 to 10.

12. The projection control unit controls the projection device to project guidance information for working on the container to a position visible to the worker. The guidance control device according to any one of claims 2 to 11.

13. The guidance control device according to any one of claims 2 to 11, wherein the projection control unit causes a display device to display information that guides the work on the container.

14. The first projection image is a specific image, and the second projection image is a guide image. A guidance control device according to any one of claims 1 to 13.

15. An information processing system that can communicate with a projection device, A projection control unit generates a first projection image to be projected onto a first projection range corresponding to a first position where a worker performs a predetermined action, and generates a second projection image to be projected onto a second projection range corresponding to a second position visible to the worker to guide the worker through the predetermined action, and controls the projection device to project the first projection image and the second projection image. The system includes an action management unit that recognizes the actions of the worker and generates a warning image if the worker's actions differ from the predetermined actions, The projection control unit controls the projection device to project the projection image including the warning image, and controls the projection device to project the second projection image into the second projection area when the position of the worker corresponding to the first projection image projected into the first projection area coincides with the first position. The first projection image is an image assigned to each worker. Information processing system.

16. A guidance control device that can communicate with the projection device, The projection device generates a first projection image to be projected onto a first projection range corresponding to a first position where the worker performs a predetermined action, and generates a second projection image to be projected onto a second projection range corresponding to a second position visible to the worker to guide the worker through the predetermined action, and controls the projection device to project the first projection image and the second projection image. The system recognizes the actions of the worker, and if the worker's actions differ from the predetermined actions, it generates a warning image. The projection device is controlled to project the projection image including the warning image, and when the position of the worker corresponding to the first projection image projected in the first projection range coincides with the first position, the projection device is controlled to project the second projection image in the second projection range. The first projection image is an image assigned to each worker. Information processing methods.

17. A computer for a guidance control device that can communicate with the projection device, A projection control means that generates a first projection image to be projected onto a first projection range corresponding to a first position where a worker performs a predetermined action, and generates a second projection image to be projected onto a second projection range corresponding to a second position visible to the worker to guide the worker through the predetermined action, and controls the projection device to project the first projection image and the second projection image. This system recognizes the actions of the worker and functions as an action management means that generates a warning image if the worker's actions differ from the predetermined actions. The projection control means controls the projection device to project the projection image including the warning image, and controls the projection device to project the second projection image into the second projection area when the position of the worker corresponding to the first projection image projected into the first projection area coincides with the first position. The first projection image is an image assigned to each worker. program.

Images