Information processing device, information processing method, information processing system, and program

Abstract

To improve efficiency of load sorting work.SOLUTION: An assistance system 100 comprises a processing device 8, a camera 2, and a projector. The processing device 8 comprises: a determination unit 81 that determines the storage position and the storage area of a load 20 from an image captured by the camera 2 of a shelf 1 on which the load 20 is placed; and a projection control unit 83 that causes the projector 6 to project, to the shelf 1, an area image 30 displaying the address of each load 20 on the basis of load information associating the address, the storage position, and the storage area acquired from the load 20. The width of the area image 30 is set in accordance with the width of the storage area of each load 20.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, an information processing system, and a program. [Background technology]

[0002] In recent years, technological developments have been progressing to support the delivery of mail and other packages. Mail delivery services handle mail of various types and sizes. Among these packages, there are packages whose addresses cannot be read by machines and packages that do not meet the standard size, and such packages must be sorted manually. Furthermore, packages that do not meet the standard size may not fit within the frame of the shelves prepared for sorting. As a result, there has been a limit to the improvement of work efficiency.

[0003] Patent Document 1 describes a system that uses a camera to read markers attached to boxes prepared for storing luggage, determines the size and storage position of the marker, identifies the position of the box stored on a shelf, and identifies the corresponding box. Feelings The document discloses a mechanism for projecting information onto shelves using a projector. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-132375 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the configuration of Patent Document 1, it is assumed that boxes with markers attached in advance are registered in the system and packages stored in the boxes are removed. Therefore, it is necessary to prepare boxes with markers attached in advance. This makes it difficult to store packages of various shapes and sizes on shelves in real time, such as in mail sorting. In addition, when packages must be stored in prepared boxes, the storage area is fixed depending on the size of the box, which creates the problem of limiting the size of the items that can be stored.

[0006] In view of the above circumstances, an object of the present invention is to improve the efficiency of luggage sorting work. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides Address given a determination unit that determines a storage location and a storage area of ​​the luggage from a first image of a shelf on which the luggage is placed, and a projection control unit that projects a second image that displays the address of the luggage for each luggage on the basis of luggage information that associates the address, the storage location, and the storage area acquired from the luggage onto the shelf. And, Preparation, The determination unit determines a storage area for the luggage to be stored in the storage area based on address information of luggage placed in the adjacent storage area, and the projection control unit For each piece of luggage, the width of the second image is set to a width corresponding to the width of the storage area. and projecting the second image onto a storage area on the shelf corresponding to the luggage, and further, the projection control unit highlights and displays the storage area corresponding to the luggage having the same address information as the acquired address. It is an information processing device.

[0008] The present invention also provides an information processing device, Address given A step of determining a storage location and a storage area of ​​the luggage from a first image obtained by photographing a shelf on which the luggage is placed, and a step of projecting a second image onto the shelf, the second image displaying the address of the luggage for each luggage based on luggage information that associates the address acquired from the luggage, the storage location, and the storage area, In the determining step, a storage area for the luggage to be stored in the storage area is determined based on address information of luggage placed in the adjacent storage area, and in the projecting step, For each piece of luggage, the width of the second image is set to a width corresponding to the width of the storage area. and projecting the second image onto a storage area on the shelf corresponding to the package, and further displaying the storage area corresponding to the package having the same address information as the acquired address in an emphasized manner in the projecting step. It is an information processing method.

[0009] The present invention also provides an information processing device, a camera, and a projection device, the information processing device comprising: Address givena determination unit that determines a storage position and a storage area of ​​the luggage from a first image of a shelf on which the luggage is placed, the first image being captured by the camera; and a projection control unit that causes the projection device to project onto the shelf a second image that displays the address of the luggage for each luggage based on luggage information that associates the address acquired from the luggage, the storage position, and the storage area. And, Preparation, The determination unit determines a storage area for the luggage to be stored in the storage area based on address information of luggage placed in the adjacent storage area, and the projection control unit For each piece of luggage, the width of the second image is set to a width corresponding to the width of the storage area. and projecting the second image onto a storage area on the shelf corresponding to the luggage, and further, the projection control unit highlights and displays the storage area corresponding to the luggage having the same address information as the acquired address. It is an information processing system.

[0010] The present invention also provides a computer of an information processing device, Address given a determination unit that determines a storage location and storage area of ​​the luggage from a first image of a shelf on which the luggage is placed, and a projection control unit that projects a second image that displays the address of the luggage for each luggage on the basis of luggage information that associates the address acquired from the luggage, the storage location, and the storage area, onto the shelf, The determination unit determines a storage area for a package to be stored in the storage area based on address information of packages placed in the adjacent storage areas, and the projection control unit For each piece of luggage, the width of the second image is set to a width corresponding to the width of the storage area. and projecting the second image onto a storage area on the shelf corresponding to the package, and further causing the projection control unit to highlight and display the storage area corresponding to the package having the same address information as the acquired address. It is a program. Effect of the Invention

[0011] According to the present invention, the efficiency of luggage sorting work can be improved. [Brief description of the drawings]

[0012] [Figure 1] 1 is an overall configuration diagram of a support system according to a first embodiment. [Diagram 2] FIG. [Diagram 3] 13 is a flowchart of a process when storing luggage. [Figure 4] FIG. 13 is an explanatory diagram of a storage determination. [Diagram 5] FIG. 13A is an explanatory diagram of the projection of a region image, and FIG. 13B is an explanatory diagram of the update of a storage region. [Figure 6] 1A and 1B are explanatory diagrams of storage area integration, where (a) is before integration and (b) is after integration. [Figure 7]13 is a flowchart of a process for integrating storage areas. [Figure 8] FIG. 13 is an explanatory diagram for storing long and narrow luggage. [Figure 9] 13 is a flowchart of a process when sorting luggage. [Figure 10] FIG. 13 is an explanatory diagram of a case where an address to be sorted is selected. [Figure 11] 13 is a flowchart of a storage location determination process. [Figure 12] FIG. 13 is an explanatory diagram for a case where the hand position determination is NG. [Figure 13] FIG. 13 is an explanatory diagram of a case where the hand position determination is OK. [Figure 14] FIG. 11 is an overall configuration diagram of a support system according to a second embodiment. [Figure 15] This is an explanatory diagram showing how the overall shape of a shelf can be grasped using AR glasses. [Figure 16] FIG. 11 is an explanatory diagram of a display of a region image according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The support system of each embodiment displays information on a sorting shelf used to prepare packages for delivery and manages storage locations for packages. Packages are, for example, mail, but are not limited to this. The support system is a specific example of an "information processing system" in the claims. In describing each embodiment, the front of the shelf is referred to as the "front", and the up, down, left, right and rear directions are as shown in the figures.

[0014] [First embodiment] <Configuration> 1, a support system 100 of the first embodiment is provided for a shelf 1. The support system 100 includes two cameras 2, a reading device 3, a tablet terminal 4, a projector 6, and a processing device 8. The camera 2, the reading device 3, the tablet terminal 4, the projector 6, and the processing device 8 are connected to each other so as to be able to communicate with each other via wire or wirelessly.

[0015] The shelf 1 is a sorting shelf for storing luggage 20. As shown in Figures 1 and 2, the shelf 1 includes a plurality of plates 11 on which the luggage 20 is placed, a support 12 for holding the plates 11 at a predetermined height, and an arm 13 to which the camera 2 is attached. The package 20 is wrapped and has a destination address written on it (a label with the destination address written on it is attached to the package). The destination address may be written, for example, in a bar code or in text format.

[0016] A label 5 is attached to the front surface of the board 11. The label 5 is a mark for identifying the size and position of the luggage 20 stored on the shelf 1. For example, the pattern of the label 5 can be, but is not limited to, a pattern in which two or more colors are alternately arranged at intervals of 1 cm (centimeter).

[0017] Camera 2 is attached to the front end of arm 13, and the lens is directed toward the front surface of shelf 1. Thus, camera 2 can capture an image of the entire front surface of shelf 1. In particular, camera 2 can capture an image of label 5 affixed to the front surface of board 11. The image captured by the camera 2 is a specific example of a "first image" in the claims.

[0018] The reading device 3 is a device used by a worker, and reads the destination address written on the package 20. The reading device 3 can be, for example, a camera or a barcode reader, but is not limited to these.

[0019] The tablet terminal 4 is a mobile terminal used by a worker, and the worker inputs predetermined information and displays predetermined information to the worker. For example, the worker can input an address from the tablet terminal 4, and perform operations when storing the luggage 20 and when sorting the luggage 20.

[0020] The projector 6 projects a predetermined image onto the front surface of the shelf 1. The projector 6 is a specific example of a "projection device" in the claims. The mirror 7 is a light reflector that adjusts the projection location of the projector 6 . As shown in Figs. 1 and 2, for example, projector 6 is disposed on the top shelf of shelf 1 and in front of shelf 1. Mirror 7 is disposed substantially above projector 6. When projector 6 projects a predetermined image onto mirror 7, the predetermined image is projected onto the front surface of plate 11 of shelf 1 by reflection from mirror 7. The predetermined image may be projected onto the remaining area of ​​the front surface of plate 11 excluding the area of ​​label 5 affixed to the front surface of plate 11. Furthermore, the predetermined image may be projected so as to overlap label 5 as long as the visibility of the image itself is good. It should be noted that the projector 6 may directly project a predetermined image onto the front surface of the shelf 1 without using the mirror 7.

[0021] The processing device 8 controls the camera 2 and the projector 6, and also controls the information display on the tablet terminal 4.

[0022] The tablet terminal 4 and the processing device 8 are configured as a computer having a CPU (Central Processing unit), memory, a storage means (storage unit, storage device) such as a hard disk, and a network interface. Various functions of this computer are realized by the CPU executing a program loaded onto the memory.

[0023] The processing device 8 includes a determination unit 81, a calculation unit 82, a projection control unit 83, and a notification unit 84. The processing device 8 also stores a package information DB 85.

[0024] The determination unit 81 performs various determinations based on the image captured by the camera 2 when a worker stores a package 20 on the shelf 1 or removes a package 20 from the shelf 1.

[0025] The calculation unit 82 calculates the storage area for the luggage 20 stored on the shelf 1 based on the label 5 in the image captured by the camera 2. The storage area is an area occupied by luggage placed on the board 11. For example, the calculation unit 82 can set the dimensions of the storage area to be the same as the width dimension (placing width) of the luggage 20. The calculation unit 82 can also calculate one storage area for multiple luggage 20. The calculation unit 82 can also calculate the storage area for luggage 20 already stored on the shelf 1. By making the interval between the colors used in the label 5 smaller, the calculation accuracy of the storage area for the stored luggage 20 by the calculation unit 82 can be improved.

[0026] The projection control unit 83 controls the projection of the projector 6, and generates and displays an image to be projected on the front surface of the shelf 1.

[0027] The notification unit 84 issues a predetermined notification to the worker. For example, the notification unit 84 issues a notification to alert the worker to an error in storing or removing the luggage 20. In addition, the notification unit 84 can issue a notification by a notification means (not shown) arranged in the work area or via the tablet terminal 4, for example.

[0028] The luggage information DB 85 is a database that registers information about the luggage 20 stored on the shelf 1 as luggage information. The luggage information may include, but is not limited to, for example, the address of the destination written on the luggage 20 (hereinafter, may be referred to as the "luggage address"), the storage position of the luggage 20 stored on the shelf 1, and the storage area of ​​the luggage 20 stored on the shelf 1. The storage position may indicate, for example, which position on which level of the board 11 the luggage is placed. Furthermore, the package information DB 85 can register package information for each package 20 . The projection control unit 83 can include the address of the destination of the package information in the image projected on the front surface of the shelf 1. The address projected on the front surface of the shelf 1 may be called a "shelf address."

[0029] <Processing> Next, the processing of the processing device 8 will be described.

[0030] (When luggage is stored) A case where a worker stores a baggage 20 on the shelf 1 will be described.

[0031] As shown in FIG. 3, the processing device 8 acquires the luggage address of the luggage 20 to be stored (step A1). Specifically, a worker uses the reading device 3 to photograph the address written on the luggage 20 or to read the barcode written on the luggage 20, so that the reading device 3 reads out the luggage address and transmits it to the processing device 8. Also, the worker uses the tablet terminal 4 that displays a list of addresses to select a desired address, and the tablet terminal 4 transmits the selected address to the processing device 8 as the luggage address. The worker attempts to store the luggage 20 in a desired position on the shelf 1. The desired position may be an empty position, or may be a position where other luggage 20 is already stored (due to stacking). The camera 2 photographs the luggage 20 to be stored.

[0032] Next, the processing device 8 acquires a real-time image of the worker storing the luggage from the camera 2 (step A2). As shown in Fig. 4, when the worker stores the luggage 20, a part of the label 5 is hidden by the luggage 20. The camera 2 can capture an image of the partially hidden label 5. The processing device 8 can acquire an image of the partially hidden label 5 from the camera 2.

[0033] Next, the determination unit 81 of the processing device 8 performs a storage determination process (step A3). Specifically, the determination unit 81 identifies a storage position of the luggage 20 from the image of the partially hidden label 5. The calculation unit 82 can also calculate the width dimension of the luggage 20 from the image of the partially hidden label 5. Thus, the determination unit 81 identifies a storage area for the luggage 20 that has the same dimensions as the calculated width dimension.

[0034] Next, the determination unit 81 determines whether the identified storage location has already been registered (step A4). Specifically, the determination unit 81 refers to the luggage information DB 85 to determine whether there is another luggage 20 that is located in the same location as the identified storage location. If it has not been registered (No in step A4), the identified storage location is originally vacant, and the determination unit 81 creates a new storage area for the luggage 20 to be stored (step A5). At this time, the dimensions of the created storage area are the dimensions calculated by the calculation unit 82, and can be the same as the width dimension of the luggage 20 to be stored.

[0035] Next, the determination unit 81 registers the package information about the package 20 to be stored in the package information DB 85 (step A6). The package information includes the address of the destination of the package 20, the storage location of the package 20, and the storage area of ​​the package 20.

[0036] Next, the projection control unit 83 of the processing device 8 generates an area image including the shelf address (step A7). Specifically, the projection control unit 83 generates an area image to be projected onto the front surface of the shelf 1, and includes the destination address included in the package information registered in step A6 in the generated area image. Incidentally, the area image is a specific example of a "second image" in the claims.

[0037] Next, the projection control unit 83 executes a projection display process by the projector 6 (step A8). Specifically, the projection control unit 83 controls the projector 6 to display an area image in a storage area included in the luggage information of the luggage 20 to be stored on the front surface of the shelf 1. As shown in Fig. 5(a), the area image 30 is displayed on the front surface of the board 11 of the shelf 1 in the storage area for the luggage 20 (having the same dimensions as the width dimension of the luggage 20). This ends the process of Fig. 3.

[0038] On the other hand, if the identified storage location has not been registered (Yes in step A4), another piece of luggage 20 has been placed in the identified storage location, and the luggage is about to be stacked. Here, the determination unit 81 determines whether or not the luggage address of the luggage 20 to be stored matches the shelf address of the other piece of luggage 20 that has already been placed (step A9). The shelf address of the other piece of luggage 20 that has already been placed is an address included in the luggage information of the other piece of luggage stored in the luggage information DB 85, ​​and is an address displayed in the storage area of ​​the other piece of luggage 20 that has already been placed by the projector 6.

[0039] If there is no match (No in step A9), the notification unit 84 performs error processing (step A10). Specifically, the notification unit 84 issues a warning to prohibit the placement of the luggage 20 to be stored in the identified storage position determined in step A4. After that, when the worker stores the luggage 20 in another position on the shelf 1, the process returns to step A2 and continues the process in FIG. 3.

[0040] On the other hand, if there is a match (Yes in step A9), the determination unit 81 determines whether the luggage 20 to be stored is larger than the storage area of ​​the other luggage 20 that has already been placed (step A11). The storage area of ​​the other luggage 20 that has already been placed is acquired from the luggage information of the other luggage registered in the luggage information DB 85. If the luggage 20 to be stored is larger (Yes in step A11), the determination unit 81 updates the storage areas for the luggage 20 to be stored and the other luggage 20 that has already been placed (step A12). Specifically, the storage area of ​​the other luggage 20 that has already been placed is expanded to be the same as the width dimension of the luggage 20 to be stored.

[0041] As shown in FIG. 5(b), when a larger piece of luggage 20 to be stored is placed on top of an already placed piece of luggage 20 (see the second board 11 from the bottom and the third board 11 from the bottom), the dimensions of the storage area for the already placed luggage 20 are enlarged to be the same as the storage area for the luggage 20 to be stored, and an area image 30 corresponding to the enlarged storage area is projected. The stacked luggage 20 are specific examples of the "first luggage" and the "second luggage" in the claims, respectively. After that, returning to step A6, the determination unit 81 registers the baggage information about the baggage 20 to be stored in the baggage information DB 85, ​​and registers the baggage information including the expanded storage area about the other baggage 20 that has already been placed in the baggage information DB 85. Then, the process of FIG. 3 is continued.

[0042] On the other hand, if the baggage 20 to be stored is not large (No in step A11), the determination unit 81 does not process any other baggage 20 that has already been placed, returns to step A6, and registers the baggage information about the baggage 20 to be stored in the baggage information DB 85. Then, the process of FIG. 3 is continued.

[0043] As shown in FIG. 5(b), when a smaller piece of luggage 20 to be stored is placed on top of an already placed piece of luggage 20 (see the bottom board 11), an area image 30 corresponding to the original storage area is projected without enlarging the dimensions of the storage area for the already placed luggage 20. The process of FIG. 3 is executed for each piece of luggage 20 to be stored.

[0044] (Storage area integration) When storage areas for multiple packages 20 with the same address are adjacent to each other, the processing device 8 can consolidate the storage areas into one. As shown in Fig. 6(a), a storage area is set for each of the two luggage 20 placed on the bottom board 11 of the shelf 1, and an area image 30 is projected onto the storage area. If the distance between the two luggage 20 is sufficiently small, the processing device merges the two storage areas into one. As a result, as shown in Fig. 6(b), one storage area is set for the two luggage 20, 20 on the bottom level, and one area image 30 is projected onto the storage area.

[0045] On the other hand, for the two pieces of luggage 20 placed on the board 11 on the first level from the bottom of the shelf 1, the space between them is sufficiently large, so the storage areas are not consolidated. By consolidating storage areas, the cargo information for the group of cargo 20 stored on shelf 1 can be simplified. The packages 20 that are the subject of storage area consolidation are specific examples of the "third package" and the "fourth package" in the claims.

[0046] The following describes the process of integrating storage areas when a new package 20 to be stored is stored on the shelf 1. The processing device 8 refers to the registered package information of the package 20 to be stored (see step A6 in FIG. 3) and knows the storage position of the package 20 to be stored.

[0047] 7, the determination unit 81 of the processing device 8 refers to the luggage information DB 85 and checks the storage area adjacent to the storage area of ​​the luggage 20 to be stored (step B1). The "adjacent storage area" refers to the storage area of ​​luggage 20 on the same level placed on the plate 11 on which the luggage 20 to be stored is placed. Note that the check in step B1 can be performed for each level of the shelf 1.

[0048] Next, the determination unit 81 of the processing device 8 determines whether the address (baggage address) of the baggage 20 to be stored is the same as the address (shelf address) of another baggage 20 having an adjacent storage area (step B2). If they are not the same (No in step B2), the storage areas are not integrated, and the projection control unit 83 of the processing device 8 performs projection display processing of a new area image for the baggage 20 to be stored following step A8 (FIG. 3) (step B3). Then, the processing in FIG. 7 ends.

[0049] On the other hand, if the addresses are the same (Yes in step B2), the determination unit 81 determines whether the distance between the storage area of ​​the baggage to be stored and the adjacent storage area is equal to or smaller than a predetermined value (sufficiently small value) (step B4). If it is not equal to or smaller than the predetermined value (No in step B4), the process proceeds to step B3, where a projection display process of a new area image is performed.

[0050] On the other hand, if it is equal to or less than the predetermined value (Yes in step B4), the determination unit 81 merges the storage area of ​​the baggage 20 to be stored with the adjacent storage area (step B5). Note that the baggage information of the baggage 20 to be stored and the baggage information of other baggage 20 having adjacent storage areas are updated for the merged storage area.

[0051] Next, the projection control unit 83 performs a projection display process of one area image 30 on the integrated storage area (step B6), after which the process of FIG.

[0052] (When storing unstable luggage) For example, if the luggage 20 has an elongated shape and is unstable when placed on the plate 11 of the shelf 1, the storage area for the luggage 20 may not be determined, resulting in a problem of losing the reliability of the luggage information. In this case, it is advisable to prepare a pair of partition plates 40, 40 that can be attached to the shelf 1, as shown in FIG.

[0053] The partition plates 40, 40 can partially divide the space of one level of the shelf 1. By storing the luggage 20 in the partial space between the partition plates 40, 40, the luggage 20 can be leaned against the partition plates 40, 40, and the luggage 20 can be stably placed on the plate 11 of the shelf 1. In addition, marks 41, 41 are attached to the front of the partition plates 40, 40. The processing device 8 can grasp the position of the partition plates 40, 40 by capturing the marks 41, 41 in the image captured by the camera 2. As shown in FIG. 8, it is preferable that the marks 41, 41 overlap the labels 5 when the partition plates 40, 40 are attached to the shelf 1. This makes it easier to match the judgment of the luggage 20 based on the marks 41, 41 with the judgment of the luggage 20 based on the labels 5.

[0054] The determination unit 81 of the processing device 8 can grasp the positions of the partitions 40, 40, and can therefore recognize the storage area between the partitions 40, 40. Thus, the determination unit 81 can determine that the storage area for the luggage 20 placed between the partitions 40, 40 is the storage area between the partitions 40, 40. The storage area included in the luggage information for the luggage 20 can be the storage area between the partitions 40, 40. Since the luggage 20 is stably placed between the partitions 40, 40, the storage area for the luggage 20 can be determined, and the reliability of the luggage information can be ensured.

[0055] The partition plates 40, 40 are movable, and the storage area between the partition plates 40, 40 can be changed as appropriate. Furthermore, as shown in FIG. 8, a plurality of packages 20 having the same address can be placed side by side in the storage area between the partitions 40, 40.

[0056] (When sorting luggage) A case where a worker takes out baggage 20 from shelf 1 as part of a sorting operation will be described.

[0057] 9, the processor 8 acquires the addresses of the items to be sorted (Step C1). Specifically, the worker selects a desired address using the tablet terminal 4 that displays a list of addresses, and the tablet terminal 4 transmits the selected address to the processor 8.

[0058] The projection control unit 83 of the processing device 8 performs an emphasis projection display process for the acquired address (step C2). Specifically, the projection control unit 83 causes the projector 6 to light and display the acquired address on the front of the shelf 1 in the storage area of ​​the luggage 20 that has the same shelf address as the address acquired in step C1 among the luggage 20 stored on the shelf 1. This lighted display may be appropriately highlighted so that it is easy for the worker to understand.

[0059] For example, as shown in Fig. 10, assume that the worker selects "X△1-chome" from the tablet terminal 4. In this case, the projector 6 can light up and display the area image 30 with the description "X△1-chome" in the storage area of ​​the baggage 20 whose shelf address is "X△1-chome". Furthermore, the worker can select multiple addresses from the tablet terminal 4, and the projector 6 can light up and display the selected multiple addresses on the front of the shelf 1. After this, the worker picks up the packages 20 from the addresses that are lighted up.

[0060] Next, the processing device 8 obtains a real-time image of the worker removing the luggage from the camera 2 (step C3). When the worker removes the luggage 20 that is indicated as being lit, the camera 2 can capture an image of the worker's hand that enters the storage area of ​​the luggage 20. The camera 2 can also capture an image of the label 5 that is partially hidden by the hand.

[0061] Next, the determination unit 81 of the processing device 8 performs a pick-up position determination process (step C4). Specifically, the determination unit 81 determines whether or not the pick-up position of the package 20 is correct based on the image of the label 5 partially hidden by the hand.

[0062] If the pick-up position is not correct (No in step C5), the notification unit 84 of the processing device 8 performs error processing (step C6). Specifically, the notification unit 84 issues a warning to the worker that he or she is picking up the wrong baggage 20. If the worker then attempts to pick up another baggage 20 stored on the shelf 1, the process returns to step C2 and continues the process in FIG. 9.

[0063] On the other hand, if the removal position is correct (Yes in step C5), it means that the worker has properly removed the corresponding luggage 20. At this point, the processing device 8 deletes the luggage information of the removed luggage 20 from the luggage information DB 85 (step C7). The camera 2 can capture an image of the empty state of the shelf 1 after the luggage 20 has been removed. The determination unit 81 of the processing device 8 determines that removal of the luggage 20 has been completed from the captured image, and deletes the luggage information. If there are multiple pieces of luggage 20 to be removed, steps C2 to C7 are repeated.

[0064] After the worker removes all of the corresponding packages 20 (packages 20 with illuminated addresses), the processor 8 acquires a sorting end operation from the worker (step C8). Specifically, when the worker selects a sorting end button on the tablet terminal 4, the processor 8 acquires a sorting end signal and ends the process of FIG.

[0065] (Storage location determination) When a worker stores or retrieves baggage 20, the processing device 8 can determine a specific space on the shelf 1 as a storage location and determine whether the worker has put his or her hand in or taken out of that storage location.

[0066] 11, the processing device 8 acquires a storage location designated by, for example, a worker (step D1). Specifically, the worker can designate a storage location by touching a desired location on the entire image of the shelf 1 displayed on the tablet terminal 4. The designated storage location is transmitted to the processing device 8. After that, the worker puts his or her hand into the desired storage location on the actual shelf 1.

[0067] Next, the processing device 8 acquires a real-time image of the worker's insertion of his / her hand from the camera 2 (step D2). The camera 2 can capture an image of the inserted hand and the label 5 partially hidden by the hand.

[0068] Next, the determination unit 81 of the processing device 8 performs hand position determination processing (step D3). Specifically, the determination unit 81 determines whether or not the hand insertion position matches the specified storage location from the image of the label 5 partially hidden by the hand.

[0069] If it does not match the specified storage location (No in step D4), the notification unit 84 of the processing device 8 performs error processing (step D5). Specifically, the notification unit 84 issues a warning to the worker that he has inserted his hand in the wrong position (for example, an announcement that "the storage location is wrong"). If the worker then tries to insert his hand into another location on the shelf 1, the process returns to step D2 and continues the process in FIG. 11.

[0070] For example, as shown in Fig. 12, when a worker retrieves a package 20 with address A (shelf address displayed in area image 30) in a specified storage location, the worker may approach the package 20 next to address B (shelf address displayed in area image 30) as shown in Fig. 12(a), or approach a location far away from the specified storage location as shown in Fig. 12(b). In this case, it is determined that the specified storage location does not match, and the notification unit 84 performs error processing.

[0071] On the other hand, if it matches the specified storage location (Yes in step D4), it means that the worker has stored the baggage 20 in the specified storage location or removed the baggage 20 from the specified storage location.

[0072] For example, as shown in Fig. 13, when a worker takes out a package 20 with address A (shelf address displayed in area image 30) in a specified storage location, the worker may approach the package 20 with address A as shown in Fig. 12(a) or approach a location slightly away from the specified storage location as shown in Fig. 12(b). In this case, the package matches the specified storage location.

[0073] Thereafter, the processing device 8 updates the package information of the target package 20 in the package information DB 85 (step D6). Specifically, when the package 20 is stored, the package address, storage area, and package width dimension calculated by the calculation unit 82 are associated and registered for the target package 20. When the package 20 is removed, the package information of the target package 20 is deleted from the package information DB. This ends the processing of FIG. 11.

[0074] According to the first embodiment, the processing device 8 can display to the worker an image display in which the storage location and address of the luggage 20 are associated. Therefore, when sorting, it is possible to assist the worker in picking up the luggage 20, and it is possible to improve the efficiency of the luggage sorting work. In addition, since the storage area is determined based on the size of the luggage 20, a sorting operation can be realized that is not dependent on the size or amount of luggage 20. In addition, since the storage area can be updated as the luggage 20 is piled up, the efficiency of managing luggage 20 with the same address can be improved. In addition, by consolidating the storage areas for adjacent packages 20, the efficiency of managing packages 20 with the same address can be improved. Furthermore, by providing the partition plate 40, it is possible to reliably store luggage 20 having an unstable shape when placed on the shelf 1, and the efficiency of managing the luggage 20 can be improved. Furthermore, by using the label 5, it is possible to reliably determine whether the package 20 has been stored on the shelf 1. In particular, since the package 20 can be stored in any vacant location, the area of ​​the shelf 1 can be effectively utilized, and space can be saved. Furthermore, by using the label 5, removal of the baggage 20 from the shelf 1 can be reliably determined. Furthermore, by using a projection device such as the projector 6, an image display correlating the storage location of the baggage 20 with the address can be displayed in an easy-to-understand manner to the worker.

[0075] [Second embodiment] In describing the second embodiment, differences from the first embodiment will be mainly described, and overlapping descriptions will be omitted. In the support system 200 of the second embodiment, AR (Augmented Reality) glasses 9 are used as a projection device instead of the projector 6 and the mirror 7. Also, a plurality of cords 51 are attached to the front surface of the shelf 1.

[0076] The AR glasses 9 are wearable terminals used by workers, and can display predetermined information to the worker wearing the AR glasses 9. The camera 2, the reading device 3, the tablet terminal 4, the processing device 8, and the AR glasses 9 (a computing device (not shown) provided in the AR glasses 9) are connected to each other so as to be able to communicate with each other via wire or wirelessly. The AR glasses 9 include a camera 91, a prism 92, and a lens 93.

[0077] The camera 91 is attached to the AR glasses 9. The camera 91 can capture an image of the foreground that is equivalent to the field of vision of a worker. The prism 92 is a light guiding means for displaying a predetermined image on the lens 93 . The lens 93 is a display medium on which information output by the arithmetic device of the AR glasses 9 is projected via the prism 92. The information to be displayed is equivalent to the image projected by the projector 6 in the first embodiment.

[0078] The codes 51 are affixed to the front of the support posts 12 of the shelf 1 and to the left and right of each of the labels 5. The codes 51 may be, for example, but are not limited to, a QR code. Each of the codes 51 includes information describing its location on the shelf 1. The processing device 8 can control the AR glasses 9 and the camera 91. However, if the AR glasses 9 have sufficient calculation and control functions, the control by the processing device 8 may not be necessary.

[0079] 15, the camera 91 captures an image of the code 51 and reads the information of the code 51, thereby being able to grasp the overall shape of the shelf 1. Therefore, the AR glasses 9 can grasp each position and each area of ​​the shelf 1 by analyzing the image captured by the camera 91. Furthermore, the processing device 8 can transmit the luggage information registered in the luggage information DB 85 to the AR glasses 9. As a result, as shown in Fig. 16, when a worker looks at the shelf 1, the AR glasses 9 can project an area image 30 indicating the shelf address and storage area for each luggage 20 stored on the shelf 1 onto the lens 93 on which the shelf 1 is displayed, in the same manner as in the first embodiment. By replacing the projection by the projector 6 with the projection by the AR glasses 9, the various processes described in the first embodiment can also be executed in the second embodiment.

[0080] According to the second embodiment, the same effects as those of the first embodiment can be obtained. Furthermore, by using the AR glasses 9, the projector 6 is no longer necessary, and the task of installing the projector 6 and the mirror 7 on the shelf 1 is no longer necessary, so the scale of the support system 100 can be reduced. In addition, the accuracy of grasping the storage area can be improved by using the camera 91 of the AR glasses 9. In addition, the camera 91 can read the address of the baggage 20 instead of the reading device 3.

[0081] <Modification> (a) The width of the area image 30 for each piece of luggage 20 stored on the shelf 1 does not need to be the same as the width of the storage area for the luggage 20, and may be larger or smaller than the width of the storage area. In other words, the width of the area image 30 may be a width corresponding to the width of the storage area for the luggage 20. (b) Instead of using the label 5, an application for measuring the dimensions of an object in a captured image may be installed in the processing device 8. By using such an application, the width dimension of the baggage 20 in the image can be measured and the storage area can be identified. (c) Software-implementable means can be implemented in hardware and hardware-implementable means can be implemented in software. [Explanation of symbols]

[0082] 100,200 Support System 1 Shelf 2 Camera 3 Reading device 4 Tablet devices4 5. Labels 51 Code 6. Projector 7. Mirror 8 Processing Equipment 81 Judgment section 82 Calculation section 83 Projection control unit 84 Notification Department 85 Baggage Information DB 9. AR Glasses 91 Camera 92 Prism 93 Lens

Claims

1. A determination unit that determines a storage location and a storage area of ​​a package having an address from a first image of a shelf on which the package is placed; a projection control unit that projects a second image onto the shelf, the second image displaying the address of each of the packages based on package information that associates the address acquired from the package, the storage location, and the storage area; Equipped with The determination unit determines a storage area for the luggage to be stored in the storage area based on address information of luggage placed in the adjacent storage area, the projection control unit projects, for each of the packages, the second image onto a storage area of ​​the shelf corresponding to the package with a width of the second image corresponding to a width of the storage area; Furthermore, the projection control unit displays the storage area corresponding to the package having the same address information as the acquired address in an emphasized manner. Information processing device.

2. The load includes a first load and a second load; 2. The information processing device according to claim 1, wherein when the second luggage is stacked on the first luggage, when the loading width of the second luggage is larger than the loading width of the first luggage, the projection control unit updates the width of the second image for the first luggage to the width of the second image for the second luggage.

3. The luggage includes a third luggage and a fourth luggage, 3 . The information processing device according to claim 1 , wherein when a distance between the third piece of luggage and the fourth piece of luggage is equal to or less than a predetermined value, the projection control unit integrates the second image for the third piece of luggage and the second image for the fourth piece of luggage.

4. A pair of partition plates are provided to divide the space of the shelf to form partial spaces; The information processing device according to claim 1 , wherein the determination unit determines a storage area for the baggage placed in the partial space as a storage area corresponding to the partial space.

5. providing a label on the front surface of the shelf that allows the dimensions of the luggage to be measured; The information processing device according to claim 1 , wherein the determination unit determines the storage area for the luggage from the first image captured when the luggage is stored, the first image showing how the label is covered by the luggage.

6. providing a label on the front surface of the shelf that allows the dimensions of the luggage to be measured; The information processing device according to any one of claims 1 to 5, wherein the determination unit determines whether the removal position of the luggage specified by the worker is correct or not from the first image that captures the degree to which the label is hidden by the worker's hand when the luggage is removed.

7. The information processing apparatus according to claim 1 , wherein the projection control unit projects the second image onto the shelf using a projector attached to the shelf.

8. The information processing device according to claim 1 , wherein the projection control unit projects the second image onto a lens of AR glasses worn by a worker.

9. An information processing device, determining a storage location and a storage area of ​​the addressed package from a first image of a shelf on which the addressed package is placed; a step of projecting a second image onto the shelf, the second image displaying the address of each of the packages based on package information that associates the address acquired from the package, the storage location, and the storage area; In the determining step, a storage area for the luggage to be stored in the storage area is determined based on address information of luggage placed in the adjacent storage area; In the projecting step, the second image is projected onto a storage area of ​​the shelf corresponding to the luggage with a width of the second image corresponding to a width of the storage area for each luggage; Furthermore, in the projecting step, the storage area corresponding to the package having the same address information as the acquired address is highlighted and displayed. Information processing methods.

10. The device includes an information processing device, a camera, and a projection device, The information processing device, a determination unit that determines a storage location and a storage area of ​​the package from a first image of a shelf on which the package with an address is placed, the first image being captured by the camera; a projection control unit that causes the projection device to project onto the shelf a second image that displays the address of the package for each package based on package information that associates the address acquired from the package, the storage location, and the storage area; Equipped with The determination unit determines a storage area for the luggage to be stored in the storage area based on address information of luggage placed in the adjacent storage area, the projection control unit projects, for each of the packages, the second image onto a storage area of ​​the shelf corresponding to the package with a width of the second image corresponding to a width of the storage area; Furthermore, the projection control unit displays the storage area corresponding to the package having the same address information as the acquired address in an emphasized manner. Information processing system.

11. The computer of the information processing device, a determination unit that determines a storage position and a storage area of ​​the addressed package from a first image of a shelf on which the addressed package is placed; a projection control unit that projects a second image, which displays the address of the package for each package, on the shelf based on package information that associates the address acquired from the package, the storage location, and the storage area; causing the determination unit to determine a storage area for a package to be stored in the storage area based on address information of packages placed in adjacent storage areas; cause the projection control unit to project, for each of the packages, the second image onto a storage area of ​​the shelf corresponding to the package with a width of the second image corresponding to a width of the storage area; and further causing the projection control unit to highlight and display the storage area corresponding to the package having the same address information as the acquired address. program.

Images