Baggage Position Management System, Baggage Position Management Procedure and Program

Abstract

A system for managing object positioning, a baggage positioning procedure, and a program that can improve comfort will be provided.A baggage position management system includes: an image capture unit that captures a captured image obtained by mapping a baggage placement area; an identification information capture unit that captures identification information of the baggage entry at a terminal; an image recognition unit that recognizes a predetermined part of a worker contained in the captured image and estimates an image area of ​​the baggage contained in the captured image; a specification unit that specifies the image area with a predetermined positional relationship to the predetermined part as a baggage area of ​​the baggage; and a storage unit that stores positional information based on a pixel position of the baggage area in the captured image and the identification information in mutual association.

Description

GENERAL STATE OF THE ART [Technical area]

[0001] The present disclosure relates to a baggage position management system, a baggage position management procedure and a program. [State of the art type]

[0002] Patent literature 1 discloses a baggage location management device in which a computer captures and processes information such as the position of a manned forklift and baggage identification (ID), recording in a database where each piece of baggage is placed while the forklift performs loading and unloading operations. This technology is less prone to human error than a method of manually entering baggage storage locations into a terminal and the like. [List of quotations] [Patent literature]

[0003] [Patent Literature 1] Japanese unexamined patent application, first publication no. 2011-219229 [Summary of the invention] / [Technical problem]

[0004] In a real warehouse, however, workers might use a hand pallet truck or similar equipment to move luggage. It is difficult to equip simple devices like a hand pallet truck with highly accurate positioning capabilities. In a situation where a worker moves luggage manually, not limited to a hand pallet truck, positioning is difficult without using a forklift. In other words, the luggage location management device disclosed in patent literature 1 has a problem: its convenience can be reduced in situations where a hand pallet truck is used, where a worker moves luggage manually, or similar situations.

[0005] The present disclosure was made to solve the problems described above, and one objective of the present disclosure is to provide an object position management system, a baggage position management procedure, and a program that can improve convenience. [Solution to the problem]

[0006] To solve the problems described above, a baggage position management system is provided according to the present disclosure, comprising: an image acquisition unit configured to capture a captured image obtained by mapping a baggage placement area; an identification information acquisition unit configured to capture identification information of baggage entry at a terminal; an image recognition unit configured to recognize a predetermined part of a worker included in the captured image and to estimate an image area of ​​the baggage included in the captured image; a specification unit configured to specify the image area with a predetermined positional relationship to the predetermined part as a baggage area of ​​the baggage;and a storage unit configured to store position information based on a pixel position of the luggage area in the captured image and the identification information in mutual association.

[0007] According to the present disclosure, a baggage position management procedure is provided which includes: a step for capturing a captured image obtained by mapping a baggage placement area; a step for capturing identification information of the baggage entry at a terminal; a step for recognizing a predetermined part of a worker that is included in the captured image and for estimating an image area of ​​the baggage that is included in the captured image; a step for specifying the image area with a predetermined positional relationship to the predetermined part as a baggage area of ​​the baggage; and a step for storing positional information based on a pixel position of the baggage area in the captured image and the identification information in mutual association.

[0008] According to the present disclosure, a program is provided that causes a computer to perform the following: a step to capture a recorded image obtained by mapping a baggage placement area; a step to capture identification information of the baggage entry at a terminal; a step to recognize a predetermined part of a worker that is included in the recorded image and to estimate an image area of ​​the baggage that is included in the recorded image; a step to specify the image area with a predetermined positional relationship to the predetermined part as a baggage area of ​​the baggage; and a step to store positional information based on a pixel position of the baggage area in the recorded image and the identification information in mutual association. [Advantageous effects of the invention]

[0009] The baggage position management system, the baggage position management procedure and the program according to the present disclosure can improve comfort. [Brief description of the drawings] Fig. Figure 1 is a block diagram showing a configuration example of a baggage position management system according to a first embodiment of the present disclosure. Fig. Figure 2 is a side view which schematically shows an installation example and the like of a camera according to the first embodiment of the present disclosure. Fig. Figure 3 is a schematic diagram showing a configuration example of a position information table according to an embodiment of the present disclosure. Fig. Figure 4 is a schematic diagram showing a configuration example of a camera information table according to an embodiment of the present disclosure. Fig. Figure 5 is a flowchart showing an operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 6 is a schematic diagram showing an example of a display screen of a terminal according to the first embodiment of the present disclosure. Fig. Figure 7 is a flowchart showing an operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 8 is a flowchart showing another operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 9 is a schematic diagram to describe an operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 10 is a schematic diagram to describe an operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 11 is a schematic diagram to describe an operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 12 is a block diagram showing a configuration example of a baggage position management system according to a second embodiment of the present disclosure. Fig. Figure 13 is a side view which schematically shows an installation example and the like of a camera according to the second embodiment of the present disclosure. Fig. Figure 14 is a flowchart showing an operational example of the baggage position management system according to the second embodiment of the present disclosure. Fig. Figure 15 is a schematic diagram to describe an operational example of the baggage position management system according to the second embodiment of the present disclosure. Fig. Figure 16 is a flowchart showing an operational example of a baggage position management system according to a third embodiment of the present disclosure. Fig. Figure 17 is a diagram showing a modification example of each embodiment of the present disclosure. Fig. Figure 18 is a schematic block diagram showing a computer configuration according to at least one embodiment. [Description of the embodiments]

[0010] A baggage position management system, a baggage position management procedure, and a program according to an embodiment of the present disclosure are described below with reference to the drawings. In each drawing, the same reference numerals are used for the same or corresponding components, and their descriptions are omitted where appropriate. <Erste Ausführungsform> (Configuration of the baggage position management system)

[0011] Fig. Figure 1 is a block diagram showing a configuration example of a baggage position management system according to a first embodiment of the present disclosure. Fig. Figure 2 is a side view which schematically shows an installation example and the like of a camera according to the first embodiment of the present disclosure.

[0012] According to the representation in Fig. 1 includes a baggage tracking system 100 according to the first embodiment of the present disclosure, comprising a server 1, one or more cameras 2, and one or more terminals 3. The baggage tracking system 100 of the illustration in Fig. 1 is a system that manages (registers, updates, discards, etc.) the position (position information) at which the luggage 6 is placed in a luggage placement area 4, as shown in Fig. 2.

[0013] The luggage placement area 4 of the illustration in Fig. For example, 2 is a temporary luggage storage area and includes a ceiling 41, a floor surface 42, and a column 43. The luggage 6 is moved into and out of the luggage storage area 4 by a worker 5 using a hand pallet truck or by the worker 5 carrying the luggage 6 by hand, for example. However, the method for moving the luggage 6 is not limited to one example and may, for example, include a method for moving the luggage 6 using a forklift or the like.

[0014] In the example of the representation in Fig. 2. The luggage 6, on a pallet 7 carried by worker 5 (e.g., using a hand pallet truck, not shown), is placed on the floor surface 42 in the luggage placement area 4. A camera 2 is installed on the ceiling 41. The camera 2 records a moving image of the entire area in the luggage placement area 4 where the luggage 6 is placed, as an imaging area. The camera 2 captures images of the luggage 6 placed in the luggage placement area 4, the worker 5 moving the luggage 6, the luggage 6 and the worker 5 while it is being moved, and the like. Image information specifying the moving images captured by the camera 2 is continuously transmitted in near real-time to the server 1 via a predetermined wired or wireless communication line. Furthermore, identification information 44 of the luggage placement area 4 (“L900” in the example shown in Figure 1) is also transmitted. Fig. 2) (hereinafter also referred to as "Baggage Placement Area ID" or "Area ID") is displayed on column 43. The Baggage Placement Area ID is information used to identify Baggage Placement Area 4. In the example shown in Fig. Only one camera 2 is shown, but multiple cameras 2 can be installed in the baggage placement area 4. In such cases, it is desirable to define a baggage placement area ID for each camera 2. With a baggage placement area ID defined for each camera 2, the camera 2 that captures an image of the baggage 6 whose position is to be recorded can be specified by defining the area ID. However, if the baggage position management system 100 is equipped with only one camera 2, the baggage placement area ID does not need to be defined. Furthermore, the location where the baggage placement area ID is displayed is not limited to column 43 but could, for example, be a wall surface or the like.Furthermore, the installation position of camera 2 is not limited to the ceiling 41, but can be, for example, the column 43, a wall surface or the like.

[0015] Each piece of luggage 6 to be managed by the baggage tracking system 100 is assigned a baggage ID, which is a unique identifier for each piece of luggage 6 within a predetermined unit. This predetermined unit is an identification unit of the baggage 6 and could be, for example, each individual piece of luggage 6, each product name of the baggage 6, each lot of baggage 6, each predetermined group of baggage 6, and so on. Furthermore, the baggage tracking system 100 manages the location information of each piece of luggage 6 in association with the baggage ID. The baggage ID consists, for example, of a variety of alphanumeric characters and can be printed on a paper medium or the like as the baggage ID itself or as a barcode encoding the baggage ID and attached to the baggage 6, or it can be attached to the baggage 6 as an RFID tag recording the baggage ID.

[0016] Terminal 3, for example, is a terminal such as a tablet or smartphone, carried by worker 5. Terminal 3 includes a central processing unit (CPU), an input / output unit such as a touch panel, a communication unit for transmitting and receiving predefined information to and from server 1, and similar components. For example, in response to an operation by worker 5, Terminal 3 enters the baggage ID of baggage 6, which has been moved to baggage placement area 4, or it enters the area ID of baggage placement area 4. Additionally, Terminal 3 transmits information specifying the entered baggage ID and area ID to server 1.The baggage ID and the area ID can be manually entered into Terminal 3 by worker 5, or can be entered automatically using a camera or RFID reader provided at Terminal 3, or an external camera or RFID reader.

[0017] Server 1 of the representation in Fig. 1 includes, as functional blocks formed by a combination of hardware, such as a computer, and software, such as a program executed by the computer, an image acquisition unit 11, an identification information acquisition unit 12, an image recognition unit 13, a specification unit 14, and a storage unit 15. The storage unit 15 manages data stored in a database 151, calculates data to be stored in the database 151, and stores image information captured by the camera 2 for a specific period of time, and the like.

[0018] The image acquisition unit 11 captures (data, representational) images of the baggage placement area 4, which are taken by one or more cameras 2. The identification information acquisition unit 12 captures the baggage ID, which is identification information for the baggage, and (data, representational) the area ID entry at terminal 3. However, there are cases in which the identification information acquisition unit 12 does not capture the area ID.

[0019] The image recognition unit 13 recognizes a predetermined part of the worker 5 contained in the captured image and estimates an image area of ​​the luggage 6 contained in the captured image. Here, the predetermined parts of the worker 5 contained in the captured image are, for example, parts of the body, such as the arm (the part between the elbow and the wrist or from the shoulder to the wrist), hand, fingertips, fingers, eyes, face, head, and the like, or any attached object, such as a hat, helmet, and the like. Furthermore, the image area of ​​the luggage 6 contained in the captured image is an area in the image that could possibly be the luggage 6 (or where the probability is equal to or greater than a predetermined value).The image recognition unit 13 performs image recognition processing, such as pattern matching, to recognize predetermined parts of the worker 5 and image areas of the luggage 6, or recognizes predetermined parts of the worker 5 and image areas of the luggage 6 using a known algorithm that employs, for example, artificial intelligence (AI). The image recognition unit 13 can recognize predetermined parts of the worker 5 by analyzing a person's pose using, for example, a deep learning technology such as OpenPose. The predetermined part is not limited to a single location but can be a multitude of locations, such as two locations, for example, the elbow and the wrist.Alternatively, a predetermined marker (optical marker) can be placed on a predetermined part of the worker 5 (for example, the wrist and elbow), and the image recognition unit 13 can recognize the predetermined part by recognizing the marker.

[0020] Alternatively, the image recognition unit 13 can estimate the image area of ​​the luggage 6 based on a difference image between an image taken before the luggage ID (identification information) was entered and an image taken after the luggage ID was entered. The difference image is an image represented by the difference in pixel values ​​between the images being compared. Each image to be compared can be a single image or an image generated by statistical processing of a large number of single images. Furthermore, it is desirable that the image taken before the luggage ID is entered is an image taken before the luggage 6, for which position information is to be recorded, is placed in the luggage placement area 4.For example, in a case where the baggage ID is to be entered into Terminal 3 after baggage 6 has been placed (if baggage 6 is stationary), the image taken before the baggage ID is entered could be an image taken several seconds to several tens of seconds before the baggage ID is entered, in a case where baggage 6 is moved manually, or an image taken several minutes before the baggage ID is entered, in a case where baggage 6 is moved using a hand pallet truck.

[0021] Specification unit 14 specifies an image area, estimated to be the baggage 6 with a predetermined positional relationship to a predetermined part of the worker 5, as the baggage area of ​​the baggage 6. Here, the image area of ​​the baggage 6 is an image area that is estimated to correspond to the baggage 6 contained in the captured image, and in a case where a multitude of baggage items 6 are located in the baggage placement area 4, either being worked on or placed therein, a multitude of image areas can be recognized. Furthermore, the baggage area of ​​the baggage 6 is the image area of ​​the baggage 6 to be recorded that corresponds to the baggage ID entered at terminal 3, and is, for example, a specific image area among a multitude of estimated image areas of the baggage 6.Furthermore, the predetermined positional relationship is, for example, a relationship in which, in a case where an arm 51 of worker 5 is included as the predetermined part, according to the representation in . Fig. 2. The image area is located near the detected arm 51. Being near means, for example, that the distance between pixels in the captured image is within a predetermined value, or that the distance between the actual luggage 6 and a predetermined part, estimated from the distance between pixels in the captured image, is within a predetermined value. The predetermined positional relationship can also be one in which the predetermined part and the luggage 6 overlap in the captured image. Alternatively, the positional relationship can be one in which the posture of the worker 5 relative to a predetermined part is a predetermined posture, and the predetermined part and the luggage 6 are located within a predetermined area.For example, the predetermined parts can be the elbow and the fingertip, and the predetermined position relationship can be a relationship in which the image area of ​​baggage 6 is in the direction the fingertip points, and the distance between the fingertip and baggage 6 is within a predetermined value.

[0022] Storage unit 15 stores the position information in database 151 based on the pixel position of the luggage area specified by specification unit 14 in the captured image and the luggage ID in mutual association. If the position and orientation of camera 2 are known, the position of the luggage 6 on the ground surface 42 can be calculated based on the pixel positions of the luggage area. The process of determining the x and y coordinates on the ground surface 42 from the pixel position (mapping process) can be carried out using the following equation, which is derived from the perspective projection camera model. Here, a world coordinate system W is defined as a coordinate system in which the z-axis points vertically upwards (ground surface: z = 0), the x-axis is perpendicular to the z-axis and the y-axis, and indicates the direction of representation. Fig. 2 and the y-axis is perpendicular to the z-axis and the x-axis and indicates the direction of the representation in Fig. 2 and a camera coordinate system C is defined as a coordinate system in which the z-axis (Zc-axis) is the direction of the line of sight and the x-axis (Xc-axis) and the y-axis (Yc-axis) are the horizontal and vertical directions of the image. The position of camera 2 with respect to the world coordinate system W is represented by the translation vector of equation (1), and the orientation of camera 2 with respect to the world coordinate system W is represented by the rotation matrix of equation (2). [Math. 1] wtc=[t1t2t3] [Math. 2] wRc=[R11R12R13R21R22R23R31R32R33]

[0023] If the focal length of camera 2 is f, the length of one side of a pixel on the image sensor of camera 2 is p, and the center position of the camera image is (cx, cy) (unit: pixel), the point on the ground surface 42 corresponding to the pixel (u, v) on the image can be found in the world coordinate system W as follows. [Math. 3] [xy]=−t3(u−cx)R31+(v−cy)R32+fpR33.[R11R12R13R21R22R23][u−cxv−cyf / p]+[t1t2]

[0024] The position information based on the pixel position of the luggage area can be represented, for example, by coordinates in the world coordinate system of the display in Fig. 2. Alternatively, the position information could be, for example, information specifying each area obtained by subdividing the ground surface 42 into a plurality of areas, or information indicating the direction and distance from a predetermined reference point. The position information could, for example, be calculated in the storage unit 15, or it could be calculated in the specification unit 14, or the like.

[0025] Fig. Figure 3 is a schematic diagram showing a configuration example of a position information table according to an embodiment of the present disclosure. Fig. Figure 4 is a schematic diagram showing a configuration example of a camera information table according to an embodiment of the present disclosure. A position information table 1511 as shown in Figure 4 is shown. Fig. 3 and a camera information table 1512 in the representation in Fig. 4 are stored in database 151, in the representation in Fig. 1. In the position information table 1511, each record (each row) contains a field for a baggage ID, a field for a baggage placement area ID, and a field for position information of the baggage 6 identified by the baggage ID. The data in each field is updated by the storage unit 15. Each record in the camera information table 1512 contains a field for a baggage placement area ID, a field for a camera ID, and a field for camera information. The camera ID is identification information for camera 2, and the image information output by camera 2 is associated with the camera ID. The camera information is information used to specify the position of baggage 6 on the ground surface 42 and information indicating the position, orientation, and the like of camera 2.Each data element in the camera information table 1512 is predefined. (Operational example of the baggage position management system)

[0026] Next, an operational example of the baggage position management system 100 will be presented with reference to the Fig. 5 to 11 and the like are described. Fig. Figure 5 is a flowchart showing an operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 6 is a schematic diagram showing an example of a display screen of a terminal according to the first embodiment of the present disclosure. Fig. Figure 7 is a flowchart showing an operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figure 8 is a flowchart showing a further operational example of the baggage position management system according to the first embodiment of the present disclosure. Fig. Figures 9 to 11 are schematic diagrams to describe an operational example of the baggage position management system according to the first embodiment of the present disclosure.

[0027] Fig. Figure 5 shows a processing flow from the time worker 5 moves luggage 6 to luggage placement area 4 until server 1 records the position information of luggage 6 in database 151 (position information table 1511). The processing of the representation in Fig. 5 Camera 2 sequentially captures and transmits images (repeat of step S21), and the image acquisition unit 11 of server 1 sequentially captures the transmitted images (repeat of step S11). It is now assumed that worker 5 has finished moving the luggage 6 to the luggage placement area 4, as shown in Fig. 2, and while the worker 5 rests his / her arm 51 on the luggage 6, he / she performs an input operation on a screen 301, as shown in Fig. 6, and Terminal 3 enters the baggage ID and the baggage placement area ID (step S31). Fig. Figure 6 shows an example of display screen 301 of terminal 3. Display screen 301 of the representation in Fig. 6 includes an input field 302 for a baggage ID, an input field 303 for an area ID, and a registration button 304 for instructing an entry. If, for example, worker 5 in step S31 enters the baggage ID (in this example, "P1000") and the area ID ("L900") on screen 301 of the display in Fig. When the user enters 6 and presses the registration button 304, Terminal 3 enters the baggage ID and the area ID. Next, Terminal 3 transmits (outputs) the baggage ID and the area ID input in step S31 to Server 1 (step S32).

[0028] In server 1, the identification information acquisition unit 12 captures (receives) the baggage ID and the area ID input at terminal 3 (step S12). Next, the image acquisition unit 11 captures an image from camera 2, taken after the baggage ID has been entered at terminal 3 (repeating steps S22 and S13). Then, in step S14, the image recognition unit 13 estimates the area of ​​baggage 6 from the captured image (step S141) and recognizes a predetermined part of worker 5 (step S142). Next, the specification unit 14 specifies an area with a predetermined positional relationship to a predetermined part as a baggage area (step S15). Finally, the storage unit 15 calculates, for example, positional information based on the pixel position of the specified baggage area (step S16).Next, storage unit 15 stores the position information and the baggage ID in mutual association in database 151 (step S17).

[0029] Next, a detailed example of the processing in steps S141, S142, S15 and S16 will be presented in Fig. 5 with reference to Fig. 7 described. Fig. 7. Steps S101, S102, S103 and S104 correspond to steps S141, S142, S15 and S16 in Fig. 5. In the operational example of the presentation in Fig. 7. The image recognition unit 13 first estimates an area (image area) that may correspond to the luggage from the captured image (current image) (step S101). Next, the image recognition unit 13 recognizes the arm 51 contained in the current image (step S102). Next, the specification unit 14 specifies an image area with a predetermined positional relationship to the position and attitude of the arm 51 as a luggage area (step S103). Next, the storage unit 15 calculates the luggage position on the ground surface 42 based on the luggage area (step S104).

[0030] Next, another detailed example of the processing in steps S141, S142, S15 and S16 will be presented in Fig. 5 with reference to the Fig. 8 to 11 described. In Fig. 8. Steps S111, S112, S113 and S114 correspond to steps S141, S142, S15 and S16 in Fig. 5. In the operational example of the presentation in Fig. 8 The image recognition unit 13 first calculates a difference image between the captured image (current image) and a predetermined image that was captured before receiving the baggage ID (step S111). Fig. Figure 9 shows an example of a difference image. XP and YP are the horizontal and vertical axes of a pixel position in a difference image I1. The difference image I1 of the representation in Fig. 9 includes an image area (cluster) 801 of the luggage 6, an image area 802 corresponding to the head of worker 5, an image area 803 corresponding to the torso, image areas 804 and 805 corresponding to the legs, an image area 806 corresponding to the arm 51, and image areas 807 and 808 corresponding to the legs of other workers.

[0031] Next, the image recognition unit 13 detects arm 51, which is contained in the current image (step S112). For example, as shown in Fig. 9 The image recognition unit 13 recognizes a pixel position 809, indicated by a star, as the pixel position of arm 51 (in this example, the wrist).

[0032] Next, the image recognition unit 13 performs clustering on the difference image I1 by performing an EDM / dilation processing or the like, and specifies the cluster that is closest to arm 51 as the baggage area (step S113). Fig. Figure 10 shows an example of an image that was created by performing dilation processing (processing the removal of the boundary sections (surfaces) of each surface) on the difference image I1 of the representation in Fig. 9. A difference image I2 after dilation processing of the representation in Fig. 10 includes an image area 811 after processing image area 801, an image area 812 after processing image area 802, an image area 813 after processing image area 803, and an image area 816 after processing image area 806, which are in the difference image I1 in Fig. 9 are included. In the example of the representation in Fig. Step 10 specifies an image area 811, which is closest to the pixel position 809 of the arm 51, as the luggage area. Next, the memory unit 15 calculates the luggage position on the floor surface 42 based on the luggage area (step S114).

[0033] Fig. Figure 11 shows an example of a difference image where the predetermined parts are two locations: the wrist and the elbow. A difference image I3 of the representation in Fig. 11 includes an image area 821 of luggage 6, an image area 822 corresponding to the head of worker 5, an image area 823 corresponding to the torso, and an image area 826 corresponding to arm 51. In the difference image I3, a pixel position 809a, indicated by an asterisk, is the pixel position of arm 51 (in this example, the wrist), and a pixel position 809b is the pixel position of arm 51 (in this example, the elbow). The specification unit 14 can, for example, specify image area 821 on a line connecting pixel position 809a and pixel position 809b as the luggage area of ​​luggage 6. (Operation and effect)

[0034] The baggage position management system 100 according to the first embodiment comprises: the image acquisition unit 11, which captures a recorded image obtained by mapping the baggage placement area 4; the identification information acquisition unit 12, which captures a baggage ID (identification information) of the baggage entry at terminal 3; the image recognition unit 13, which recognizes a predetermined part (for example, the arm 51) of the worker 5 contained in the captured image and estimates an image area of ​​the baggage contained in the captured image; the specification unit 14, which specifies the image area with a predetermined positional relationship to the predetermined part as a baggage area of ​​the baggage; and the storage unit 15, which stores position information based on a pixel position of the baggage area in the captured image and the baggage ID (identification information) in mutual association.According to this configuration, for example, the comfort of the system can be easily improved, since no high-precision positioning means need to be used.

[0035] Additionally, according to the first embodiment, the image recognition unit 13 can estimate the image area based on a difference image between the captured image taken before the baggage ID (identification information) is entered and the captured image taken after the baggage ID (identification information) is entered, and the specification unit 14 can specify the image area within a predetermined range of the predetermined part as the baggage area. <Zweite Ausführungsform>

[0036] Next, a baggage position management system according to a second embodiment of the present disclosure is described below with reference to the Fig. 12 to 15 described. Fig. Figure 12 is a block diagram showing a configuration example of a baggage position management system according to the second embodiment of the present disclosure. Fig. Figure 13 is a side view which schematically shows an installation example and the like of a camera according to the second embodiment of the present disclosure. Fig. Figure 14 is a flowchart showing an operational example of the baggage position management system according to the second embodiment of the present disclosure. Fig. Figure 15 is a schematic diagram to describe an operational example of the baggage position management system according to the second embodiment of the present disclosure.

[0037] According to the representation in Fig. 12. The baggage position management system 100a according to the second embodiment differs from the baggage position management system 100 according to the first embodiment of the illustration in Fig. 1 in the following respect. That is, does a server 1a differ in its representation in Fig. 12 corresponding to server 1, the representation in Fig. 1 partially in operation of a specification unit 14a of the representation in Fig. 12 corresponding to the specification unit 14 of the representation in Fig. 1. Furthermore, server 1a includes the representation in Fig. 12 new terminal image acquisition unit 16 and terminal image recognition unit 17. In addition, terminal 3a includes the representation in Fig. 12 corresponding to terminal 3 of the representation in Fig. 1 a camera 31. According to the illustration in Fig. 13 A two-dimensional code 45 is attached to column 43 of baggage placement area 4. The two-dimensional code 45 has a predetermined shape and size. Furthermore, the two-dimensional code 45 can, for example, contain a baggage placement area ID or information that can be assigned to a baggage placement area ID in a coded form. The two-dimensional code 45 is a configuration example of a “marker” according to the present disclosure.

[0038] In the second embodiment, as shown in Fig. 13, worker 5 uses camera 31 of terminal 3a to capture an image of a two-dimensional code 45 and transmits the captured image (hereinafter referred to as the terminal image) to server 1. In server 1, the terminal image capture unit 16 captures a terminal image containing a two-dimensional code 45 (a predetermined marker) attached to the baggage placement area 4, which is captured by terminal 3a. If the size and position of the two-dimensional code 45 are known, server 1 can estimate the approximate position of terminal 3a from the terminal image. The terminal image recognition unit 17 estimates the position of terminal 3a based on the terminal image. Furthermore, the terminal image recognition unit 17 decodes the two-dimensional code 45 from the terminal image and recognizes the baggage placement area ID. In this case, it is not necessary to enter the baggage placement area ID at terminal 3a.Furthermore, the terminal image recognition unit 17 calculates the pixel position of terminal 3a in the captured image taken by camera 2, using the mapping process described above based on the estimated position of terminal 3a.

[0039] While the specification unit 14 of the representation in Fig. 1. Additionally, an image area with a predetermined positional relationship to a predetermined part of the worker 5 is specified as a luggage area, the specification unit 14a of the representation in Fig. 12. A picture surface with a predetermined positional relationship to a predetermined part of worker 5 and terminal 3a as a baggage area. The fact that a predetermined part of worker 5 and terminal 3a have a predetermined positional relationship to the picture surface means, for example, that a distance between the predetermined part and the picture surface and a distance between terminal 3a and the picture surface both lie within a predetermined range; that an average value of a distance between the predetermined part and the picture surface and a distance between terminal 3a and the picture surface lie within a predetermined range; that a distance between the predetermined part and the picture surface and a distance between terminal 3a and the picture surface, or an average value thereof, is shortest compared to other picture surfaces; and the like.However, the predetermined relationship is not limited to these examples and can, for instance, be a relationship that takes into account a direction determined by a multitude of predetermined parts, as illustrated in the first embodiment.

[0040] In the flowchart of the representation in Fig. 14, which is an operational example of the baggage position management system 100a according to the second embodiment compared to Fig. Step S301, capturing a terminal image, and step S302, transmitting the terminal image to server 1a, are additionally added to terminal 3a. Additionally, step S121, in which terminal image recognition unit 17 captures a terminal image, step S122, in which terminal image recognition unit 17 recognizes an area ID, and step S123, in which terminal image recognition unit 17 estimates the position of terminal 3a, are added to server 1a. Step S15a corresponds to step S15 of the representation in Fig. 3 specifies the specification unit 14a as an image area with a predetermined positional relationship to a predetermined part and terminal 3a as a baggage area.

[0041] Fig. Figure 15 shows an example of a captured image, taken by camera 2 as image I4. In this example, the image recognition unit 13 recognizes the areas of worker 5, luggage 6, and the like, for example, through image recognition processing such as pattern matching. The image recognition unit 13 performs predetermined image recognition processing on the captured image I4, thereby recognizing image area 801 of luggage 6, image area 802 corresponding to worker 5's head, image area 803 corresponding to the torso, image areas 804 and 805 corresponding to the legs, image area 806 corresponding to arm 51, and image areas 807 and 808 corresponding to the legs of other workers.Furthermore, the image recognition unit 13 also recognizes an image area 831 of another piece of luggage 6, an image area 832 corresponding to the head of another worker 5, an image area 833 corresponding to the torso, image areas 834 and 835 corresponding to the legs, and an image area 836 corresponding to the arm. Additionally, the image recognition unit 13 recognizes a pixel position 809 of arm 51 and a pixel position 839 of the arm of another worker 5, and the terminal image recognition unit 17 calculates and recognizes a pixel position 841 of terminal 3a.

[0042] In this case, the captured image I4 includes image areas 801 and 831 of the two pieces of luggage, and the pixel positions 809 and 839 of the predetermined parts are detected near each of image areas 801 and 831. However, specification unit 14a can specify that image area 801 of destination luggage 6 is the luggage area based on pixel position 841 of terminal 3a.

[0043] According to the second embodiment, baggage positions can be recorded in the database without malfunction, even when multiple workers are present in the same baggage placement area. Furthermore, entering the baggage placement area ID is simpler than manual entry. <Dritte Ausführungsform>

[0044] Next, a baggage position management system according to a third embodiment of the present disclosure is described with reference to Fig. 16 described. Fig. Figure 16 is a flowchart showing an operational example of a baggage position management system according to the third embodiment of the present disclosure. Fig. Figure 16 shows a modification example of the processing of the representation in Fig. 7, which serves as a detailed example of the processing in steps S141, S142, S15 and S16 in Fig. 5 in the first embodiment. Fig. Steps S103A and S103B are redeployed between steps S103 and S104. In step S103A, specification unit 14 determines whether a plurality of baggage areas is specified in step S103. If there is a plurality of baggage areas (step S103A: YES), specification unit 14 in step S103B processes the transmission of an image containing the plurality of baggage areas to terminal 3 and instructs terminal 3 to select a baggage area.

[0045] In the third embodiment, in a case where there are a plurality of image areas that are candidates for the baggage area, the specification unit 14 transmits image information containing the plurality of image areas that are candidates to the terminal 3 and causes the terminal 3 to select the baggage area.

[0046] According to the third embodiment, for example, the recorded image I4, the representation in Fig. 15 is transferred to Terminal 3, and on the Terminal 3 side, either baggage area 801 or baggage area 831 can be selected. The configuration for selecting a baggage area in the third embodiment can be combined with the first and second embodiments. <Andere Ausführungsformen>

[0047] Although the embodiments of the present disclosure have been described in detail above with reference to the drawings, the specific configuration is not limited to the present embodiment, and design changes and the like that do not deviate from the core of the present disclosure are also included. For example, part of the configuration of server 1 or 1a may be included in camera 2 or terminal 3 or 3a.

[0048] According to the representation in Fig. 17. The baggage ID, baggage area position information, and the like, captured by the baggage position management system 100 (100a), can also be configured to be transmitted to a warehouse management system 200, which manages the position information and the like of other manned or unmanned forklifts operating in a logistics warehouse, including a baggage placement area. With this configuration, other manned or unmanned forklifts can determine the current position of baggage moved by a worker using a pallet jack or manually, and can work in conjunction with the worker to perform loading and unloading operations within the warehouse.Furthermore, the baggage position management system 100 (100a) can be configured separately from the warehouse management system 200, which manages the position information and the like of manned or unmanned forklifts, or it can be integrated into the warehouse management system 200. This is shown in the configuration example in [reference to relevant document]. Fig. In section 17, the warehouse management system 200 includes a database 201 that stores position information for manned or unmanned forklifts, position information for each piece of luggage, and the like. For example, if the luggage position management system 100 (100a) records, updates, or otherwise performs actions on the luggage position information, data specifying the luggage ID and position information are sent from the luggage position management system 100 (100a) to the warehouse management system 200, and the contents of database 201 are updated. The warehouse management system 200 and database 201 can be partially or fully networked. The configuration example shown in [reference missing] Fig. 17. The position information captured by the baggage position management system 100 (100a) is managed together with at least one intermediary between the position information of a manned forklift and the position information of an unmanned forklift. In this case, the management includes, for example, unified management, centralized management, and concentrated management of information. <computerkonfiguration>

[0049] Fig. Figure 18 is a schematic block diagram showing a configuration of a computer according to at least one embodiment.

[0050] A computer 90 includes a processor 91, a main memory 92, a storage 93 and an interface 94.

[0051] Servers 1 and 1a, described above, are implemented in computer 90. Additionally, a process for each of the processing units described above is stored as a program in memory 93. Processor 91 reads the program from memory 93, loads the program into main memory 92, and executes the processing described above according to the program. Processor 91 also allocates a memory area in main memory 92, corresponding to each of the memory units described above, according to the program.

[0052] The program can be used to implement some of the functions of the Computer 90. For example, the program can perform its function in combination with other programs already stored in memory or in combination with other programs implemented in other devices. In other embodiments, the computer can include, in addition to or instead of the configuration described above, a user-defined large-scale integrated circuit (LSI), such as a programmable logic device (PLD). Examples of PLDs include programmable array logic (PAL), generic array logic (GAL), complex programmable logic device (CPLD), and field programmable gate array (FPGA).In this case, some or all of the functions implemented by the processor can be implemented by the integrated circuit.

[0053] Examples of memory 93 include a hard disk drive (HDD), a solid-state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. Memory 93 can be an internal medium directly connected to a bus of computer 90, or it can be an external medium connected to computer 90 via interface 94 or a communication line. Additionally, if the program is distributed to computer 90 via the communication line, the computer 90 receiving the distribution can load the program into main memory 92 and perform the processing described above. In at least one embodiment, memory 93 is a non-volatile physical storage medium. <Ergänzende Anmerkungen>

[0054] The baggage position management systems 100 and 100a described in each embodiment can be understood, for example, as follows. (1) The baggage position management systems 100 and 100a according to a first aspect include: an image acquisition unit configured to capture a captured image obtained by mapping a baggage placement area; an identification information acquisition unit configured to capture identification information of baggage entry at a terminal; an image recognition unit configured to recognize a predetermined part of a worker included in the captured image and to estimate an image area of ​​the baggage included in the captured image; a specification unit configured to specify the image area with a predetermined positional relationship to the predetermined part as a baggage area of ​​the baggage;and a storage unit configured to store positional information based on a pixel position of the luggage area in the captured image and the identification information in mutual association. Comfort can be improved according to this aspect and each of the following aspects. (2) The baggage position management systems 100 and 100a according to a second aspect are the baggage position management systems 100 and 100a according to (1) wherein the image recognition unit estimates the image area on the basis of a difference image between the image taken before the input of the identification information and the image taken after the input of the identification information, and the specification unit specifies the image area within a predetermined range of the predetermined part as the baggage area. (3) The baggage position management system 100a according to a third aspect is the baggage position management system 100a according to (1) or (2), further comprising: a terminal image acquisition unit configured to acquire a terminal image containing a predetermined marker attached to the baggage placement area captured by the terminal; and a terminal image recognition unit configured to estimate a position of the terminal based on the terminal image, wherein the specification unit specifies the image area with a predetermined positional relationship to the predetermined part and the terminal as the baggage area. (4) The baggage position management systems 100 and 100a according to a fourth aspect are the baggage position management systems 100 and 100a according to (1) to (3), wherein, in a case where there is a plurality of image areas that are candidates for the baggage area, the specification unit transmits image information including the plurality of image areas that are candidates to the terminal and causes the terminal to select the baggage area. (5) The baggage position management systems 100 and 100a according to a fifth aspect are the baggage position management systems 100 and 100a according to (1) to (4) in which the position information is managed together with at least one between position information of a manned forklift and position information of an unmanned forklift.

[0055] Although preferred embodiments of the invention have been described and illustrated above, it is understood that these are exemplary of the invention and are not to be considered limiting. Additions, omissions, substitutions, and other modifications may be made without deviating from the scope of the invention. Accordingly, the invention is not to be considered limited by the foregoing description and is limited only by the scope of the appended claims. [List of reference symbols] 100, 100a Baggage Position Management System 1, 1a Server 2 cameras 3, 3a Terminal 4 luggage storage areas 5 workers 6 pieces of luggage 11 Image capture unit 12 Identification Information Acquisition Unit 13 Image recognition unit 14 Specification unit 15 storage units 16 Terminal image capture unit 17 Terminal image recognition unit 44 Identification information 45 Two-dimensional code 51 Arm QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2011-219229

[0003] < / computerkonfiguration>

Claims

[1] Baggage position management system, comprising: an image capture unit configured to capture a recorded image obtained by mapping a baggage placement area; an identification information capture unit configured to capture identification information of baggage entry at a terminal; an image recognition unit configured to recognize a predetermined part of a worker included in the captured image and to estimate an image area of ​​the luggage included in the captured image; a specification unit configured to specify the image area with a predetermined positional relationship to the predetermined part as a luggage area of ​​the luggage; and a storage unit configured to store position information based on a pixel position of the luggage area in the captured image and the identification information in mutual association. [2] Baggage position management system according to claim 1, wherein the image recognition unit estimates the image area based on a difference image between the captured image taken before the input of the identification information and the captured image taken after the input of the identification information, and The specification unit specifies the image area within a predetermined range of the predetermined part as the luggage area. [3] Baggage position management system according to claim 1 or 2, further comprising: a terminal image capture unit configured to capture a terminal image that includes a predetermined marker attached to the baggage placement area, which is captured by the terminal; and a terminal image recognition unit configured to estimate the terminal's position based on the terminal image, where the specification unit specifies the image area with a predetermined positional relationship to the predetermined part and the terminal as the baggage area. [4] Baggage position management system according to any one of claims 1 to 3, wherein in a case where there is a plurality of image areas that are candidates for the baggage area, the specification unit transmits image information containing the plurality of image areas that are the candidates to the terminal and causes the terminal to select the baggage area. [5] Baggage position management system according to one of claims 1 to 4, wherein the position information is managed together with at least one between position information of a manned forklift and position information of an unmanned forklift. [6] Baggage position management procedures, including: a step towards capturing a recorded image obtained by mapping a luggage placement area; a step towards capturing identification information of baggage entry at a terminal; a step towards recognizing a predetermined part of a worker that is included in the recorded image and estimating an image area of ​​the luggage that is included in the recorded image; a step towards specifying the image area with a predetermined positional relationship to the predetermined part as the luggage area of ​​the luggage; and a step to store position information based on a pixel position of the luggage area in the recorded image and the identification information in mutual association. [7] Computer program that causes a computer to perform the following: a step towards capturing a recorded image obtained by mapping a luggage placement area; a step towards capturing identification information of baggage entry at a terminal; a step towards recognizing a predetermined part of a worker that is included in the recorded image and estimating an image area of ​​the luggage that is included in the recorded image; a step towards specifying the image area with a predetermined positional relationship to the predetermined part as the luggage area of ​​the luggage; and a step to store position information based on a pixel position of the luggage area in the recorded image and the identification information in mutual association.

Images